Method: Dual-model architecture integrating Projected Fusion block for adaptive inter-layer feature weighting and Differential Feature Learning mechanism that identifies discriminative features by computing differences between parallel encoders learning complementary representations from identical inputs.

Result: Achieved significant accuracy improvements on question answering and dialogue tasks. Analysis revealed a hierarchical ‘funnel pattern’ where shallow layers show high feature diversity while deep layers demonstrate concentrated usage, enabling detection with only 1% of feature dimensions with minimal performance degradation.

Conclusion: Hallucination signals are more concentrated than previously assumed, offering a pathway toward computationally efficient detection systems that could reduce inference costs while maintaining accuracy.

Abstract: Large language model hallucination represents a critical challenge where outputs deviate from factual accuracy due to distributional biases in training data. While recent investigations establish that specific hidden layers exhibit differences between hallucinatory and factual content, the precise localization of hallucination signals within layers remains unclear, limiting the development of efficient detection methods. We propose a dual-model architecture integrating a Projected Fusion (PF) block for adaptive inter-layer feature weighting and a Differential Feature Learning (DFL) mechanism that identifies discriminative features by computing differences between parallel encoders learning complementary representations from identical inputs. Through systematic experiments across HaluEval’s question answering, dialogue, and summarization datasets, we demonstrate that hallucination signals concentrate in highly sparse feature subsets, achieving significant accuracy improvements on question answering and dialogue tasks. Notably, our analysis reveals a hierarchical “funnel pattern” where shallow layers exhibit high feature diversity while deep layers demonstrate concentrated usage, enabling detection performance to be maintained with minimal degradation using only 1% of feature dimensions. These findings suggest that hallucination signals are more concentrated than previously assumed, offering a pathway toward computationally efficient detection systems that could reduce inference costs while maintaining accuracy.

Method: Reconceptualize context quality assessment as inference-time data valuation using CI value metric, develop parameterized surrogate model with hierarchical architecture capturing local query-context relevance and global inter-context interactions, trained through oracle CI value supervision and end-to-end generator feedback.

Result: Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate significant outperformance over state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information.

Conclusion: The CI value approach provides a comprehensive solution for context quality assessment in RAG systems, eliminating complex hyperparameter tuning and achieving superior performance through holistic utilization of available information.

Abstract: Retrieval-Augmented Generation (RAG) addresses large language model (LLM) hallucinations by grounding responses in external knowledge, but its effectiveness is compromised by poor-quality retrieved contexts containing irrelevant or noisy information. While existing approaches attempt to improve performance through context selection based on predefined context quality assessment metrics, they show limited gains over standard RAG. We attribute this limitation to their failure in holistically utilizing available information (query, context list, and generator) for comprehensive quality assessment. Inspired by recent advances in data selection, we reconceptualize context quality assessment as an inference-time data valuation problem and introduce the Contextual Influence Value (CI value). This novel metric quantifies context quality by measuring the performance degradation when removing each context from the list, effectively integrating query-aware relevance, list-aware uniqueness, and generator-aware alignment. Moreover, CI value eliminates complex selection hyperparameter tuning by simply retaining contexts with positive CI values. To address practical challenges of label dependency and computational overhead, we develop a parameterized surrogate model for CI value prediction during inference. The model employs a hierarchical architecture that captures both local query-context relevance and global inter-context interactions, trained through oracle CI value supervision and end-to-end generator feedback. Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate that our context selection method significantly outperforms state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information. Code is available at https://github.com/SJTU-DMTai/RAG-CSM.

Method: Defined maximum effective context window concept, formulated testing methods for context window effectiveness across various sizes and problem types, and created standardized comparison methods to identify failure points.

Result: Found significant differences between reported MCW and MECW across multiple models, with most models showing severe accuracy degradation by 1000 tokens and some failing with as few as 100 tokens. MECW varies based on problem type.

Conclusion: Maximum effective context window is drastically different from maximum context window and shifts based on problem type, providing actionable insights for improving model accuracy and reducing hallucinations.

Abstract: Large language model (LLM) providers boast big numbers for maximum context window sizes. To test the real world use of context windows, we 1) define a concept of maximum effective context window, 2) formulate a testing method of a context window’s effectiveness over various sizes and problem types, and 3) create a standardized way to compare model efficacy for increasingly larger context window sizes to find the point of failure. We collected hundreds of thousands of data points across several models and found significant differences between reported Maximum Context Window (MCW) size and Maximum Effective Context Window (MECW) size. Our findings show that the MECW is, not only, drastically different from the MCW but also shifts based on the problem type. A few top of the line models in our test group failed with as little as 100 tokens in context; most had severe degradation in accuracy by 1000 tokens in context. All models fell far short of their Maximum Context Window by as much as 99 percent. Our data reveals the Maximum Effective Context Window shifts based on the type of problem provided, offering clear and actionable insights into how to improve model accuracy and decrease model hallucination rates.

Method: Propose using human-crafted symbols as a compass to guide large language models’ intuition.

Result: Not specified in the abstract - this appears to be a position paper proposing a conceptual framework.

Conclusion: The future of AI requires more than scaling; it needs human-crafted symbols to provide direction and enable genuine discovery by guiding LLMs’ powerful but blind intuition.

Abstract: We argue that AI’s future requires more than scaling. To unlock genuine discovery, large language models need a compass: human-crafted symbols to guide their powerful but blind intuition.

Method: Translated two established moral reasoning benchmarks into five culturally and typologically diverse languages for multilingual zero-shot evaluation, combined with carefully constructed research questions to analyze underlying drivers of moral judgment disparities.

Result: Revealed significant inconsistencies in LLMs’ moral judgments across languages, often reflecting cultural misalignment, and identified various drivers including disagreements and reasoning strategy differences.

Conclusion: Developed a structured typology of moral reasoning errors that emphasizes the need for more culturally-aware AI systems to address cross-lingual moral judgment inconsistencies.

Abstract: Large Language Models (LLMs) are increasingly deployed in multilingual and multicultural environments where moral reasoning is essential for generating ethically appropriate responses. Yet, the dominant pretraining of LLMs on English-language data raises critical concerns about their ability to generalize judgments across diverse linguistic and cultural contexts. In this work, we systematically investigate how language mediates moral decision-making in LLMs. We translate two established moral reasoning benchmarks into five culturally and typologically diverse languages, enabling multilingual zero-shot evaluation. Our analysis reveals significant inconsistencies in LLMs’ moral judgments across languages, often reflecting cultural misalignment. Through a combination of carefully constructed research questions, we uncover the underlying drivers of these disparities, ranging from disagreements to reasoning strategies employed by LLMs. Finally, through a case study, we link the role of pretraining data in shaping an LLM’s moral compass. Through this work, we distill our insights into a structured typology of moral reasoning errors that calls for more culturally-aware AI.

Method: Evaluated GPT-5 using a simulation framework with synthetic cases aligned with ADA Standards of Care 2025, inspired by public datasets (NHANES, Pima Indians, EyePACS, MIMIC-IV). Tested five scenarios: symptom recognition, lab interpretation, gestational diabetes screening, remote monitoring, and multimodal complication detection.

Result: GPT-5 showed strong alignment with ADA-defined criteria across all tested scenarios. It successfully classified cases, generated clinical rationales, produced patient explanations, and output structured JSON summaries.

Conclusion: GPT-5 may function as a dual-purpose tool for clinicians and patients in diabetes care, while underscoring the importance of reproducible evaluation frameworks for responsibly assessing LLMs in healthcare.

Abstract: Diabetes mellitus is a major global health challenge, affecting over half a billion adults worldwide with prevalence projected to rise. Although the American Diabetes Association (ADA) provides clear diagnostic thresholds, early recognition remains difficult due to vague symptoms, borderline laboratory values, gestational complexity, and the demands of long-term monitoring. Advances in large language models (LLMs) offer opportunities to enhance decision support through structured, interpretable, and patient-friendly outputs. This study evaluates GPT-5, the latest generative pre-trained transformer, using a simulation framework built entirely on synthetic cases aligned with ADA Standards of Care 2025 and inspired by public datasets including NHANES, Pima Indians, EyePACS, and MIMIC-IV. Five representative scenarios were tested: symptom recognition, laboratory interpretation, gestational diabetes screening, remote monitoring, and multimodal complication detection. For each, GPT-5 classified cases, generated clinical rationales, produced patient explanations, and output structured JSON summaries. Results showed strong alignment with ADA-defined criteria, suggesting GPT-5 may function as a dual-purpose tool for clinicians and patients, while underscoring the importance of reproducible evaluation frameworks for responsibly assessing LLMs in healthcare.

Method: The authors introduce a MORL taxonomy, examine advantages and limitations of various MORL methods for LLM optimization, and propose a MORL benchmarking framework to evaluate different methods’ effects on diverse objective relationships.

Result: The paper identifies the need for improved MORL approaches for LLMs and proposes a vision for future research focusing on meta-policy MORL development with bi-level learning paradigms to enhance efficiency and flexibility.

Conclusion: The conclusion emphasizes the importance of developing meta-policy MORL approaches that can improve LLM performance through bi-level learning, highlighting key research questions and potential solutions for advancing MORL applications in LLM optimization.

Abstract: Multi-Objective Reinforcement Learning (MORL) presents significant challenges and opportunities for optimizing multiple objectives in Large Language Models (LLMs). We introduce a MORL taxonomy and examine the advantages and limitations of various MORL methods when applied to LLM optimization, identifying the need for efficient and flexible approaches that accommodate personalization functionality and inherent complexities in LLMs and RL. We propose a vision for a MORL benchmarking framework that addresses the effects of different methods on diverse objective relationships. As future research directions, we focus on meta-policy MORL development that can improve efficiency and flexibility through its bi-level learning paradigm, highlighting key research questions and potential solutions for improving LLM performance.

Method: Analyzed the mechanisms of performance trade-off through the lens of forgetting and fairness objectives, examining how selective forgetting of stereotypes affects overall forgetting levels.

Result: Found that downstream performance is driven by overall forgetting level, selective stereotype forgetting increases overall forgetting, and general forgetting mitigation solutions are ineffective.

Conclusion: Current fairness objectives have limitations in achieving performance trade-offs due to their tendency to cause excessive overall forgetting that cannot be effectively mitigated.

Abstract: Moral alignment has emerged as a widely adopted approach for regulating the behavior of pretrained language models (PLMs), typically through fine-tuning or model editing on curated datasets. However, this process often comes at the cost of degraded downstream task performance. Prior studies commonly aim to achieve a performance trade-off by encouraging PLMs to selectively forget stereotypical knowledge through carefully designed fairness objectives, while preserving their helpfulness. In this short paper, we investigate the underlying mechanisms of the performance trade-off in the context of mitigating gender stereotypes, through the lens of forgetting and the fairness objective. Our analysis reveals the limitations of current fairness objective in achieving trade-off by demonstrating that: (1) downstream task performance is primarily driven by the overall forgetting level; (2) selective forgetting of stereotypes tends to increase overall forgetting; and (3) general solutions for mitigating forgetting are ineffective at reducing overall forgetting and fail to improve downstream task performance.

Method: A simple training recipe involving careful prompt and model selection, warm-up using offline RL approach called TAO, followed by rigorous online RLVR training. No additional training data beyond BIRD training set and no proprietary models used.

Result: Achieved state-of-the-art accuracy on BIRD private test set: 73.56% without self-consistency and 75.68% with self-consistency. Required fewer generations than the second-best approach.

Conclusion: The simplicity of the RLVR framework makes it broadly applicable to enterprise domains like business intelligence, data science, and coding, demonstrating strong potential for custom reasoning models in enterprise settings.

Abstract: Developing custom reasoning models via Reinforcement Learning (RL) that can incorporate organization-specific knowledge has great potential to address problems faced by enterprise customers. In many of these problems, the reward function is verifiable, a setting termed RL with Verifiable Rewards (RLVR). We apply RLVR to a popular data science benchmark called BIRD that measures the ability of an AI agent to convert a natural language query for a database to SQL executions. We apply a simple and general-purpose training recipe involving careful prompt and model selection, a warm-up stage using our offline RL approach called TAO, followed by rigorous online RLVR training. With no additional training data beyond the BIRD training set and no use of proprietary models, our very first submission to the BIRD leaderboard reached state-of-the-art accuracy on the private test set: 73.56% without self-consistency and 75.68% with self-consistency. In the latter case, our model also required fewer generations than the second-best approach. While BIRD is only a proxy task, the simplicity of our framework makes it broadly applicable to enterprise domains such as business intelligence, data science, and coding.

Method: Propose mixture-of-token generation (MoT-G) in RLVR, extending it to operate directly in continuous mixture space for chain-of-thought generation, including training-free methods using weighted sums over token embeddings.

Result: MoT-G methods achieve 5-35% gains on 7 out of 10 reasoning tasks compared to standard decoding, reaching comparable accuracy with half the trajectories, suggesting improved training efficiency.

Conclusion: MoT-G’s benefits stem from maintaining higher hidden-state entropy and promoting exploration in token space, demonstrating the value of preserving distributional information in RLVR.

Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a leading approach for improving large language model (LLM) reasoning capabilities. Most current methods follow variants of Group Relative Policy Optimization, which samples multiple reasoning completions, scores them relative to each other, and adjusts the policy accordingly. However, these approaches invariably sample discrete tokens at each reasoning step, discarding the rich distributional information in the model’s probability distribution over candidate tokens. While preserving and utilizing this distributional information has proven beneficial in non-RL settings, current RLVR methods seem to be unnecessarily constraining the reasoning search space by not using this information. To address this limitation, we investigate mixture-of-token generation (MoT-G) in RLVR. We present a unified framework that generalizes existing MoT-G approaches, including existing training-free methods that construct mixture embeddings as weighted sums over token embeddings, and extend RLVR to operate directly in this continuous mixture space for generating chain-of-thought. Evaluating two MoT-G variants on Reasoning-Gym, a suite of reasoning-intensive language tasks, we find that MoT–G methods achieve substantial improvements (5–35 % gains on 7 out of 10 tasks) compared to standard decoding with the Qwen2.5-1.5B model, while reaching comparable accuracy with half the number of trajectories, suggesting improved training efficiency. Through comprehensive hidden-state and token-level analyses, we provide evidence that MoT–G’s benefits may stem from its ability to maintain higher hidden-state entropy throughout the reasoning process and promote exploration in token space.

Method: Task-decomposed reasoning framework that breaks empathy into a pipeline for analysis (building comprehensive understanding) and synthesis (generation), uniting deep analysis with expressive generation.

Result: Significantly outperforms strong baselines in both automatic and LLM-based evaluations, demonstrating the effectiveness of structured decomposition for empathetic agents.

Conclusion: Structured decomposition is a promising paradigm for creating more capable and interpretable empathetic agents, successfully bridging the gap between analytical depth and generative fluency.

Abstract: Empathetic response generation is a crucial task for creating more human-like and supportive conversational agents. However, existing methods face a core trade-off between the analytical depth of specialized models and the generative fluency of Large Language Models (LLMs). To address this, we propose TRACE, Task-decomposed Reasoning for Affective Communication and Empathy, a novel framework that models empathy as a structured cognitive process by decomposing the task into a pipeline for analysis and synthesis. By building a comprehensive understanding before generation, TRACE unites deep analysis with expressive generation. Experimental results show that our framework significantly outperforms strong baselines in both automatic and LLM-based evaluations, confirming that our structured decomposition is a promising paradigm for creating more capable and interpretable empathetic agents. Our code is available at https://anonymous.4open.science/r/TRACE-18EF/README.md.

Method: Adds a pooled classification head and a reasoning head supervised by teacher rationales, trained with weighted sum of label cross-entropy and token-level LM loss over input-plus-rationale sequences, then disables reasoning head at test time.

Result: Achieves 0.65-5.47% relative gains over pooled baselines on seven SuperGLUE tasks, with larger gains on entailment/causal tasks, while matching pooled classifier throughput and exceeding CoT decoding by 96-142 times in QPS.

Conclusion: DHRD successfully decouples reasoning benefits from inference costs, enabling accuracy improvements without throughput penalties.

Abstract: Chain-of-Thought (CoT) prompting often improves classification accuracy, but it introduces a significant throughput penalty with rationale generation (Wei et al., 2022; Cheng and Van Durme, 2024). To resolve this trade-off, we introduce Dual-Head Reasoning Distillation (DHRD), a simple training method for decoder-only language models (LMs) that adds (i) a pooled classification head used during training and inference and (ii) a reasoning head supervised by teacher rationales used only in training. We train with a loss function that is a weighted sum of label cross-entropy and token-level LM loss over input-plus-rationale sequences. On seven SuperGLUE tasks, DHRD yields relative gains of 0.65-5.47% over pooled baselines, with notably larger gains on entailment/causal tasks. Since we disable the reasoning head at test time, inference throughput matches pooled classifiers and exceeds CoT decoding on the same backbones by 96-142 times in QPS.

Method: Systematic framework using parallel instruction datasets in 10 programming languages with controlled perturbations disrupting structural or semantic properties, finetuning LLMs from 5 families and 8 scales, evaluating on 3,331 experiments across natural language, math, and code tasks.

Result: Structural perturbations hurt performance more than semantic ones, especially on math/code tasks; pseudocode/flowcharts work as well as code; corrupted code with surface regularities remains competitive; different languages favor different tasks (Python for natural language, Java/Rust for math).

Conclusion: Different code properties influence reasoning in distinct ways, providing insights for designing training data to enhance LLM reasoning capabilities.

Abstract: Code data has been shown to enhance the reasoning capabilities of large language models (LLMs), but it remains unclear which aspects of code are most responsible. We investigate this question with a systematic, data-centric framework. We construct parallel instruction datasets in ten programming languages and apply controlled perturbations that selectively disrupt structural or semantic properties of code. We then finetune LLMs from five model families and eight scales on each variant and evaluate their performance on natural language, math, and code tasks. Across 3,331 experiments, our results show that LLMs are more vulnerable to structural perturbations than semantic ones, particularly on math and code tasks. Appropriate abstractions like pseudocode and flowcharts can be as effective as code, while encoding the same information with fewer tokens without adhering to original syntax can often retain or even improve performance. Remarkably, even corrupted code with misleading signals remains competitive when surface-level regularities persist. Finally, syntactic styles also shape task-specific gains with Python favoring natural language reasoning and lower-level languages such as Java and Rust favoring math. Through our systematic framework, we aim to provide insight into how different properties of code influence reasoning and inform the design of training data for enhancing LLM reasoning capabilities.

Method: Built a chatbot using sentence embedding model from Kisan Call Center dataset. Incorporated entity extraction and eliminated synonyms to improve accuracy.

Result: Initial accuracy of 56% with sentence embedding model. After improvements, accuracy increased to 86%. System provides 24/7 access through any electronic device.

Conclusion: The chatbot enables easier access to farming information, improves agricultural output, and reduces workload for call center staff by redirecting their efforts to more valuable tasks.

Abstract: India is an agro-based economy and proper information about agricultural practices is the key to optimal agricultural growth and output. In order to answer the queries of the farmer, we have build an agricultural chatbot based on the dataset from Kisan Call Center. This system is robust enough to answer queries related to weather, market rates, plant protection and government schemes. This system is available 24* 7, can be accessed through any electronic device and the information is delivered with the ease of understanding. The system is based on a sentence embedding model which gives an accuracy of 56%. After eliminating synonyms and incorporating entity extraction, the accuracy jumps to 86%. With such a system, farmers can progress towards easier information about farming related practices and hence a better agricultural output. The job of the Call Center workforce would be made easier and the hard work of various such workers can be redirected to a better goal.

Method: Evaluated multiple production and open systems using strict DER protocol with overlap scoring. Conducted error analysis and tested lightweight domain adaptation by fine-tuning segmentation module on accent-matched data.

Result: Consistent domain penalty appears for clinical speech across all models, attributed to false alarms and missed detections from short turns and frequent overlap. Domain adaptation reduces error but doesn’t eliminate the performance gap.

Conclusion: Results point to overlap-aware segmentation and balanced clinical resources as practical next steps for improving speaker diarization in clinical domains.

Abstract: This study examines domain effects in speaker diarization for African-accented English. We evaluate multiple production and open systems on general and clinical dialogues under a strict DER protocol that scores overlap. A consistent domain penalty appears for clinical speech and remains significant across models. Error analysis attributes much of this penalty to false alarms and missed detections, aligning with short turns and frequent overlap. We test lightweight domain adaptation by fine-tuning a segmentation module on accent-matched data; it reduces error but does not eliminate the gap. Our contributions include a controlled benchmark across domains, a concise approach to error decomposition and conversation-level profiling, and an adaptation recipe that is easy to reproduce. Results point to overlap-aware segmentation and balanced clinical resources as practical next steps.

Method: Comprehensive evaluation of both citation paradigms from zero-shot to advanced retrieval-augmented methods across four attribution datasets, analyzing trade-offs between coverage and citation correctness.

Result: P-Cite methods achieve high coverage with competitive correctness and moderate latency, while G-Cite methods prioritize precision at the cost of coverage and speed. Retrieval is the main driver of attribution quality in both paradigms.

Conclusion: Recommend retrieval-centric, P-Cite-first approach for high-stakes applications, reserving G-Cite for precision-critical settings like strict claim verification.

Abstract: Trustworthy Large Language Models (LLMs) must cite human-verifiable sources in high-stakes domains such as healthcare, law, academia, and finance, where even small errors can have severe consequences. Practitioners and researchers face a choice: let models generate citations during decoding, or let models draft answers first and then attach appropriate citations. To clarify this choice, we introduce two paradigms: Generation-Time Citation (G-Cite), which produces the answer and citations in one pass, and Post-hoc Citation (P-Cite), which adds or verifies citations after drafting. We conduct a comprehensive evaluation from zero-shot to advanced retrieval-augmented methods across four popular attribution datasets and provide evidence-based recommendations that weigh trade-offs across use cases. Our results show a consistent trade-off between coverage and citation correctness, with retrieval as the main driver of attribution quality in both paradigms. P-Cite methods achieve high coverage with competitive correctness and moderate latency, whereas G-Cite methods prioritize precision at the cost of coverage and speed. We recommend a retrieval-centric, P-Cite-first approach for high-stakes applications, reserving G-Cite for precision-critical settings such as strict claim verification. Our codes and human evaluation results are available at https://anonymous.4open.science/r/Citation_Paradigms-BBB5/

Method: TwS combines linguistic reasoning with on-the-fly audio-domain analysis, enabling models to actively think with audio signals through numerical analysis and digital manipulation via multimodal reasoning.

Result: On the MELD-Hard1k benchmark with acoustic perturbations, TwS achieved substantial robustness improvements: small models gained 24.73% absolute accuracy, with improvements scaling up to 36.61% for larger models, while baseline LALMs suffered over 50% performance degradation.

Conclusion: Audio Chain-of-Thought reasoning can significantly enhance audio model robustness without retraining, opening new directions for developing more robust audio understanding systems.

Abstract: Recent Large Audio-Language Models (LALMs) have shown strong performance on various audio understanding tasks such as speech translation and Audio Q&A. However, they exhibit significant limitations on challenging audio reasoning tasks in complex acoustic scenarios. These situations would greatly benefit from the use of acoustic tools like noise suppression, source separation, and precise temporal alignment, but current LALMs lack access to such tools. To address this limitation, we introduce Thinking-with-Sound (TwS), a framework that equips LALMs with Audio CoT by combining linguistic reasoning with on-the-fly audio-domain analysis. Unlike existing approaches that treat audio as static input, TwS enables models to actively think with audio signals, performing numerical analysis and digital manipulation through multimodal reasoning. To evaluate this approach, we construct MELD-Hard1k, a new robustness benchmark created by introducing various acoustic perturbations. Experiments reveal that state-of-the-art LALMs suffer dramatic performance degradation on MELD-Hard1k, with accuracy dropping by more than $50%$ compared to clean audio. TwS achieves substantial improvements in robustness, demonstrating both effectiveness and scalability: small models gain $24.73%$ absolute accuracy, with improvements scaling consistently up to $36.61%$ for larger models. Our findings demonstrate that Audio CoT can significantly enhance robustness without retraining, opening new directions for developing more robust audio understanding systems.

Method: ComPSum generates structured analysis of users by comparing their preferences with other users’ preferences, then uses this analysis to guide personalized summary generation. Also proposes AuthorMap evaluation framework and constructs PerMSum dataset.

Result: ComPSum outperforms strong baselines on the PerMSum dataset when evaluated using the AuthorMap framework.

Conclusion: Comparative analysis of user preferences enables effective personalization in multi-document summarization, and the proposed ComPSum framework with AuthorMap evaluation provides a robust approach for personalized MDS.

Abstract: Personalized multi-document summarization (MDS) is essential for meeting individual user preferences of writing style and content focus for summaries. In this paper, we propose that for effective personalization, it is important to identify fine-grained differences between users’ preferences by comparing the given user’s preferences with other users’ preferences.Motivated by this, we propose ComPSum, a personalized MDS framework. It first generates a structured analysis of a user by comparing their preferences with other users’ preferences. The generated structured analysis is then used to guide the generation of personalized summaries. To evaluate the performance of ComPSum, we propose AuthorMap, a fine-grained reference-free evaluation framework for personalized MDS. It evaluates the personalization of a system based on the authorship attribution between two personalized summaries generated for different users. For robust evaluation of personalized MDS, we construct PerMSum, a personalized MDS dataset in the review and news domain. We evaluate the performance of ComPSum on PerMSum using AuthorMap, showing that it outperforms strong baselines.

Method: Developed five VLM-as-formalizer pipelines for one-shot, open-vocabulary, multimodal PDDL formalization. Evaluated on existing benchmark and introduced two new benchmarks with authentic, multi-view, low-quality images. Used intermediate representations like captions and scene graphs.

Result: VLM-as-formalizer significantly outperforms end-to-end plan generation. Vision is the main bottleneck - VLMs often miss necessary object relations. Intermediate representations provide partial improvement but inconsistent gains.

Conclusion: VLM-as-formalizer approach is promising for multimodal planning but vision capabilities need improvement. Future research should focus on better multimodal planning formalization methods.

Abstract: The advancement of vision language models (VLMs) has empowered embodied agents to accomplish simple multimodal planning tasks, but not long-horizon ones requiring long sequences of actions. In text-only simulations, long-horizon planning has seen significant improvement brought by repositioning the role of LLMs. Instead of directly generating action sequences, LLMs translate the planning domain and problem into a formal planning language like the Planning Domain Definition Language (PDDL), which can call a formal solver to derive the plan in a verifiable manner. In multimodal environments, research on VLM-as-formalizer remains scarce, usually involving gross simplifications such as predefined object vocabulary or overly similar few-shot examples. In this work, we present a suite of five VLM-as-formalizer pipelines that tackle one-shot, open-vocabulary, and multimodal PDDL formalization. We evaluate those on an existing benchmark while presenting another two that for the first time account for planning with authentic, multi-view, and low-quality images. We conclude that VLM-as-formalizer greatly outperforms end-to-end plan generation. We reveal the bottleneck to be vision rather than language, as VLMs often fail to capture an exhaustive set of necessary object relations. While generating intermediate, textual representations such as captions or scene graphs partially compensate for the performance, their inconsistent gain leaves headroom for future research directions on multimodal planning formalization.

Method: Evaluated 87 LLM-generated translations of e-commerce marketing emails across 24 regional dialects of 20 languages using human reviewers who provided quantitative ratings and qualitative feedback on faithfulness to tone, meaning, and intended audience.

Result: Leading models produce grammatically correct translations but struggle with culturally nuanced language, even in high-resource languages. Figurative expressions and wordplay were frequently mistranslated, requiring substantial human refinement.

Conclusion: Cultural appropriateness is a key determinant of multilingual LLM performance, challenging the assumption that data volume alone predicts translation quality. Current systems have limitations for real-world localization, highlighting the need for expanded research in culturally diverse contexts.

Abstract: This pilot study explores the localisation capabilities of state-of-the-art multilingual AI models when translating figurative language, such as idioms and puns, from English into a diverse range of global languages. It expands on existing LLM translation research and industry benchmarks, which emphasise grammatical accuracy and token-level correctness, by focusing on cultural appropriateness and overall localisation quality - critical factors for real-world applications like marketing and e-commerce. To investigate these challenges, this project evaluated a sample of 87 LLM-generated translations of e-commerce marketing emails across 24 regional dialects of 20 languages. Human reviewers fluent in each target language provided quantitative ratings and qualitative feedback on faithfulness to the original’s tone, meaning, and intended audience. Findings suggest that, while leading models generally produce grammatically correct translations, culturally nuanced language remains a clear area for improvement, often requiring substantial human refinement. Notably, even high-resource global languages, despite topping industry benchmark leaderboards, frequently mistranslated figurative expressions and wordplay. This work challenges the assumption that data volume is the most reliable predictor of machine translation quality and introduces cultural appropriateness as a key determinant of multilingual LLM performance - an area currently underexplored in existing academic and industry benchmarks. As a proof of concept, this pilot highlights limitations of current multilingual AI systems for real-world localisation use cases. Results of this pilot support the opportunity for expanded research at greater scale to deliver generalisable insights and inform deployment of reliable machine translation workflows in culturally diverse contexts.

Method: Created VoiceAssistant-Eval benchmark with 10,497 curated examples spanning 13 task categories covering listening (natural sounds, music, spoken dialogue), speaking (multi-turn dialogue, role-play imitation, various scenarios), and viewing (highly heterogeneous images). Evaluated 21 open-source models and GPT-4o-Audio.

Result: Three key findings: (1) proprietary models don’t universally outperform open-source models; (2) most models excel at speaking but lag in audio understanding; (3) well-designed smaller models can rival much larger ones (Step-Audio-2-mini 7B achieved more than double the listening accuracy of LLaMA-Omni2-32B-Bilingual). Challenges remain in multimodal input and role-play voice imitation.

Conclusion: VoiceAssistant-Eval identifies gaps in current AI assistants and establishes a rigorous framework for evaluating and guiding development of next-generation AI assistants, with significant work needed in robustness and safety alignment.

Abstract: The growing capabilities of large language models and multimodal systems have spurred interest in voice-first AI assistants, yet existing benchmarks are inadequate for evaluating the full range of these systems’ capabilities. We introduce VoiceAssistant-Eval, a comprehensive benchmark designed to assess AI assistants across listening, speaking, and viewing. VoiceAssistant-Eval comprises 10,497 curated examples spanning 13 task categories. These tasks include natural sounds, music, and spoken dialogue for listening; multi-turn dialogue, role-play imitation, and various scenarios for speaking; and highly heterogeneous images for viewing. To demonstrate its utility, we evaluate 21 open-source models and GPT-4o-Audio, measuring the quality of the response content and speech, as well as their consistency. The results reveal three key findings: (1) proprietary models do not universally outperform open-source models; (2) most models excel at speaking tasks but lag in audio understanding; and (3) well-designed smaller models can rival much larger ones. Notably, the mid-sized Step-Audio-2-mini (7B) achieves more than double the listening accuracy of LLaMA-Omni2-32B-Bilingual. However, challenges remain: multimodal (audio plus visual) input and role-play voice imitation tasks are difficult for current models, and significant gaps persist in robustness and safety alignment. VoiceAssistant-Eval identifies these gaps and establishes a rigorous framework for evaluating and guiding the development of next-generation AI assistants. Code and data will be released at https://mathllm.github.io/VoiceAssistantEval/ .

Method: Hybrid storage policy preserves first and most recent tokens in full-rank while compressing intermediate tokens using low-rank projection with online subspace adaptation via Oja’s algorithm. Includes comprehensive updates during prefilling and lightweight periodic updates during decoding.

Result: Maintains or improves zero-shot accuracy at high compression ratios, with strongest gains on very long-context benchmarks requiring complex reasoning. Fully compatible with modern attention modules like FlashAttention.

Conclusion: OjaKV provides a practical, plug-and-play solution for memory-efficient long-context inference that dynamically tracks context shifts through online adaptation, eliminating the need for model fine-tuning.

Abstract: The expanding long-context capabilities of large language models are constrained by a significant memory bottleneck: the key-value (KV) cache required for autoregressive generation. This bottleneck is substantial; for instance, a Llama-3.1-8B model processing a 32K-token prompt at a batch size of 4 requires approximately 16GB for its KV cache, a size exceeding the model’s weights. While KV-cache compression via low-rank projection is a promising direction, existing methods rely on a static, offline-learned subspace that performs poorly under data distribution shifts. To overcome these limitations, we introduce OjaKV, a novel framework that integrates a strategic hybrid storage policy with online subspace adaptation. First, OjaKV recognizes that not all tokens are equally important for compression; it preserves the crucial first and most recent tokens in full-rank, maintaining high-fidelity anchors for attention. Second, for the vast majority of intermediate tokens, it applies low-rank compression by incrementally adapting the projection basis using Oja’s algorithm for online principal component analysis. This adaptation involves a comprehensive update during prompt prefilling and lightweight periodic updates during decoding, ensuring the subspace remains aligned with the evolving context. Crucially, our framework is fully compatible with modern attention modules like FlashAttention. Experiments demonstrate that OjaKV maintains or even improves zero-shot accuracy at high compression ratios. In particular, OjaKV achieves its strongest gains on very long-context benchmarks that require complex reasoning, highlighting the importance of online subspace adaptation in dynamically tracking context shifts. These results establish our hybrid framework as a practical, plug-and-play solution for memory-efficient long-context inference without requiring model fine-tuning.

Method: The survey organizes XAI techniques according to transformer architectures and analyzes how methods are adapted to each architecture type while assessing their strengths and limitations. Evaluation is done through dual lenses of plausibility and faithfulness.

Result: The paper provides a structured perspective on XAI technique effectiveness for different LM architectures, identifying how methods perform across various transformer model types.

Conclusion: The survey identifies open research challenges and outlines future directions to guide development of robust, transparent, and interpretable XAI methods specifically tailored for language models.

Abstract: Language Models (LMs) have significantly advanced natural language processing and enabled remarkable progress across diverse domains, yet their black-box nature raises critical concerns about the interpretability of their internal mechanisms and decision-making processes. This lack of transparency is particularly problematic for adoption in high-stakes domains, where stakeholders need to understand the rationale behind model outputs to ensure accountability. On the other hand, while explainable artificial intelligence (XAI) methods have been well studied for non-LMs, they face many limitations when applied to LMs due to their complex architectures, considerable training corpora, and broad generalization abilities. Although various surveys have examined XAI in the context of LMs, they often fail to capture the distinct challenges arising from the architectural diversity and evolving capabilities of these models. To bridge this gap, this survey presents a comprehensive review of XAI techniques with a particular emphasis on LMs, organizing them according to their underlying transformer architectures: encoder-only, decoder-only, and encoder-decoder, and analyzing how methods are adapted to each while assessing their respective strengths and limitations. Furthermore, we evaluate these techniques through the dual lenses of plausibility and faithfulness, offering a structured perspective on their effectiveness. Finally, we identify open research challenges and outline promising future directions, aiming to guide ongoing efforts toward the development of robust, transparent, and interpretable XAI methods for LMs.

Method: Define misinformed review points as weaknesses with incorrect premises or questions already answered by paper. Build human-annotated dataset and test LLMs on premise-level factuality evaluation.

Result: 15.2% of weaknesses and 26.4% of questions are misinformed. LLMs show moderate agreement with humans on ReviewScore evaluation, with premise-level factuality showing higher agreement than weakness-level.

Conclusion: Automated ReviewScore evaluation using LLMs is feasible, with premise-level analysis being more reliable than weakness-level evaluation.

Abstract: Peer review serves as a backbone of academic research, but in most AI conferences, the review quality is degrading as the number of submissions explodes. To reliably detect low-quality reviews, we define misinformed review points as either “weaknesses” in a review that contain incorrect premises, or “questions” in a review that can be already answered by the paper. We verify that 15.2% of weaknesses and 26.4% of questions are misinformed and introduce ReviewScore indicating if a review point is misinformed. To evaluate the factuality of each premise of weaknesses, we propose an automated engine that reconstructs every explicit and implicit premise from a weakness. We build a human expert-annotated ReviewScore dataset to check the ability of LLMs to automate ReviewScore evaluation. Then, we measure human-model agreements on ReviewScore using eight current state-of-the-art LLMs and verify moderate agreements. We also prove that evaluating premise-level factuality shows significantly higher agreements than evaluating weakness-level factuality. A thorough disagreement analysis further supports a potential of fully automated ReviewScore evaluation.

Method: Built on Hidden Profile paradigm with 65 tasks from custom designs, human studies, and automatic generation; evaluated 15 LLMs across four model families using asymmetric information integration tasks.

Result: GPT-4.1 groups fail to integrate distributed knowledge, showing human-like collective reasoning failures; some models (Gemini-2.5-Flash/Pro) perform better, but scale and reasoning are not reliable indicators of collective reasoning strength.

Conclusion: HiddenBench provides the first reproducible benchmark for collective reasoning in multi-agent LLMs, offering diagnostic insights and foundation for future artificial collective intelligence research.

Abstract: Multi-agent systems built on large language models (LLMs) promise enhanced problem-solving through distributed information integration, but may also replicate collective reasoning failures observed in human groups. Yet the absence of a theory-grounded benchmark makes it difficult to systematically evaluate and improve such reasoning. We introduce HiddenBench, the first benchmark for evaluating collective reasoning in multi-agent LLMs. It builds on the Hidden Profile paradigm from social psychology, where individuals each hold asymmetric pieces of information and must communicate to reach the correct decision. To ground the benchmark, we formalize the paradigm with custom tasks and show that GPT-4.1 groups fail to integrate distributed knowledge, exhibiting human-like collective reasoning failures that persist even with varied prompting strategies. We then construct the full benchmark, spanning 65 tasks drawn from custom designs, prior human studies, and automatic generation. Evaluating 15 LLMs across four model families, HiddenBench exposes persistent limitations while also providing comparative insights: some models (e.g., Gemini-2.5-Flash/Pro) achieve higher performance, yet scale and reasoning are not reliable indicators of stronger collective reasoning. Our work delivers the first reproducible benchmark for collective reasoning in multi-agent LLMs, offering diagnostic insight and a foundation for future research on artificial collective intelligence.

Method: Created GRAB benchmark with 1.61M sentences from 8,247 filings using automated labeling combining FinBERT token attention, YAKE keyphrase signals, and taxonomy-aware collocation matching anchored in a risk taxonomy of 193 terms mapped to 21 fine-grained types.

Result: Developed a unified evaluation framework with fixed dataset splits and robust metrics including Accuracy, Macro-F1, Topic BERTScore, and Effective Number of Topics.

Conclusion: GRAB enables reproducible, standardized comparison across various topic models on financial disclosures, providing dataset, labels, and code for the research community.

Abstract: Risk categorization in 10-K risk disclosures matters for oversight and investment, yet no public benchmark evaluates unsupervised topic models for this task. We present GRAB, a finance-specific benchmark with 1.61M sentences from 8,247 filings and span-grounded sentence labels produced without manual annotation by combining FinBERT token attention, YAKE keyphrase signals, and taxonomy-aware collocation matching. Labels are anchored in a risk taxonomy mapping 193 terms to 21 fine-grained types nested under five macro classes; the 21 types guide weak supervision, while evaluation is reported at the macro level. GRAB unifies evaluation with fixed dataset splits and robust metrics–Accuracy, Macro-F1, Topic BERTScore, and the entropy-based Effective Number of Topics. The dataset, labels, and code enable reproducible, standardized comparison across classical, embedding-based, neural, and hybrid topic models on financial disclosures.

Method: Proposes Think-on-Graph 3.0 (ToG-3) with Multi-Agent Context Evolution and Retrieval (MACER) mechanism. Features dynamic construction of Chunk-Triplets-Community heterogeneous graph index with dual-evolution of Evolving Query and Evolving Sub-Graph. Uses multi-agent system (Constructor, Retriever, Reflector, Responser) for iterative evidence retrieval, answer generation, and graph refinement.

Result: Extensive experiments show ToG-3 outperforms baselines on both deep and broad reasoning benchmarks. Ablation studies confirm the efficacy of MACER framework components.

Conclusion: ToG-3 successfully addresses limitations of static graph construction in Graph-based RAG by enabling adaptive, targeted graph index building during reasoning, allowing deep precise reasoning even with lightweight LLMs.

Abstract: Retrieval-Augmented Generation (RAG) and Graph-based RAG has become the important paradigm for enhancing Large Language Models (LLMs) with external knowledge. However, existing approaches face a fundamental trade-off. While graph-based methods are inherently dependent on high-quality graph structures, they face significant practical constraints: manually constructed knowledge graphs are prohibitively expensive to scale, while automatically extracted graphs from corpora are limited by the performance of the underlying LLM extractors, especially when using smaller, local-deployed models. This paper presents Think-on-Graph 3.0 (ToG-3), a novel framework that introduces Multi-Agent Context Evolution and Retrieval (MACER) mechanism to overcome these limitations. Our core innovation is the dynamic construction and refinement of a Chunk-Triplets-Community heterogeneous graph index, which pioneeringly incorporates a dual-evolution mechanism of Evolving Query and Evolving Sub-Graph for precise evidence retrieval. This approach addresses a critical limitation of prior Graph-based RAG methods, which typically construct a static graph index in a single pass without adapting to the actual query. A multi-agent system, comprising Constructor, Retriever, Reflector, and Responser agents, collaboratively engages in an iterative process of evidence retrieval, answer generation, sufficiency reflection, and, crucially, evolving query and subgraph. This dual-evolving multi-agent system allows ToG-3 to adaptively build a targeted graph index during reasoning, mitigating the inherent drawbacks of static, one-time graph construction and enabling deep, precise reasoning even with lightweight LLMs. Extensive experiments demonstrate that ToG-3 outperforms compared baselines on both deep and broad reasoning benchmarks, and ablation studies confirm the efficacy of the components of MACER framework.

Method: ProPerSim simulation framework where a user agent with rich persona interacts with the assistant and provides preference ratings. ProPerAssistant uses retrieval-augmented learning and continuously adapts through user feedback.

Result: Experiments across 32 diverse personas show ProPerAssistant successfully adapts its strategy and steadily improves user satisfaction over time.

Conclusion: The framework demonstrates the promise of combining proactivity and personalization in AI assistants, enabling them to make timely, personalized recommendations that align with user preferences.

Abstract: As large language models (LLMs) become increasingly integrated into daily life, there is growing demand for AI assistants that are not only reactive but also proactive and personalized. While recent advances have pushed forward proactivity and personalization individually, their combination remains underexplored. To bridge this gap, we introduce ProPerSim, a new task and simulation framework for developing assistants capable of making timely, personalized recommendations in realistic home scenarios. In our simulation environment, a user agent with a rich persona interacts with the assistant, providing ratings on how well each suggestion aligns with its preferences and context. The assistant’s goal is to use these ratings to learn and adapt to achieve higher scores over time. Built on ProPerSim, we propose ProPerAssistant, a retrieval-augmented, preference-aligned assistant that continually learns and adapts through user feedback. Experiments across 32 diverse personas show that ProPerAssistant adapts its strategy and steadily improves user satisfaction, highlighting the promise of uniting proactivity and personalization.

Method: Fine-tune voice conversion model on English dysarthric speech (UASpeech) to encode speaker characteristics and prosodic distortions, then apply it to convert healthy non-English speech (FLEURS) into dysarthric-like speech. Use generated data to fine-tune multilingual ASR model (MMS).

Result: VC with both speaker and prosody conversion significantly outperforms off-the-shelf MMS and conventional augmentation techniques (speed/tempo perturbation) on PC-GITA (Spanish), EasyCall (Italian), and SSNCE (Tamil) datasets. Generated speech simulates dysarthric characteristics confirmed by objective and subjective analyses.

Conclusion: Voice conversion approach effectively generates non-English dysarthric-like speech, enabling improved multilingual ASR performance for dysarthric speech recognition.

Abstract: Automatic speech recognition (ASR) for dysarthric speech remains challenging due to data scarcity, particularly in non-English languages. To address this, we fine-tune a voice conversion model on English dysarthric speech (UASpeech) to encode both speaker characteristics and prosodic distortions, then apply it to convert healthy non-English speech (FLEURS) into non-English dysarthric-like speech. The generated data is then used to fine-tune a multilingual ASR model, Massively Multilingual Speech (MMS), for improved dysarthric speech recognition. Evaluation on PC-GITA (Spanish), EasyCall (Italian), and SSNCE (Tamil) demonstrates that VC with both speaker and prosody conversion significantly outperforms the off-the-shelf MMS performance and conventional augmentation techniques such as speed and tempo perturbation. Objective and subjective analyses of the generated data further confirm that the generated speech simulates dysarthric characteristics.

Method: Conducted extensive experiments benchmarking both proprietary and public LLMs on answering multiple questions based on the same conversational context, including fine-tuning public models.

Result: Strong proprietary LLMs like GPT-4o achieve best overall performance, but fine-tuned public LLMs with up to 8B parameters can surpass GPT-4o in accuracy.

Conclusion: Fine-tuned public LLMs demonstrate potential for transparent and cost-effective deployment in real-world applications, providing viable alternatives to proprietary models.

Abstract: Deploying Large Language Models (LLMs) for question answering (QA) over lengthy contexts is a significant challenge. In industrial settings, this process is often hindered by high computational costs and latency, especially when multiple questions must be answered based on the same context. In this work, we explore the capabilities of LLMs to answer multiple questions based on the same conversational context. We conduct extensive experiments and benchmark a range of both proprietary and public models on this challenging task. Our findings highlight that while strong proprietary LLMs like GPT-4o achieve the best overall performance, fine-tuned public LLMs with up to 8 billion parameters can surpass GPT-4o in accuracy, which demonstrates their potential for transparent and cost-effective deployment in real-world applications.

Method: Uses most recent output as draft for growing input context, biases verification towards draft tokens for higher acceptance rate, and continues from divergence point after verification.

Result: Achieves up to 1.7x speedup compared to conventional auto-regressive re-translation without quality compromise, and reduces flickering by 80% using display-only mask-k technique.

Conclusion: Provides model-agnostic, plug-and-play solution for streaming applications that eliminates draft computations and significantly improves performance.

Abstract: Large Language Models (LLMs) have recently demonstrated impressive capabilities in various text generation tasks. However, it remains challenging to use them off-the-shelf in streaming applications (such as live translation), where the output must continually update as the input context expands, while still maintaining a reasonable computational cost to meet the latency requirement. In this work, we reexamine the re-translation approach to simultaneous translation and propose Self-Speculative Biased Decoding, a novel inference paradigm designed to avoid repeatedly generating output from scratch for a consistently growing input stream. We propose using the most recent output as a draft for the current growing input context. During the verification stage, the output will be biased towards the draft token for a higher draft acceptance rate. This strategy not only minimizes flickering that might distract users but also leads to higher speedups. Conventional decoding may take charge from the point of divergence after draft verification and continue until the end condition is met. Unlike existing speculative decoding strategies, our approach eliminates the need for draft computations, making it a model-agnostic and plug-and-play solution for accelerating latency-sensitive streaming applications. Experimental results on simultaneous text-to-text re-translation demonstrate that our approach achieves up to 1.7x speedup compared to conventional auto-regressive re-translation without compromising quality. Additionally, it significantly reduces flickering by 80% by incorporating the display-only mask-k technique.

Method: Uses a decoder-only Transformer trained on behavioral sequences with joint optimization: InfoNCE for retrieval and hybrid pointwise-pairwise loss for ranking. Introduces novel time-aware rotary positional embedding to incorporate time information into attention mechanism.

Result: Achieves significant improvements on widely adopted recommendation and search benchmarks compared to strong generative recommender and joint search and recommendation baselines.

Conclusion: Demonstrates the viability of a single generative foundation model for industrial-scale unified information access, allowing semantic signals from search to improve recommendation and vice versa.

Abstract: The dominant retrieve-then-rank pipeline in large-scale recommender systems suffers from mis-calibration and engineering overhead due to its architectural split and differing optimization objectives. While recent generative sequence models have shown promise in unifying retrieval and ranking by auto-regressively generating ranked items, existing solutions typically address either personalized search or query-free recommendation, often exhibiting performance trade-offs when attempting to unify both. We introduce \textit{SynerGen}, a novel generative recommender model that bridges this critical gap by providing a single generative backbone for both personalized search and recommendation, while simultaneously excelling at retrieval and ranking tasks. Trained on behavioral sequences, our decoder-only Transformer leverages joint optimization with InfoNCE for retrieval and a hybrid pointwise-pairwise loss for ranking, allowing semantic signals from search to improve recommendation and vice versa. We also propose a novel time-aware rotary positional embedding to effectively incorporate time information into the attention mechanism. \textit{SynerGen} achieves significant improvements on widely adopted recommendation and search benchmarks compared to strong generative recommender and joint search and recommendation baselines. This work demonstrates the viability of a single generative foundation model for industrial-scale unified information access.

Method: Used causal inference with five potential causal structures across eight reasoning tasks to analyze structured output’s impact on GPT-4o.

Result: Found no causal impact in 43/48 scenarios; only 5 showed effects, with 3 involving multifaceted causal structures influenced by instructions.

Conclusion: Structured output generally has minimal causal impact on LLM generation quality, with effects mainly emerging through complex instruction interactions.

Abstract: Structured output from large language models (LLMs) has enhanced efficiency in processing generated information and is increasingly adopted in industrial applications. Prior studies have investigated the impact of structured output on LLMs’ generation quality, often presenting one-way findings. Some suggest that structured format enhances completeness and factual accuracy, while others argue that it restricts the reasoning capacity of LLMs and leads to reductions in standard evaluation metrics. Potential limitations of these assessments include restricted testing scenarios, weakly controlled comparative settings, and reliance on coarse metrics. In this work, we present a refined analysis using causal inference. Based on one assumed and two guaranteed constraints, we derive five potential causal structures characterizing the influence of structured output on LLMs’ generation: (1) collider without m-bias, (2) collider with m-bias, (3) single cause from instruction, (4) single cause from output format, and (5) independence. Across seven public and one developed reasoning tasks, we find that coarse metrics report positive, negative, or neutral effects of structured output on GPT-4o’s generation. However, causal inference reveals no causal impact in 43 out of 48 scenarios. In the remaining 5, 3 involve multifaceted causal structures influenced by concrete instructions.

Method: Proposed CARB benchmark covering 10 cultures across 4 domains, and Think-as-Locals approach using reinforcement learning from verifiable rewards to elicit culturally grounded reasoning.

Result: Evaluation shows current RMs have deficiencies in cultural awareness modeling and rely on surface-level features rather than authentic cultural understanding. Think-as-Locals effectively mitigates spurious correlations.

Conclusion: CARB enables better evaluation of cultural awareness in RMs, and Think-as-Locals with RLVR advances culture-aware reward modeling by reducing reliance on superficial features.

Abstract: Reward models (RMs) are crucial for aligning large language models (LLMs) with diverse cultures. Consequently, evaluating their cultural awareness is essential for further advancing global alignment of LLMs. However, existing RM evaluations fall short in assessing cultural awareness due to the scarcity of culturally relevant evaluation datasets. To fill this gap, we propose Cultural Awareness Reward modeling Benchmark (CARB), covering 10 distinct cultures across 4 cultural domains. Our extensive evaluation of state-of-the-art RMs reveals their deficiencies in modeling cultural awareness and demonstrates a positive correlation between performance on CARB and downstream multilingual cultural alignment tasks. Further analysis identifies the spurious correlations within culture-aware reward modeling, wherein RM’s scoring relies predominantly on surface-level features rather than authentic cultural nuance understanding. To address these, we propose Think-as-Locals to elicit deeper culturally grounded reasoning from generative RMs via reinforcement learning from verifiable rewards (RLVR) and employ well-designed rewards to ensure accurate preference judgments and high-quality structured evaluation criteria generation. Experimental results validate its efficacy in mitigating spurious features interference and advancing culture-aware reward modeling.

Method: Implemented four adaptive actions in a decoder-only LLM framework with action-aware prompting for training, and developed a latency-aware TTS pipeline for realistic timing evaluation.

Result: Experiments on ACL60/60 benchmarks show consistent improvements in semantic metrics (COMET-KIWI) and lower delay (Average Lagging) compared to reference translations and salami-based baselines, with DROP and SENTENCE_CUT combination yielding the best balance.

Conclusion: Enriching the action space of LLM-based SiMT provides a promising direction for bridging the gap between human and machine interpretation.

Abstract: Simultaneous Machine Translation (SiMT) requires high-quality translations under strict real-time constraints, which traditional encoder-decoder policies with only READ/WRITE actions cannot fully address. We extend the action space of SiMT with four adaptive actions: SENTENCE_CUT, DROP, PARTIAL_SUMMARIZATION and PRONOMINALIZATION, which enable real-time restructuring, omission, and simplification while preserving semantic fidelity. We implement these actions in a decoder-only large language model (LLM) framework and construct training references through action-aware prompting. To evaluate both quality and latency, we further develop a latency-aware TTS pipeline that maps textual outputs to speech with realistic timing. Experiments on the ACL60/60 English-Chinese and English-German benchmarks show that our framework consistently improves semantic metrics (e.g., COMET-KIWI) and achieves lower delay (measured by Average Lagging) compared to reference translations and salami-based baselines. Notably, combining DROP and SENTENCE_CUT yields the best overall balance between fluency and latency. These results demonstrate that enriching the action space of LLM-based SiMT provides a promising direction for bridging the gap between human and machine interpretation.

Method: Applies information bottleneck to filter unimodal inputs, uses parameterized mask generator to disentangle multimodal representations into causal and shortcut subrepresentations, incorporates instrumental variable constraint for global consistency, and adopts backdoor adjustment by randomly recombining features.

Result: Extensive experiments on multimodal sentiment analysis, humor detection, and sarcasm detection with OOD test sets demonstrate the effectiveness of CaMIB, showing improved generalization performance.

Conclusion: CaMIB provides an effective causal approach for multimodal language understanding that enhances out-of-distribution generalization while maintaining interpretability and theoretical soundness.

Abstract: Human Multimodal Language Understanding (MLU) aims to infer human intentions by integrating related cues from heterogeneous modalities. Existing works predominantly follow a ``learning to attend" paradigm, which maximizes mutual information between data and labels to enhance predictive performance. However, such methods are vulnerable to unintended dataset biases, causing models to conflate statistical shortcuts with genuine causal features and resulting in degraded out-of-distribution (OOD) generalization. To alleviate this issue, we introduce a Causal Multimodal Information Bottleneck (CaMIB) model that leverages causal principles rather than traditional likelihood. Concretely, we first applies the information bottleneck to filter unimodal inputs, removing task-irrelevant noise. A parameterized mask generator then disentangles the fused multimodal representation into causal and shortcut subrepresentations. To ensure global consistency of causal features, we incorporate an instrumental variable constraint, and further adopt backdoor adjustment by randomly recombining causal and shortcut features to stabilize causal estimation. Extensive experiments on multimodal sentiment analysis, humor detection, and sarcasm detection, along with OOD test sets, demonstrate the effectiveness of CaMIB. Theoretical and empirical analyses further highlight its interpretability and soundness.

Method: Extend existing benchmark for linguistic puzzles, test LLMs including OpenAI’s o1 across various linguistic topics, analyze performance gaps, and apply findings to puzzle generation.

Result: LLMs outperform humans on most puzzle types except writing systems and understudied languages, demonstrating potential for automated puzzle generation.

Conclusion: Automated linguistic puzzle generation can promote linguistics education and disseminate knowledge about rare languages, making it an important research task.

Abstract: In this paper, we introduce a combination of novel and exciting tasks: the solution and generation of linguistic puzzles. We focus on puzzles used in Linguistic Olympiads for high school students. We first extend the existing benchmark for the task of solving linguistic puzzles. We explore the use of Large Language Models (LLMs), including recent state-of-the-art models such as OpenAI’s o1, for solving linguistic puzzles, analyzing their performance across various linguistic topics. We demonstrate that LLMs outperform humans on most puzzles types, except for those centered on writing systems, and for the understudied languages. We use the insights from puzzle-solving experiments to direct the novel task of puzzle generation. We believe that automating puzzle generation, even for relatively simple puzzles, holds promise for expanding interest in linguistics and introducing the field to a broader audience. This finding highlights the importance of linguistic puzzle generation as a research task: such puzzles can not only promote linguistics but also support the dissemination of knowledge about rare and understudied languages.

Method: ResT reshapes policy gradients through entropy-informed token reweighting, progressively upweighting reasoning tokens during training. This enables a smooth transition from structural correctness to semantic reasoning.

Result: ResT achieves state-of-the-art results on BFCL and API-Bank benchmarks, outperforming prior methods by up to 8.76%. When fine-tuned on a 4B LLM, it surpasses GPT-4o by 4.11% on single-turn tasks and 1.50% on multi-turn tasks.

Conclusion: The entropy-aware token reweighting scheme in ResT effectively stabilizes convergence in multi-turn tool-use tasks and demonstrates superior performance compared to existing methods and even large commercial models like GPT-4o.

Abstract: Large language models (LLMs) transcend passive generation and act as goal-directed agents by invoking external tools. Reinforcement learning (RL) offers a principled framework for optimizing these emergent tool-use policies, yet the prevailing paradigm relies exclusively on sparse outcome rewards and lacks consideration of the particularity of tool-use tasks, inflating policy-gradient variance and resulting in inefficient training. To better understand and address these challenges, we first establish a theoretical link between policy entropy and training stability of tool-use tasks, which reveals that structured, low-entropy tokens are primary determinants of rewards. Motivated by this insight, we propose \textbf{Res}haped \textbf{T}oken-level policy gradients (\textbf{ResT}) for tool-use tasks. ResT reshapes the policy gradient through entropy-informed token reweighting, progressively upweighting reasoning tokens as training proceeds. This entropy-aware scheme enables a smooth shift from structural correctness to semantic reasoning and stabilizes convergence in multi-turn tool-use tasks. Evaluation on BFCL and API-Bank shows that ResT achieves state-of-the-art results, outperforming prior methods by up to $8.76%$. When fine-tuned on a 4B base LLM, ResT further surpasses GPT-4o by $4.11%$ on single-turn tasks and $1.50%$ on multi-turn base tasks.

Method: Propose semantic agreement (meaning-level consensus between ensemble outputs) as a training-free signal for reliable deferral, which works without model internals and across black-box APIs.

Result: Semantic cascades match or surpass target-model quality at 40% of the cost, reduce latency by up to 60%, and remain robust to model updates.

Conclusion: Semantic agreement provides a practical baseline for real-world LLM deployment, offering stronger reliability signals than token-level confidence while being training-free and model-agnostic.

Abstract: Cascade systems route computational requests to smaller models when possible and defer to larger models only when necessary, offering a promising approach to balance cost and quality in LLM deployment. However, they face a fundamental challenge in open-ended text generation: determining output reliability when generation quality lies on a continuous spectrum, often with multiple valid responses. To address this, we propose semantic agreement – meaning-level consensus between ensemble outputs – as a training-free signal for reliable deferral. We show that when diverse model outputs agree semantically, their consensus is a stronger reliability signal than token-level confidence. Evaluated from 500M to 70B-parameter models, we find that semantic cascades match or surpass target-model quality at 40% of the cost and reduce latency by up to 60%. Our method requires no model internals, works across black-box APIs, and remains robust to model updates, making it a practical baseline for real-world LLM deployment.

Method: Created KnowMT-Bench with dynamic evaluation where models generate their own multi-turn dialogue histories from progressive question sequences, then evaluated final-turn answers using human-validated automated pipeline across medicine, finance, and law domains.

Result: Multi-turn contexts degrade performance: factual capability declines due to contextual noise from self-generated histories, and information efficiency drops as models become more verbose with longer dialogues.

Conclusion: RAG can effectively alleviate factual degradation, highlighting the importance of KnowMT-Bench for evaluating and enhancing conversational factual capabilities of LLMs in real-world knowledge-intensive applications.

Abstract: Multi-Turn Long-Form Question Answering (MT-LFQA) is a key application paradigm of Large Language Models (LLMs) in knowledge-intensive domains. However, existing benchmarks are limited to single-turn dialogue, while multi-turn dialogue benchmarks typically assess other orthogonal capabilities rather than knowledge-intensive factuality. To bridge this critical gap, we introduce \textbf{KnowMT-Bench}, the \textit{first-ever} benchmark designed to systematically evaluate MT-LFQA for LLMs across knowledge-intensive fields, including medicine, finance, and law. To faithfully assess the model’s real-world performance, KnowMT-Bench employs a dynamic evaluation setting where models generate their own multi-turn dialogue histories given logically progressive question sequences. The factual capability and information delivery efficiency of the \textit{final-turn} answer are then evaluated using a human-validated automated pipeline. Our experiments reveal that multi-turn contexts degrade performance: factual capability declines due to the contextual noise from self-generated histories, while information efficiency drops as models become more verbose with increasing dialogue length. We then investigate mitigation strategies, demonstrating that retrieval-augmented generation (RAG) can effectively alleviate and even reverse this factual degradation. These findings underscore the importance of our benchmark in evaluating and enhancing the conversational factual capabilities of LLMs in real-world knowledge-intensive applications. Code is available at \href{https://github.com/hardenyu21/KnowMT-Bench}{\textcolor{cyan}{\texttt{KnowMT-Bench}}}.

Method: LoRAN applies lightweight non-linear transformations to low-rank updates and uses Sinter, a sine-based activation that adds structured perturbations without increasing parameters.

Result: Experiments show LoRAN consistently outperforms QLoRA across summarization and classification tasks, with Sinter surpassing standard activations like Sigmoid, ReLU, and Tanh.

Conclusion: The study highlights the importance of activation design in low-rank tuning, demonstrating that non-linear extensions and specialized activations can significantly improve parameter-efficient fine-tuning.

Abstract: Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient fine-tuning method for large language models. However, its linear nature limits expressiveness. We propose LoRAN, a non-linear extension of LoRA that applies lightweight transformations to the low-rank updates. We further introduce Sinter, a sine-based activation that adds structured perturbations without increasing parameter count. Experiments across summarization and classification tasks show that LoRAN consistently improves over QLoRA. Ablation studies reveal that Sinter outperforms standard activations such as Sigmoid, ReLU, and Tanh, highlighting the importance of activation design in lowrank tuning.

Method: Quantifies external context utilization via distributional distance, measures internal knowledge utilization by tracking predicted token evolution across transformer layers, and introduces statistical validation framework.

Result: Achieves consistently high AUROC and AUPRC scores, outperforming prior methods by up to +13% AUROC on HalluRAG benchmark, and remains robust under relaxed assumptions about retrieval quality and model matching.

Conclusion: LUMINA offers an effective and practical solution for hallucination detection in RAG systems, combining statistical validation with robust performance across different scenarios.

Abstract: Retrieval-Augmented Generation (RAG) aims to mitigate hallucinations in large language models (LLMs) by grounding responses in retrieved documents. Yet, RAG-based LLMs still hallucinate even when provided with correct and sufficient context. A growing line of work suggests that this stems from an imbalance between how models use external context and their internal knowledge, and several approaches have attempted to quantify these signals for hallucination detection. However, existing methods require extensive hyperparameter tuning, limiting their generalizability. We propose LUMINA, a novel framework that detects hallucinations in RAG systems through context-knowledge signals: external context utilization is quantified via distributional distance, while internal knowledge utilization is measured by tracking how predicted tokens evolve across transformer layers. We further introduce a framework for statistically validating these measurements. Experiments on common RAG hallucination benchmarks and four open-source LLMs show that LUMINA achieves consistently high AUROC and AUPRC scores, outperforming prior utilization-based methods by up to +13% AUROC on HalluRAG. Moreover, LUMINA remains robust under relaxed assumptions about retrieval quality and model matching, offering both effectiveness and practicality.

Method: RL-ZVP extracts learning signals from zero-variance prompts by directly rewarding correctness and penalizing errors even without contrasting responses, using token-level characteristics to modulate feedback and preserve nuanced signals.

Result: Across six math reasoning benchmarks, RL-ZVP achieved significant improvements of up to 8.61 points in accuracy and 7.77 points in pass rate over GRPO, consistently outperforming baselines that filter out zero-variance prompts.

Conclusion: Zero-variance prompts have untapped potential for learning in RLVR, and RL-ZVP successfully leverages them to improve LLM reasoning abilities.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is a powerful framework for improving the reasoning abilities of Large Language Models (LLMs). However, current methods such as GRPO rely only on problems where the model responses to the same input differ in correctness, while ignoring those where all responses receive the same reward - so-called zero-variance prompts. In this work, we argue that such prompts are not useless but can, in fact, provide meaningful feedback for policy optimization. To this end, we introduce RL with Zero-Variance Prompts (RL-ZVP), a novel algorithm that extract learning signals from zero-variance prompts. RL-ZVP directly rewards correctness and penalizes errors even without contrasting responses, modulating feedback with token-level characteristics to preserve informative, nuanced signals. Across six math reasoning benchmarks, RL-ZVP achieves significant improvements of up to 8.61 points in accuracy and 7.77 points in pass rate over GRPO, while consistently outperforming other baselines that filter out zero-variance prompts. These results highlight the untapped potential of learning from zero-variance prompts in RLVR.

Method: QoNext identifies experiential factors, conducts controlled experiments with human ratings under varied configurations, builds a QoE-oriented database, and trains predictive models to estimate user experience from system parameters.

Result: QoNext enables proactive and fine-grained evaluation of foundation models and provides actionable guidance for optimizing productized services.

Conclusion: The QoNext framework successfully bridges the gap in foundation model evaluation by focusing on user experience through QoE principles, offering practical optimization guidance.

Abstract: Existing evaluations of foundation models, including recent human-centric approaches, fail to capture what truly matters: user’s experience during interaction. Current methods treat evaluation as a matter of output correctness alone, overlooking that user satisfaction emerges from the interplay between response quality and interaction, which limits their ability to account for the mechanisms underlying user experience. To address this gap, we introduce QoNext, the first framework that adapts Quality of Experience (QoE) principles from networking and multimedia to the assessment of foundation models. QoNext identifies experiential factors that shape user experience and incorporates them into controlled experiments, where human ratings are collected under varied configurations. From these studies we construct a QoE-oriented database and train predictive models that estimate perceived user experience from measurable system parameters. Our results demonstrate that QoNext not only enables proactive and fine-grained evaluation but also provides actionable guidance for productized services of optimizing foundation models in practice.

Method: Elastic Mixture-of-Experts (EMoE) trains experts to collaborate in diverse combinations and improves router selection quality without additional training overhead.

Result: EMoE significantly expands performance-scaling range to 2-3x training-time k and pushes peak performance to higher levels across various MoE settings.

Conclusion: EMoE effectively addresses expert collaboration issues in MoE models, enabling flexible inference-time scaling while maintaining robust performance across computational budgets.

Abstract: Mixture-of-Experts (MoE) models typically fix the number of activated experts $k$ at both training and inference. Intuitively, activating more experts at inference $k’$ (where $k’> k$) means engaging a larger set of model parameters for the computation and thus is expected to improve performance. However, contrary to this intuition, we find the scaling range to be so narrow that performance begins to degrade rapidly after only a slight increase in the number of experts. Further investigation reveals that this degradation stems from a lack of learned collaboration among experts. To address this, we introduce Elastic Mixture-of-Experts (EMoE), a novel training framework that enables MoE models to scale the number of activated experts at inference without incurring additional training overhead. By simultaneously training experts to collaborate in diverse combinations and encouraging the router for high-quality selections, EMoE ensures robust performance across computational budgets at inference. We conduct extensive experiments on various MoE settings. Our results show that EMoE significantly expands the effective performance-scaling range, extending it to as much as 2-3$\times$ the training-time $k$, while also pushing the model’s peak performance to a higher level.

Method: Created a new Turkish citation intent dataset, evaluated standard ICL with LLMs, introduced DSPy framework for automated prompt optimization, and used stacked ensemble with XGBoost meta-model for classification.

Result: Achieved state-of-the-art accuracy of 91.3% using the optimized ensemble approach, significantly outperforming standard in-context learning methods.

Conclusion: Provides Turkish NLP community with foundational dataset and robust classification framework for qualitative citation studies, paving way for future research in this domain.

Abstract: Understanding the qualitative intent of citations is essential for a comprehensive assessment of academic research, a task that poses unique challenges for agglutinative languages like Turkish. This paper introduces a systematic methodology and a foundational dataset to address this problem. We first present a new, publicly available dataset of Turkish citation intents, created with a purpose-built annotation tool. We then evaluate the performance of standard In-Context Learning (ICL) with Large Language Models (LLMs), demonstrating that its effectiveness is limited by inconsistent results caused by manually designed prompts. To address this core limitation, we introduce a programmable classification pipeline built on the DSPy framework, which automates prompt optimization systematically. For final classification, we employ a stacked generalization ensemble to aggregate outputs from multiple optimized models, ensuring stable and reliable predictions. This ensemble, with an XGBoost meta-model, achieves a state-of-the-art accuracy of 91.3%. Ultimately, this study provides the Turkish NLP community and the broader academic circles with a foundational dataset and a robust classification framework paving the way for future qualitative citation studies.

Method: A two-agent framework: first agent extracts rubric-relevant components and encodes them into structured representation, second agent uses this to assign final scores, mimicking human grading process.

Result: AutoSCORE consistently improved scoring accuracy, human-machine agreement (QWK, correlations), and error metrics (MAE, RMSE) across diverse tasks and rubrics, with strong benefits on complex rubrics and smaller LLMs.

Conclusion: Structured component recognition combined with multi-agent design provides a scalable, reliable, and interpretable solution for automated scoring in education.

Abstract: Automated scoring plays a crucial role in education by reducing the reliance on human raters, offering scalable and immediate evaluation of student work. While large language models (LLMs) have shown strong potential in this task, their use as end-to-end raters faces challenges such as low accuracy, prompt sensitivity, limited interpretability, and rubric misalignment. These issues hinder the implementation of LLM-based automated scoring in assessment practice. To address the limitations, we propose AutoSCORE, a multi-agent LLM framework enhancing automated scoring via rubric-aligned Structured COmponent REcognition. With two agents, AutoSCORE first extracts rubric-relevant components from student responses and encodes them into a structured representation (i.e., Scoring Rubric Component Extraction Agent), which is then used to assign final scores (i.e., Scoring Agent). This design ensures that model reasoning follows a human-like grading process, enhancing interpretability and robustness. We evaluate AutoSCORE on four benchmark datasets from the ASAP benchmark, using both proprietary and open-source LLMs (GPT-4o, LLaMA-3.1-8B, and LLaMA-3.1-70B). Across diverse tasks and rubrics, AutoSCORE consistently improves scoring accuracy, human-machine agreement (QWK, correlations), and error metrics (MAE, RMSE) compared to single-agent baselines, with particularly strong benefits on complex, multi-dimensional rubrics, and especially large relative gains on smaller LLMs. These results demonstrate that structured component recognition combined with multi-agent design offers a scalable, reliable, and interpretable solution for automated scoring.

Method: Proposes SimulSense framework that continuously reads input speech and triggers write decisions when new sense units are perceived, mimicking human interpreter behavior without requiring specialized training data or expensive LLM inference.

Result: Achieves superior quality-latency tradeoff compared to two state-of-the-art baseline systems, with decision-making up to 9.6x faster than the baselines.

Conclusion: SimulSense provides a more efficient and practical approach to simultaneous speech translation by mimicking human interpreter decision-making processes, substantially improving real-time efficiency while maintaining translation quality.

Abstract: How to make human-interpreter-like read/write decisions for simultaneous speech translation (SimulST) systems? Current state-of-the-art systems formulate SimulST as a multi-turn dialogue task, requiring specialized interleaved training data and relying on computationally expensive large language model (LLM) inference for decision-making. In this paper, we propose SimulSense, a novel framework for SimulST that mimics human interpreters by continuously reading input speech and triggering write decisions to produce translation when a new sense unit is perceived. Experiments against two state-of-the-art baseline systems demonstrate that our proposed method achieves a superior quality-latency tradeoff and substantially improved real-time efficiency, where its decision-making is up to 9.6x faster than the baselines.

Method: Large-scale study assessing 95 advanced LLMs on 87 real-world clinical text tasks across 9 languages and 8 task types, with fine-grained analyses of reasoning length, medical concept alignment, and error profiles using both LLM-as-a-judge and clinical expert evaluation.

Result: 86.3% of models suffered consistent performance degradation with CoT; more capable models remained relatively robust while weaker ones declined substantially.

Conclusion: CoT enhances interpretability but may undermine reliability in clinical text tasks, highlighting a critical paradox and the need for transparent and trustworthy approaches in clinical reasoning.

Abstract: Large language models (LLMs) are increasingly being applied to clinical care, a domain where both accuracy and transparent reasoning are critical for safe and trustworthy deployment. Chain-of-thought (CoT) prompting, which elicits step-by-step reasoning, has demonstrated improvements in performance and interpretability across a wide range of tasks. However, its effectiveness in clinical contexts remains largely unexplored, particularly in the context of electronic health records (EHRs), the primary source of clinical documentation, which are often lengthy, fragmented, and noisy. In this work, we present the first large-scale systematic study of CoT for clinical text understanding. We assess 95 advanced LLMs on 87 real-world clinical text tasks, covering 9 languages and 8 task types. Contrary to prior findings in other domains, we observe that 86.3% of models suffer consistent performance degradation in the CoT setting. More capable models remain relatively robust, while weaker ones suffer substantial declines. To better characterize these effects, we perform fine-grained analyses of reasoning length, medical concept alignment, and error profiles, leveraging both LLM-as-a-judge evaluation and clinical expert evaluation. Our results uncover systematic patterns in when and why CoT fails in clinical contexts, which highlight a critical paradox: CoT enhances interpretability but may undermine reliability in clinical text tasks. This work provides an empirical basis for clinical reasoning strategies of LLMs, highlighting the need for transparent and trustworthy approaches.

Method: Uses counterfactual data augmentation and rationale-based supervision to disentangle sentiment from stance, with a model-agnostic framework that doesn’t require fine-tuning.

Result: Significantly reduces spurious correlations, enhances zero-shot generalization, and improves fairness across multiple LLMs. Also releases the first high-quality Thai political stance dataset.

Conclusion: Highlights the importance of culturally grounded debiasing techniques for underrepresented languages like Thai.

Abstract: Political stance detection in low-resource and culturally complex settings poses a critical challenge for large language models (LLMs). In the Thai political landscape - marked by indirect language, polarized figures, and entangled sentiment and stance - LLMs often display systematic biases such as sentiment leakage and favoritism toward entities. These biases undermine fairness and reliability. We present ThaiFACTUAL, a lightweight, model-agnostic calibration framework that mitigates political bias without requiring fine-tuning. ThaiFACTUAL uses counterfactual data augmentation and rationale-based supervision to disentangle sentiment from stance and reduce bias. We also release the first high-quality Thai political stance dataset, annotated with stance, sentiment, rationales, and bias markers across diverse entities and events. Experimental results show that ThaiFACTUAL significantly reduces spurious correlations, enhances zero-shot generalization, and improves fairness across multiple LLMs. This work highlights the importance of culturally grounded debiasing techniques for underrepresented languages.

Method: Combines Motivational Knowledge Graph (storing problem, challenge, solution nodes) with dual-agent Socratic Ideator using questioning to refine ideas.

Result: Outperforms state-of-the-art approaches on ICLR25 paper topics dataset across LLM-based scoring, ELO ranking, and human evaluation metrics.

Conclusion: The framework successfully addresses LLM ideation limitations by providing essential grounding and practical improvement steps through motivational graphs and Socratic questioning.

Abstract: Large Language Models (LLMs) hold substantial potential for accelerating academic ideation but face critical challenges in grounding ideas and mitigating confirmation bias for further refinement. We propose integrating motivational knowledge graphs and socratic dialogue to address these limitations in enhanced LLM ideation (MotivGraph-SoIQ). This novel framework provides essential grounding and practical idea improvement steps for LLM ideation by integrating a Motivational Knowledge Graph (MotivGraph) with a Q-Driven Socratic Ideator. The MotivGraph structurally stores three key node types(problem, challenge and solution) to offer motivation grounding for the LLM ideation process. The Ideator is a dual-agent system utilizing Socratic questioning, which facilitates a rigorous refinement process that mitigates confirmation bias and improves idea quality across novelty, experimental rigor, and motivational rationality dimensions. On the ICLR25 paper topics dataset, MotivGraph-SoIQ exhibits clear advantages over existing state-of-the-art approaches across LLM-based scoring, ELO ranking, and human evaluation metrics.

Method: Propose a simple black-box metric based on analyzing LLM behavior under uncertainty expression. Found that LLMs generate consistent responses for factual content but inconsistent responses for non-factual content.

Result: The proposed metric is more predictive of factuality in model responses than baselines that use internal knowledge of LLMs.

Conclusion: Expression of uncertainty can effectively detect hallucinations in black-box settings, providing a practical solution without requiring internal model access.

Abstract: Despite the great advancement of Language modeling in recent days, Large Language Models (LLMs) such as GPT3 are notorious for generating non-factual responses, so-called “hallucination” problems. Existing methods for detecting and alleviating this hallucination problem require external resources or the internal state of LLMs, such as the output probability of each token. Given the LLM’s restricted external API availability and the limited scope of external resources, there is an urgent demand to establish the Black-Box approach as the cornerstone for effective hallucination detection. In this work, we propose a simple black-box hallucination detection metric after the investigation of the behavior of LLMs under expression of uncertainty. Our comprehensive analysis reveals that LLMs generate consistent responses when they present factual responses while non-consistent responses vice versa. Based on the analysis, we propose an efficient black-box hallucination detection metric with the expression of uncertainty. The experiment demonstrates that our metric is more predictive of the factuality in model responses than baselines that use internal knowledge of LLMs.

Method: GraphSearch organizes retrieval into a modular framework with six modules for multi-turn interactions and iterative reasoning, plus a dual-channel retrieval strategy that issues semantic queries over chunk-based text data and relational queries over structural graph data.

Result: Experimental results across six multi-hop RAG benchmarks demonstrate that GraphSearch consistently improves answer accuracy and generation quality over traditional strategies.

Conclusion: GraphSearch represents a promising direction for advancing graph retrieval-augmented generation by enabling comprehensive utilization of both text and graph modalities through its dual-channel retrieval approach.

Abstract: Graph Retrieval-Augmented Generation (GraphRAG) enhances factual reasoning in LLMs by structurally modeling knowledge through graph-based representations. However, existing GraphRAG approaches face two core limitations: shallow retrieval that fails to surface all critical evidence, and inefficient utilization of pre-constructed structural graph data, which hinders effective reasoning from complex queries. To address these challenges, we propose \textsc{GraphSearch}, a novel agentic deep searching workflow with dual-channel retrieval for GraphRAG. \textsc{GraphSearch} organizes the retrieval process into a modular framework comprising six modules, enabling multi-turn interactions and iterative reasoning. Furthermore, \textsc{GraphSearch} adopts a dual-channel retrieval strategy that issues semantic queries over chunk-based text data and relational queries over structural graph data, enabling comprehensive utilization of both modalities and their complementary strengths. Experimental results across six multi-hop RAG benchmarks demonstrate that \textsc{GraphSearch} consistently improves answer accuracy and generation quality over the traditional strategy, confirming \textsc{GraphSearch} as a promising direction for advancing graph retrieval-augmented generation.

Method: Using vector embeddings from state-of-the-art language models and a cumulative clustering approach to track outlier evolution over time in French and English news datasets on corporate social responsibility and climate change.

Result: Outliers consistently evolve into coherent topics over time across both models and languages, revealing a consistent pattern of topic emergence.

Conclusion: Outliers should not be dismissed as noise but rather recognized as valuable indicators of emerging topics that can provide early insights into topic evolution in dynamic text corpora.

Abstract: This paper examines how outliers, often dismissed as noise in topic modeling, can act as weak signals of emerging topics in dynamic news corpora. Using vector embeddings from state-of-the-art language models and a cumulative clustering approach, we track their evolution over time in French and English news datasets focused on corporate social responsibility and climate change. The results reveal a consistent pattern: outliers tend to evolve into coherent topics over time across both models and languages.

Method: Developed TACOS, a fine-grained 21-class taxonomy that explicitly models varying safety thresholds and external tool dependencies, integrating safety filtering and tool selection into unified intent classification.

Result: Created a TACOS-annotated dataset and experiments showed the value of specialized taxonomy for clinical agents, revealing insights about training data distribution and base model pretrained knowledge.

Conclusion: TACOS provides an effective framework for comprehensive safety in clinical chatbots, demonstrating the importance of domain-specific safety taxonomies that integrate multiple safety considerations.

Abstract: Safety is a paramount concern in clinical chatbot applications, where inaccurate or harmful responses can lead to serious consequences. Existing methods–such as guardrails and tool calling–often fall short in addressing the nuanced demands of the clinical domain. In this paper, we introduce TACOS (TAxonomy of COmprehensive Safety for Clinical Agents), a fine-grained, 21-class taxonomy that integrates safety filtering and tool selection into a single user intent classification step. TACOS is a taxonomy that can cover a wide spectrum of clinical and non-clinical queries, explicitly modeling varying safety thresholds and external tool dependencies. To validate our framework, we curate a TACOS-annotated dataset and perform extensive experiments. Our results demonstrate the value of a new taxonomy specialized for clinical agent settings, and reveal useful insights about train data distribution and pretrained knowledge of base models.

Method: FRC integrates LLM semantic priors with continuous fuzzy membership degrees, creating explicit interaction between probability-based reasoning and fuzzy membership reasoning to gradually transform ambiguous inputs into clear decisions.

Result: Validated on sentiment analysis tasks, FRC ensures stable reasoning and facilitates knowledge transfer across different model scales, capturing conflicting or uncertain signals that traditional methods miss.

Conclusion: FRC provides a general mechanism for managing subtle and ambiguous expressions with improved interpretability and robustness in NLP tasks.

Abstract: With the rapid advancement of large language models (LLMs), natural language processing (NLP) has achieved remarkable progress. Nonetheless, significant challenges remain in handling texts with ambiguity, polysemy, or uncertainty. We introduce the Fuzzy Reasoning Chain (FRC) framework, which integrates LLM semantic priors with continuous fuzzy membership degrees, creating an explicit interaction between probability-based reasoning and fuzzy membership reasoning. This transition allows ambiguous inputs to be gradually transformed into clear and interpretable decisions while capturing conflicting or uncertain signals that traditional probability-based methods cannot. We validate FRC on sentiment analysis tasks, where both theoretical analysis and empirical results show that it ensures stable reasoning and facilitates knowledge transfer across different model scales. These findings indicate that FRC provides a general mechanism for managing subtle and ambiguous expressions with improved interpretability and robustness.

Method: Created RedNote-Vibe dataset from Xiaohongshu platform with 5 years of data including user engagement metrics, and proposed PLAD framework using psycholinguistic features for interpretable AIGT detection.

Result: PLAD achieves superior detection performance and provides insights into signatures distinguishing human vs AI content, revealing complex relationships between linguistic features and social media engagement.

Conclusion: The RedNote-Vibe dataset enables temporal AIGT analysis, and PLAD offers an effective interpretable approach for social media AIGT detection with insights into linguistic-engagement relationships.

Abstract: The proliferation of Large Language Models (LLMs) has led to widespread AI-Generated Text (AIGT) on social media platforms, creating unique challenges where content dynamics are driven by user engagement and evolve over time. However, existing datasets mainly depict static AIGT detection. In this work, we introduce RedNote-Vibe, the first longitudinal (5-years) dataset for social media AIGT analysis. This dataset is sourced from Xiaohongshu platform, containing user engagement metrics (e.g., likes, comments) and timestamps spanning from the pre-LLM period to July 2025, which enables research into the temporal dynamics and user interaction patterns of AIGT. Furthermore, to detect AIGT in the context of social media, we propose PsychoLinguistic AIGT Detection Framework (PLAD), an interpretable approach that leverages psycholinguistic features. Our experiments show that PLAD achieves superior detection performance and provides insights into the signatures distinguishing human and AI-generated content. More importantly, it reveals the complex relationship between these linguistic features and social media engagement. The dataset is available at https://github.com/testuser03158/RedNote-Vibe.

Method: Derived a standard set of quality criterion names and definitions from three surveys of NLP evaluations, structured into a hierarchy where parent nodes capture common aspects of child nodes.

Result: Developed QCET (Quality Criteria for Evaluation Taxonomy) - a standardized taxonomy that provides clear definitions and hierarchical structure for NLP quality criteria.

Conclusion: QCET enables establishing comparability of existing evaluations, guiding new evaluation designs, and assessing regulatory compliance, addressing a long-standing issue in NLP evaluation methodology.

Abstract: Prior work has shown that two NLP evaluation experiments that report results for the same quality criterion name (e.g. Fluency) do not necessarily evaluate the same aspect of quality, and the comparability implied by the name can be misleading. Not knowing when two evaluations are comparable in this sense means we currently lack the ability to draw reliable conclusions about system quality on the basis of multiple, independently conducted evaluations. This in turn hampers the ability of the field to progress scientifically as a whole, a pervasive issue in NLP since its beginning (Sparck Jones, 1981). It is hard to see how the issue of unclear comparability can be fully addressed other than by the creation of a standard set of quality criterion names and definitions that the several hundred quality criterion names actually in use in the field can be mapped to, and grounded in. Taking a strictly descriptive approach, the QCET Quality Criteria for Evaluation Taxonomy derives a standard set of quality criterion names and definitions from three surveys of evaluations reported in NLP, and structures them into a hierarchy where each parent node captures common aspects of its child nodes. We present QCET and the resources it consists of, and discuss its three main uses in (i) establishing comparability of existing evaluations, (ii) guiding the design of new evaluations, and (iii) assessing regulatory compliance.

Method: Proposed LocFT-BF method that restores fine-tuning to standard breadth-first pipeline with mini-batch optimization and systematically analyzes tuning locations for localized editing.

Result: Outperforms state-of-the-art methods by large margins, sustains 100K edits and 72B-parameter models (10x beyond prior practice) without sacrificing general capabilities.

Conclusion: Fine-tuning can be advanced from an underestimated baseline to a leading method for model editing by using proper pipeline design and localized tuning strategy.

Abstract: Fine-tuning, a foundational method for adapting large language models, has long been considered ineffective for model editing. Here, we challenge this belief, arguing that the reported failure arises not from the inherent limitation of fine-tuning itself, but from adapting it to the sequential nature of the editing task, a single-pass depth-first pipeline that optimizes each sample to convergence before moving on. While intuitive, this depth-first pipeline coupled with sample-wise updating over-optimizes each edit and induces interference across edits. Our controlled experiments reveal that simply restoring fine-tuning to the standard breadth-first (i.e., epoch-based) pipeline with mini-batch optimization substantially improves its effectiveness for model editing. Moreover, fine-tuning in editing also suffers from suboptimal tuning parameter locations inherited from prior methods. Through systematic analysis of tuning locations, we derive LocFT-BF, a simple and effective localized editing method built on the restored fine-tuning framework. Extensive experiments across diverse LLMs and datasets demonstrate that LocFT-BF outperforms state-of-the-art methods by large margins. Notably, to our knowledge, it is the first to sustain 100K edits and 72B-parameter models,10 x beyond prior practice, without sacrificing general capabilities. By clarifying a long-standing misconception and introducing a principled localized tuning strategy, we advance fine-tuning from an underestimated baseline to a leading method for model editing, establishing a solid foundation for future research.

Method: Replaces low-rank decomposition with structured sparse factorization using a dense dictionary and column-sparse coefficient matrix, optimized with calibration data to minimize functional reconstruction error.

Result: Consistently superior to state-of-the-art low-rank methods in accuracy and perplexity across Llama and Qwen models at 20-50% compression ratios.

Conclusion: Structured sparse dictionary learning is a powerful alternative to conventional low-rank approaches for efficient LLM deployment, offering better expressiveness and model fidelity.

Abstract: Post-training compression of large language models (LLMs) largely relies on low-rank weight approximation, which represents each column of a weight matrix in a shared low-dimensional subspace. While this is a computationally efficient strategy, the imposed structural constraint is rigid and can lead to a noticeable model accuracy drop. In this work, we propose CoSpaDi (Compression via Sparse Dictionary Learning), a novel training-free compression framework that replaces low-rank decomposition with a more flexible structured sparse factorization in which each weight matrix is represented with a dense dictionary and a column-sparse coefficient matrix. This formulation enables a union-of-subspaces representation: different columns of the original weight matrix are approximated in distinct subspaces spanned by adaptively selected dictionary atoms, offering greater expressiveness than a single invariant basis. Crucially, CoSpaDi leverages a small calibration dataset to optimize the factorization such that the output activations of compressed projection layers closely match those of the original ones, thereby minimizing functional reconstruction error rather than mere weight approximation. This data-aware strategy preserves better model fidelity without any fine-tuning under reasonable compression ratios. Moreover, the resulting structured sparsity allows efficient sparse-dense matrix multiplication and is compatible with post-training quantization for further memory and latency gains. We evaluate CoSpaDi across multiple Llama and Qwen models under per-layer and per-group settings at 20-50% compression ratios, demonstrating consistent superiority over state-of-the-art data-aware low-rank methods both in accuracy and perplexity. Our results establish structured sparse dictionary learning as a powerful alternative to conventional low-rank approaches for efficient LLM deployment.

Method: Proposes Dialogue Act Script (DAS), a structured framework for encoding, localizing, and generating multilingual dialogues from abstract intent representations instead of direct translation.

Result: Human evaluations across Italian, German, and Chinese show DAS-generated dialogues outperform both machine and human translators on cultural relevance, coherence, and situational appropriateness.

Conclusion: DAS enables generation of culturally appropriate multilingual dialogues by using structured dialogue act representations, mitigating translationese and improving fluency.

Abstract: Non-English dialogue datasets are scarce, and models are often trained or evaluated on translations of English-language dialogues, an approach which can introduce artifacts that reduce their naturalness and cultural appropriateness. This work proposes Dialogue Act Script (DAS), a structured framework for encoding, localizing, and generating multilingual dialogues from abstract intent representations. Rather than translating dialogue utterances directly, DAS enables the generation of new dialogues in the target language that are culturally and contextually appropriate. By using structured dialogue act representations, DAS supports flexible localization across languages, mitigating translationese and enabling more fluent, naturalistic conversations. Human evaluations across Italian, German, and Chinese show that DAS-generated dialogues consistently outperform those produced by both machine and human translators on measures of cultural relevance, coherence, and situational appropriateness.

Method: Proposes Solve-to-Judge (S2J) approach that simultaneously leverages both solving and judging capabilities on a single GRM’s output for supervision, explicitly linking problem-solving and evaluation abilities during model optimization to narrow the solve-to-judge gap.

Result: S2J effectively reduces the solve-to-judge gap by 16.2% and enhances judgment performance by 5.8%, achieving state-of-the-art performance among GRMs built on the same base model while using significantly smaller training datasets, accomplished through self-evolution without external model distillation.

Conclusion: The Solve-to-Judge approach successfully bridges the solve-to-judge gap in generative reward models by explicitly connecting problem-solving and judgment capabilities during optimization, leading to improved performance with less data and no reliance on external models.

Abstract: With the rapid development of large language models (LLMs), generative reward models (GRMs) have been widely adopted for reward modeling and evaluation. Previous studies have primarily focused on training specialized GRMs by optimizing them on preference datasets with the judgment correctness as supervision. While it’s widely accepted that GRMs with stronger problem-solving capabilities typically exhibit superior judgment abilities, we first identify a significant solve-to-judge gap when examining individual queries. Specifically, the solve-to-judge gap refers to the phenomenon where GRMs struggle to make correct judgments on some queries (14%-37%), despite being fully capable of solving them. In this paper, we propose the Solve-to-Judge (S2J) approach to address this problem. Specifically, S2J simultaneously leverages both the solving and judging capabilities on a single GRM’s output for supervision, explicitly linking the GRM’s problem-solving and evaluation abilities during model optimization, thereby narrowing the gap. Our comprehensive experiments demonstrate that S2J effectively reduces the solve-to-judge gap by 16.2%, thereby enhancing the model’s judgment performance by 5.8%. Notably, S2J achieves state-of-the-art (SOTA) performance among GRMs built on the same base model while utilizing a significantly smaller training dataset. Moreover, S2J accomplishes this through self-evolution without relying on more powerful external models for distillation.

Method: The approach uses test-time compute scaling to elicit multiple reasoning paths from LLMs, trains a verifier model (VERIFIERFC) to select the best reasoning path, and introduces an adaptive mechanism for selective TTS based on claim complexity.

Result: The method achieves 1.8x higher efficiency than standard TTS and delivers 18.8% performance improvement over single-shot claim verification methods, while effectively mitigating reasoning drift issues.

Conclusion: Test-time compute scaling with verifier models and adaptive mechanisms significantly improves fact-checking of complex numerical claims by addressing reasoning drift and enhancing computational efficiency.

Abstract: Fact-checking real-world claims, particularly numerical claims, is inherently complex that require multistep reasoning and numerical reasoning for verifying diverse aspects of the claim. Although large language models (LLMs) including reasoning models have made tremendous advances, they still fall short on fact-checking real-world claims that require a combination of compositional and numerical reasoning. They are unable to understand nuance of numerical aspects, and are also susceptible to the reasoning drift issue, where the model is unable to contextualize diverse information resulting in misinterpretation and backtracking of reasoning process. In this work, we systematically explore scaling test-time compute (TTS) for LLMs on the task of fact-checking complex numerical claims, which entails eliciting multiple reasoning paths from an LLM. We train a verifier model (VERIFIERFC) to navigate this space of possible reasoning paths and select one that could lead to the correct verdict. We observe that TTS helps mitigate the reasoning drift issue, leading to significant performance gains for fact-checking numerical claims. To improve compute efficiency in TTS, we introduce an adaptive mechanism that performs TTS selectively based on the perceived complexity of the claim. This approach achieves 1.8x higher efficiency than standard TTS, while delivering a notable 18.8% performance improvement over single-shot claim verification methods. Our code and data can be found at https://github.com/VenkteshV/VerifierFC

Method: Uni-LAP integrates SCMs and LLMs: SCM uses novel Top-K loss function to generate candidate articles, while LLM employs syllogism-inspired reasoning to refine final predictions.

Result: Empirical evaluation on multiple jurisdiction datasets shows Uni-LAP consistently outperforms existing baselines, demonstrating effectiveness and generalizability.

Conclusion: The proposed Uni-LAP framework successfully addresses limitations of existing methods by combining strengths of SCMs and LLMs, showing improved performance and broader applicability across different legal systems.

Abstract: Legal Article Prediction (LAP) is a critical task in legal text classification, leveraging natural language processing (NLP) techniques to automatically predict relevant legal articles based on the fact descriptions of cases. As a foundational step in legal decision-making, LAP plays a pivotal role in determining subsequent judgments, such as charges and penalties. Despite its importance, existing methods face significant challenges in addressing the complexities of LAP. Supervised classification models (SCMs), such as CNN and BERT, struggle to fully capture intricate fact patterns due to their inherent limitations. Conversely, large language models (LLMs), while excelling in generative tasks, perform suboptimally in predictive scenarios due to the abstract and ID-based nature of legal articles. Furthermore, the diversity of legal systems across jurisdictions exacerbates the issue, as most approaches are tailored to specific countries and lack broader applicability. To address these limitations, we propose Uni-LAP, a universal framework for legal article prediction that integrates the strengths of SCMs and LLMs through tight collaboration. Specifically, in Uni-LAP, the SCM is enhanced with a novel Top-K loss function to generate accurate candidate articles, while the LLM employs syllogism-inspired reasoning to refine the final predictions. We evaluated Uni-LAP on datasets from multiple jurisdictions, and empirical results demonstrate that our approach consistently outperforms existing baselines, showcasing its effectiveness and generalizability.

Method: Comprehensive review of 31 models and 21 benchmarks, analyzing encoder-only and generative architectures, training methods (contrastive learning), and evaluation approaches (translation-based vs culturally grounded).

Result: Found current training methods favor language neutrality over cultural awareness, with two-thirds of benchmarks using translation-based approaches. Identified discrepancies in cross-lingual capabilities and gaps between training objectives and evaluation goals.

Conclusion: There is a fundamental tension between language neutrality and cultural awareness in multilingual vision-language models, with current approaches prioritizing neutrality through contrastive learning, while cultural awareness remains underdeveloped and evaluation methods need better alignment with real-world cultural contexts.

Abstract: This survey examines multilingual vision-language models that process text and images across languages. We review 31 models and 21 benchmarks, spanning encoder-only and generative architectures, and identify a key tension between language neutrality (consistent cross-lingual representations) and cultural awareness (adaptation to cultural contexts). Current training methods favor neutrality through contrastive learning, while cultural awareness depends on diverse data. Two-thirds of evaluation benchmarks use translation-based approaches prioritizing semantic consistency, though recent work incorporates culturally grounded content. We find discrepancies in cross-lingual capabilities and gaps between training objectives and evaluation goals.

Method: Uses instruction-response fine-tuning on food-annotated corpora to link food entities to multiple ontologies, comparing against zero-shot, one-shot, and few-shot LLM baselines.

Result: Achieves state-of-the-art performance with F1 scores reaching 98% on some ontologies and datasets, significantly outperforming non-fine-tuned versions.

Conclusion: Provides three main contributions: food-annotated corpora in IR format, a robust food semantic understanding model, and a strong baseline for future food NEL benchmarking.

Abstract: This paper introduces FoodSEM, a state-of-the-art fine-tuned open-source large language model (LLM) for named-entity linking (NEL) to food-related ontologies. To the best of our knowledge, food NEL is a task that cannot be accurately solved by state-of-the-art general-purpose (large) language models or custom domain-specific models/systems. Through an instruction-response (IR) scenario, FoodSEM links food-related entities mentioned in a text to several ontologies, including FoodOn, SNOMED-CT, and the Hansard taxonomy. The FoodSEM model achieves state-of-the-art performance compared to related models/systems, with F1 scores even reaching 98% on some ontologies and datasets. The presented comparative analyses against zero-shot, one-shot, and few-shot LLM prompting baselines further highlight FoodSEM’s superior performance over its non-fine-tuned version. By making FoodSEM and its related resources publicly available, the main contributions of this article include (1) publishing a food-annotated corpora into an IR format suitable for LLM fine-tuning/evaluation, (2) publishing a robust model to advance the semantic understanding of text in the food domain, and (3) providing a strong baseline on food NEL for future benchmarking.

Method: Uses Information Bottleneck principle to compress high-level plans into minimal yet sufficient latent tokens (Reasoning Capsules). Employs dual objective: primary task loss for accuracy and auxiliary plan-reconstruction loss to ground latent space and improve interpretability.

Result: Reduces visible token footprint of reasoning while maintaining or improving accuracy on complex benchmarks. Achieves better balance between efficiency, accuracy, and interpretability compared to standard CoT.

Conclusion: R-Capsule framework successfully combines efficiency of latent reasoning with transparency of explicit CoT, offering a practical solution to CoT’s verbosity and error propagation issues while maintaining interpretability.

Abstract: Chain-of-Thought (CoT) prompting helps Large Language Models (LLMs) tackle complex reasoning by eliciting explicit step-by-step rationales. However, CoT’s verbosity increases latency and memory usage and may propagate early errors across long chains. We propose the Reasoning Capsule (R-Capsule), a framework that aims to combine the efficiency of latent reasoning with the transparency of explicit CoT. The core idea is to compress the high-level plan into a small set of learned latent tokens (a Reasoning Capsule) while keeping execution steps lightweight or explicit. This hybrid approach is inspired by the Information Bottleneck (IB) principle, where we encourage the capsule to be approximately minimal yet sufficient for the task. Minimality is encouraged via a low-capacity bottleneck, which helps improve efficiency. Sufficiency is encouraged via a dual objective: a primary task loss for answer accuracy and an auxiliary plan-reconstruction loss that encourages the capsule to faithfully represent the original textual plan. The reconstruction objective helps ground the latent space, thereby improving interpretability and reducing the use of uninformative shortcuts. Our framework strikes a balance between efficiency, accuracy, and interpretability, thereby reducing the visible token footprint of reasoning while maintaining or improving accuracy on complex benchmarks. Our codes are available at: https://anonymous.4open.science/r/Reasoning-Capsule-7BE0

Method: Group Tree Optimization (GTO) with two components: (1) Draft Tree Reward - sampling-free objective measuring expected acceptance length; (2) Group-based Draft Policy Training - stable optimization contrasting trees from current and reference models using debiased advantages and PPO-style updates.

Result: Across dialogue (MT-Bench), code (HumanEval), and math (GSM8K) tasks with multiple LLMs, GTO increases acceptance length by 7.4% and yields additional 7.7% speedup over prior state-of-the-art EAGLE-3.

Conclusion: By bridging draft policy misalignment, GTO offers a practical, general solution for efficient LLM inference through better alignment between training objectives and decoding-time tree policies.

Abstract: Speculative decoding accelerates large language model (LLM) inference by letting a lightweight draft model propose multiple tokens that the target model verifies in parallel. Yet existing training objectives optimize only a single greedy draft path, while decoding follows a tree policy that re-ranks and verifies multiple branches. This draft policy misalignment limits achievable speedups. We introduce Group Tree Optimization (GTO), which aligns training with the decoding-time tree policy through two components: (i) Draft Tree Reward, a sampling-free objective equal to the expected acceptance length of the draft tree under the target model, directly measuring decoding performance; (ii) Group-based Draft Policy Training, a stable optimization scheme that contrasts trees from the current and a frozen reference draft model, forming debiased group-standardized advantages and applying a PPO-style surrogate along the longest accepted sequence for robust updates. We further prove that increasing our Draft Tree Reward provably improves acceptance length and speedup. Across dialogue (MT-Bench), code (HumanEval), and math (GSM8K), and multiple LLMs (e.g., LLaMA-3.1-8B, LLaMA-3.3-70B, Vicuna-1.3-13B, DeepSeek-R1-Distill-LLaMA-8B), GTO increases acceptance length by 7.4% and yields an additional 7.7% speedup over prior state-of-the-art EAGLE-3. By bridging draft policy misalignment, GTO offers a practical, general solution for efficient LLM inference.

Method: Development and hosting of twelve shared tasks covering diverse challenges in scholarly document processing, integrated into the consortium’s research data infrastructure.

Result: Fostered methodological innovations and contributed open-access datasets, models, and tools for the broader research community.

Conclusion: Shared tasks are powerful tools for advancing research through community-based standardized evaluation and promoting FAIR research practices.

Abstract: Shared tasks are powerful tools for advancing research through community-based standardised evaluation. As such, they play a key role in promoting findable, accessible, interoperable, and reusable (FAIR), as well as transparent and reproducible research practices. This paper presents an updated overview of twelve shared tasks developed and hosted under the German National Research Data Infrastructure for Data Science and Artificial Intelligence (NFDI4DS) consortium, covering a diverse set of challenges in scholarly document processing. Hosted at leading venues, the tasks foster methodological innovations and contribute open-access datasets, models, and tools for the broader research community, which are integrated into the consortium’s research data infrastructure.

Method: Multiround Adaptive Chain-of-Thought Compression (MACC) uses token elasticity phenomenon and multiround refinement to progressively compress CoTs, adaptively determining optimal compression depth for each input.

Result: Achieves 5.6% average accuracy improvement over state-of-the-art baselines, reduces CoT length by 47 tokens on average, significantly lowers latency, and enables reliable performance prediction using interpretable features.

Conclusion: CoT compression is both effective and predictable, enabling efficient model selection and forecasting without repeated fine-tuning across different models.

Abstract: Chain-of-Thought (CoT) reasoning improves performance on complex tasks but introduces significant inference latency due to verbosity. We propose Multiround Adaptive Chain-of-Thought Compression (MACC), a framework that leverages the token elasticity phenomenon–where overly small token budgets can paradoxically increase output length–to progressively compress CoTs via multiround refinement. This adaptive strategy allows MACC to determine the optimal compression depth for each input. Our method achieves an average accuracy improvement of 5.6 percent over state-of-the-art baselines, while also reducing CoT length by an average of 47 tokens and significantly lowering latency. Furthermore, we show that test-time performance–accuracy and token length–can be reliably predicted using interpretable features like perplexity and compression rate on the training set. Evaluated across different models, our method enables efficient model selection and forecasting without repeated fine-tuning, demonstrating that CoT compression is both effective and predictable. Our code will be released in https://github.com/Leon221220/MACC.

Method: Introduces BMAS English dataset for binary classification (human vs machine text), multiclass classification (identifying specific generators), sentence-level segmentation (detecting human-AI collaborative text boundaries), and adversarial attack scenarios.

Result: A comprehensive dataset and framework for machine-generated text detection that addresses multiple detection scenarios including classification, attribution, segmentation, and adversarial settings.

Conclusion: This work aims to address previous limitations in Machine-Generated Text Detection (MGTD) in a more meaningful way by providing a multi-faceted approach to detecting and analyzing machine-generated content.

Abstract: Large Language Models (LLMs) are gearing up to surpass human creativity. The veracity of the statement needs careful consideration. In recent developments, critical questions arise regarding the authenticity of human work and the preservation of their creativity and innovative abilities. This paper investigates such issues. This paper addresses machine-generated text detection across several scenarios, including document-level binary and multiclass classification or generator attribution, sentence-level segmentation to differentiate between human-AI collaborative text, and adversarial attacks aimed at reducing the detectability of machine-generated text. We introduce a new work called BMAS English: an English language dataset for binary classification of human and machine text, for multiclass classification, which not only identifies machine-generated text but can also try to determine its generator, and Adversarial attack addressing where it is a common act for the mitigation of detection, and Sentence-level segmentation, for predicting the boundaries between human and machine-generated text. We believe that this paper will address previous work in Machine-Generated Text Detection (MGTD) in a more meaningful way.

Method: Meta-learning framework that translates context into adapter parameters with compositional structure, enabling algebraic merging of adapters from different inputs.

Result: Outperforms ICL and prior generator-based methods on multiple-choice and extractive QA tasks, especially when scaling to more inputs. Provides lower inference cost, robustness to long-context instability, and reversible encoding.

Conclusion: Composable adapter generation is a practical and efficient alternative for scaling LLM deployment, offering benefits in cost, robustness, and handling context window limitations.

Abstract: Large language models (LLMs) often seamlessly adapt to new tasks through in-context learning (ICL) or supervised fine-tuning (SFT). However, both of these approaches face key limitations: ICL is inefficient when handling many demonstrations, and SFT incurs training overhead while sacrificing flexibility. Mapping instructions or demonstrations from context directly into adapter parameters offers an appealing alternative. While prior work explored generating adapters based on a single input context, it has overlooked the need to integrate multiple chunks of information. To address this gap, we introduce CompAs, a meta-learning framework that translates context into adapter parameters with a compositional structure. Adapters generated this way can be merged algebraically, enabling instructions, demonstrations, or retrieved passages to be seamlessly combined without reprocessing long prompts. Critically, this approach yields three benefits: lower inference cost, robustness to long-context instability, and establishes a principled solution when input exceeds the model’s context window. Furthermore, CompAs encodes information into adapter parameters in a reversible manner, enabling recovery of input context through a decoder, facilitating safety and security. Empirical results on diverse multiple-choice and extractive question answering tasks show that CompAs outperforms ICL and prior generator-based methods, especially when scaling to more inputs. Our work establishes composable adapter generation as a practical and efficient alternative for scaling LLM deployment.

Method: Used synthetic data distillation framework to conduct large-scale supervised study comparing Instruction Fine-Tuning (IFT) and reasoning models of varying sizes on math-centric and general-purpose tasks in multiple-choice and open-ended formats.

Result: Reasoning consistently improves model performance, often matching or surpassing significantly larger IFT systems. Reasoning models become increasingly valuable as model size scales, overcoming IFT performance limits on reasoning-intensive and open-ended tasks.

Conclusion: While IFT remains Pareto-optimal in training and inference costs, reasoning capabilities provide significant performance benefits, especially at larger model scales and for reasoning-intensive tasks.

Abstract: Large Language Models (LLMs) with reasoning capabilities have achieved state-of-the-art performance on a wide range of tasks. Despite its empirical success, the tasks and model scales at which reasoning becomes effective, as well as its training and inference costs, remain underexplored. In this work, we rely on a synthetic data distillation framework to conduct a large-scale supervised study. We compare Instruction Fine-Tuning (IFT) and reasoning models of varying sizes, on a wide range of math-centric and general-purpose tasks, evaluating both multiple-choice and open-ended formats. Our analysis reveals that reasoning consistently improves model performance, often matching or surpassing significantly larger IFT systems. Notably, while IFT remains Pareto-optimal in training and inference costs, reasoning models become increasingly valuable as model size scales, overcoming IFT performance limits on reasoning-intensive and open-ended tasks.

Method: Argument based on two premises: (1) specific intentions are needed for literal meaning, and (2) LLMs lack these intentions. Defense against semantic externalist and internalist counterarguments.

Result: The paper concludes that LLM outputs are indeed meaningless in the literal sense, despite their apparent meaningfulness and utility.

Conclusion: While LLM outputs lack literal meaning due to absence of proper intentions, they can still appear meaningful and serve as tools for acquiring true beliefs and knowledge.

Abstract: In this paper, we offer a simple argument for the conclusion that the outputs of large language models (LLMs) are meaningless. Our argument is based on two key premises: (a) that certain kinds of intentions are needed in order for LLMs’ outputs to have literal meanings, and (b) that LLMs cannot plausibly have the right kinds of intentions. We defend this argument from various types of responses, for example, the semantic externalist argument that deference can be assumed to take the place of intentions and the semantic internalist argument that meanings can be defined purely in terms of intrinsic relations between concepts, such as conceptual roles. We conclude the paper by discussing why, even if our argument is sound, the outputs of LLMs nevertheless seem meaningful and can be used to acquire true beliefs and even knowledge.

Method: Recursive Thematic Partitioning (RTP) leverages LLMs to interactively build a binary tree where each node is a natural language question that semantically partitions the data, creating interpretable taxonomies.

Result: RTP’s question-driven hierarchy is more interpretable than BERTopic’s keyword-based topics and serves as powerful features in downstream classification tasks, especially when themes correlate with task labels.

Conclusion: RTP shifts text analysis from statistical pattern discovery to knowledge-driven thematic analysis and enables structured, controllable prompts for generative models, transforming analysis into synthesis tools.

Abstract: Unsupervised analysis of text corpora is challenging, especially in data-scarce domains where traditional topic models struggle. While these models offer a solution, they typically describe clusters with lists of keywords that require significant manual effort to interpret and often lack semantic coherence. To address this critical interpretability gap, we introduce Recursive Thematic Partitioning (RTP), a novel framework that leverages Large Language Models (LLMs) to interactively build a binary tree. Each node in the tree is a natural language question that semantically partitions the data, resulting in a fully interpretable taxonomy where the logic of each cluster is explicit. Our experiments demonstrate that RTP’s question-driven hierarchy is more interpretable than the keyword-based topics from a strong baseline like BERTopic. Furthermore, we establish the quantitative utility of these clusters by showing they serve as powerful features in downstream classification tasks, particularly when the data’s underlying themes correlate with the task labels. RTP introduces a new paradigm for data exploration, shifting the focus from statistical pattern discovery to knowledge-driven thematic analysis. Furthermore, we demonstrate that the thematic paths from the RTP tree can serve as structured, controllable prompts for generative models. This transforms our analytical framework into a powerful tool for synthesis, enabling the consistent imitation of specific characteristics discovered in the source corpus.

Method: Proposes StableToken with multi-branch architecture that processes audio in parallel and merges representations through bit-wise voting mechanism to form stable token sequences.

Result: Sets new SOTA in token stability, drastically reducing Unit Edit Distance under diverse noise conditions, and significantly improves robustness of SpeechLLMs on various tasks.

Conclusion: StableToken’s consensus-driven mechanism effectively addresses tokenizer instability, providing foundational stability that directly benefits downstream speech language models.

Abstract: Prevalent semantic speech tokenizers, designed to capture linguistic content, are surprisingly fragile. We find they are not robust to meaning-irrelevant acoustic perturbations; even at high Signal-to-Noise Ratios (SNRs) where speech is perfectly intelligible, their output token sequences can change drastically, increasing the learning burden for downstream LLMs. This instability stems from two flaws: a brittle single-path quantization architecture and a distant training signal indifferent to intermediate token stability. To address this, we introduce StableToken, a tokenizer that achieves stability through a consensus-driven mechanism. Its multi-branch architecture processes audio in parallel, and these representations are merged via a powerful bit-wise voting mechanism to form a single, stable token sequence. StableToken sets a new state-of-the-art in token stability, drastically reducing Unit Edit Distance (UED) under diverse noise conditions. This foundational stability translates directly to downstream benefits, significantly improving the robustness of SpeechLLMs on a variety of tasks.

Method: Introduces Composite Reasoning (CR) approach that allows LLMs to dynamically explore and combine multiple reasoning styles adaptively based on domain requirements.

Result: Outperforms existing baselines (CoT, DeepSeek-R1) on scientific and medical QA benchmarks, with superior sample efficiency and token usage. Adaptively prioritizes different reasoning styles for different domains.

Conclusion: Cultivating internal reasoning style diversity enables LLMs to develop more robust, adaptive, and efficient problem-solving abilities.

Abstract: Large Language Models (LLMs), despite their remarkable capabilities, rely on singular, pre-dominant reasoning paradigms, hindering their performance on intricate problems that demand diverse cognitive strategies. To address this, we introduce Composite Reasoning (CR), a novel reasoning approach empowering LLMs to dynamically explore and combine multiple reasoning styles like deductive, inductive, and abductive for more nuanced problem-solving. Evaluated on scientific and medical question-answering benchmarks, our approach outperforms existing baselines like Chain-of-Thought (CoT) and also surpasses the accuracy of DeepSeek-R1 style reasoning (SR) capabilities, while demonstrating superior sample efficiency and adequate token usage. Notably, CR adaptively emphasizes domain-appropriate reasoning styles. It prioritizes abductive and deductive reasoning for medical question answering, but shifts to causal, deductive, and inductive methods for scientific reasoning. Our findings highlight that by cultivating internal reasoning style diversity, LLMs acquire more robust, adaptive, and efficient problem-solving abilities.

Method: Proposed Reverse Speculative Decoding (RSD) where the teacher model proposes candidate tokens but the student model determines acceptance based on its own probability distributions, filtering out low probability tokens to create student-friendly reasoning traces.

Result: Direct distillation degraded performance by 20.5%, while RSD-generated reasoning traces achieved 4.9% improvement across major reasoning benchmarks. RSD traces are model-specific and must be tailored for each student architecture.

Conclusion: Low probability tokens constitute the critical bottleneck in reasoning ability transfer, and distributional alignment through methods like RSD must be specifically tailored for each student model’s unique internal representation.

Abstract: Transferring reasoning capabilities from larger language models to smaller ones through supervised fine-tuning often fails counterintuitively, with performance degrading despite access to high-quality teacher demonstrations. We identify that this failure stems from distributional misalignment: reasoning traces from larger models contain tokens that are low probability under the student’s distribution, exceeding the internal representation capacity of smaller architectures and creating learning barriers rather than helpful guidance. We propose Reverse Speculative Decoding (RSD), a mechanism for generating student-friendly reasoning traces in which the teacher model proposes candidate tokens but the student model determines acceptance based on its own probability distributions, filtering low probability tokens. When applied to Qwen3-0.6B, direct distillation of s1K-1.1 reasoning trace data degrades average performance across major reasoning benchmarks by 20.5%, while the same model trained on RSD-generated reasoning traces achieves meaningful improvements of 4.9%. Our analysis reveals that low probability tokens constitute the critical bottleneck in reasoning ability transfer. However, cross-model experiments demonstrate that RSD traces are model-specific rather than universally applicable, indicating that distributional alignment must be tailored for each student architecture’s unique internal representation.

Method: FeatBench uses pure natural language prompts without code hints, employs a multi-level filtering pipeline for quality, includes F2P and P2P tests for verification, and covers diverse application domains.

Result: Evaluation shows feature implementation in vibe coding is challenging, with the highest success rate only 29.94%. Analysis reveals “aggressive implementation” strategy that causes failures but leads to superior software design.

Conclusion: FeatBench addresses the gap in evaluating vibe coding capabilities and reveals the significant challenge of feature implementation in this paradigm, with the benchmark and tools released for community research.

Abstract: The rapid advancement of Large Language Models (LLMs) has given rise to a novel software development paradigm known as “vibe coding,” where users interact with coding agents through high-level natural language. However, existing evaluation benchmarks for code generation inadequately assess an agent’s vibe coding capabilities. Existing benchmarks are misaligned, as they either require code-level specifications or focus narrowly on issue-solving, neglecting the critical scenario of feature implementation within the vibe coding paradiam. To address this gap, we propose FeatBench, a novel benchmark for vibe coding that focuses on feature implementation. Our benchmark is distinguished by several key features: 1. Pure Natural Language Prompts. Task inputs consist solely of abstract natural language descriptions, devoid of any code or structural hints. 2. A Rigorous & Evolving Data Collection Process. FeatBench is built on a multi-level filtering pipeline to ensure quality and a fully automated pipeline to evolve the benchmark, mitigating data contamination. 3. Comprehensive Test Cases. Each task includes Fail-to-Pass (F2P) and Pass-to-Pass (P2P) tests to verify correctness and prevent regressions. 4. Diverse Application Domains. The benchmark includes repositories from diverse domains to ensure it reflects real-world scenarios. We evaluate two state-of-the-art agent frameworks with four leading LLMs on FeatBench. Our evaluation reveals that feature implementation within the vibe coding paradigm is a significant challenge, with the highest success rate of only 29.94%. Our analysis also reveals a tendency for “aggressive implementation,” a strategy that paradoxically leads to both critical failures and superior software design. We release FeatBench, our automated collection pipeline, and all experimental results to facilitate further community research.

Method: Introduced FLEXI benchmark with six diverse human-LLM interaction scenarios to systematically evaluate latency, quality, and conversational effectiveness, focusing on model interruption in emergencies.

Result: Revealed significant gaps between open source and commercial models in emergency awareness, turn terminating, and interaction latency.

Conclusion: Next token-pair prediction offers a promising path toward achieving truly seamless and human-like full-duplex interaction.

Abstract: Full-Duplex Speech-to-Speech Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling real-time spoken dialogue systems. However, benchmarking and modeling these models remains a fundamental challenge. We introduce FLEXI, the first benchmark for full-duplex LLM-human spoken interaction that explicitly incorporates model interruption in emergency scenarios. FLEXI systematically evaluates the latency, quality, and conversational effectiveness of real-time dialogue through six diverse human-LLM interaction scenarios, revealing significant gaps between open source and commercial models in emergency awareness, turn terminating, and interaction latency. Finally, we suggest that next token-pair prediction offers a promising path toward achieving truly seamless and human-like full-duplex interaction.

Method: 1) Develop a new benchmark for safety compliance by generating realistic LLM safety scenarios seeded with legal statutes; 2) Align Qwen3-8B using Group Policy Optimization (GRPO) to construct Compliance Reasoner, which aligns LLMs with legal standards.

Result: Compliance Reasoner achieves superior performance on the new benchmark, with average improvements of +10.45% for EU AI Act and +11.85% for GDPR compliance.

Conclusion: The legal compliance approach provides a rigorous, systematic framework for LLM safety, effectively bridging the gap between LLM safety and legal standards through the proposed Compliance Reasoner.

Abstract: The proliferation of Large Language Models (LLMs) has demonstrated remarkable capabilities, elevating the critical importance of LLM safety. However, existing safety methods rely on ad-hoc taxonomy and lack a rigorous, systematic protection, failing to ensure safety for the nuanced and complex behaviors of modern LLM systems. To address this problem, we solve LLM safety from legal compliance perspectives, named safety compliance. In this work, we posit relevant established legal frameworks as safety standards for defining and measuring safety compliance, including the EU AI Act and GDPR, which serve as core legal frameworks for AI safety and data security in Europe. To bridge the gap between LLM safety and legal compliance, we first develop a new benchmark for safety compliance by generating realistic LLM safety scenarios seeded with legal statutes. Subsequently, we align Qwen3-8B using Group Policy Optimization (GRPO) to construct a safety reasoner, Compliance Reasoner, which effectively aligns LLMs with legal standards to mitigate safety risks. Our comprehensive experiments demonstrate that the Compliance Reasoner achieves superior performance on the new benchmark, with average improvements of +10.45% for the EU AI Act and +11.85% for GDPR.

Method: Proposes SSKG-LLM with three modules: Knowledge Graph Retrieval (KGR) and Knowledge Graph Encoding (KGE) to preserve semantics while utilizing structure, and Knowledge Graph Adaptation (KGA) to enable LLMs to understand KG embeddings.

Result: Extensive experiments show that incorporating structural information from KGs enhances the factual reasoning abilities of LLMs, though specific quantitative results are not provided in the abstract.

Conclusion: SSKG-LLM successfully bridges the gap between KG structural information and LLM reasoning processes, providing an effective solution to reduce hallucination in LLMs through better KG integration.

Abstract: Currently, the main approach for Large Language Models (LLMs) to tackle the hallucination issue is incorporating Knowledge Graphs(KGs).However, LLMs typically treat KGs as plain text, extracting only semantic information and limiting their use of the crucial structural aspects of KGs. Another challenge is the gap between the embedding spaces of KGs encoders and LLMs text embeddings, which hinders the effective integration of structured knowledge. To overcome these obstacles, we put forward the SSKG-LLM, an innovative model architecture that is designed to efficiently integrate both the Structural and Semantic information of KGs into the reasoning processes of LLMs. SSKG-LLM incorporates the Knowledge Graph Retrieval (KGR) module and the Knowledge Graph Encoding (KGE) module to preserve semantics while utilizing structure. Then, the Knowledge Graph Adaptation (KGA) module is incorporated to enable LLMs to understand KGs embeddings. We conduct extensive experiments and provide a detailed analysis to explore how incorporating the structural information of KGs can enhance the factual reasoning abilities of LLMs. Our code are available at https://github.com/yfangZhang/SSKG-LLM.

Method: Conducted the first systematic study of explainability-fairness relationship in hate speech detection using both encoder- and decoder-only models. Examined three dimensions: identifying biased predictions, selecting fair models, and mitigating bias during training.

Result: Input-based explanations can effectively detect biased predictions and serve as useful supervision for reducing bias during training, but they are unreliable for selecting fair models among candidates.

Conclusion: While explanations show promise for bias detection and mitigation in hate speech detection, they should not be solely relied upon for model selection due to their unreliability in this context.

Abstract: Natural language processing (NLP) models often replicate or amplify social bias from training data, raising concerns about fairness. At the same time, their black-box nature makes it difficult for users to recognize biased predictions and for developers to effectively mitigate them. While some studies suggest that input-based explanations can help detect and mitigate bias, others question their reliability in ensuring fairness. Existing research on explainability in fair NLP has been predominantly qualitative, with limited large-scale quantitative analysis. In this work, we conduct the first systematic study of the relationship between explainability and fairness in hate speech detection, focusing on both encoder- and decoder-only models. We examine three key dimensions: (1) identifying biased predictions, (2) selecting fair models, and (3) mitigating bias during model training. Our findings show that input-based explanations can effectively detect biased predictions and serve as useful supervision for reducing bias during training, but they are unreliable for selecting fair models among candidates.

Method: Systematic evaluation of fine-tuned LLMs using MALLS and Willow datasets, comparing encoder-decoder vs decoder-only architectures, with techniques like vocabulary extension, predicate conditioning, and multilingual training.

Result: Fine-tuned Flan-T5-XXL achieves 70% accuracy with predicate lists, outperforming GPT-4o and DeepSeek-R1-0528, and shows 15-20% performance boost from predicate availability. Models generalize to unseen logical arguments without specific training.

Conclusion: Structural logic translation is robust, but predicate extraction remains the main bottleneck. T5 models surpass larger decoder-only LLMs, and predicate availability significantly boosts performance.

Abstract: Automating the translation of natural language to first-order logic (FOL) is crucial for knowledge representation and formal methods, yet remains challenging. We present a systematic evaluation of fine-tuned LLMs for this task, comparing architectures (encoder-decoder vs. decoder-only) and training strategies. Using the MALLS and Willow datasets, we explore techniques like vocabulary extension, predicate conditioning, and multilingual training, introducing metrics for exact match, logical equivalence, and predicate alignment. Our fine-tuned Flan-T5-XXL achieves 70% accuracy with predicate lists, outperforming GPT-4o and even the DeepSeek-R1-0528 model with CoT reasoning ability as well as symbolic systems like ccg2lambda. Key findings show: (1) predicate availability boosts performance by 15-20%, (2) T5 models surpass larger decoder-only LLMs, and (3) models generalize to unseen logical arguments (FOLIO dataset) without specific training. While structural logic translation proves robust, predicate extraction emerges as the main bottleneck.

Method: Generated directed graphs to train transformer models of varying sizes, evaluating their ability to infer transitive relations (connectivity) for different graph sizes and structures, focusing on grid-like graphs vs. graphs with disconnected components.

Result: Transformers successfully learn connectivity on low-dimensional grid graphs where nodes can be embedded in low-dimensional subspaces. Higher-dimensional grid graphs are more challenging. Model scaling improves generalization for grid graphs, but transformers struggle with graphs containing many disconnected components.

Conclusion: Transformers can learn transitive relations effectively for structured grid-like graphs but face limitations with complex graph structures containing many disconnected components, with graph dimensionality being a key predictor of learning difficulty.

Abstract: Reasoning capability is essential to ensure the factual correctness of the responses of transformer-based Large Language Models (LLMs), and robust reasoning about transitive relations is instrumental in many settings, such as causal inference. Hence, it is essential to investigate the capability of transformers in the task of inferring transitive relations (e.g., knowing A causes B and B causes C, then A causes C). The task of inferring transitive relations is equivalent to the task of connectivity in directed graphs (e.g., knowing there is a path from A to B, and there is a path from B to C, then there is a path from A to C). Past research focused on whether transformers can learn to infer transitivity from in-context examples provided in the input prompt. However, transformers’ capability to infer transitive relations from training examples and how scaling affects the ability is unexplored. In this study, we seek to answer this question by generating directed graphs to train transformer models of varying sizes and evaluate their ability to infer transitive relations for various graph sizes. Our findings suggest that transformers are capable of learning connectivity on “grid-like’’ directed graphs where each node can be embedded in a low-dimensional subspace, and connectivity is easily inferable from the embeddings of the nodes. We find that the dimensionality of the underlying grid graph is a strong predictor of transformers’ ability to learn the connectivity task, where higher-dimensional grid graphs pose a greater challenge than low-dimensional grid graphs. In addition, we observe that increasing the model scale leads to increasingly better generalization to infer connectivity over grid graphs. However, if the graph is not a grid graph and contains many disconnected components, transformers struggle to learn the connectivity task, especially when the number of components is large.

Method: Created workflow from raw data to annotation, built InviTE corpus with expert annotations, and compared fine-tuned BERT models with zero-shot prompted LLMs.

Result: Models pre-trained on historical data and fine-tuned for invective detection performed better than zero-shot LLMs.

Conclusion: Fine-tuned historical BERT models are superior to zero-shot LLMs for detecting invective language in historical texts.

Abstract: In this paper, we aim at the application of Natural Language Processing (NLP) techniques to historical research endeavors, particularly addressing the study of religious invectives in the context of the Protestant Reformation in Tudor England. We outline a workflow spanning from raw data, through pre-processing and data selection, to an iterative annotation process. As a result, we introduce the InviTE corpus – a corpus of almost 2000 Early Modern English (EModE) sentences, which are enriched with expert annotations regarding invective language throughout 16th-century England. Subsequently, we assess and compare the performance of fine-tuned BERT-based models and zero-shot prompted instruction-tuned large language models (LLMs), which highlights the superiority of models pre-trained on historical data and fine-tuned to invective detection.

Method: The study applies Bayesian framework to relevance-theoretic pragmatics by examining paradigmatic pragmatic phenomena, particularly the communication of implicit meaning through conversational implicatures.

Result: The paper demonstrates how Bayesian approaches can be extended beyond Gricean frameworks to model relevance-theoretic accounts of pragmatic communication.

Conclusion: Bayesian probability theory provides a promising computational framework for modeling relevance-theoretic pragmatics, particularly for understanding how implicit meaning is conveyed through conversational implicatures.

Abstract: Recent advances in Bayesian probability theory and its application to cognitive science in combination with the development of a new generation of computational tools and methods for probabilistic computation have led to a ‘probabilistic turn’ in pragmatics and semantics. In particular, the framework of Rational Speech Act theory has been developed to model broadly Gricean accounts of pragmatic phenomena in Bayesian terms, starting with fairly simple reference games and covering ever more complex communicative exchanges such as verbal syllogistic reasoning. This paper explores in which way a similar Bayesian approach might be applied to relevance-theoretic pragmatics (Sperber & Wilson, 1995) by study a paradigmatic pragmatic phenomenon: the communication of implicit meaning by ways of (conversational) implicatures.

Method: Curated books from Project Gutenberg with temporal annotations, used time-sensitive Valence-Arousal-Dominance analysis to quantify lexical semantic change, and constructed historically calibrated affective lexicons.

Result: Modern LLM-based tools struggle to detect discriminatory language and contextualize sentiment across different time periods, and language models trained on CHRONOBERG have difficulty encoding diachronic meaning shifts.

Conclusion: There is a need for temporally aware training and evaluation pipelines, and CHRONOBERG serves as a scalable resource for studying linguistic change and temporal generalization.

Abstract: Large language models (LLMs) excel at operating at scale by leveraging social media and various data crawled from the web. Whereas existing corpora are diverse, their frequent lack of long-term temporal structure may however limit an LLM’s ability to contextualize semantic and normative evolution of language and to capture diachronic variation. To support analysis and training for the latter, we introduce CHRONOBERG, a temporally structured corpus of English book texts spanning 250 years, curated from Project Gutenberg and enriched with a variety of temporal annotations. First, the edited nature of books enables us to quantify lexical semantic change through time-sensitive Valence-Arousal-Dominance (VAD) analysis and to construct historically calibrated affective lexicons to support temporally grounded interpretation. With the lexicons at hand, we demonstrate a need for modern LLM-based tools to better situate their detection of discriminatory language and contextualization of sentiment across various time-periods. In fact, we show how language models trained sequentially on CHRONOBERG struggle to encode diachronic shifts in meaning, emphasizing the need for temporally aware training and evaluation pipelines, and positioning CHRONOBERG as a scalable resource for the study of linguistic change and temporal generalization. Disclaimer: This paper includes language and display of samples that could be offensive to readers. Open Access: Chronoberg is available publicly on HuggingFace at ( https://huggingface.co/datasets/spaul25/Chronoberg). Code is available at (https://github.com/paulsubarna/Chronoberg).

Method: An exploratory framework using large language models (LLMs) to perform deep semantic analysis through four analytical workflows: failure mode identification, causal chain inference, comparative site analysis, and data quality auditing.

Result: LLMs successfully functioned as “reliability co-pilots,” synthesizing textual information to generate actionable, expert-level hypotheses beyond simple labeling.

Conclusion: The work provides a novel methodology for using LLMs as reasoning tools to unlock insights from unstructured data, offering a new pathway to enhance operational intelligence in the wind energy sector.

Abstract: A wealth of operational intelligence is locked within the unstructured free-text of wind turbine maintenance logs, a resource largely inaccessible to traditional quantitative reliability analysis. While machine learning has been applied to this data, existing approaches typically stop at classification, categorising text into predefined labels. This paper addresses the gap in leveraging modern large language models (LLMs) for more complex reasoning tasks. We introduce an exploratory framework that uses LLMs to move beyond classification and perform deep semantic analysis. We apply this framework to a large industrial dataset to execute four analytical workflows: failure mode identification, causal chain inference, comparative site analysis, and data quality auditing. The results demonstrate that LLMs can function as powerful “reliability co-pilots,” moving beyond labelling to synthesise textual information and generate actionable, expert-level hypotheses. This work contributes a novel and reproducible methodology for using LLMs as a reasoning tool, offering a new pathway to enhance operational intelligence in the wind energy sector by unlocking insights previously obscured in unstructured data.

Method: Analyzed OLMO2’s pre- and post-training corpora by drawing large random samples, automatically annotating documents for political orientation, and examining source domains and content, then correlating with model stance on policy issues.

Result: Left-leaning documents predominate across datasets, pre-training corpora contain more politically engaged content, left/right documents frame topics through distinct values and legitimacy sources, and training data stance strongly correlates with model political biases.

Conclusion: Political content analysis should be integrated into data curation pipelines and filtering strategies should be thoroughly documented for transparency.

Abstract: Large language models (LLMs) are known to generate politically biased text, yet how such biases arise remains unclear. A crucial step toward answering this question is the analysis of training data, whose political content remains largely underexplored in current LLM research. To address this gap, we present in this paper an analysis of the pre- and post-training corpora of OLMO2, the largest fully open-source model released together with its complete dataset. From these corpora, we draw large random samples, automatically annotate documents for political orientation, and analyze their source domains and content. We then assess how political content in the training data correlates with models’ stance on specific policy issues. Our analysis shows that left-leaning documents predominate across datasets, with pre-training corpora containing significantly more politically engaged content than post-training data. We also find that left- and right-leaning documents frame similar topics through distinct values and sources of legitimacy. Finally, the predominant stance in the training data strongly correlates with models’ political biases when evaluated on policy issues. These findings underscore the need to integrate political content analysis into future data curation pipelines as well as in-depth documentation of filtering strategies for transparency.

Method: Created Chimera test suite with 7,500 Wikipedia diagrams annotated with semantic triples and multi-level questions assessing entity recognition, relation understanding, knowledge grounding, and visual reasoning. Evaluated 15 VLMs from 7 families for three shortcut types.

Result: VLMs’ strong performance largely stems from shortcut behaviors: visual-memorization shortcuts have slight impact, knowledge-recall shortcuts play moderate role, and Clever-Hans shortcuts contribute significantly.

Conclusion: Current VLMs have critical limitations in genuine diagram comprehension and rely heavily on shortcuts, underscoring the need for more robust evaluation protocols that benchmark true understanding of complex visual inputs.

Abstract: Diagrams convey symbolic information in a visual format rather than a linear stream of words, making them especially challenging for AI models to process. While recent evaluations suggest that vision-language models (VLMs) perform well on diagram-related benchmarks, their reliance on knowledge, reasoning, or modality shortcuts raises concerns about whether they genuinely understand and reason over diagrams. To address this gap, we introduce Chimera, a comprehensive test suite comprising 7,500 high-quality diagrams sourced from Wikipedia; each diagram is annotated with its symbolic content represented by semantic triples along with multi-level questions designed to assess four fundamental aspects of diagram comprehension: entity recognition, relation understanding, knowledge grounding, and visual reasoning. We use Chimera to measure the presence of three types of shortcuts in visual question answering: (1) the visual-memorization shortcut, where VLMs rely on memorized visual patterns; (2) the knowledge-recall shortcut, where models leverage memorized factual knowledge instead of interpreting the diagram; and (3) the Clever-Hans shortcut, where models exploit superficial language patterns or priors without true comprehension. We evaluate 15 open-source VLMs from 7 model families on Chimera and find that their seemingly strong performance largely stems from shortcut behaviors: visual-memorization shortcuts have slight impact, knowledge-recall shortcuts play a moderate role, and Clever-Hans shortcuts contribute significantly. These findings expose critical limitations in current VLMs and underscore the need for more robust evaluation protocols that benchmark genuine comprehension of complex visual inputs (e.g., diagrams) rather than question-answering shortcuts.

Method: Identify a direction in model’s activation space that captures unanswerability through activation additions during inference, then use projection onto this direction for classification.

Result: Method effectively detects unanswerable questions, generalizes better across datasets than prompt-based and classifier-based approaches, and extends to other unanswerability types like lack of consensus and subjectivity.

Conclusion: Activation space directions provide a reliable way to detect unanswerability and can control model abstention behavior through causal interventions.

Abstract: Large language models (LLMs) often respond confidently to questions even when they lack the necessary information, leading to hallucinated answers. In this work, we study the problem of (un)answerability detection, focusing on extractive question answering (QA) where the model should determine if a passage contains sufficient information to answer a given question. We propose a simple approach for identifying a direction in the model’s activation space that captures unanswerability and uses it for classification. This direction is selected by applying activation additions during inference and measuring their impact on the model’s abstention behavior. We show that projecting hidden activations onto this direction yields a reliable score for (un)answerability classification. Experiments on two open-weight LLMs and four extractive QA benchmarks show that our method effectively detects unanswerable questions and generalizes better across datasets than existing prompt-based and classifier-based approaches. Moreover, the obtained directions extend beyond extractive QA to unanswerability that stems from factors, such as lack of scientific consensus and subjectivity. Last, causal interventions show that adding or ablating the directions effectively controls the abstention behavior of the model.

Method: Used multilingual legal and non-legal benchmarks with adversarial robustness testing through character/word-level perturbations. Employed LLM-as-a-Judge approach for evaluation and developed an open-source modular pipeline for multilingual benchmarking of legal tasks including classification, summarization, and reasoning.

Result: Legal tasks pose significant challenges with accuracies often below 50% on legal reasoning benchmarks vs over 70% on general tasks. English provides more stable but not always higher accuracy. Gemini outperforms LLaMA by ~24 percentage points. Performance correlates with syntactic similarity to English. Prompt sensitivity and adversarial vulnerability persist across languages.

Conclusion: Despite improvements in newer LLMs, significant challenges remain for reliable deployment in critical, multilingual legal applications, highlighting the need for continued research and development in this domain.

Abstract: In an era dominated by Large Language Models (LLMs), understanding their capabilities and limitations, especially in high-stakes fields like law, is crucial. While LLMs such as Meta’s LLaMA, OpenAI’s ChatGPT, Google’s Gemini, DeepSeek, and other emerging models are increasingly integrated into legal workflows, their performance in multilingual, jurisdictionally diverse, and adversarial contexts remains insufficiently explored. This work evaluates LLaMA and Gemini on multilingual legal and non-legal benchmarks, and assesses their adversarial robustness in legal tasks through character and word-level perturbations. We use an LLM-as-a-Judge approach for human-aligned evaluation. We moreover present an open-source, modular evaluation pipeline designed to support multilingual, task-diverse benchmarking of any combination of LLMs and datasets, with a particular focus on legal tasks, including classification, summarization, open questions, and general reasoning. Our findings confirm that legal tasks pose significant challenges for LLMs with accuracies often below 50% on legal reasoning benchmarks such as LEXam, compared to over 70% on general-purpose tasks like XNLI. In addition, while English generally yields more stable results, it does not always lead to higher accuracy. Prompt sensitivity and adversarial vulnerability is also shown to persist across languages. Finally, a correlation is found between the performance of a language and its syntactic similarity to English. We also observe that LLaMA is weaker than Gemini, with the latter showing an average advantage of about 24 percentage points across the same task. Despite improvements in newer LLMs, challenges remain in deploying them reliably for critical, multilingual legal applications.

Method: A neural-agent framework that first grounds agents in real lexical systems (e.g., English), then systematically manipulates communicative needs using color naming tasks, with different supervised and reinforcement learning pipelines.

Result: Neural agents trained to ‘speak’ an existing language can reproduce human-like patterns in color naming to a remarkable extent, simulating the evolution of lexical systems within a single generation.

Conclusion: The framework supports further use of NeLLCom-Lex to elucidate the mechanisms of semantic change, showing that agents can develop human-like naming behavior and change lexicons according to communicative needs.

Abstract: Lexical semantic change has primarily been investigated with observational and experimental methods; however, observational methods (corpus analysis, distributional semantic modeling) cannot get at causal mechanisms, and experimental paradigms with humans are hard to apply to semantic change due to the extended diachronic processes involved. This work introduces NeLLCom-Lex, a neural-agent framework designed to simulate semantic change by first grounding agents in a real lexical system (e.g. English) and then systematically manipulating their communicative needs. Using a well-established color naming task, we simulate the evolution of a lexical system within a single generation, and study which factors lead agents to: (i) develop human-like naming behavior and lexicons, and (ii) change their behavior and lexicons according to their communicative needs. Our experiments with different supervised and reinforcement learning pipelines show that neural agents trained to ‘speak’ an existing language can reproduce human-like patterns in color naming to a remarkable extent, supporting the further use of NeLLCom-Lex to elucidate the mechanisms of semantic change.

Method: Propose solution divergence as a novel metric that supports both SFT and RL strategies, tested across three representative problem domains.

Result: Using solution divergence consistently improves success rates in problem-solving tasks across various models.

Conclusion: Solution divergence is a simple but effective tool for advancing LLM training and evaluation.

Abstract: Large language models (LLMs) have been widely used for problem-solving tasks. Most recent work improves their performance through supervised fine-tuning (SFT) with labeled data or reinforcement learning (RL) from task feedback. In this paper, we study a new perspective: the divergence in solutions generated by LLMs for a single problem. We show that higher solution divergence is positively related to better problem-solving abilities across various models. Based on this finding, we propose solution divergence as a novel metric that can support both SFT and RL strategies. We test this idea on three representative problem domains and find that using solution divergence consistently improves success rates. These results suggest that solution divergence is a simple but effective tool for advancing LLM training and evaluation.

Method: Jointly fine-tuned Qwen2.5-3B-Instruct models using parameter-efficient finetuning for both tasks, integrated additional translation and QA data, used retrieval-augmented generation for Ukrainian QA, and applied ensembling for Upper and Lower Sorbian QA.

Result: The developed models outperformed the baseline on both machine translation and question answering tasks across all three Slavic languages.

Conclusion: The approach demonstrates that parameter-efficient finetuning combined with task-specific enhancements like retrieval-augmented generation and ensembling can effectively improve performance for low-resource Slavic languages in both translation and QA tasks.

Abstract: This paper presents the JGU Mainz submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: Machine Translation and Question Answering, focusing on Ukrainian, Upper Sorbian, and Lower Sorbian. For each language, we jointly fine-tune a Qwen2.5-3B-Instruct model for both tasks with parameter-efficient finetuning. Our pipeline integrates additional translation and multiple-choice question answering (QA) data. For Ukrainian QA, we further use retrieval-augmented generation. We also apply ensembling for QA in Upper and Lower Sorbian. Experiments show that our models outperform the baseline on both tasks.

Method: Uses closed-form computation of geometrical properties to represent LLMs as linear operators within prompts’ semantic task space, providing an efficient training-free approach.

Result: Achieves competitive or state-of-the-art results on success prediction and model selection tasks, with notable performance in out-of-sample scenarios.

Conclusion: The proposed method offers highly interpretable LLM representations with exceptional scalability and real-time adaptability to dynamically expanding model repositories.

Abstract: Large language models (LLMs) achieve impressive results over various tasks, and ever-expanding public repositories contain an abundance of pre-trained models. Therefore, identifying the best-performing LLM for a given task is a significant challenge. Previous works have suggested learning LLM representations to address this. However, these approaches present limited scalability and require costly retraining to encompass additional models and datasets. Moreover, the produced representation utilizes distinct spaces that cannot be easily interpreted. This work presents an efficient, training-free approach to representing LLMs as linear operators within the prompts’ semantic task space, thus providing a highly interpretable representation of the models’ application. Our method utilizes closed-form computation of geometrical properties and ensures exceptional scalability and real-time adaptability to dynamically expanding repositories. We demonstrate our approach on success prediction and model selection tasks, achieving competitive or state-of-the-art results with notable performance in out-of-sample scenarios.

Method: Two-stage framework: Stage I computes shared post-attention hidden states; Stage II clones these into parallel branches with policy-reference steering for objective-specific control while maintaining consistency.

Result: On DeepSeek-7B, AMBS improves average HHH alignment scores by +32.4%, reduces unsafe outputs by 11.0% vs naive 1-to-N baseline, and remains competitive with SOTA methods.

Conclusion: AMBS provides an effective approach for unified multi-objective LLM alignment that maintains consistency across objectives while improving performance.

Abstract: Alignment of Large Language Models (LLMs) along multiple objectives-helpfulness, harmlessness, and honesty (HHH)-is critical for safe and reliable deployment. Prior work has used steering vector-small control signals injected into hidden states-to guide LLM outputs, typically via one-to-one (1-to-1) Transformer decoders. In this setting, optimizing a single alignment objective can inadvertently overwrite representations learned for other objectives, leading to catastrophic forgetting. More recent approaches extend steering vectors via one-to-many (1-to-N) Transformer decoders. While this alleviates catastrophic forgetting, naive multi-branch designs optimize each objective independently, which can cause inference fragmentation-outputs across HHH objectives may become inconsistent. We propose Adaptive Multi-Branch Steering (AMBS), a two-stage 1-to-N framework for unified and efficient multi-objective alignment. In Stage I, post-attention hidden states of the Transformer layer are computed once to form a shared representation. In Stage II, this representation is cloned into parallel branches and steered via a policy-reference mechanism, enabling objective-specific control while maintaining cross-objective consistency. Empirical evaluations on Alpaca, BeaverTails, and TruthfulQA show that AMBS consistently improves HHH alignment across multiple 7B LLM backbones. For example, on DeepSeek-7B, AMBS improves average alignment scores by +32.4% and reduces unsafe outputs by 11.0% compared to a naive 1-to-N baseline, while remaining competitive with state-of-the-art methods.

Method: Fine-grained, hybrid-granularity quantization strategy for FP8 training that seamlessly integrates continual pre-training and supervised fine-tuning while maintaining numerical fidelity.

Result: The FP8 recipe is remarkably stable and essentially lossless, achieving performance on par with BF16 baseline across reasoning benchmarks, with 22% reduction in training time, 14% decrease in peak memory usage, and 19% increase in throughput.

Conclusion: FP8 is established as a practical and robust alternative to BF16 for large-scale model training, with code release to democratize access.

Abstract: The immense computational cost of training Large Language Models (LLMs) presents a major barrier to innovation. While FP8 training offers a promising solution with significant theoretical efficiency gains, its widespread adoption has been hindered by the lack of a comprehensive, open-source training recipe. To bridge this gap, we introduce an end-to-end FP8 training recipe that seamlessly integrates continual pre-training and supervised fine-tuning. Our methodology employs a fine-grained, hybrid-granularity quantization strategy to maintain numerical fidelity while maximizing computational efficiency. Through extensive experiments, including the continue pre-training of models on a 160B-token corpus, we demonstrate that our recipe is not only remarkably stable but also essentially lossless, achieving performance on par with the BF16 baseline across a suite of reasoning benchmarks. Crucially, this is achieved with substantial efficiency improvements, including up to a 22% reduction in training time, a 14% decrease in peak memory usage, and a 19% increase in throughput. Our results establish FP8 as a practical and robust alternative to BF16, and we will release the accompanying code to further democratize large-scale model training.

Method: Proposes CogFlow framework: curates cognitive flow dataset via tree-structured planning, uses supervised fine-tuning for basic capability, then reinforcement learning with multi-objective rewards for self-improvement.

Result: Extensive experiments show CogFlow effectively enhances social cognitive capabilities of LLMs and even humans, leading to more effective social decision-making.

Conclusion: Cognitive Reasoning paradigm and CogFlow framework successfully bridge the gap between logical reasoning and social cognition in AI systems.

Abstract: LLMs trained for logical reasoning excel at step-by-step deduction to reach verifiable answers. However, this paradigm is ill-suited for navigating social situations, which induce an interpretive process of analyzing ambiguous cues that rarely yield a definitive outcome. To bridge this gap, we introduce Cognitive Reasoning, a paradigm modeled on human social cognition. It formulates the interpretive process into a structured cognitive flow of interconnected cognitive units (e.g., observation or attribution), which combine adaptively to enable effective social thinking and responses. We then propose CogFlow, a complete framework that instills this capability in LLMs. CogFlow first curates a dataset of cognitive flows by simulating the associative and progressive nature of human thought via tree-structured planning. After instilling the basic cognitive reasoning capability via supervised fine-tuning, CogFlow adopts reinforcement learning to enable the model to improve itself via trial and error, guided by a multi-objective reward that optimizes both cognitive flow and response quality. Extensive experiments show that CogFlow effectively enhances the social cognitive capabilities of LLMs, and even humans, leading to more effective social decision-making.

Method: Created a clinically grounded error ontology (5 domains, 59 codes), developed RAEC pipeline using retrieval of similar historical message-response pairs, and implemented two-stage DSPy prompting for hierarchical error detection.

Result: Retrieval context improved error identification in clinical completeness and workflow appropriateness. Human validation showed superior agreement (50% vs 33%) and performance (F1=0.500 vs 0.256) compared to baseline.

Conclusion: The RAEC pipeline serves as effective AI guardrails for patient messaging, demonstrating that context-enhanced evaluation significantly improves error detection in LLM-generated clinical responses.

Abstract: Asynchronous patient-clinician messaging via EHR portals is a growing source of clinician workload, prompting interest in large language models (LLMs) to assist with draft responses. However, LLM outputs may contain clinical inaccuracies, omissions, or tone mismatches, making robust evaluation essential. Our contributions are threefold: (1) we introduce a clinically grounded error ontology comprising 5 domains and 59 granular error codes, developed through inductive coding and expert adjudication; (2) we develop a retrieval-augmented evaluation pipeline (RAEC) that leverages semantically similar historical message-response pairs to improve judgment quality; and (3) we provide a two-stage prompting architecture using DSPy to enable scalable, interpretable, and hierarchical error detection. Our approach assesses the quality of drafts both in isolation and with reference to similar past message-response pairs retrieved from institutional archives. Using a two-stage DSPy pipeline, we compared baseline and reference-enhanced evaluations on over 1,500 patient messages. Retrieval context improved error identification in domains such as clinical completeness and workflow appropriateness. Human validation on 100 messages demonstrated superior agreement (concordance = 50% vs. 33%) and performance (F1 = 0.500 vs. 0.256) of context-enhanced labels vs. baseline, supporting the use of our RAEC pipeline as AI guardrails for patient messaging.

Method: Constructed a benchmark with 1,000+ human-annotated examples, proposed free-form textual descriptions for error representation, and evaluated four large-scale LLMs using optimal prompting strategies.

Result: The benchmark proved challenging - best model achieved only 0.67 F1 score. Key challenges identified: LLMs incorrectly flag missing details as inconsistent and struggle with factually correct outputs that align with parametric knowledge but aren’t verifiable from source.

Conclusion: LLMs show promise for hallucination localization but face significant challenges, particularly with distinguishing between actual inconsistencies and missing details, and handling outputs that align with their parametric knowledge but lack source verification.

Abstract: Context-grounded hallucinations are cases where model outputs contain information not verifiable against the source text. We study the applicability of LLMs for localizing such hallucinations, as a more practical alternative to existing complex evaluation pipelines. In the absence of established benchmarks for meta-evaluation of hallucinations localization, we construct one tailored to LLMs, involving a challenging human annotation of over 1,000 examples. We complement the benchmark with an LLM-based evaluation protocol, verifying its quality in a human evaluation. Since existing representations of hallucinations limit the types of errors that can be expressed, we propose a new representation based on free-form textual descriptions, capturing the full range of possible errors. We conduct a comprehensive study, evaluating four large-scale LLMs, which highlights the benchmark’s difficulty, as the best model achieves an F1 score of only 0.67. Through careful analysis, we offer insights into optimal prompting strategies for the task and identify the main factors that make it challenging for LLMs: (1) a tendency to incorrectly flag missing details as inconsistent, despite being instructed to check only facts in the output; and (2) difficulty with outputs containing factually correct information absent from the source - and thus not verifiable - due to alignment with the model’s parametric knowledge.

Method: Collected over 8,500 job advertisements from Egypt, Jordan, Saudi Arabia, and UAE, comprising 550,000+ words, and applied analyses on gender representation, occupational structure, and dialectal variation.

Result: The dataset enables analyses of gender bias, profession classification, salary estimation, and job category normalization using large language models, demonstrating utility for labor market research.

Conclusion: ArabJobs provides valuable resources for fairness-aware Arabic NLP and labor market studies, with the dataset publicly available on GitHub for future research.

Abstract: ArabJobs is a publicly available corpus of Arabic job advertisements collected from Egypt, Jordan, Saudi Arabia, and the United Arab Emirates. Comprising over 8,500 postings and more than 550,000 words, the dataset captures linguistic, regional, and socio-economic variation in the Arab labour market. We present analyses of gender representation and occupational structure, and highlight dialectal variation across ads, which offers opportunities for future research. We also demonstrate applications such as salary estimation and job category normalisation using large language models, alongside benchmark tasks for gender bias detection and profession classification. The findings show the utility of ArabJobs for fairness-aware Arabic NLP and labour market research. The dataset is publicly available on GitHub: https://github.com/drelhaj/ArabJobs.

Method: Representing subregular language classes by their deciding predicates and using linear models for learning, with experiments on synthetic data and English morphology.

Result: Perfect separability under noise-free conditions in synthetic experiments, and learned features aligning with well-known linguistic constraints in real-data experiments on English morphology.

Conclusion: The subregular hierarchy provides a rigorous and interpretable foundation for modeling natural language structure, with demonstrated linear separability and learnability.

Abstract: We prove that all standard subregular language classes are linearly separable when represented by their deciding predicates. This establishes finite observability and guarantees learnability with simple linear models. Synthetic experiments confirm perfect separability under noise-free conditions, while real-data experiments on English morphology show that learned features align with well-known linguistic constraints. These results demonstrate that the subregular hierarchy provides a rigorous and interpretable foundation for modeling natural language structure. Our code used in real-data experiments is available at https://github.com/UTokyo-HayashiLab/subregular.

Method: Collected opinions from diverse individuals to create a self-sourced dataset, simulated deliberation using product presentations with striking facts, and compared multiple NLP techniques including Frequency-Based Discourse Modulation and Quantum-Deliberation Framework.

Result: The Frequency-Based Discourse Modulation and Quantum-Deliberation Framework models outperformed existing state-of-the-art models in interpreting deliberative discourse and producing meaningful insights about opinion shifts.

Conclusion: The study demonstrates practical applications in public policy-making, debate evaluation, decision-support frameworks, and large-scale social media opinion mining, highlighting the effectiveness of advanced NLP techniques for deliberation modeling.

Abstract: Deliberation plays a crucial role in shaping outcomes by weighing diverse perspectives before reaching decisions. With recent advancements in Natural Language Processing, it has become possible to computationally model deliberation by analyzing opinion shifts and predicting potential outcomes under varying scenarios. In this study, we present a comparative analysis of multiple NLP techniques to evaluate how effectively models interpret deliberative discourse and produce meaningful insights. Opinions from individuals of varied backgrounds were collected to construct a self-sourced dataset that reflects diverse viewpoints. Deliberation was simulated using product presentations enriched with striking facts, which often prompted measurable shifts in audience opinions. We have given comparative analysis between two models namely Frequency-Based Discourse Modulation and Quantum-Deliberation Framework which outperform the existing state of art models. The findings highlight practical applications in public policy-making, debate evaluation, decision-support frameworks, and large-scale social media opinion mining.

Method: Resampling-based methods that account for both model- and data-related sources of variation in performance scores.

Result: Shows that accounting for both sources of variation is necessary to avoid substantially underestimating overall variability; demonstrates utility for computing sampling distributions for leaderboard quantities like averages, pairwise differences, and rankings.

Conclusion: Resampling methods are effective for comprehensive uncertainty quantification in NLP benchmarks, providing better statistical precision estimates for common leaderboard metrics.

Abstract: In this paper, we introduce a set of resampling-based methods for quantifying uncertainty and statistical precision of evaluation metrics in multilingual and/or multitask NLP benchmarks. We show how experimental variation in performance scores arises from both model- and data-related sources, and that accounting for both of them is necessary to avoid substantially underestimating the overall variability over hypothetical replications. Using multilingual question answering, machine translation, and named entity recognition as example tasks, we also demonstrate how resampling methods are useful for computing sampling distributions for various quantities used in leaderboards such as the average/median, pairwise differences between models, and rankings.

Method: Design post-training architectural modifications for linear attention and state space models to scale up state size with minimal parameter increase, using a training pipeline called StateX.

Result: Experiments on models up to 1.3B parameters show StateX efficiently enhances recall and in-context learning abilities without high post-training costs or compromising other capabilities.

Conclusion: StateX provides an effective solution for improving RNNs’ long-context recall abilities through efficient post-training state expansion, bridging the gap between RNN efficiency and Transformer-like recall performance.

Abstract: While Transformer-based models have demonstrated remarkable language modeling performance, their high complexities result in high costs when processing long contexts. In contrast, recurrent neural networks (RNNs) such as linear attention and state space models have gained popularity due to their constant per-token complexities. However, these recurrent models struggle with tasks that require accurate recall of contextual information from long contexts, because all contextual information is compressed into a constant-size recurrent state. Previous works have shown that recall ability is positively correlated with the recurrent state size, yet directly training RNNs with larger recurrent states results in high training costs. In this paper, we introduce StateX, a training pipeline for efficiently expanding the states of pre-trained RNNs through post-training. For two popular classes of RNNs, linear attention and state space models, we design post-training architectural modifications to scale up the state size with no or negligible increase in model parameters. Experiments on models up to 1.3B parameters demonstrate that StateX efficiently enhances the recall and in-context learning ability of RNNs without incurring high post-training costs or compromising other capabilities.

Method: Proposes a variational reasoning framework using thinking traces as latent variables, extends ELBO to multi-trace objectives, introduces forward-KL formulation for training stability, and shows connections to rejection sampling and binary-reward RL.

Result: Empirically validated on Qwen 2.5 and Qwen 3 model families across various reasoning tasks, demonstrating stable objectives and improved reasoning capabilities.

Conclusion: The work provides a unified probabilistic framework that connects variational inference with RL methods, revealing previously unnoticed biases and offering stable training objectives for enhancing language model reasoning.

Abstract: We introduce a variational reasoning framework for language models that treats thinking traces as latent variables and optimizes them through variational inference. Starting from the evidence lower bound (ELBO), we extend it to a multi-trace objective for tighter bounds and propose a forward-KL formulation that stabilizes the training of the variational posterior. We further show that rejection sampling finetuning and binary-reward RL, including GRPO, can be interpreted as local forward-KL objectives, where an implicit weighting by model accuracy naturally arises from the derivation and reveals a previously unnoticed bias toward easier questions. We empirically validate our method on the Qwen 2.5 and Qwen 3 model families across a wide range of reasoning tasks. Overall, our work provides a principled probabilistic perspective that unifies variational inference with RL-style methods and yields stable objectives for improving the reasoning ability of language models. Our code is available at https://github.com/sail-sg/variational-reasoning.

Method: Feedback-conditional policy (FCP) learns from response-feedback pairs via maximum likelihood training on offline data, plus online bootstrapping where policy generates under positive conditions and refines with fresh feedback.

Result: FCP reframes feedback-driven learning as conditional generation rather than reward optimization, providing a more expressive approach for LLMs to learn directly from verbal feedback.

Conclusion: The proposed method offers a novel paradigm for LLM training that preserves the richness of verbal feedback through conditional generation techniques.

Abstract: LLMs are often trained with RL from human or AI feedback, yet such methods typically compress nuanced feedback into scalar rewards, discarding much of their richness and inducing scale imbalance. We propose treating verbal feedback as a conditioning signal. Inspired by language priors in text-to-image generation, which enable novel outputs from unseen prompts, we introduce the feedback-conditional policy (FCP). FCP learns directly from response-feedback pairs, approximating the feedback-conditional posterior through maximum likelihood training on offline data. We further develop an online bootstrapping stage where the policy generates under positive conditions and receives fresh feedback to refine itself. This reframes feedback-driven learning as conditional generation rather than reward optimization, offering a more expressive way for LLMs to directly learn from verbal feedback. Our code is available at https://github.com/sail-sg/feedback-conditional-policy.

Method: Used 7542 expert writer annotations of novelty, pragmaticality, and sensicality via close reading of human and AI-generated text. Tested zero-shot, few-shot, and finetuned models on creative expression identification.

Result: 91% of top-quartile n-gram novelty expressions weren’t judged creative. Higher n-gram novelty in LLMs correlates with lower pragmaticality. Frontier LLMs perform better than random but struggle with non-pragmatic expressions.

Conclusion: N-gram novelty alone is inadequate for measuring creativity. LLMs need improvement in identifying creative and non-pragmatic expressions, though LLM-as-a-Judge novelty scores show promise for predicting expert preferences.

Abstract: N-gram novelty is widely used to evaluate language models’ ability to generate text outside of their training data. More recently, it has also been adopted as a metric for measuring textual creativity. However, theoretical work on creativity suggests that this approach may be inadequate, as it does not account for creativity’s dual nature: novelty (how original the text is) and appropriateness (how sensical and pragmatic it is). We investigate the relationship between this notion of creativity and n-gram novelty through 7542 expert writer annotations (n=26) of novelty, pragmaticality, and sensicality via close reading of human and AI-generated text. We find that while n-gram novelty is positively associated with expert writer-judged creativity, ~91% of top-quartile expressions by n-gram novelty are not judged as creative, cautioning against relying on n-gram novelty alone. Furthermore, unlike human-written text, higher n-gram novelty in open-source LLMs correlates with lower pragmaticality. In an exploratory study with frontier close-source models, we additionally confirm that they are less likely to produce creative expressions than humans. Using our dataset, we test whether zero-shot, few-shot, and finetuned models are able to identify creative expressions (a positive aspect of writing) and non-pragmatic ones (a negative aspect). Overall, frontier LLMs exhibit performance much higher than random but leave room for improvement, especially struggling to identify non-pragmatic expressions. We further find that LLM-as-a-Judge novelty scores from the best-performing model were predictive of expert writer preferences.

Method: Uses visual language model (VLM) to generate detailed descriptions and suggestions from screenshots and GUI-agent testing, with backtracking and select-best mechanism. Also introduces Step-GRPO training using screenshot and GUI-agent scores as dense process supervision.

Result: On WebGen-Bench dataset, increased Claude-3.5-Sonnet accuracy from 26.4% to 51.9% and appearance score from 3.0 to 3.9. Step-GRPO training increased Qwen2.5-Coder-7B-Instruct accuracy from 38.9% to 45.4% and appearance score from 3.4 to 3.7.

Conclusion: Comprehensive visual feedback and Step-GRPO training significantly improve website-generation performance, demonstrating the importance of visual evaluation for code quality assessment in web development tasks.

Abstract: Agent systems powered by large language models (LLMs) have demonstrated impressive performance on repository-level code-generation tasks. However, for tasks such as website codebase generation, which depend heavily on visual effects and user-interaction feedback, current code agents rely only on simple code execution for feedback and verification. This approach fails to capture the actual quality of the generated code. In this paper, we propose WebGen-Agent, a novel website-generation agent that leverages comprehensive and multi-level visual feedback to iteratively generate and refine the website codebase. Detailed and expressive text descriptions and suggestions regarding the screenshots and GUI-agent testing of the websites are generated by a visual language model (VLM), together with scores that quantify their quality. The screenshot and GUI-agent scores are further integrated with a backtracking and select-best mechanism, enhancing the performance of the agent. Utilizing the accurate visual scores inherent in the WebGen-Agent workflow, we further introduce \textit{Step-GRPO with Screenshot and GUI-agent Feedback} to improve the ability of LLMs to act as the reasoning engine of WebGen-Agent. By using the screenshot and GUI-agent scores at each step as the reward in Step-GRPO, we provide a dense and reliable process supervision signal, which effectively improves the model’s website-generation ability. On the WebGen-Bench dataset, WebGen-Agent increases the accuracy of Claude-3.5-Sonnet from 26.4% to 51.9% and its appearance score from 3.0 to 3.9, outperforming the previous state-of-the-art agent system. Additionally, our Step-GRPO training approach increases the accuracy of Qwen2.5-Coder-7B-Instruct from 38.9% to 45.4% and raises the appearance score from 3.4 to 3.7.

Method: Reformat constituency parsing as seq2seq generation; evaluate LLMs under zero-shot, few-shot, and fine-tuning paradigms; propose two strategies: learning from errors and multi-agent output refinement.

Result: LLMs achieve acceptable improvements but have substantial limitations; proposed methods effectively reduce invalid/unfaithful trees and enhance parsing performance across learning paradigms.

Conclusion: The proposed error-learning and multi-agent collaboration strategies successfully improve constituency parsing by ensuring more valid and faithful constituent tree generation.

Abstract: Constituency parsing is a fundamental yet unsolved challenge in natural language processing. In this paper, we examine the potential of recent large language models (LLMs) to address this challenge. We reformat constituency parsing as a sequence-to-sequence generation problem and evaluate the performance of a diverse range of LLMs under zero-shot, few-shot, and supervised fine-tuning learning paradigms. We observe that while LLMs achieve acceptable improvements, they still encounter substantial limitations, due to the absence of mechanisms to guarantee the validity and faithfulness of the generated constituent trees. Motivated by this observation, we propose two strategies to guide LLMs to generate more accurate constituent trees by learning from erroneous samples and refining outputs in a multi-agent collaboration way, respectively. The experimental results demonstrate that our methods effectively reduce the occurrence of invalid and unfaithful trees, thereby enhancing overall parsing performance and achieving promising results across different learning paradigms.

Method: Curated a novel Text2Afford dataset with in-the-wild sentences annotated with objects and affordances, then evaluated PTLMs and VLMs on affordance reasoning tasks and conducted few-shot fine-tuning experiments.

Result: PTLMs show limited reasoning for uncommon object affordances, pre-trained VLMs don’t effectively capture object affordances, but few-shot fine-tuning improves affordance knowledge in both model types.

Conclusion: The research contributes a novel dataset for language grounding tasks and provides insights into LM capabilities regarding object affordances, advancing understanding of this important aspect of grounded reasoning.

Abstract: We investigate the knowledge of object affordances in pre-trained language models (LMs) and pre-trained Vision-Language models (VLMs). A growing body of literature shows that PTLMs fail inconsistently and non-intuitively, demonstrating a lack of reasoning and grounding. To take a first step toward quantifying the effect of grounding (or lack thereof), we curate a novel and comprehensive dataset of object affordances – Text2Afford, characterized by 15 affordance classes. Unlike affordance datasets collected in vision and language domains, we annotate in-the-wild sentences with objects and affordances. Experimental results reveal that PTLMs exhibit limited reasoning abilities when it comes to uncommon object affordances. We also observe that pre-trained VLMs do not necessarily capture object affordances effectively. Through few-shot fine-tuning, we demonstrate improvement in affordance knowledge in PTLMs and VLMs. Our research contributes a novel dataset for language grounding tasks, and presents insights into LM capabilities, advancing the understanding of object affordances. Codes and data are available at https://github.com/sayantan11995/Text2Afford

Method: Classified neurons into four categories (all-shared, partial-shared, specific, non-activated) based on their responses across languages. Conducted experiments on three tasks across nine languages using several LLMs.

Result: Deactivating all-shared neurons significantly decreases performance; shared neurons play vital role in response generation; neuron activation patterns are highly sensitive and vary across tasks, LLMs, and languages.

Conclusion: The findings provide insights into internal workings of multilingual LLMs and pave way for future research. Code is released to foster further investigation.

Abstract: Large language models (LLMs) have revolutionized the field of natural language processing (NLP), and recent studies have aimed to understand their underlying mechanisms. However, most of this research is conducted within a monolingual setting, primarily focusing on English. Few studies have attempted to explore the internal workings of LLMs in multilingual settings. In this study, we aim to fill this research gap by examining how neuron activation is shared across tasks and languages. We classify neurons into four distinct categories based on their responses to a specific input across different languages: all-shared, partial-shared, specific, and non-activated. Building upon this categorisation, we conduct extensive experiments on three tasks across nine languages using several LLMs and present an in-depth analysis in this work. Our findings reveal that: (i) deactivating the all-shared neurons significantly decreases performance; (ii) the shared neurons play a vital role in generating responses, especially for the all-shared neurons; (iii) neuron activation patterns are highly sensitive and vary across tasks, LLMs, and languages. These findings shed light on the internal workings of multilingual LLMs and pave the way for future research. We release the code to foster research in this area.

Method: LLMAEL uses off-the-shelf, tuning-free LLMs as context augmenters to generate entity descriptions that serve as additional input for specialized entity linking models.

Result: Sets new state-of-the-art results across 6 EL benchmarks, achieving an absolute 8.9% gain in EL accuracy compared to prior LLM integration methods.

Conclusion: The framework successfully leverages LLMs’ context generation capabilities to enhance specialized EL models, demonstrating significant performance improvements without requiring LLM fine-tuning.

Abstract: Specialized entity linking (EL) models are well-trained at mapping mentions to unique knowledge base (KB) entities according to a given context. However, specialized EL models struggle to disambiguate long-tail entities due to their limited training data. Meanwhile, extensively pre-trained large language models (LLMs) possess broader knowledge of uncommon entities. Yet, with a lack of specialized EL training, LLMs frequently fail to generate accurate KB entity names, limiting their standalone effectiveness in EL. With the observation that LLMs are more adept at context generation instead of EL execution, we introduce LLM-Augmented Entity Linking (LLMAEL), the first framework to enhance specialized EL models with LLM data augmentation. LLMAEL leverages off-the-shelf, tuning-free LLMs as context augmenters, generating entity descriptions to serve as additional input for specialized EL models. Experiments show that LLMAEL sets new state-of-the-art results across 6 widely adopted EL benchmarks: compared to prior methods that integrate tuning-free LLMs into EL, LLMAEL achieves an absolute 8.9% gain in EL accuracy. We release our code and datasets.

Method: Enhanced Position Layout (EPL) adjusts position IDs to minimize distance between context tokens and special tokens while maintaining sequence order between all tokens.

Result: EPL improves ROUGE-1 F1 by 1.9 points on out-of-domain QA datasets and boosts vision compression LLM accuracy by 2.6 points in multimodal scenarios.

Conclusion: Simple adjustments to position IDs can significantly enhance context compression capabilities in LLMs across both text and multimodal domains.

Abstract: Using special tokens (e.g., gist, memory, or compressed tokens) to compress context information is a common practice for large language models (LLMs). However, existing approaches often neglect that position encodings inherently induce local inductive biases in models, causing the compression process to ignore holistic contextual dependencies. We propose \textbf{Enhanced Position Layout (EPL)}, a simple yet effective method that improves the context compression capability of LLMs by only adjusting position IDs, the numerical identifiers that specify token positions. EPL minimizes the distance between context tokens and their corresponding special tokens and at the same time maintains the sequence order in position IDs between context tokens, special tokens, and the subsequent tokens. Integrating EPL into our best performing context compression model results in a 1.9 ROUGE-1 F1 improvement on out-of-domain question answering datasets on average. When extended to multimodal scenarios, EPL leads to an average accuracy gain of 2.6 points for vision compression LLMs.

Method: Analyzed Mamba-based models’ forgetting capabilities, investigated the relationship between training context length and state size, and measured how minimum training length and maximum context length for accurate retrieval scale with state size.

Result: Models fail to learn effective forgetting when trained on contexts shorter than state size. Minimum training length required for learning forgetting scales linearly with state size, while maximum context length for accurate passkey retrieval scales exponentially with state size.

Conclusion: Current RNN architectures have critical limitations in long-context modeling. Future designs must account for interplay between state size, training length, and forgetting mechanisms to achieve robust performance in long-context tasks.

Abstract: Recent advancements in recurrent architectures, such as Mamba and RWKV, have showcased strong language capabilities. Unlike transformer-based models, these architectures encode all contextual information into a fixed-size state, leading to great inference efficiency. However, this approach can cause information interference, where different token data conflicts, resulting in performance degradation and incoherent outputs beyond a certain context length. To prevent this, most RNNs incorporate mechanisms designed to “forget” earlier tokens. In this paper, we reveal that Mamba-based models struggle to effectively forget earlier tokens even with built-in forgetting mechanisms. We demonstrate that this issue stems from training on contexts that are too short for the state size, enabling the model to perform well without needing to learn how to forget. Then, we show that the minimum training length required for the model to learn forgetting scales linearly with the state size, and the maximum context length for accurate retrieval of a 5-digit passkey scales exponentially with the state size, indicating that the model retains some information beyond the point where forgetting begins. These findings highlight a critical limitation in current RNN architectures and provide valuable insights for improving long-context modeling. Our work suggests that future RNN designs must account for the interplay between state size, training length, and forgetting mechanisms to achieve robust performance in long-context tasks.

Method: Frame NLU as a reinforcement learning environment where token generation is treated as actions, and use Proximal Policy Optimization (PPO) to optimize for reward signals based on alignment with ground-truth labels.

Result: PPO consistently outperforms supervised fine-tuning with 6.3 point average improvement on GLUE, surpasses zero-shot and few-shot prompting by 38.7 and 26.1 points respectively, and beats GPT-4o by over 4% on sentiment and NLI tasks.

Conclusion: Reframing NLU tasks as reinforcement learning problems enables effective adaptation of LLMs using simple end-task rewards rather than extensive data curation, showing promising direction for task adaptation.

Abstract: Instruction-fine-tuned large language models (LLMs) under 14B parameters continue to underperform on natural language understanding (NLU) tasks, often trailing smaller models like BERT-base on benchmarks such as GLUE and SuperGLUE. Motivated by the success of reinforcement learning in reasoning tasks (e.g., DeepSeek), we explore Proximal Policy Optimization (PPO) as a framework to improve the NLU capabilities of LLMs. We frame NLU as a reinforcement learning environment, treating token generation as a sequence of actions and optimizing for reward signals based on alignment with ground-truth labels. PPO consistently outperforms supervised fine-tuning, yielding an average improvement of 6.3 points on GLUE, and surpasses zero-shot and few-shot prompting by 38.7 and 26.1 points, respectively. Notably, PPO-tuned models outperform GPT-4o by over 4% on average across sentiment and natural language inference tasks, including gains of 7.3% on the Mental Health dataset and 10.9% on SIGA-nli. This work highlights a promising direction for adapting LLMs to new tasks by reframing them as reinforcement learning problems, enabling learning through simple end-task rewards rather than extensive data curation.

Method: Analyzed impact of vertical text input on various LLMs across multiple text classification datasets, examined tokenization and attention matrices, and tested mitigation strategies like CoT and few-shot learning.

Result: Vertical text input significantly reduces LLM accuracy in text classification; CoT doesn’t help but few-shot learning with careful analysis does; vulnerability stems from tokenization and attention matrix issues.

Conclusion: LLMs are vulnerable to vertical text formatting, posing risks in real-world applications, and require specific mitigation strategies beyond standard reasoning approaches.

Abstract: Vertical text input is commonly encountered in various real-world applications, such as mathematical computations and word-based Sudoku puzzles. While current large language models (LLMs) have excelled in natural language tasks, they remain vulnerable to variations in text formatting. Recent research demonstrates that modifying input formats, such as vertically aligning words for encoder-based models, can substantially lower accuracy in text classification tasks. While easily understood by humans, these inputs can significantly mislead models, posing a potential risk of bypassing detection in real-world scenarios involving harmful or sensitive information. With the expanding application of LLMs, a crucial question arises: Do decoder-based LLMs exhibit similar vulnerabilities to vertically formatted text input? In this paper, we investigate the impact of vertical text input on the performance of various LLMs across multiple text classification datasets and analyze the underlying causes. Our findings are as follows: (i) Vertical text input significantly degrades the accuracy of LLMs in text classification tasks. (ii) Chain-of-Thought (CoT) reasoning does not help LLMs recognize vertical input or mitigate its vulnerability, but few-shot learning with careful analysis does. (iii) We explore the underlying cause of the vulnerability by analyzing the inherent issues in tokenization and attention matrices.

Method: SCA introduces a decomposition step to the clustering-based topic modeling framework, allowing it to discover multiple topics per sample by breaking down documents into semantic components.

Result: SCA achieves competitive coherence and diversity compared to BERTopic while uncovering at least double the topics and maintaining near-zero noise rate. It also outperforms LLM-based TopicGPT with similar compute budgets.

Conclusion: SCA provides an effective and efficient approach for topic modeling of large datasets, overcoming key limitations of existing methods.

Abstract: Topic modeling is a key method in text analysis, but existing approaches fail to efficiently scale to large datasets or are limited by assuming one topic per document. Overcoming these limitations, we introduce Semantic Component Analysis (SCA), a topic modeling technique that discovers multiple topics per sample by introducing a decomposition step to the clustering-based topic modeling framework. We evaluate SCA on Twitter datasets in English, Hausa and Chinese. There, it achieves competitive coherence and diversity compared to BERTopic, while uncovering at least double the topics and maintaining a noise rate close to zero. We also find that SCA outperforms the LLM-based TopicGPT in scenarios with similar compute budgets. SCA thus provides an effective and efficient approach for topic modeling of large datasets.

Method: Surveyed academic compute resources, created a benchmark to measure pre-training time on various GPUs, identified optimal training settings, and conducted experiments using 2,000 GPU-hours across different models and academic GPUs.

Result: Found that models like Pythia-1B can be replicated with 3x fewer GPU-days (4 GPUs in 18 days vs original 64 GPUs in 3 days) using optimized configurations.

Conclusion: Academic pre-training is feasible with proper optimization, and the benchmark provides guidance for researchers to conduct larger-scale training experiments despite resource constraints.

Abstract: Pre-training is notoriously compute-intensive and academic researchers are notoriously under-resourced. It is, therefore, commonly assumed that academics can’t pre-train models. In this paper, we seek to clarify this assumption. We first survey academic researchers to learn about their available compute and then empirically measure the time to replicate models on such resources. We introduce a benchmark to measure the time to pre-train models on given GPUs and also identify ideal settings for maximizing training speed. We run our benchmark on a range of models and academic GPUs, spending 2,000 GPU-hours on our experiments. Our results reveal a brighter picture for academic pre-training: for example, although Pythia-1B was originally trained on 64 GPUs for 3 days, we find it is also possible to replicate this model (with the same hyper-parameters) in 3x fewer GPU-days: i.e. on 4 GPUs in 18 days. We conclude with a cost-benefit analysis to help clarify the trade-offs between price and pre-training time. We believe our benchmark will help academic researchers conduct experiments that require training larger models on more data. We fully release our codebase at: https://github.com/apoorvkh/academic-pretraining.

Method: Proposes three atomic knowledge operators for retrieving and manipulating heterogeneous knowledge. First decomposes questions into fine-grained reasoning trees with atomic operators, then executes each operator to flexibly retrieve and operate atomic-level knowledge from multiple sources.

Result: Significant improvements over state-of-the-art baselines: 9.4% F1 improvement on 2WikiMultihop and 9.5% on BlendQA benchmark. Also introduces BlendQA, a challenging benchmark for heterogeneous knowledge reasoning.

Conclusion: AtomR effectively addresses limitations in compositional reasoning and knowledge integration through atomic-level operators, demonstrating superior performance on complex knowledge reasoning tasks across multiple datasets.

Abstract: Despite the outstanding capabilities of large language models (LLMs), knowledge-intensive reasoning still remains a challenging task due to LLMs’ limitations in compositional reasoning and the hallucination problem. A prevalent solution is to employ chain-of-thought (CoT) with retrieval-augmented generation (RAG), which first formulates a reasoning plan by decomposing complex questions into simpler sub-questions, and then applies iterative RAG at each sub-question. However, prior works exhibit two crucial problems: inadequate reasoning planning and poor incorporation of heterogeneous knowledge. In this paper, we introduce AtomR, a framework for LLMs to conduct accurate heterogeneous knowledge reasoning at the atomic level. Inspired by how knowledge graph query languages model compositional reasoning through combining predefined operations, we propose three atomic knowledge operators, a unified set of operators for LLMs to retrieve and manipulate knowledge from heterogeneous sources. First, in the reasoning planning stage, AtomR decomposes a complex question into a reasoning tree where each leaf node corresponds to an atomic knowledge operator, achieving question decomposition that is highly fine-grained and orthogonal. Subsequently, in the reasoning execution stage, AtomR executes each atomic knowledge operator, which flexibly selects, retrieves, and operates atomic level knowledge from heterogeneous sources. We also introduce BlendQA, a challenging benchmark specially tailored for heterogeneous knowledge reasoning. Experiments on three single-source and two multi-source datasets show that AtomR outperforms state-of-the-art baselines by a large margin, with F1 score improvements of 9.4% on 2WikiMultihop and 9.5% on BlendQA. We release our code and datasets.

Method: Proposes GraphJudge framework with entity-centric strategy to eliminate noise and fine-tunes an LLM as a graph judge to enhance KG quality.

Result: Achieves state-of-the-art performance on two general and one domain-specific text-graph pair datasets with strong generalization abilities.

Conclusion: GraphJudge effectively addresses key challenges in KG construction from unstructured data through noise elimination and quality enhancement via a specialized graph judge LLM.

Abstract: In real-world scenarios, most of the data obtained from the information retrieval (IR) system is unstructured. Converting natural language sentences into structured Knowledge Graphs (KGs) remains a critical challenge. We identified three limitations with respect to existing KG construction methods: (1) There could be a large amount of noise in real-world documents, which could result in extracting messy information. (2) Naive LLMs usually extract inaccurate knowledge from some domain-specific documents. (3) Hallucination phenomenon cannot be overlooked when directly using LLMs to construct KGs. In this paper, we propose \textbf{GraphJudge}, a KG construction framework to address the aforementioned challenges. In this framework, we designed an entity-centric strategy to eliminate the noise information in the documents. And we fine-tuned a LLM as a graph judge to finally enhance the quality of generated KGs. Experiments conducted on two general and one domain-specific text-graph pair datasets demonstrate state-of-the-art performance against various baseline methods with strong generalization abilities. Our code is available at \href{https://github.com/hhy-huang/GraphJudge}{https://github.com/hhy-huang/GraphJudge}.

Method: Four-component framework: FinCap defines required capabilities, FinRec provides training recipe with continual pre-training and instruction-following plus preference data distillation using generative reward model signals, FinTrain offers curated datasets, and FinEval provides comprehensive evaluation suite.

Result: Llama-Fin achieves state-of-the-art performance across a wide range of financial tasks. Analysis shows how each post-training stage contributes to distinct capabilities and reveals specific challenges and effective solutions.

Conclusion: The FINDAP framework provides valuable insights for domain adaptation of LLMs, demonstrating systematic approaches to address challenges in specialized domain post-training.

Abstract: Domain-adaptive post-training of large language models (LLMs) has emerged as a promising approach for specialized domains such as medicine and finance. However, significant challenges remain in identifying optimal adaptation criteria and training strategies across varying data and model configurations. To address these challenges, we introduce FINDAP, a systematic and fine-grained investigation into domain-adaptive post-training of LLMs for the finance domain. Our approach consists of four key components: FinCap, which defines the core capabilities required for the target domain; FinRec, an effective training recipe that jointly optimizes continual pre-training and instruction-following, along with a novel preference data distillation method leveraging process signals from a generative reward model; FinTrain, a curated set of training datasets supporting FinRec; and FinEval, a comprehensive evaluation suite aligned with FinCap. The resulting model, Llama-Fin, achieves state-of-the-art performance across a wide range of financial tasks. Our analysis also highlights how each post-training stage contributes to distinct capabilities, uncovering specific challenges and effective solutions, providing valuable insights for domain adaptation of LLMs

Method: Assembles diverse open-source LLMs in an ensemble approach that balances agreement and disagreement, using relevancy scoring based on embedding distances between topic descriptions and LLM reasoning.

Result: Ensemble approach achieved highest accuracy and optimal precision-sensitivity trade-off compared to individual LLMs. Relevancy scoring effectively mitigated LLM heterogeneity.

Conclusion: The ensemble framework with locally-deployable LLMs provides an effective solution for privacy-preserving labeling of free-text data in psychological research.

Abstract: Free-text responses are commonly collected in psychological studies, providing rich qualitative insights that quantitative measures may not capture. Labeling curated topics of research interest in free-text data by multiple trained human coders is typically labor-intensive and time-consuming. Though large language models (LLMs) excel in language processing, LLM-assisted labeling techniques relying on closed-source LLMs cannot be directly applied to free-text data, without explicit consent for external use. In this study, we propose a framework of assembling locally-deployable LLMs to enhance the labeling of predetermined topics in free-text data under privacy constraints. Analogous to annotation by multiple human raters, this framework leverages the heterogeneity of diverse open-source LLMs. The ensemble approach seeks a balance between the agreement and disagreement across LLMs, guided by a relevancy scoring methodology that utilizes embedding distances between topic descriptions and LLMs’ reasoning. We evaluated the ensemble approach using both publicly accessible Reddit data from eating disorder related forums, and free-text responses from eating disorder patients, both complemented by human annotations. We found that: (1) there is heterogeneity in the performance of labeling among same-sized LLMs, with some showing low sensitivity but high precision, while others exhibit high sensitivity but low precision. (2) Compared to individual LLMs, the ensemble of LLMs achieved the highest accuracy and optimal precision-sensitivity trade-off in predicting human annotations. (3) The relevancy scores across LLMs showed greater agreement than dichotomous labels, indicating that the relevancy scoring method effectively mitigates the heterogeneity in LLMs’ labeling.

Method: Trained multilingual PRMs on a dataset spanning 7 languages (translated from English) and evaluated on two reasoning benchmarks across 11 languages. Analyzed sensitivity to training languages, English data volume, candidate responses, and model parameters.

Result: Multilingual PRMs improved average accuracy and reduced early-stage reasoning errors across multiple languages. Performance was sensitive to training language diversity and English data volume, with benefits from more candidate responses and trainable parameters.

Conclusion: The work successfully extends process reward modeling to multilingual settings, opening avenues for robust multilingual applications in complex reasoning tasks. The authors release code to support further research.

Abstract: Large language models (LLMs) are designed to perform a wide range of tasks. To improve their ability to solve complex problems requiring multi-step reasoning, recent research leverages process reward modeling to provide fine-grained feedback at each step of the reasoning process for reinforcement learning (RL), but it predominantly focuses on English. In this paper, we tackle the critical challenge of extending process reward models (PRMs) to multilingual settings. To achieve this, we train multilingual PRMs on a dataset spanning seven languages, which is translated from English. Through comprehensive evaluations on two widely used reasoning benchmarks across 11 languages, we demonstrate that multilingual PRMs not only improve average accuracy but also reduce early-stage reasoning errors. Furthermore, our results highlight the sensitivity of multilingual PRMs to both the number of training languages and the volume of English data, while also uncovering the benefits arising from more candidate responses and trainable parameters. This work opens promising avenues for robust multilingual applications in complex, multi-step reasoning tasks. In addition, we release the code to foster research along this line.

Method: Applied mechanistic interpretability tools to identify activations encoding sociodemographic information (gender, race) in LLMs, using MLP layer analysis and inference-time patching interventions.

Result: Gender information is highly localized in MLP layers and can be reliably manipulated via patching, altering clinical vignettes and downstream predictions like depression risk. Race representation is more distributed but also intervenable.

Conclusion: This is the first application of mechanistic interpretability to LLMs in healthcare, demonstrating the ability to identify and surgically alter sociodemographic biases in clinical contexts.

Abstract: We know from prior work that LLMs encode social biases, and that this manifests in clinical tasks. In this work we adopt tools from mechanistic interpretability to unveil sociodemographic representations and biases within LLMs in the context of healthcare. Specifically, we ask: Can we identify activations within LLMs that encode sociodemographic information (e.g., gender, race)? We find that gender information is highly localized in MLP layers and can be reliably manipulated at inference time via patching. Such interventions can surgically alter generated clinical vignettes for specific conditions, and also influence downstream clinical predictions which correlate with gender, e.g., patient risk of depression. We find that representation of patient race is somewhat more distributed, but can also be intervened upon, to a degree. To our knowledge, this is the first application of mechanistic interpretability methods to LLMs for healthcare.

Method: LoRA-MGPO incorporates Momentum-Guided Perturbation Optimization (MGPO) that uses momentum vectors from optimizer’s state to guide weight perturbations without dual gradient computations, plus adaptive normalization using EMA of gradient norms to scale perturbation magnitudes.

Result: Experiments on natural language understanding and generation benchmarks show LoRA-MGPO consistently achieves superior performance over LoRA and other PEFT methods, with smoother loss curves, faster convergence, and improved generalization.

Conclusion: LoRA-MGPO effectively stabilizes training dynamics by mitigating double descent phenomenon and avoiding sharp minima, leading to more stable optimization and better fine-tuning performance.

Abstract: Parameter-efficient fine-tuning (PEFT), particularly Low-Rank Adaptation (LoRA), adapts large language models (LLMs) by training only a small fraction of parameters. However, as the rank of the low-rank matrices used for adaptation increases, LoRA often exhibits an unstable “double descent” phenomenon, characterized by transient divergence in the training loss, which delays convergence and impairs generalization by causing instability due to the attraction to sharp local minima. To address this, we introduce LoRA-MGPO, a framework that incorporates Momentum-Guided Perturbation Optimization (MGPO). MGPO stabilizes training dynamics by mitigating the double descent phenomenon and guiding weight perturbations using momentum vectors from the optimizer’s state, thus avoiding dual gradient computations. Additionally, an adaptive normalization scheme scales the magnitude of perturbations based on an exponential moving average (EMA) of gradient norms, further enhancing stability. While EMA controls the magnitude of the perturbations, MGPO guides their direction, ensuring a more stable optimization trajectory. Experiments on a suite of natural language understanding and generation benchmarks show that LoRA-MGPO consistently achieves superior performance over LoRA and other PEFT methods. The analysis indicates that LoRA-MGPO leads to smoother loss curves, faster convergence, and improved generalization by stabilizing the training process and mitigating the attraction to sharp minima.

Method: Created a new Russian ICD coding dataset with EHR diagnosis fields, benchmarked BERT, LLaMA with LoRA, and RAG models, conducted transfer learning experiments across domains and terminologies, and applied the best model to label in-house EHR data.

Result: Training with automated predicted codes significantly improved accuracy compared to manually annotated data from physicians on a curated test set.

Conclusion: Automated clinical coding shows strong potential for resource-limited languages like Russian, offering enhanced clinical efficiency and data accuracy.

Abstract: This study investigates the feasibility of automating clinical coding in Russian, a language with limited biomedical resources. We present a new dataset for ICD coding, which includes diagnosis fields from electronic health records (EHRs) annotated with over 10,000 entities and more than 1,500 unique ICD codes. This dataset serves as a benchmark for several state-of-the-art models, including BERT, LLaMA with LoRA, and RAG, with additional experiments examining transfer learning across domains (from PubMed abstracts to medical diagnosis) and terminologies (from UMLS concepts to ICD codes). We then apply the best-performing model to label an in-house EHR dataset containing patient histories from 2017 to 2021. Our experiments, conducted on a carefully curated test set, demonstrate that training with the automated predicted codes leads to a significant improvement in accuracy compared to manually annotated data from physicians. We believe our findings offer valuable insights into the potential for automating clinical coding in resource-limited languages like Russian, which could enhance clinical efficiency and data accuracy in these contexts. Our code and dataset are available at https://github.com/auto-icd-coding/ruccod.

Method: Two-step chain-of-thought prompting approach where models first identify credible sources for claims, then generate persuasive responses using three LLMs (ChatGPT, Gemini, Claude).

Result: Models struggle to ground responses in real news sources, prefer citing left-leaning sources, and show varying response diversity.

Conclusion: Prompt-engineering alone is insufficient for LLM-based fact-checking; more robust guardrails are needed for both researchers and non-technical users.

Abstract: While Large Language Models (LLMs) can amplify online misinformation, they also show promise in tackling misinformation. In this paper, we empirically study the capabilities of three LLMs – ChatGPT, Gemini, and Claude – in countering political misinformation. We implement a two-step, chain-of-thought prompting approach, where models first identify credible sources for a given claim and then generate persuasive responses. Our findings suggest that models struggle to ground their responses in real news sources, and tend to prefer citing left-leaning sources. We also observe varying degrees of response diversity among models. Our findings highlight concerns about using LLMs for fact-checking through only prompt-engineering, emphasizing the need for more robust guardrails. Our results have implications for both researchers and non-technical users.

Method: Task elicitation - an automated method that builds new evaluations to profile model behavior by finding natural-language tasks where models exhibit systematic failures.

Result: The method found hundreds of natural-language tasks (order of magnitude more than prior work) where frontier models show systematic failures in domains like forecasting and online harassment. Specific examples include Sonnet 3.5 over-associating quantum computing with AGI, and o3-mini hallucinating when fabrications are repeated in-context.

Conclusion: Task elicitation is an effective automated approach for discovering systematic model failures across diverse domains, significantly expanding evaluation coverage beyond manual benchmark creation.

Abstract: Language model evaluations often fail to characterize consequential failure modes, forcing experts to inspect outputs and build new benchmarks. We introduce task elicitation, a method that automatically builds new evaluations to profile model behavior. Task elicitation finds hundreds of natural-language tasks – an order of magnitude more than prior work – where frontier models exhibit systematic failures, in domains ranging from forecasting to online harassment. For example, we find that Sonnet 3.5 over-associates quantum computing and AGI and that o3-mini is prone to hallucination when fabrications are repeated in-context.

Method: Uses a generator to interact with the environment and an assistant to examine each action, triggering rollback operations when incorrect actions are detected, with additional strategies for rollback scenarios.

Result: Achieves significant improvements over strong baselines on three widely used benchmarks and functions as a robust plug-and-play module.

Conclusion: GA-Rollback effectively mitigates error propagation in LLM agents through its generator-assistant rollback mechanism.

Abstract: Large language model (LLM) agents typically adopt a step-by-step reasoning framework, in which they interleave the processes of thinking and acting to accomplish the given task. However, this paradigm faces a deep-rooted one-pass issue whereby each generated intermediate thought is plugged into the trajectory regardless of its correctness, which can cause irreversible error propagation. To address the issue, this paper proposes a novel framework called Generator-Assistant Stepwise Rollback (GA-Rollback) to induce better decision-making for LLM agents. Particularly, GA-Rollback utilizes a generator to interact with the environment and an assistant to examine each action produced by the generator, where the assistant triggers a rollback operation upon detection of incorrect actions. Moreover, we introduce two additional strategies tailored for the rollback scenario to further improve its effectiveness. Extensive experiments show that GA-Rollback achieves significant improvements over several strong baselines on three widely used benchmarks. Our analysis further reveals that GA-Rollback can function as a robust plug-and-play module, integrating seamlessly with other methods.

Method: Compare different inference methods for extracting preferences from LLM judgment distributions, including taking the mean vs mode, risk-averse approaches, and analyzing chain-of-thought prompting effects on distribution spread.

Result: Taking the mean of judgment distributions consistently outperforms greedy decoding across all evaluation settings (pointwise, pairwise, listwise). Risk-averse methods often improve performance, while chain-of-thought prompting collapses distribution spread and harms performance.

Conclusion: Leveraging distributional outputs from LLM judges provides better performance than using the text interface alone, with mean-based inference and risk-averse methods being particularly effective.

Abstract: Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) has become a standard practice applicable to many tasks. A judgment is often extracted from the judge’s textual output alone, typically with greedy decoding. However, LLM judges naturally provide distributions over judgment tokens, inviting a breadth of inference methods for extracting fine-grained preferences. We find that taking the mean of the judgment distribution consistently outperforms taking the mode (i.e. greedy decoding) in all evaluation settings (i.e. pointwise, pairwise, and listwise). We further explore novel methods of deriving preferences from judgment distributions, and find that methods incorporating risk aversion often improve performance. Lastly, we analyze LLM-as-a-judge paired with chain-of-thought (CoT) prompting, showing that CoT can collapse the spread of the judgment distribution, often harming performance. Our findings show that leveraging distributional output improves LLM-as-a-judge, as opposed to using the text interface alone.

Method: Transform monolithic reasoning into iterative process with short reasoning segments interleaved with concise progress summaries, creating a sawtooth memory pattern. Reconstruct long-context datasets into iterative format.

Result: Reduces computational costs while improving performance - Qwen2.5-Math-7B shows 3-13% improvements across MATH500, AIME24, and GPQA_diamond benchmarks. Created 333K training instances from OpenR1-Math.

Conclusion: Challenges the assumed trade-off between reasoning depth and computational efficiency, providing scalable approach to complex reasoning without architectural modifications.

Abstract: Advanced reasoning in large language models has achieved remarkable performance on challenging tasks, but the prevailing long-context reasoning paradigm faces critical limitations: quadratic computational scaling with sequence length, reasoning constrained by maximum context boundaries, and performance degradation beyond pre-training context windows. Existing approaches primarily compress reasoning chains without addressing the fundamental scaling problem. To overcome these challenges, we introduce InftyThink, a paradigm that transforms monolithic reasoning into an iterative process with intermediate summarization. By interleaving short reasoning segments with concise progress summaries, our approach enables unbounded reasoning depth while maintaining bounded computational costs. This creates a characteristic sawtooth memory pattern that significantly reduces computational complexity compared to traditional approaches. Furthermore, we develop a methodology for reconstructing long-context reasoning datasets into our iterative format, transforming OpenR1-Math into 333K training instances. Experiments across multiple model architectures demonstrate that our approach reduces computational costs while improving performance, with Qwen2.5-Math-7B showing 3-13% improvements across MATH500, AIME24, and GPQA_diamond benchmarks. Our work challenges the assumed trade-off between reasoning depth and computational efficiency, providing a more scalable approach to complex reasoning without architectural modifications.

Method: 1) Decouple total head size from hidden size for flexible attention FLOPs control; 2) Jointly optimize model size and GQA configuration to better allocate inference resources between attention layers and other components.

Result: Common GQA configurations are highly suboptimal for long contexts. The proposed recipe shows that for long-context scenarios, fewer attention heads with larger model size is optimal, achieving >50% reduction in memory usage and FLOPs compared to Llama-3’s GQA with no performance degradation.

Conclusion: The findings provide valuable insights for designing efficient long-context LLMs, with a recipe for deriving cost-optimal GQA configurations that significantly improve efficiency without compromising model capabilities.

Abstract: Grouped-Query Attention (GQA) is a widely adopted strategy for reducing the computational cost of attention layers in large language models (LLMs). However, current GQA configurations are often suboptimal because they overlook how context length influences inference cost. Since inference cost grows with context length, the most cost-efficient GQA configuration should also vary accordingly. In this work, we analyze the relationship among context length, model size, GQA configuration, and model loss, and introduce two innovations: (1) we decouple the total head size from the hidden size, enabling more flexible control over attention FLOPs; and (2) we jointly optimize the model size and the GQA configuration to arrive at a better allocation of inference resources between attention layers and other components. Our analysis reveals that commonly used GQA configurations are highly suboptimal for long-context scenarios. More importantly, we propose a recipe for deriving cost-optimal GQA configurations. Our results show that for long-context scenarios, one should use fewer attention heads while scaling up model size. Configurations selected by our recipe can reduce both memory usage and FLOPs by more than 50% compared to Llama-3’s GQA, with no degradation in model capabilities. Our findings offer valuable insights for designing efficient long-context LLMs. The code is available at https://www.github.com/THUNLP/cost-optimal-gqa .

Method: HiRAG utilizes hierarchical knowledge to enhance semantic understanding and structure capturing capabilities in the indexing and retrieval processes of RAG systems.

Result: Extensive experiments demonstrate that HiRAG achieves significant performance improvements over state-of-the-art baseline methods.

Conclusion: Hierarchical knowledge utilization in RAG systems leads to enhanced performance in domain-specific tasks.

Abstract: Graph-based Retrieval-Augmented Generation (RAG) methods have significantly enhanced the performance of large language models (LLMs) in domain-specific tasks. However, existing RAG methods do not adequately utilize the naturally inherent hierarchical knowledge in human cognition, which limits the capabilities of RAG systems. In this paper, we introduce a new RAG approach, called HiRAG, which utilizes hierarchical knowledge to enhance the semantic understanding and structure capturing capabilities of RAG systems in the indexing and retrieval processes. Our extensive experiments demonstrate that HiRAG achieves significant performance improvements over the state-of-the-art baseline methods.

Method: Created Font Recognition Benchmark (FRB) with 15 common fonts in easy (10 sentences) and hard (font names as text, creating stroop effect) versions. Evaluated various VLMs on font recognition tasks.

Result: Current VLMs show poor font recognition performance, are easily affected by stroop effects, and few-shot learning/CoT prompting provide minimal accuracy improvements.

Conclusion: VLMs have inherent limitations in capturing semantic features for font recognition, highlighting their current shortcomings in fine-grained visual tasks.

Abstract: Modern Vision-Language Models (VLMs) exhibit remarkable visual and linguistic capabilities, achieving impressive performance in various tasks such as image recognition and object localization. However, their effectiveness in fine-grained tasks remains an open question. In everyday scenarios, individuals encountering design materials, such as magazines, typography tutorials, research papers, or branding content, may wish to identify aesthetically pleasing fonts used in the text. Given their multimodal capabilities and free accessibility, many VLMs are often considered potential tools for font recognition. This raises a fundamental question: Do VLMs truly possess the capability to recognize fonts? To investigate this, we introduce the Font Recognition Benchmark (FRB), a compact and well-structured dataset comprising 15 commonly used fonts. FRB includes two versions: (i) an easy version, where 10 sentences are rendered in different fonts, and (ii) a hard version, where each text sample consists of the names of the 15 fonts themselves, introducing a stroop effect that challenges model perception. Through extensive evaluation of various VLMs on font recognition tasks, we arrive at the following key findings: (i) Current VLMs exhibit limited font recognition capabilities, with many state-of-the-art models failing to achieve satisfactory performance and being easily affected by the stroop effect introduced by textual information. (ii) Few-shot learning and Chain-of-Thought (CoT) prompting provide minimal benefits in improving font recognition accuracy across different VLMs. (iii) Attention analysis sheds light on the inherent limitations of VLMs in capturing semantic features.

Method: Created CLASH dataset with 345 high-impact dilemmas and 3,795 individual perspectives of diverse values, then benchmarked 14 non-thinking and thinking models on understanding decision ambivalence, psychological discomfort, and value shifts.

Result: Key findings: (1) Strong models struggle with ambivalent decisions (GPT-5: 24.06%, Claude-4-Sonnet: 51.01% accuracy), (2) LLMs predict discomfort but not value shifts, (3) Cognitive behaviors don’t transfer to value reasoning, (4) Steerability correlates with value preferences, (5) Third-party reasoning increases steerability.

Conclusion: LLMs face significant challenges in high-stakes value reasoning, exhibiting specific failure patterns and limited understanding of value dynamics, highlighting the need for specialized approaches to value-based decision-making in AI.

Abstract: Navigating dilemmas involving conflicting values is challenging even for humans in high-stakes domains, let alone for AI, yet prior work has been limited to everyday scenarios. To close this gap, we introduce CLASH (Character perspective-based LLM Assessments in Situations with High-stakes), a meticulously curated dataset consisting of 345 high-impact dilemmas along with 3,795 individual perspectives of diverse values. CLASH enables the study of critical yet underexplored aspects of value-based decision-making processes, including understanding of decision ambivalence and psychological discomfort as well as capturing the temporal shifts of values in the perspectives of characters. By benchmarking 14 non-thinking and thinking models, we uncover several key findings. (1) Even strong proprietary models, such as GPT-5 and Claude-4-Sonnet, struggle with ambivalent decisions, achieving only 24.06 and 51.01 accuracy. (2) Although LLMs reasonably predict psychological discomfort, they do not adequately comprehend perspectives involving value shifts. (3) Cognitive behaviors that are effective in the math-solving and game strategy domains do not transfer to value reasoning. Instead, new failure patterns emerge, including early commitment and overcommitment. (4) The steerability of LLMs towards a given value is significantly correlated with their value preferences. (5) Finally, LLMs exhibit greater steerability when reasoning from a third-party perspective, although certain values (e.g., safety) benefit uniquely from first-person framing.

Method: Propose SOLAR framework that observes value conflicts and trade-offs in user-generated texts to better represent subjective ground of individuals on social media.

Result: Empirical results show SOLAR improves overall inference results and performance on controversial situations, and provides explanations about individuals’ value preferences.

Conclusion: SOLAR framework effectively infers individual moral judgments and provides value preference explanations, demonstrating LLMs’ capability to account for individual-level subjectivity.

Abstract: Large Language Models (LLMs) not only have solved complex reasoning problems but also exhibit remarkable performance in tasks that require subjective decision making. Existing studies suggest that LLM generations can be subjectively grounded to some extent, yet exploring whether LLMs can account for individual-level subjectivity has not been sufficiently studied. In this paper, we characterize subjectivity of individuals on social media and infer their moral judgments using LLMs. We propose a framework, SOLAR (Subjective Ground with Value Abstraction), that observes value conflicts and trade-offs in the user-generated texts to better represent subjective ground of individuals. Empirical results show that our framework improves overall inference results as well as performance on controversial situations. Additionally, we qualitatively show that SOLAR provides explanations about individuals’ value preferences, which can further account for their judgments.

Method: Three-stage approach: (1) synthetic multilingual data generation with cultural/linguistic nuances, (2) supervised fine-tuning, and (3) curriculum-based Group Relative Policy Optimization (GRPO) framework.

Result: MrGuard consistently outperforms recent baselines by more than 15% across both in-domain and out-of-domain languages, and maintains robustness to multilingual variations like code-switching and low-resource language distractors.

Conclusion: The multilingual guardrail with reasoning capability effectively addresses LLM safety vulnerabilities in diverse linguistic contexts and provides explanations for language-specific risks in content moderation.

Abstract: Large Language Models (LLMs) are susceptible to adversarial attacks such as jailbreaking, which can elicit harmful or unsafe behaviors. This vulnerability is exacerbated in multilingual settings, where multilingual safety-aligned data is often limited. Thus, developing a guardrail capable of detecting and filtering unsafe content across diverse languages is critical for deploying LLMs in real-world applications. In this work, we introduce a multilingual guardrail with reasoning for prompt classification. Our method consists of: (1) synthetic multilingual data generation incorporating culturally and linguistically nuanced variants, (2) supervised fine-tuning, and (3) a curriculum-based Group Relative Policy Optimization (GRPO) framework that further improves performance. Experimental results demonstrate that our multilingual guardrail, MrGuard, consistently outperforms recent baselines across both in-domain and out-of-domain languages by more than 15%. We also evaluate MrGuard’s robustness to multilingual variations, such as code-switching and low-resource language distractors in the prompt, and demonstrate that it preserves safety judgments under these challenging conditions. The multilingual reasoning capability of our guardrail enables it to generate explanations, which are particularly useful for understanding language-specific risks and ambiguities in multilingual content moderation.

Method: Conducted systematic survey of prior UQ and calibration methods for LLMs, created rigorous benchmark, and empirically evaluated six methods using two reliability datasets.

Result: The empirical evaluation justified significant findings from the review, providing insights into the effectiveness of different UQ and calibration approaches.

Conclusion: This is the first dedicated study reviewing calibration methods and relevant metrics for LLMs, with identified future directions and open challenges in the field.

Abstract: Large Language Models (LLMs) have been transformative across many domains. However, hallucination – confidently outputting incorrect information – remains one of the leading challenges for LLMs. This raises the question of how to accurately assess and quantify the uncertainty of LLMs. Extensive literature on traditional models has explored Uncertainty Quantification (UQ) to measure uncertainty and employed calibration techniques to address the misalignment between uncertainty and accuracy. While some of these methods have been adapted for LLMs, the literature lacks an in-depth analysis of their effectiveness and does not offer a comprehensive benchmark to enable insightful comparison among existing solutions. In this work, we fill this gap via a systematic survey of representative prior works on UQ and calibration for LLMs and introduce a rigorous benchmark. Using two widely used reliability datasets, we empirically evaluate six related methods, which justify the significant findings of our review. Finally, we provide outlooks for key future directions and outline open challenges. To the best of our knowledge, this survey is the first dedicated study to review the calibration methods and relevant metrics for LLMs.

Method: Leverages LLMs to automatically detect non-convex components and transform them into solvable forms, with integrated error correction and feasibility validation mechanisms.

Result: Achieves 96% execution rate and 80% success rate on GPT-4, significantly outperforming baseline approaches in complex optimization tasks across diverse scenarios.

Conclusion: LLM-OptiRA enables fully automated resolution of non-convex resource allocation problems, reducing reliance on expert knowledge while ensuring robustness through validation mechanisms.

Abstract: Solving non-convex resource allocation problems poses significant challenges in wireless communication systems, often beyond the capability of traditional optimization techniques. To address this issue, we propose LLM-OptiRA, the first framework that leverages large language models (LLMs) to automatically detect and transform non-convex components into solvable forms, enabling fully automated resolution of non-convex resource allocation problems in wireless communication systems. LLM-OptiRA not only simplifies problem-solving by reducing reliance on expert knowledge, but also integrates error correction and feasibility validation mechanisms to ensure robustness. Experimental results show that LLM-OptiRA achieves an execution rate of 96% and a success rate of 80% on GPT-4, significantly outperforming baseline approaches in complex optimization tasks across diverse scenarios.

Method: Fine-tune instruction-tuned LLMs on 3.9K factually grounded reasoning traces sourced from large reasoning models and conditioned on KG paths.

Result: fs1-tuned model (32B) outperforms instruction-tuned counterparts by 6-14 absolute points (pass@16), with most improvements on complex questions requiring 3+ hops and numerical answers.

Conclusion: Anchoring reasoning to factual KG paths is critical for transforming LLMs into reliable systems for knowledge-intensive tasks, showing effectiveness beyond STEM domains.

Abstract: We introduce fs1, a simple yet effective method that improves the factuality of reasoning traces by sourcing them from large reasoning models (e.g., DeepSeek-R1) and grounding them by conditioning on knowledge graph (KG) paths. We fine-tune eight instruction-tuned Large Language Models (LLMs) on 3.9K factually grounded reasoning traces and rigorously evaluate them on six complex open-domain question-answering (QA) benchmarks encompassing 23.9K questions. Our results demonstrate that our fs1-tuned model (32B parameters) consistently outperforms instruction-tuned counterparts with parallel sampling by 6-14 absolute points (pass@$16$). Our detailed analysis shows that fs1 considerably improves model performance over more complex questions (requiring 3 or more hops on KG paths) and numerical answer types compared to the baselines. Furthermore, in single-pass inference, we notice that smaller LLMs show the most improvements. While prior works demonstrate the effectiveness of reasoning traces primarily in the STEM domains, our work shows strong evidence that anchoring reasoning to factual KG paths is a critical step in transforming LLMs for reliable knowledge-intensive tasks.

Method: Fine-tuned Qwen2.5-Coder-7B-Instruct with reinforcement learning using a reward function that combines correctness and performance speedup, evaluated on a large-scale benchmark of 8,072 real-world assembly programs.

Result: SuperCoder achieved 95.0% correctness and 1.46x average speedup over gcc -O3, significantly outperforming Claude-opus-4 baseline (51.5% correctness, 1.43x speedup).

Conclusion: LLMs can effectively serve as superoptimizers for assembly programs, establishing a new foundation for program performance optimization beyond traditional compiler approaches.

Abstract: Superoptimization is the task of transforming a program into a faster one while preserving its input-output behavior. In this work, we investigate whether large language models (LLMs) can serve as superoptimizers, generating assembly programs that outperform code already optimized by industry-standard compilers. We construct the first large-scale benchmark for this problem, consisting of 8,072 real-world assembly programs averaging 130 lines, in contrast to prior datasets restricted to 2-15 straight-line, loop-free programs. We evaluate 23 LLMs on this benchmark and find that the strongest baseline, Claude-opus-4, achieves a 51.5% test-passing rate and a 1.43x average speedup over gcc -O3. To further enhance performance, we fine-tune models with reinforcement learning, optimizing a reward function that integrates correctness and performance speedup. Starting from Qwen2.5-Coder-7B-Instruct (61.4% correctness, 1.10x speedup), the fine-tuned model SuperCoder attains 95.0% correctness and 1.46x average speedup. Our results demonstrate, for the first time, that LLMs can be applied as superoptimizers for assembly programs, establishing a foundation for future research in program performance optimization beyond compiler heuristics.

Method: ZeroTuning adds lightweight biases to the initial token’s attention logits, which monotonically controls the entropy of downstream attention distribution. It comes in two variants: supervised mode calibrated on validation examples, and unsupervised mode that directly minimizes model output entropy.

Result: Achieves broad gains across 15 datasets with Llama-3.1-8B: 19.9% improvement on classification, 4.5% on question answering, and 2.1% on dialogue. Works with quantized inference and maintains performance with increasing context lengths.

Conclusion: ZeroTuning provides a simpler, more elegant alternative to previous attention tuning methods that is lightweight, kernel-agnostic, and outperforms more complex approaches while requiring minimal code changes.

Abstract: Token-level attention tuning, a class of training-free methods including Post-hoc Attention Steering (PASTA) and Attention Calibration (ACT), has emerged as a promising way to improve frozen LLMs with interpretable interventions. However, these methods depend on auxiliary heuristics to identify “important” task-specific tokens, which can introduce bias and limit applicability when token importance is unclear or when using optimized kernels where attention maps are inaccessible. We propose a simpler and more elegant alternative: acting only on the initial token (e.g., in LLaMA). We show theoretically that adding lightweight biases to this token’s attention logits monotonically controls the entropy of the downstream attention distribution - an effect amplified by its natural function as an attention sink. Our empirical analysis reveals that this tuning process can positively affect LLMs and better unlock their pretrained knowledge, with stronger effects in early layers and distinct scaling preferences across attention heads. Building on these insights, we introduce ZeroTuning: a training-free method that improves LLM performance by applying head-specific attention adjustments to the initial token, requiring zero parameter updates. We present two variants: a supervised mode that calibrates on validation examples, and a novel unsupervised mode that directly minimizes the model’s output entropy. The method is lightweight, kernel-agnostic, and requires only four lines of modification to the standard LlamaAttention code. It achieves broad gains across 15 datasets and outperforms previous, more complex methods; for instance, with Llama-3.1-8B, it yields relative improvements of 19.9% on classification, 4.5% on question answering, and 2.1% on dialogue. ZeroTuning also works out-of-the-box with quantized inference and maintains its performance improvements with increasing context lengths.

Method: HBO uses bilevel optimization with Global Actors that balance data sampling across datasets and Local Actors that optimize data usage within each dataset based on difficulty levels, guided by reward functions from the LLM’s training state.

Result: HBO consistently outperforms existing baselines across three LLM backbones and nine diverse tasks in multilingual and multitask setups, achieving significant accuracy gains.

Conclusion: HBO provides a comprehensive solution to data imbalance and heterogeneity in LLM fine-tuning, enabling more effective training across diverse datasets through its hierarchical balancing approach.

Abstract: Fine-tuning large language models (LLMs) on a mixture of diverse datasets poses challenges due to data imbalance and heterogeneity. Existing methods often address these issues across datasets (globally) but overlook the imbalance and heterogeneity within individual datasets (locally), which limits their effectiveness. We introduce Hierarchical Balancing Optimization (HBO), a novel method that enables LLMs to autonomously adjust data allocation during fine-tuning both across datasets (globally) and within each individual dataset (locally). HBO employs a bilevel optimization strategy with two types of actors: a Global Actor, which balances data sampling across different subsets of the training mixture, and several Local Actors, which optimizes data usage within each subset based on difficulty levels. These actors are guided by reward functions derived from the LLM’s training state, which measure learning progress and relative performance improvement. We evaluate HBO on three LLM backbones across nine diverse tasks in multilingual and multitask setups. Results show that HBO consistently outperforms existing baselines, achieving significant accuracy gains. Our in-depth analysis further demonstrates that both the global actor and local actors of HBO effectively adjust data usage during fine-tuning. HBO provides a comprehensive solution to the challenges of data imbalance and heterogeneity in LLM fine-tuning, enabling more effective training across diverse datasets.

Method: A four-step process: 1) align representation dimensions with auto-encoders, 2) identify intervention layer pairs via mutual information analysis, 3) generate steering vectors from expert model using Recursive Feature Machines, 4) apply vectors during inference without updating parameters.

Result: Significantly outperforms established baselines across 15 benchmarks in four domains using three different LLMs, achieving improved performance at minimal cost.

Conclusion: ExpertSteer successfully enables cross-model knowledge transfer for activation steering, allowing arbitrary expert models to guide target LLMs without parameter updates, demonstrating superior effectiveness over model-specific steering approaches.

Abstract: Large Language Models (LLMs) exhibit remarkable capabilities across various tasks, yet guiding them to follow desired behaviours during inference remains a significant challenge. Activation steering offers a promising method to control the generation process of LLMs by modifying their internal activations. However, existing methods commonly intervene in the model’s behaviour using steering vectors generated by the model itself, which constrains their effectiveness to that specific model and excludes the possibility of leveraging powerful external expert models for steering. To address these limitations, we propose ExpertSteer, a novel approach that leverages arbitrary specialized expert models to generate steering vectors, enabling intervention in any LLMs. ExpertSteer transfers the knowledge from an expert model to a target LLM through a cohesive four-step process: first aligning representation dimensions with auto-encoders to enable cross-model transfer, then identifying intervention layer pairs based on mutual information analysis, next generating steering vectors from the expert model using Recursive Feature Machines, and finally applying these vectors on the identified layers during inference to selectively guide the target LLM without updating model parameters. We conduct comprehensive experiments using three LLMs on 15 popular benchmarks across four distinct domains. Experiments demonstrate that ExpertSteer significantly outperforms established baselines across diverse tasks at minimal cost.

Method: Fine-tune the Base model and directly graft the learned weight updates to the Instruct model, introducing no additional parameters.

Result: Shadow-FT consistently outperforms conventional full-parameter and parameter-efficient tuning approaches across 19 benchmarks covering coding, reasoning, and mathematical tasks.

Conclusion: Shadow-FT is effective for mainstream LLMs, can be applied to multimodal LLMs, and combined with DPO, providing a simple yet powerful fine-tuning approach.

Abstract: Large language models (LLMs) consistently benefit from further fine-tuning on various tasks. However, we observe that directly tuning the Instruct (i.e., instruction-tuned) models often leads to marginal improvements and even performance degeneration. Notably, paired Base models, the foundation for these Instruct variants, contain highly similar weight values (i.e., less than 2% on average for Llama 3.1 8B). The Base model tends to be a good learner yet a weak backbone without post-training. Therefore, we propose a novel Shadow-FT framework to tune the Instruct models by leveraging the corresponding Base models. The key insight is to fine-tune the Base model, and then \textit{directly} graft the learned weight updates to the Instruct model. Our proposed Shadow-FT introduces no additional parameters, is easy to implement, and significantly improves performance. We conduct extensive experiments on tuning mainstream LLMs, such as Qwen 3 and Llama 3 series, and evaluate them across 19 benchmarks covering coding, reasoning, and mathematical tasks. Experimental results demonstrate that Shadow-FT consistently outperforms conventional full-parameter and parameter-efficient tuning approaches. Further analyses indicate that Shadow-FT can be applied to multimodal large language models (MLLMs) and combined with direct preference optimization~(DPO). Codes and weights are available at \href{https://github.com/wutaiqiang/Shadow-FT}{Github}.

Method: Measured representation similarity between languages and applied logit lens to interpret implicit steps in multilingual multi-choice reasoning questions.

Result: LLMs predict inconsistently due to dissimilar representations across languages. Larger models are more multilingual but have more dissociated hidden states, though better at retrieving cross-lingual knowledge. Small models can be improved by steering towards shared semantic space.

Conclusion: Knowledge sharing in small models can be enhanced by steering processing towards shared semantic space, improving multilingual reasoning performance through better knowledge transfer and output consistency with English.

Abstract: Large language models (LLMs) are demonstrably capable of cross-lingual transfer, but can produce inconsistent output when prompted with the same queries written in different languages. To understand how language models are able to generalize knowledge from one language to the others, we measure representation similarity between languages, and apply the logit lens to interpret the implicit steps taken by LLMs to solve multilingual multi-choice reasoning questions. Our analyses reveal LLMs predict inconsistently and are less accurate because they rely on representations that are dissimilar across languages, rather than working in a shared semantic space. While larger models are more multilingual, we show their hidden states are more likely to dissociate from the shared representation compared to smaller models, but are nevertheless more capable of retrieving knowledge embedded across different languages. Finally, we demonstrate that knowledge sharing in small models can be facilitated by steering their latent processing towards the shared semantic space. This improves the models’ multilingual reasoning performance, as a result of more knowledge transfer from, and better output consistency with English.

Method: Computes parameter shifts in one step using only a hidden state and its gradient, employs lifelong normalization strategy to continuously update feature statistics across turns for adapting to distributional shifts.

Result: Achieves over 7x faster editing speed than previous SOTA, uses less than 1/4 VRAM, supports up to 2M edits while maintaining high accuracy, and can edit 7B LLMs on 24GB consumer GPUs.

Conclusion: UltraEdit represents significant progress toward safe and scalable lifelong learning, demonstrating superior performance across diverse model editing scenarios.

Abstract: Lifelong learning enables large language models (LLMs) to adapt to evolving information by continually updating their internal knowledge. An ideal system should support efficient, wide-ranging updates while preserving existing capabilities and ensuring reliable deployment. Model editing stands out as a promising solution for this goal, offering a focused and efficient way to revise a model’s internal knowledge. Although recent paradigms have made notable progress, they often struggle to meet the demands of practical lifelong adaptation at scale. To bridge this gap, we propose UltraEdit, a training-, subject-, and memory-free approach that is well-suited for ultra-scalable, real-world lifelong model editing. UltraEdit fundamentally differs from traditional paradigms by computing parameter shifts in one step using only a hidden state and its gradient, making the approach simple yet efficient. To improve scalability in lifelong settings, UltraEdit employs a lifelong normalization strategy that continuously updates feature statistics across turns, allowing it to adapt to distributional shifts and maintain consistency over time. UltraEdit achieves editing speeds over 7x faster than the previous state-of-the-art method, which was also the fastest known approach, while using less than 1/4 the VRAM. This makes it the only method currently capable of editing a 7B LLM on a 24GB consumer-grade GPU. Furthermore, we construct UltraEditBench, the largest dataset in the field to date with over 2M editing pairs, and demonstrate that our method supports up to 2M edits while maintaining high accuracy. Comprehensive experiments on five datasets and six models show that UltraEdit consistently achieves superior performance across diverse model editing scenarios, taking a further step towards safe and scalable lifelong learning. Our code is available at: https://github.com/XiaojieGu/UltraEdit

Method: Introduces Unlearning Token optimized to steer LLMs toward forgetting space, and lightweight Unlearning Edit to efficiently associate unlearning targets with this meta-token, modifying only ~3.66% of LLM parameters.

Result: Outperforms 8 baselines on TOFU benchmark: ~4.01× better model ability than best-forgetting baseline with higher unlearning efficacy, and 35.96% better unlearning efficacy than best-retaining method with better ability retention.

Conclusion: UniErase demonstrates balanced and dual top-tier performances in knowledge unlearning and ability retention, establishing a new paradigm for precise LLM unlearning.

Abstract: Large language models (LLMs) require iterative updates to address the outdated information problem, where LLM unlearning offers an approach for selective removal. However, mainstream unlearning methods primarily rely on fine-tuning techniques, which often lack precision in targeted unlearning and struggle to balance unlearning efficacy with general ability under massive and sequential settings. To bridge this gap, in this work, we introduce UniErase, a novel unlearning framework that demonstrates precision and balanced performances between knowledge unlearning and ability retaining. We first propose the Unlearning Token, which is optimized to steer LLMs toward a forgetting space. To achieve concrete unlearning behaviors, we further introduce the lightweight Unlearning Edit to efficiently associate the unlearning targets with this meta-token. Serving as a new unlearning paradigm via editing, UniErase achieves outstanding performances across batch, sequential, and precise unlearning tasks under fictitious and real-world knowledge scenarios. On the TOFU benchmark, compared with 8 baselines, UniErase, modifying only $\sim$ \textbf{3.66%} of the LLM parameters, outperforms the previous best-forgetting baseline by \textbf{$\sim$ 4.01$\times$} for \textbf{model ability} with even higher unlearning efficacy. Similarly, UniErase, with better ability retention, also surpasses the previous best-retaining method by \textbf{35.96%} for \textbf{unlearning efficacy}, showing balanced and dual top-tier performances in the current unlearning community.

Method: Conducted a multilingual study across five typologically distinct languages (English, Russian, German, Hindi, Vietnamese) and five model architectures, examining how position bias interacts with prompt strategies and affects output entropy.

Result: Key findings: (1) Position bias is primarily model-driven with language-specific variations; (2) Explicit instructions about context relevance unexpectedly reduce accuracy; (3) Largest accuracy drop occurs with middle-position information but without corresponding entropy peak.

Conclusion: Position bias patterns in LLMs are more complex than previously assumed, being model-dependent with language variations, and common prompt engineering practices may be counterproductive.

Abstract: Large Language Models (LLMs) exhibit position bias - a systematic tendency to neglect information at specific context positions. However, the patterns of position bias behavior, depending on the language or model, remain unexplored. We present a multilingual study across five typologically distinct languages (English, Russian, German, Hindi, and Vietnamese) and five model architectures, examining how position bias interacts with prompt strategies and affects output entropy. Our key findings are: (1) Position bias is primarily model-driven, yet exhibits language-specific variations. For instance, Qwen2.5-7B-Instruct and DeepSeek 7B Chat consistently favors late positions, challenging established assumptions of a universal early-token bias in LLMs. (2) Explicitly instructing the model that “the context is relevant to the query” unexpectedly reduces accuracy across languages, undermining common prompt-engineering practices. (3) While the largest accuracy drop occurs when relevant information is placed in the middle of the context, this is not explicitly reflected by a corresponding peak in output entropy.

Method: Proposes ARC-JSD using Jensen-Shannon Divergence to identify essential context sentences without additional fine-tuning, gradient calculation, or surrogate modeling.

Result: Superior accuracy and significant computational efficiency improvements on RAG benchmarks (TyDi QA, Hotpot QA, Musique) compared to previous surrogate-based methods. Mechanistic analysis reveals specific attention heads and MLP layers responsible for context attribution.

Conclusion: ARC-JSD provides an efficient and accurate solution for context attribution in RAG systems, with insights into model internals that affect RAG behaviors.

Abstract: Retrieval-Augmented Generation (RAG) leverages large language models (LLMs) combined with external contexts to enhance the accuracy and reliability of generated responses. However, reliably attributing generated content to specific context segments, context attribution, remains challenging due to the computationally intensive nature of current methods, which often require extensive fine-tuning or human annotation. In this work, we introduce a novel Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD), enabling efficient and accurate identification of essential context sentences without additional fine-tuning, gradient-calculation or surrogate modelling. Evaluations on a wide range of RAG benchmarks, such as TyDi QA, Hotpot QA, and Musique, using instruction-tuned LLMs in different scales demonstrate superior accuracy and significant computational efficiency improvements compared to the previous surrogate-based method. Furthermore, our mechanistic analysis reveals specific attention heads and multilayer perceptron (MLP) layers responsible for context attribution, providing valuable insights into the internal workings of RAG models and how they affect RAG behaviours. Our code is available at https://github.com/ruizheliUOA/ARC_JSD.

Method: RecInter uses agent-based simulation with robust interaction mechanisms, multidimensional user profiling, advanced agent architecture, and LLMs fine-tuned on Chain-of-Thought enriched interaction data. It includes Merchant Agents that can reply to user actions.

Result: The platform achieves significantly improved simulation credibility and successfully replicates emergent phenomena like Brand Loyalty and the Matthew Effect. Experiments show the interaction mechanism is crucial for simulating realistic system evolution.

Conclusion: RecInter establishes itself as a credible testbed for recommender systems research by providing a more realistic and evolving simulation environment through dynamic interaction mechanisms.

Abstract: Evaluating and iterating upon recommender systems is crucial, yet traditional A/B testing is resource-intensive, and offline methods struggle with dynamic user-platform interactions. While agent-based simulation is promising, existing platforms often lack a mechanism for user actions to dynamically reshape the environment. To bridge this gap, we introduce RecInter, a novel agent-based simulation platform for recommender systems featuring a robust interaction mechanism. In RecInter platform, simulated user actions (e.g., likes, reviews, purchases) dynamically update item attributes in real-time, and introduced Merchant Agents can reply, fostering a more realistic and evolving ecosystem. High-fidelity simulation is ensured through Multidimensional User Profiling module, Advanced Agent Architecture, and LLM fine-tuned on Chain-of-Thought (CoT) enriched interaction data. Our platform achieves significantly improved simulation credibility and successfully replicates emergent phenomena like Brand Loyalty and the Matthew Effect. Experiments demonstrate that this interaction mechanism is pivotal for simulating realistic system evolution, establishing our platform as a credible testbed for recommender systems research. Our codes are available at https://github.com/jinsong8/RecInter.

Method: Developed a representation-level analysis framework using PCA-based similarity/shift, centered kernel alignment (CKA), Fisher information, and mean PCA distance to measure representational drift across six unlearning methods, three data domains, and two LLMs.

Result: Identified four distinct forgetting regimes based on reversibility and catastrophicity. Found that achieving ideal irreversible, non-catastrophic forgetting is extremely challenging. Discovered one case of seemingly irreversible, targeted forgetting.

Conclusion: Current unlearning evaluation practices have fundamental gaps. The proposed representation-level framework provides a foundation for trustworthy unlearning assessment and reveals the difficulty of achieving genuine, irreversible forgetting in LLMs.

Abstract: Unlearning in large language models (LLMs) aims to remove specified data, but its efficacy is typically assessed with task-level metrics like accuracy and perplexity. We demonstrate that these metrics are often misleading, as models can appear to forget while their original behavior is easily restored through minimal fine-tuning. This phenomenon of \emph{reversibility} suggests that information is merely suppressed, not genuinely erased. To address this critical evaluation gap, we introduce a \emph{representation-level analysis framework}. Our toolkit comprises PCA-based similarity and shift, centered kernel alignment (CKA), and Fisher information, complemented by a summary metric, the mean PCA distance, to measure representational drift. Applying this framework across six unlearning methods, three data domains, and two LLMs, we identify four distinct forgetting regimes based on their \emph{reversibility} and \emph{catastrophicity}. Our analysis reveals that achieving the ideal state–irreversible, non-catastrophic forgetting–is exceptionally challenging. By probing the limits of unlearning, we identify a case of seemingly irreversible, targeted forgetting, offering new insights for designing more robust erasure algorithms. Our findings expose a fundamental gap in current evaluation practices and establish a representation-level foundation for trustworthy unlearning.

Method: Uses belief propagation on carefully constructed graphical models to capture local coherence between adjacent sentences while also grouping distant but semantically similar sentences.

Result: Experimental results on illustrative examples and long-form document datasets show favorable performance compared to competing approaches.

Conclusion: BP-Seg effectively addresses text segmentation by balancing local coherence and long-range semantic similarity through graphical modeling and belief propagation.

Abstract: Text segmentation based on the semantic meaning of sentences is a fundamental task with broad utility in many downstream applications. In this paper, we propose a graphical model-based unsupervised learning approach, named BP-Seg for efficient text segmentation. Our method not only considers local coherence, capturing the intuition that adjacent sentences are often more related, but also effectively groups sentences that are distant in the text yet semantically similar. This is achieved through belief propagation on the carefully constructed graphical models. Experimental results on both an illustrative example and a dataset with long-form documents demonstrate that our method performs favorably compared to competing approaches.

Method: Applied Information Bottleneck principle to quantitatively compare LLMs and humans, analyzing embeddings from 40+ LLMs against classic human categorization benchmarks and examining training dynamics.

Result: Three key findings: 1) LLMs align with human categories but miss fine-grained semantic distinctions; 2) LLMs achieve optimal information-theoretic compression while humans prioritize contextual richness; 3) Encoder models outperform decoder models in human alignment, suggesting distinct mechanisms for generation vs understanding.

Conclusion: LLMs and humans employ divergent strategies - LLMs optimize for compression efficiency while humans for adaptive utility, revealing fundamental differences between artificial and biological intelligence that can guide development of more human-aligned AI.

Abstract: Humans organize knowledge into compact categories that balance compression with semantic meaning preservation. Large Language Models (LLMs) demonstrate striking linguistic abilities, yet whether they achieve this same balance remains unclear. We apply the Information Bottleneck principle to quantitatively compare how LLMs and humans navigate this compression-meaning trade-off. Analyzing embeddings from 40+ LLMs against classic human categorization benchmarks, we uncover three key findings. First, LLMs broadly align with human categories but miss fine-grained semantic distinctions crucial for human understanding. Second, LLMs demonstrate aggressive statistical compression, achieving ``optimal’’ information-theoretic efficiency, while humans prioritize contextual richness and adaptive flexibility. Third, encoder models surprisingly outperform decoder models in human alignment, suggesting that generation and understanding rely on distinct mechanisms in current architectures. In addition, training dynamics analysis reveals that conceptual structure develops in distinct phases: rapid initial formation followed by architectural reorganization, with semantic processing migrating from deeper to mid-network layers as models discover more efficient encoding. These divergent strategies, where LLMs optimize for compression and humans for adaptive utility, reveal fundamental differences between artificial and biological intelligence, guiding development toward more human-aligned AI.

Method: Uses dense passage retrieval to get associated knowledge, introduces knowledge integration model that incorporates retrieval knowledge into instructions during fine-tuning, and employs ensemble-based generation for controllable output with text consistency (coherence, fluency, answer format assurance).

Result: Extensive experiments on TriviaQA, MSMARCO, and CMRC2018 datasets demonstrate GenKI’s effectiveness compared to state-of-the-art baselines. Ablation studies show linear relationship between retrieved knowledge frequency and accurate knowledge recall.

Conclusion: GenKI successfully addresses knowledge integration and controllable generation challenges in OpenQA, showing improved performance across diverse datasets and revealing insights about knowledge recall patterns.

Abstract: Open-domain question answering (OpenQA) represents a cornerstone in natural language processing (NLP), primarily focused on extracting answers from unstructured textual data. With the rapid advancements in Large Language Models (LLMs), LLM-based OpenQA methods have reaped the benefits of emergent understanding and answering capabilities enabled by massive parameters compared to traditional methods. However, most of these methods encounter two critical challenges: how to integrate knowledge into LLMs effectively and how to adaptively generate results with specific answer formats for various task situations. To address these challenges, we propose a novel framework named GenKI, which aims to improve the OpenQA performance by exploring Knowledge Integration and controllable Generation on LLMs simultaneously. Specifically, we first train a dense passage retrieval model to retrieve associated knowledge from a given knowledge base. Subsequently, we introduce a novel knowledge integration model that incorporates the retrieval knowledge into instructions during fine-tuning to intensify the model. Furthermore, to enable controllable generation in LLMs, we leverage a certain fine-tuned LLM and an ensemble based on text consistency incorporating all coherence, fluency, and answer format assurance. Finally, extensive experiments conducted on the TriviaQA, MSMARCO, and CMRC2018 datasets, featuring diverse answer formats, have demonstrated the effectiveness of GenKI with comparison of state-of-the-art baselines. Moreover, ablation studies have disclosed a linear relationship between the frequency of retrieved knowledge and the model’s ability to recall knowledge accurately against the ground truth. Our code of GenKI is available at https://github.com/USTC-StarTeam/GenKI

Method: Created BiomedSQL benchmark with 68,000 question/SQL query/answer triples grounded in a harmonized BigQuery knowledge base integrating gene-disease associations, omics data causal inference, and drug approval records. Evaluated various LLMs across prompting strategies and interaction paradigms.

Result: GPT-o3-mini achieved 59.0% execution accuracy, while the custom multi-step agent BMSQL reached 62.6%, both significantly below the expert baseline of 90.0%.

Conclusion: BiomedSQL provides a foundation for advancing text-to-SQL systems capable of supporting scientific discovery through robust reasoning over structured biomedical knowledge bases, with current systems showing substantial room for improvement.

Abstract: Biomedical researchers increasingly rely on large-scale structured databases for complex analytical tasks. However, current text-to-SQL systems often struggle to map qualitative scientific questions into executable SQL, particularly when implicit domain reasoning is required. We introduce BiomedSQL, the first benchmark explicitly designed to evaluate scientific reasoning in text-to-SQL generation over a real-world biomedical knowledge base. BiomedSQL comprises 68,000 question/SQL query/answer triples grounded in a harmonized BigQuery knowledge base that integrates gene-disease associations, causal inference from omics data, and drug approval records. Each question requires models to infer domain-specific criteria, such as genome-wide significance thresholds, effect directionality, or trial phase filtering, rather than rely on syntactic translation alone. We evaluate a range of open- and closed-source LLMs across prompting strategies and interaction paradigms. Our results reveal a substantial performance gap: GPT-o3-mini achieves 59.0% execution accuracy, while our custom multi-step agent, BMSQL, reaches 62.6%, both well below the expert baseline of 90.0%. BiomedSQL provides a new foundation for advancing text-to-SQL systems capable of supporting scientific discovery through robust reasoning over structured biomedical knowledge bases. Our dataset is publicly available at https://huggingface.co/datasets/NIH-CARD/BiomedSQL, and our code is open-source at https://github.com/NIH-CARD/biomedsql.

Method: Created EmoBench-UA dataset through crowdsourcing using Toloka.ai platform, adapted annotation schema from English-centric works, and evaluated linguistic-based baselines, synthetic data translated from English, and large language models.

Result: The findings highlight the challenges of emotion classification in non-mainstream languages like Ukrainian.

Conclusion: There is a need for further development of Ukrainian-specific models and training resources for emotion classification.

Abstract: While Ukrainian NLP has seen progress in many texts processing tasks, emotion classification remains an underexplored area with no publicly available benchmark to date. In this work, we introduce EmoBench-UA, the first annotated dataset for emotion detection in Ukrainian texts. Our annotation schema is adapted from the previous English-centric works on emotion detection (Mohammad et al., 2018; Mohammad, 2022) guidelines. The dataset was created through crowdsourcing using the Toloka.ai platform ensuring high-quality of the annotation process. Then, we evaluate a range of approaches on the collected dataset, starting from linguistic-based baselines, synthetic data translated from English, to large language models (LLMs). Our findings highlight the challenges of emotion classification in non-mainstream languages like Ukrainian and emphasize the need for further development of Ukrainian-specific models and training resources.

Method: Two approaches: 1) Distillation using reasoning traces from DeepSeek-R1 to create Table-R1-SFT model, 2) RLVR with task-specific verifiable rewards using GRPO algorithm to create Table-R1-Zero model.

Result: Table-R1-Zero model matches or exceeds GPT-4.1 and DeepSeek-R1 performance using only 7B parameters, with strong out-of-domain generalization. Shows benefits of instruction tuning, architecture choices, and emergent table reasoning skills.

Conclusion: Inference-time scaling through post-training strategies enables smaller models to achieve frontier-level performance on table reasoning tasks, with RLVR showing particularly strong results and generalization capabilities.

Abstract: In this work, we present the first study to explore inference-time scaling on table reasoning tasks. We develop and evaluate two post-training strategies to enable inference-time scaling: distillation from frontier model reasoning traces and reinforcement learning with verifiable rewards (RLVR). For distillation, we introduce a large-scale dataset of reasoning traces generated by DeepSeek-R1, which we use to fine-tune LLMs into the Table-R1-SFT model. For RLVR, we propose task-specific verifiable reward functions and apply the GRPO algorithm to obtain the Table-R1-Zero model. We evaluate our Table-R1-series models across diverse table reasoning tasks, including short-form QA, fact verification, and free-form QA. Notably, the Table-R1-Zero model matches or exceeds the performance of GPT-4.1 and DeepSeek-R1, while using only a 7B-parameter LLM. It also demonstrates strong generalization to out-of-domain datasets. Extensive ablation and qualitative analyses reveal the benefits of instruction tuning, model architecture choices, and cross-task generalization, as well as emergence of essential table reasoning skills during RL training.

Method: During SFT stage, incorporated high-quality textual reasoning data and general multimodal data alongside medical data. Synthesized reflective-pattern-injected CoT samples to equip models with initial reflective reasoning capabilities. Developed InfiMed-SFT-3B and InfiMed-RL-3B models.

Result: InfiMed-RL-3B achieved 59.2% average accuracy across seven multimodal medical benchmarks, outperforming larger models like InternVL3-8B (57.3%). Used 188K samples in SFT phase and 36K in RLVR phase.

Conclusion: The proposed training strategies effectively advance MLLM performance in medical scenarios, with both SFT and RLVR phases contributing to superior results.

Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in domains such as visual understanding and mathematical reasoning. However, their application in the medical domain is constrained by two key challenges: (1) multimodal medical datasets are scarce and often contain sparse information, limiting reasoning depth; and (2) Reinforcement Learning with Verifiable Rewards (RLVR), though effective in general domains, cannot reliably improve model performance in the medical domain. To overcome these challenges, during the supervised fine-tuning (SFT) stage, we incorporate high-quality textual reasoning data and general multimodal data alongside multimodal medical data to efficiently enhance foundational medical capabilities and restore the base model’s reasoning ability. Moreover, considering that there are some multimodal medical datasets with sparse information, we further synthesize reflective-pattern-injected chain-of-thought (CoT) in addition to general CoT samples, equipping the model with initial reflective reasoning capabilities that provide a structured foundation for subsequent RLVR training. Finally, we introduce our InfiMed-Series models, InfiMed-SFT-3B and InfiMed-RL-3B, both of which deliver state-of-the-art performance across seven multimodal medical benchmarks. Notably, InfiMed-RL-3B achieves an average accuracy of 59.2%, outperforming even larger models like InternVL3-8B, which achieves 57.3%. Specifically, during the SFT phase, we utilized 188K samples, while the RLVR phase incorporated 36K samples, demonstrating the efficacy of both training strategies in achieving superior performance. We also conducted a series of extensive experiments, which provide valuable insights that contribute to advancing the performance of MLLMs in medical scenarios.

Method: Developed RARE framework with knowledge-graph-driven synthesis pipeline (RARE-Get) that automatically extracts relations from time-sensitive corpora and generates multi-level question sets without manual intervention.

Result: Created RARE-Set dataset with 527 finance/economics/policy documents and 48,295 evolving questions; found RAG systems unexpectedly sensitive to perturbations and consistently less robust on multi-hop queries.

Conclusion: RAG systems show significant vulnerability to perturbations, particularly on complex multi-hop queries, highlighting the need for more robust retrieval-augmented generation approaches.

Abstract: Retrieval-Augmented Generation (RAG) enhances recency and factuality in answers. However, existing evaluations rarely test how well these systems cope with real-world noise, conflicting between internal and external retrieved contexts, or fast-changing facts. We introduce Retrieval-Aware Robustness Evaluation (RARE), a unified framework and large-scale benchmark that jointly stress-tests query and document perturbations over dynamic, time-sensitive corpora. One of the central features of RARE is a knowledge-graph-driven synthesis pipeline (RARE-Get) that automatically extracts single and multi-hop relations from the customized corpus and generates multi-level question sets without manual intervention. Leveraging this pipeline, we construct a dataset (RARE-Set) spanning 527 expert-level time-sensitive finance, economics, and policy documents and 48295 questions whose distribution evolves as the underlying sources change. To quantify resilience, we formalize retrieval-conditioned robustness metrics (RARE-Met) that capture a model’s ability to remain correct or recover when queries, documents, or real-world retrieval results are systematically altered. Our findings reveal that RAG systems are unexpectedly sensitive to perturbations. Moreover, they consistently demonstrate lower robustness on multi-hop queries compared to single-hop queries across all domains.

Method: Graph probing method for uncovering functional connectivity of LLM neurons and relating it to language generation performance, tested across diverse LLM families and scales with interventional experiments.

Result: Discovered universal predictability of next-token prediction performance using only neural topology, even with just 1% of connections. Topology probing outperforms activation probing by up to 130.4%. Identified default networks and hub neurons, with causal evidence showing LLMs exploit topological information.

Conclusion: Neural topology contains orders of richer information about LLM performance than neural activation and can be leveraged to improve efficiency, reliability, and safety through applications in model pruning, hallucination detection, and LLM fingerprinting.

Abstract: Probing large language models (LLMs) has yielded valuable insights into their internal mechanisms by linking neural activations to interpretable semantics. However, the complex mechanisms that link neuron’s functional co-activation with the emergent model capabilities remains largely unknown, hindering a deeper understanding and safer development of LLMs. In this work, we introduce graph probing, a method for uncovering the functional connectivity of LLM neurons and relating it to language generation performance. By probing models across diverse LLM families and scales, we discover a universal predictability of next-token prediction performance using only neural topology, which persists even when retaining just 1% of neuron connections. Strikingly, probing on topology outperforms probing on activation by up to 130.4%, suggesting that neural topology contains orders of richer information of LLM performance than neural activation, which can be easily extracted with simple linear or MLP probes. To explain the dependence between neural topology and language performance, we identify default networks and hub neurons in LLMs and provide causal evidence by interventional experiments on multiple benchmarks, showing that LLMs actually exploit these topological information. Further analyses suggest that neural topology can be effectively leveraged to improve the efficiency, reliability, and safety of LLMs through proof-of-concept applications in model pruning, hallucination detection, and LLM fingerprinting. Codes and data for the graph probing toolbox are available at https://github.com/DavyMorgan/llm-graph-probing.

Method: Uses joint sampling to generate paired responses with/without retrieval, learning local and global advantages to quantify when to ignore misleading context versus adopt it. Employs asymmetric advantage transformation to amplify exploratory behaviors toward parametric knowledge.

Result: Significantly improves robustness and reasoning accuracy in knowledge conflict scenarios and general RAG scenarios, outperforming SOTA baselines by 23% in counterfactual scenarios, with no degradation when retrieved context is accurate.

Conclusion: The framework effectively trains LLMs to use parametric knowledge to resist contextual interference while still exploiting external context when reliably helpful, enhancing RAG system reliability.

Abstract: Retrieval-augmented generation (RAG) improves performance on knowledge-intensive tasks but can be derailed by wrong, irrelevant, or conflicting retrieved text, causing models to rely on inaccurate evidence and cascade errors. We propose Knowledgeable-R1, a reinforcement-learning framework that explicitly trains large language models to use parametric knowledge (PK) to resist contextual interference while still exploiting external context when it is reliably helpful. Knowledgeable-R1 introduces a joint sampling scheme that generates paired responses with and without retrieval, and learns both local advantages (within each decoding regime) and global advantages under the same input to quantify when to ignore misleading context versus adopt it. We employ an asymmetric advantage transformation that amplifies exploratory behaviors toward parametric knowledge. Experiments show that \method significantly improves robustness and reasoning accuracy in knowledge conflict scenarios and general RAG scenarios, outperforming SOTA baselines by 23% in counterfactual scenarios, and without degradation when the retrieved context is fully accurate.Our code are available at https://github.com/lcy80366872/knowledgeable-R1.

Method: Uses QA-LIGN to decompose rewards into interpretable principle-specific evaluations through structured natural language programs, employing a draft-critic-revise pipeline with symbolic evaluation against rubrics during GRPO training.

Result: Applied to Llama-3.1-8B-Instruct, reduces attack success rates by up to 68.7% while maintaining 0.67% false refusal rate, achieving Pareto optimal safety-helpfulness performance and outperforming DPO and GRPO with state-of-the-art reward models.

Conclusion: Making reward signals interpretable and modular improves alignment effectiveness, suggesting transparency enhances LLM safety.

Abstract: Alignment of large language models (LLMs) with principles like helpfulness, honesty, and harmlessness typically relies on scalar rewards that obscure which objectives drive the training signal. We introduce QA-LIGN, which decomposes monolithic rewards into interpretable principle-specific evaluations through structured natural language programs. Models learn through a draft, critique, and revise pipeline, where symbolic evaluation against the rubrics provides transparent feedback for both initial and revised responses during GRPO training. Applied to uncensored Llama-3.1-8B-Instruct, QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate, achieving Pareto optimal safety-helpfulness performance and outperforming both DPO and GRPO with state-of-the-art reward models given equivalent training. These results demonstrate that making reward signals interpretable and modular improves alignment effectiveness, suggesting transparency enhances LLM safety.

Method: CoPe uses contrastive decoding to maximize users’ implicit reward signals after parameter-efficient fine-tuning on user-specific data, without requiring external reward models or additional training.

Result: CoPe improves personalization by an average of 10.57% in ROUGE-L across five open-ended personalized text generation tasks.

Conclusion: CoPe demonstrates that effective decoding-time algorithms can significantly enhance LLM personalization without external resources or additional training procedures.

Abstract: As large language models (LLMs) are progressively deployed in various real-world applications, personalization of LLMs has become increasingly important. While various approaches to LLM personalization such as prompt-based and training-based methods have been actively explored, the development of effective decoding-time algorithms remains largely overlooked, despite their demonstrated potential. In this paper, we propose CoPe (Contrasting Personal Preference), a novel decoding-time approach applied after performing parameter-efficient fine-tuning (PEFT) on user-specific data. Our core idea is to leverage reward-guided decoding specifically for personalization by maximizing each user’s implicit reward signal. We evaluate CoPe across five open-ended personalized text generation tasks. Our empirical results demonstrate that CoPe achieves strong performance, improving personalization by an average of 10.57% in ROUGE-L, without relying on external reward models or additional training procedures.

Method: Created MUCAR benchmark with two components: 1) multilingual dataset where ambiguous text is resolved by visual context, and 2) dual-ambiguity dataset pairing ambiguous images with ambiguous text to yield clear interpretations through mutual disambiguation.

Result: Evaluation of 19 state-of-the-art multimodal models (open-source and proprietary) shows substantial performance gaps compared to human-level performance in ambiguity resolution.

Conclusion: Current multimodal models have significant limitations in cross-modal ambiguity comprehension, highlighting the need for more sophisticated methods to advance multimodal reasoning capabilities.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated significant advances across numerous vision-language tasks. MLLMs have shown promising capability in aligning visual and textual modalities, allowing them to process image-text pairs with clear and explicit meanings. However, resolving the inherent ambiguities present in real-world language and visual contexts remains a challenge. Existing multimodal benchmarks typically overlook linguistic and visual ambiguities, relying mainly on unimodal context for disambiguation and thus failing to exploit the mutual clarification potential between modalities. To bridge this gap, we introduce MUCAR, a novel and challenging benchmark designed explicitly for evaluating multimodal ambiguity resolution across multilingual and cross-modal scenarios. MUCAR includes first a multilingual dataset where ambiguous textual expressions are uniquely resolved by corresponding visual contexts, and second a dual-ambiguity dataset that systematically pairs ambiguous images with ambiguous textual contexts, with each combination carefully constructed to yield a single, clear interpretation through mutual disambiguation. Extensive evaluations involving 19 state-of-the-art multimodal models–encompassing both open-source and proprietary architectures–reveal substantial gaps compared to human-level performance, highlighting the need for future research into more sophisticated cross-modal ambiguity comprehension methods, further pushing the boundaries of multimodal reasoning.

Method: Uses 0.5B parameter architecture with mean-pooling and bidirectional attention; implements progressive multi-stage training (pre-training, fine-tuning, contrastive distillation) with focal-style reweighting and hard-negative mining; curates over 20 pre-training and 100 fine-tuning categories with task-specific instructions and multi-class labeling.

Result: Achieves state-of-the-art performance on Massive Text Embedding Benchmark, outperforming comparable-sized models and rivaling models 3-26x larger.

Conclusion: Sets new standard for versatile compact embedding models under 1B parameters, demonstrating that superior training techniques and data quality can overcome size limitations.

Abstract: Recent advancements in Large Language Models (LLMs)-based text embedding models primarily focus on data scaling or synthesis, yet limited exploration of training techniques and data quality, thereby constraining performance. In this work, we propose KaLM-Embedding-V2, a series of versatile and compact embedding models, systematically incentivizing advanced embedding capability in LLMs by superior training techniques and high-quality data. For model architecture, we implement the models on a 0.5B compact size with simple mean-pooling to produce fixed-length embeddings and remove the causal attention mask to enable fully bidirectional representation learning. For training techniques, we propose a progressive multi-stage training pipeline: pre-training on weakly supervised large-scale datasets, fine-tuning with supervised high-quality datasets, and contrastive distillation with fine-grained soft signals, integrated with focal-style reweighting and online hard-negative mixing to emphasize difficult samples and enrich hard negatives, respectively. For training data, we curate over 20 categories for pre-training and 100 categories for fine-tuning and contrastive distillation, to improve both performance and generalization, leveraging task-specific instructions, hard-negative mining, and example-based multi-class labeling to ensure high quality. Combining these techniques, our KaLM-Embedding-V2 series achieves state-of-the-art performance on the Massive Text Embedding Benchmark, outperforming models of comparable size and rivaling models 3-26x larger, setting a new standard for versatile and compact embedding models under 1B parameters. The code, data, and models will be publicly available to facilitate academic research.

Method: Proposed VAT-KG construction pipeline with stringent filtering and alignment steps for cross-modal knowledge alignment, enabling automatic generation from any multimodal dataset. Also introduced a novel multimodal RAG framework for concept-level knowledge retrieval.

Result: Experiments on question answering tasks across various modalities demonstrated VAT-KG’s effectiveness in supporting MLLMs and its practical value in unifying multimodal knowledge.

Conclusion: VAT-KG successfully addresses limitations of existing MMKGs by providing comprehensive multimodal coverage and enabling more grounded reasoning through concept-level knowledge retrieval.

Abstract: Multimodal Knowledge Graphs (MMKGs), which represent explicit knowledge across multiple modalities, play a pivotal role by complementing the implicit knowledge of Multimodal Large Language Models (MLLMs) and enabling more grounded reasoning via Retrieval Augmented Generation (RAG). However, existing MMKGs are generally limited in scope: they are often constructed by augmenting pre-existing knowledge graphs, which restricts their knowledge, resulting in outdated or incomplete knowledge coverage, and they often support only a narrow range of modalities, such as text and visual information. These limitations restrict applicability to multimodal tasks, particularly as recent MLLMs adopt richer modalities like video and audio. Therefore, we propose the Visual-Audio-Text Knowledge Graph (VAT-KG), the first concept-centric and knowledge-intensive multimodal knowledge graph that covers visual, audio, and text information, where each triplet is linked to multimodal data and enriched with detailed descriptions of concepts. Specifically, our construction pipeline ensures cross-modal knowledge alignment between multimodal data and fine-grained semantics through a series of stringent filtering and alignment steps, enabling the automatic generation of MMKGs from any multimodal dataset. We further introduce a novel multimodal RAG framework that retrieves detailed concept-level knowledge in response to queries from arbitrary modalities. Experiments on question answering tasks across various modalities demonstrate the effectiveness of VAT-KG in supporting MLLMs, highlighting its practical value in unifying and leveraging multimodal knowledge.

Method: Systematically curates real-world chat data with diverse speaker attributes and acoustic conditions, augments with speech-specific phenomena, and designs query-aware evaluation using customized checklists and prompts for accurate automatic assessment.

Result: Comprehensive testing reveals significant performance differences across speech scenarios, with query-aware evaluation enabling finer-grained assessment of various speech-specific challenges.

Conclusion: The benchmark provides valuable insights for speech model development and evaluation, addressing critical gaps in current assessment methods for end-to-end speech LLMs.

Abstract: Recent multi-modal Large Language Models (LLMs) such as GPT-4o have demonstrated strong capabilities of direct speech interaction. However, the lack of specialized and comprehensive benchmarks for end-to-end speech LLM evaluation hinders optimizing the user experience of Audio LLMs in real-world applications. Existing evaluation methods often adapt text-based benchmarks, overlooking speech’s unique characteristics and challenges, including prosody, homophones, stuttering, and differing user expectations. Here, we introduce the first comprehensive benchmark designed to systematically evaluate end-to-end speechLLMs in practical speech conversations. We systematically curate real-world chat data relevant to spoken scenarios, introduce diversity in speaker attributes and acoustic conditions, and augment the dataset with speech-specific phenomena. We further design a query-aware evaluation method to use customized evaluation checklists and prompts to enhance the accuracy of automatic evaluation. We conduct comprehensive testing and detailed analysis of various mainstream speech models, revealing significant differences in model performance across different speech scenarios. The use of query-aware evaluation further enables a finer-grained assessment under various speech-specific scenarios. Our benchmark can provide valuable insights for speech model development and evaluation.

Method: Used jigsaw puzzles as a novel task absent from pretraining corpora, systematically studied SFT and RFT on Qwen2.5-VL models, analyzed learning dynamics through magnitude and direction of training data influence.

Result: RFT mainly reinforces correct samples aligned with base model’s probability landscape, causing weaker interference with prior knowledge. Training on RFT-simulated rollouts allows SFT to preserve knowledge while learning new tasks.

Conclusion: Distribution of training data, not algorithmic differences, plays central role in forgetting; RFT shows potential for stable continual learning in multimodal LLMs.

Abstract: Post-training algorithms such as Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT) are widely used to adapt multimodal large language models to downstream tasks. While effective at task adaptation, their impact on prior knowledge remains unclear. In this paper, we introduce jigsaw puzzles as a novel task absent from existing pretraining corpora and systematically study the behavior of SFT and RFT on open-source multimodal model, Qwen2.5-VL series. Our experiments reveal a sharp trade-off: SFT enables rapid task acquisition but leads to catastrophic forgetting, whereas RFT learns more slowly but maintains prior knowledge. We study this phenomenon through learning dynamics by examining both the magnitude and direction of how training data influence prior knowledge. Our analysis shows that RFT mainly reinforces correct samples naturally aligned with the base model’s probability landscape, leading to weaker interference with prior knowledge. Moreover, training on RFT-simulated rollouts, which exert a small magnitude of influence and are well aligned in direction to prior knowledge, allows SFT to preserve prior knowledge better while rapidly learning new tasks. These findings suggest that distribution of training data, rather than algorithmic differences, plays a central role in forgetting, and highlight RFT’s potential for stable continual learning in multimodal large language models.

Method: S³ initiates a schematic template directly in the output context as a starting state for diffusion LLMs, providing a more robust alternative to prompt optimization by leveraging the model’s innate reverse reasoning capability.

Result: Experiments show S³ substantially improves dLLMs’ performance in controllable generation across structure adherence, content fidelity, and faithfulness metrics.

Conclusion: The method establishes new perspectives and practical pathways for deploying language models in controllable generation tasks, unlocking dLLMs’ potential in this domain.

Abstract: Controllable generation is a fundamental task in NLP with many applications, providing a basis for function calling to agentic communication. However, even state-of-the-art autoregressive Large Language Models (LLMs) today exhibit unreliability when required to generate structured output. Inspired by the current new diffusion-based large language models (dLLM), we realize that the architectural difference, especially the global information-sharing mechanism for language modeling, may be the key to unlock next-level controllable generation. To explore the possibility, we propose Self-adaptive Schema Scaffolding ($S^3$), a novel framework that enables dLLM to stably generate reliable structured outputs (e.g., JSON) by utilizing its innate reverse reasoning capability and global context awareness. $S^3$ initiates a schematic template directly in the output context as a starting state for dLLM, offering a more robust and general method than intricate prompt optimization. Experiments demonstrate that our method substantially unlocks the dLLM’s potential in controllable generation in terms of structure adherence, content fidelity, and faithfulness. These results establish new perspectives and practical pathways for deploying language models in controllable generation tasks.

Method: The authors transform existing long-context datasets and incorporate newly constructed datasets into a multi-turn format to simulate incremental information processing. The benchmark systematically covers all four memory competencies through careful dataset selection and curation.

Result: Evaluation of diverse memory agents (context-based, RAG systems, agents with external memory, and tool-integrated agents) shows that current methods fail to master all four memory competencies, highlighting limitations in existing approaches.

Conclusion: There is a significant need for further research into comprehensive memory mechanisms for LLM agents, as current methods are insufficient for handling the full spectrum of memory capabilities required for effective agent performance.

Abstract: Recent benchmarks for Large Language Model (LLM) agents primarily focus on evaluating reasoning, planning, and execution capabilities, while another critical component-memory, encompassing how agents memorize, update, and retrieve long-term information-is under-evaluated due to the lack of benchmarks. We term agents with memory mechanisms as memory agents. In this paper, based on classic theories from memory science and cognitive science, we identify four core competencies essential for memory agents: accurate retrieval, test-time learning, long-range understanding, and selective forgetting. Existing benchmarks either rely on limited context lengths or are tailored for static, long-context settings like book-based QA, which do not reflect the interactive, multi-turn nature of memory agents that incrementally accumulate information. Moreover, no existing benchmarks cover all four competencies. We introduce MemoryAgentBench, a new benchmark specifically designed for memory agents. Our benchmark transforms existing long-context datasets and incorporates newly constructed datasets into a multi-turn format, effectively simulating the incremental information processing characteristic of memory agents. By carefully selecting and curating datasets, our benchmark provides comprehensive coverage of the four core memory competencies outlined above, thereby offering a systematic and challenging testbed for assessing memory quality. We evaluate a diverse set of memory agents, ranging from simple context-based and retrieval-augmented generation (RAG) systems to advanced agents with external memory modules and tool integration. Empirical results reveal that current methods fall short of mastering all four competencies, underscoring the need for further research into comprehensive memory mechanisms for LLM agents.

Method: Proposed M2A method combines multi-scale multilingual alignment with language-consistency rewards on machine-translated questions to train models to reason directly in target languages.

Result: M2A significantly enhances multilingual reasoning fidelity in both mathematical and factual reasoning tasks across multiple languages.

Conclusion: Reasoning-aware multilingual reinforcement learning is crucial for robust cross-lingual generalization in LLMs.

Abstract: Large Language Models (LLMs) have achieved strong performance in domains like mathematics, factual question answering, and code generation, yet their ability to reason on these tasks in different languages remains underdeveloped. Especially for low-resource languages such as Swahili or Thai, LLMs can often misinterpret prompts or default to reasoning in English. This implicit bias toward high-resource languages undermines factual accuracy, interpretability, and trust. We propose M2A, a novel method that combines multi-scale multilingual alignment with language-consistency rewards on machine-translated questions, training models to reason directly and accurately in the target language. Furthermore, existing multilingual benchmarks only evaluate on final answers, overlooking whether reasoning occurs in the intended language. To close this gap, we introduce GeoFact-X, a geography-based multilingual factual reasoning benchmark together with reasoning traces in five languages: English, Hindi, Japanese, Swahili, and Thai. Our results show that M2A significantly enhances multilingual reasoning fidelity in both mathematical and factual reasoning tasks, highlighting that reasoning-aware multilingual reinforcement learning is crucial for robust cross-lingual generalization. https://jd730.github.io/projects/M2A_GeoFact-X

Method: Used five LLMs and three RLLMs to assess the effects of prompting methods, agentic frameworks, and multilingual alignment methods on financial question-answering tasks through systematic evaluation.

Result: (1) Prompting methods and agent frameworks enhance LLM performance by simulating Long CoT; (2) RLLMs’ inherent Long CoT capabilities limit conventional methods’ effectiveness; (3) Multilingual alignment methods mainly improve LLM multilingual performance by extending reasoning length, with minimal benefits for RLLMs.

Conclusion: The study provides important references for enhancing LLM and RLLM performance in financial question-answering and discusses strategies that may inspire future improvements in this domain.

Abstract: Recently, the development of large language models (LLMs) and reasoning large language models (RLLMs) have gained considerable attention from many researchers. RLLMs enhance the reasoning capabilities of LLMs through Long Chain-of-Thought (Long CoT) processes, significantly improving the performance of LLMs in addressing complex problems. However, there are few works that systematically explore what methods can fully unlock the performance of LLMs and RLLMs within the financial domain. To investigate the impact of various methods on LLMs and RLLMs, we utilize five LLMs and three RLLMs to assess the effects of prompting methods, agentic frameworks, and multilingual alignment methods on financial question-answering tasks. Our research findings indicate: (1) Current prompting methods and agent frameworks enhance the performance of LLMs in financial question answering by simulating Long CoT; (2) RLLMs possess inherent Long CoT capabilities, which limits the effectiveness of conventional methods in further enhancing their performance; (3) Current advanced multilingual alignment methods primarily improve the multilingual performance of LLMs by extending the reasoning length, which yields minimal benefits for RLLMs. Additionally, we discuss strategies for enhancing the performance of LLMs and RLLMs in financial question answering, which may serve as a inspiration for future improvements. We hope that this study can serve as an important reference for LLMs and RLLMs in the field of financial question answering.

Method: Constructs steering vectors from reasoning traces (from teacher models or human annotations) and applies them directly to the key-value cache in a one-shot manner to shift model behavior.

Result: Improves both qualitative reasoning structure and quantitative task performance on diverse reasoning benchmarks, scales to larger models, and provides gains on challenging datasets like GPQA and MATH.

Conclusion: Cache steering offers substantial advantages over prior activation steering techniques in inference latency, hyperparameter stability, and API integration, while enabling controllable transfer of reasoning styles for practical behavior-level guidance.

Abstract: We propose cache steering, a lightweight method for implicit steering of language models via a one-shot intervention applied directly to the key-value cache. To validate its effectiveness, we apply cache steering to induce chain-of-thought reasoning in small language models. Our approach constructs steering vectors from reasoning traces, obtained either from teacher models (e.g., GPT-4o) or existing human annotations, that shift model behavior toward more explicit, multi-step reasoning without fine-tuning or prompt modifications. Experimental evaluations on diverse reasoning benchmarks demonstrate that cache steering improves both the qualitative structure of model reasoning and quantitative task performance. Additional experiments show that the method also scales to larger models and yields further gains on challenging datasets such as GPQA and MATH. Compared to prior activation steering techniques that require continuous interventions, our one-shot cache steering offers substantial advantages in terms of inference latency, hyperparameter stability, and ease of integration with existing inference APIs. Beyond mere reasoning induction, we show that cache steering enables controllable transfer of reasoning styles (e.g., stepwise, causal, analogical), making it a practical tool for behavior-level guidance of language models.

Method: TAIL synthesizes chain-of-thoughts data that imitate Turing Machine execution through computer programs, using linear expansion into atomic states and explicit memory fetch mechanisms.

Result: TAIL significantly improves length generalization and performance of Qwen2.5-7B on various tasks using only synthetic data, surpassing previous methods and DeepSeek-R1.

Conclusion: Turing Machine concepts (not thinking styles) are key for length generalization, enabling read-write behaviors in attention layers, providing a promising direction for LLM reasoning from synthetic data.

Abstract: Length generalization, the ability to solve problems of longer sequences than those observed during training, poses a core challenge of Transformer-based large language models (LLM). Although existing studies have predominantly focused on data-driven approaches for arithmetic operations and symbolic manipulation tasks, these approaches tend to be task-specific with limited overall performance. To pursue a more general solution, this paper focuses on a broader case of reasoning problems that are computable, i.e., problems that algorithms can solve, thus can be solved by the Turing Machine. From this perspective, this paper proposes Turing MAchine Imitation Learning (TAIL) to improve the length generalization ability of LLMs. TAIL synthesizes chain-of-thoughts (CoT) data that imitate the execution process of a Turing Machine by computer programs, which linearly expands the reasoning steps into atomic states to alleviate shortcut learning and explicit memory fetch mechanism to reduce the difficulties of dynamic and long-range data access in elementary operations. To validate the reliability and universality of TAIL, we construct a challenging synthetic dataset covering 8 classes of algorithms and 18 tasks. Without bells and whistles, TAIL significantly improves the length generalization ability as well as the performance of Qwen2.5-7B on various tasks using only synthetic data, surpassing previous methods and DeepSeek-R1. The experimental results reveal that the key concepts in the Turing Machine, instead of the thinking styles, are indispensable for TAIL for length generalization, through which the model exhibits read-and-write behaviors consistent with the properties of the Turing Machine in their attention layers. This work provides a promising direction for future research in the learning of LLM reasoning from synthetic data.

Method: Two-phase approach: 1) Online sparsification during prefilling by dynamically selecting important attention matrix parts, 2) Progressive KV compression during decoding by adaptively maintaining relevant cache based on recent output tokens. Also introduces new benchmark with 11 multi-turn datasets.

Result: Extensive experiments show LoopServe consistently achieves superior effectiveness compared to existing baselines and significantly accelerates LLM inference across wide range of long-context dialogue tasks.

Conclusion: LoopServe provides an effective adaptive acceleration framework that outperforms existing methods in multi-turn dialogue scenarios while maintaining response quality.

Abstract: Multi-turn dialogues are essential in many real-world applications of large language models, such as chatbots and virtual assistants. As conversation histories become longer, existing large language models face increasing computational and memory challenges, which hinder their ability to provide efficient and responsive interactions. Most current acceleration methods either compress the context or optimize key value caching, but they often rely on fixed or position-based heuristics that do not adapt well to the dynamic and unpredictable patterns found in actual multi-turn conversations. As a result, these models cannot accurately identify and prioritize the most relevant context, leading to degraded response quality. In this paper, we present LoopServe, an adaptive dual-phase inference acceleration framework for large language models in multi-turn dialogues. LoopServe introduces two main innovations. First, it performs online sparsification during the prefilling phase by dynamically selecting the most important parts of the attention matrix for each new input. Second, it uses progressive key value compression during decoding by adaptively maintaining a relevant and efficient cache based on the most recently generated output tokens. We also propose a new benchmark with eleven multi-turn datasets that reflect realistic query positions and conversational dependencies. Extensive experiments demonstrate that LoopServe consistently achieves superior effectiveness compared to existing baselines and significantly accelerates LLM inference across a wide range of long-context dialogue tasks.

Method: WINO employs parallel draft-and-verify mechanism that aggressively drafts multiple tokens while simultaneously using bidirectional context to verify and re-mask suspicious tokens for refinement.

Result: WINO accelerates inference by 6× while improving accuracy by 2.58% on GSM8K math benchmark, and achieves 10× speedup with higher performance on Flickr30K captioning.

Conclusion: WINO decisively improves the quality-speed trade-off in DLLMs and provides superior performance compared to existing methods.

Abstract: Diffusion Large Language Models (DLLMs) have emerged as a compelling alternative to Autoregressive models, designed for fast parallel generation. However, existing DLLMs are plagued by a severe quality-speed trade-off, where faster parallel decoding leads to significant performance degradation. We attribute this to the irreversibility of standard decoding in DLLMs, which is easily polarized into the wrong decoding direction along with early error context accumulation. To resolve this, we introduce Wide-In, Narrow-Out (WINO), a training-free decoding algorithm that enables revokable decoding in DLLMs. WINO employs a parallel draft-and-verify mechanism, aggressively drafting multiple tokens while simultaneously using the model’s bidirectional context to verify and re-mask suspicious ones for refinement. Verified in open-source DLLMs like LLaDA and MMaDA, WINO is shown to decisively improve the quality-speed trade-off. For instance, on the GSM8K math benchmark, it accelerates inference by 6$\times$ while improving accuracy by 2.58%; on Flickr30K captioning, it achieves a 10$\times$ speedup with higher performance. More comprehensive experiments are conducted to demonstrate the superiority and provide an in-depth understanding of WINO.

Method: Replace GRPO’s arithmetic mean with geometric mean of token-level rewards, which is less sensitive to outliers and maintains more stable importance sampling ratios.

Result: GMPO-7B improves average Pass@1 by up to 4.1% over GRPO on multiple mathematical reasoning benchmarks, outperforming state-of-the-art approaches.

Conclusion: GMPO provides a plug-and-play improvement to GRPO that enhances stability and performance through geometric mean optimization of token rewards.

Abstract: Group Relative Policy Optimization (GRPO) has significantly enhanced the reasoning capability of large language models by optimizing the arithmetic mean of token-level rewards. Unfortunately, GRPO is observed to suffer from unstable policy updates when facing tokens with outlier importance-weighted rewards, which manifest as extreme importance sampling ratios during training. In this study, we propose Geometric-Mean Policy Optimization (GMPO), with the aim to improve the stability of GRPO through suppressing token reward outliers. Instead of optimizing the arithmetic mean, GMPO maximizes the geometric mean of token-level rewards, which is inherently less sensitive to outliers and maintains a more stable range of importance sampling ratio. GMPO is plug-and-play-simply replacing GRPO’s arithmetic mean with the geometric mean of token-level rewards, as the latter is inherently less sensitive to outliers. GMPO is theoretically plausible-analysis reveals that both GMPO and GRPO are weighted forms of the policy gradient while the former enjoys more stable weights, which consequently benefits policy optimization and performance. Experiments on multiple mathematical reasoning benchmarks show that GMPO-7B improves the average Pass@1 of GRPO by up to 4.1%, outperforming many state-of-the-art approaches. Code is available at https://github.com/callsys/GMPO.

Method: Used persona-based LLM prompting to rewrite evaluation prompts with diverse writing styles while keeping semantic content identical.

Result: Writing style variations significantly impact LLM performance estimates, with certain styles consistently triggering either low or high performance across models and tasks.

Conclusion: Persona-based prompting offers a scalable method to augment benchmarks and improve external validity of LLM performance assessments across linguistic variations.

Abstract: Current benchmarks for evaluating Large Language Models (LLMs) often do not exhibit enough writing style diversity, with many adhering primarily to standardized conventions. Such benchmarks do not fully capture the rich variety of communication patterns exhibited by humans. Thus, it is possible that LLMs, which are optimized on these benchmarks, may demonstrate brittle performance when faced with “non-standard” input. In this work, we test this hypothesis by rewriting evaluation prompts using persona-based LLM prompting, a low-cost method to emulate diverse writing styles. Our results show that, even with identical semantic content, variations in writing style and prompt formatting significantly impact the estimated performance of the LLM under evaluation. Notably, we identify distinct writing styles that consistently trigger either low or high performance across a range of models and tasks, irrespective of model family, size, and recency. Our work offers a scalable approach to augment existing benchmarks, improving the external validity of the assessments they provide for measuring LLM performance across linguistic variations.

Method: Proposed AdaPlan paradigm for global planning-guided agents, and PilotRL framework using progressive reinforcement learning to train agents in three stages: following explicit guidance, optimizing plan quality, and joint optimization of planning-execution coordination.

Result: PilotRL achieves state-of-the-art performance with LLaMA3.1-8B-Instruct + PilotRL surpassing GPT-4o by 3.60% and showing 55.78% improvement over GPT-4o-mini at comparable parameter scale.

Conclusion: The adaptive global plan-based paradigm and progressive reinforcement learning framework effectively address limitations of current LLM agents, enabling better long-horizon decision-making and generalization capabilities.

Abstract: Large Language Models (LLMs) have shown remarkable advancements in tackling agent-oriented tasks. Despite their potential, existing work faces challenges when deploying LLMs in agent-based environments. The widely adopted agent paradigm ReAct centers on integrating single-step reasoning with immediate action execution, which limits its effectiveness in complex tasks requiring long-term strategic planning. Furthermore, the coordination between the planner and executor during problem-solving is also a critical factor to consider in agent design. Additionally, current approaches predominantly rely on supervised fine-tuning, which often leads models to memorize established task completion trajectories, thereby restricting their generalization ability when confronted with novel problem contexts. To address these challenges, we introduce an adaptive global plan-based agent paradigm AdaPlan, aiming to synergize high-level explicit guidance with execution to support effective long-horizon decision-making. Based on the proposed paradigm, we further put forward PilotRL, a global planning-guided training framework for LLM agents driven by progressive reinforcement learning. We first develop the model’s ability to follow explicit guidance from global plans when addressing agent tasks. Subsequently, based on this foundation, we focus on optimizing the quality of generated plans. Finally, we conduct joint optimization of the model’s planning and execution coordination. Experiments indicate that PilotRL could achieve state-of-the-art performances, with LLaMA3.1-8B-Instruct + PilotRL surpassing closed-sourced GPT-4o by 3.60%, while showing a more substantial gain of 55.78% comparing to GPT-4o-mini at a comparable parameter scale.

Method: Integrates LLM-guided MCTS with adaptive path evaluation, using LLM-based planner for relation selection, Transformer-based scorer for context-aware estimation, and dynamic pseudo-path refinement for continuous adaptation.

Result: Extensive experiments on multiple KGQA benchmarks show DAMR significantly outperforms SOTA methods.

Conclusion: DAMR enables efficient and context-aware KGQA through the integration of LLM-guided MCTS with adaptive evaluation mechanisms, addressing limitations of existing approaches.

Abstract: Knowledge Graph Question Answering (KGQA) aims to interpret natural language queries and perform structured reasoning over knowledge graphs by leveraging their relational and semantic structures to retrieve accurate answers. Existing methods primarily follow either the retrieve-then-reason paradigm, which relies on Graph Neural Networks or heuristic rules to extract static candidate paths, or dynamic path generation strategies that employ LLMs with prompting to jointly perform retrieval and reasoning. However, the former lacks adaptability due to static path extraction and the absence of contextual refinement, while the latter suffers from high computational costs and limited evaluation accuracy because of their dependence on fixed scoring functions and repeated LLM calls. To address these issues, this paper proposes Dynamically Adaptive MCTS-based Reasoning (DAMR), a novel framework that integrates LLM-guided Monte Carlo Tree Search (MCTS) with adaptive path evaluation to enable efficient and context-aware KGQA. DAMR leverages MCTS as a backbone, where an LLM-based planner selects the top-$k$ semantically relevant relations at each expansion step to effectively reduce the search space. To enhance evaluation accuracy, we introduce a lightweight Transformer-based scorer that performs context-aware plausibility estimation by jointly encoding the question and relation sequence through cross-attention, thereby capturing fine-grained semantic shifts during multi-hop reasoning. Furthermore, to mitigate the scarcity of high-quality supervision, DAMR incorporates a dynamic pseudo-path refinement mechanism that periodically generates training signals from partial paths explored during search, enabling the scorer to continually adapt to the evolving distribution of reasoning trajectories. Extensive experiments on multiple KGQA benchmarks show that DAMR significantly outperforms SOTA methods.

Method: Pretrain an MLP to imitate kNN retriever behavior on pretraining data, then integrate it with Transformer decoders through probability interpolation.

Result: 12.3% relative improvement on QA benchmarks, 5.2 points gain on general NLP tasks, 10 points reduction in hallucinations, and 2.5x faster inference than RAG.

Conclusion: Learning retrieval patterns parametrically bridges efficient inference and effective knowledge access, offering a practical alternative to RAG and fine-tuning.

Abstract: Modern approaches to enhancing Large Language Models’ factual accuracy and knowledge utilization face a fundamental trade-off: non-parametric retrieval-augmented generation (RAG) provides flexible access to external knowledge but suffers from high inference latency and shallow integration, while parametric fine-tuning methods like LoRA risk catastrophic forgetting and degraded general capabilities. In this work, we propose MLP Memory, a lightweight parametric module that learns to internalize retrieval patterns without explicit document access. By pretraining an MLP to imitate a $k$NN retriever’s behavior on the entire pretraining dataset, we create a differentiable memory component that captures the benefits of retrieval-based knowledge access in a fully parametric form. Our architecture integrates this pretrained MLP Memory with Transformer decoders through simple probability interpolation, achieving 12.3% relative improvement on five question-answering benchmarks and 5.2 points absolute gain across nine general NLP tasks, while reducing hallucinations by up to 10 points on HaluEval. Moreover, MLP Memory delivers 2.5$\times$ faster inference than RAG with superior accuracy. Our findings show that learning retrieval patterns parametrically bridges the gap between efficient inference and effective knowledge access, offering a practical alternative to both RAG and fine-tuning approaches.

Method: Systematic review of literature from 2020-2025, analyzing evaluation methods and mitigation techniques including instruction tuning, multi-agent reasoning, and RAG frameworks for external knowledge access.

Result: Key findings show limitations of current metrics, importance of validated external evidence, and improved factual consistency through domain-specific customization.

Conclusion: The review emphasizes the need for more accurate, understandable, and context-aware fact-checking frameworks to advance research toward more trustworthy LLM models.

Abstract: Large Language Models (LLMs) are trained on vast and diverse internet corpora that often include inaccurate or misleading content. Consequently, LLMs can generate misinformation, making robust fact-checking essential. This review systematically analyzes how LLM-generated content is evaluated for factual accuracy by exploring key challenges such as hallucinations, dataset limitations, and the reliability of evaluation metrics. The review emphasizes the need for strong fact-checking frameworks that integrate advanced prompting strategies, domain-specific fine-tuning, and retrieval-augmented generation (RAG) methods. It proposes five research questions that guide the analysis of the recent literature from 2020 to 2025, focusing on evaluation methods and mitigation techniques. Instruction tuning, multi-agent reasoning, and RAG frameworks for external knowledge access are also reviewed. The key findings demonstrate the limitations of current metrics, the importance of validated external evidence, and the improvement of factual consistency through domain-specific customization. The review underscores the importance of building more accurate, understandable, and context-aware fact-checking. These insights contribute to the advancement of research toward more trustworthy models.

Method: Introduces Dynamic Entropy Weighting with two algorithms: Group Token Policy Optimization (GTPO) assigns entropy-weighted rewards to each token, and Sequence-Level GRPO (GRPO-S). Uses policy entropy as heuristic for cognitive effort at pivotal reasoning junctures.

Result: Experimental results across challenging reasoning benchmarks show significant performance improvements over DAPO baseline, validating the effectiveness of entropy-weighting mechanism.

Conclusion: Dynamic Entropy Weighting enables true per-token credit assignment in RL for LLM reasoning, with policy entropy serving as a powerful learning signal that drives performance improvements in complex reasoning tasks.

Abstract: Reinforcement learning (RL) is a pivotal task for enhancing Large Language Model (LLM) reasoning. Conventional algorithms, however, typically adhere to a coarse-grained credit assignment paradigm, applying a uniform reward to all tokens in a sequence, a critical flaw in long-chain reasoning tasks. In this paper, we address this challenge and propose Dynamic Entropy Weighting, a novel mechanism that facilitates fine-grained rewards through two new algorithms: Group Token Policy Optimization (GTPO), which assigns an entropy-weighted reward to each token, and the analogous algorithm Sequence-Level GRPO (GRPO-S). Our approach is founded on the hypothesis that high policy entropy within a reasoning path is a powerful heuristic for cognitive effort at pivotal junctures, which can be repurposed into a learning signal. By repurposing policy entropy for reward shaping, we achieve true per-token credit assignment. Experimental results across challenging reasoning benchmarks validate the superiority of our approach, showing our methods significantly outperform a strong DAPO baseline and confirming our entropy-weighting mechanism as the key driver of this performance boost.

Method: Introduced Adaptive Self-Correction Chain-of-Thought (ASCoT) with two main components: Adaptive Verification Manager (AVM) that uses Positional Impact Score function I(k) to prioritize late-stage steps, and Multi-Perspective Self-Correction Engine (MSCE) that applies dual-path correction to identified failure parts.

Result: Extensive experiments on GSM8K and MATH benchmarks show ASCoT achieves outstanding accuracy, outperforming strong baselines including standard CoT, demonstrating the effectiveness of adaptive, vulnerability-aware correction.

Conclusion: The work emphasizes the importance of diagnosing specific failure modes in LLM reasoning and advocates for shifting from uniform verification to adaptive, vulnerability-aware correction mechanisms, particularly addressing the newly discovered Late-Stage Fragility phenomenon.

Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet the reliability of these reasoning chains remains a critical challenge. A widely held “cascading failure” hypothesis suggests that errors are most detrimental when they occur early in the reasoning process. This paper challenges that assumption through systematic error-injection experiments, revealing a counter-intuitive phenomenon we term “Late-Stage Fragility”: errors introduced in the later stages of a CoT chain are significantly more likely to corrupt the final answer than identical errors made at the beginning. To address this specific vulnerability, we introduce the Adaptive Self-Correction Chain-of-Thought (ASCoT) method. ASCoT employs a modular pipeline in which an Adaptive Verification Manager (AVM) operates first, followed by the Multi-Perspective Self-Correction Engine (MSCE). The AVM leverages a Positional Impact Score function I(k) that assigns different weights based on the position within the reasoning chains, addressing the Late-Stage Fragility issue by identifying and prioritizing high-risk, late-stage steps. Once these critical steps are identified, the MSCE applies robust, dual-path correction specifically to the failure parts. Extensive experiments on benchmarks such as GSM8K and MATH demonstrate that ASCoT achieves outstanding accuracy, outperforming strong baselines, including standard CoT. Our work underscores the importance of diagnosing specific failure modes in LLM reasoning and advocates for a shift from uniform verification strategies to adaptive, vulnerability-aware correction mechanisms.

Method: Evaluated GPT-OSS models against six contemporary open source LLMs (14.7B-235B parameters) across ten benchmarks covering general knowledge, math reasoning, code generation, multilingual understanding, and conversational ability. Used standardized inference settings with statistical validation via McNemar’s test and effect size analysis.

Result: GPT-OSS-20B consistently outperformed GPT-OSS-120B on benchmarks like HumanEval and MMLU despite requiring substantially less memory and energy. Both models showed mid-tier overall performance with strengths in code generation and weaknesses in multilingual tasks.

Conclusion: Scaling in sparse architectures may not yield proportional performance gains, highlighting the need for better optimization strategies and more efficient model selection for open source deployments.

Abstract: In August 2025, OpenAI released GPT-OSS models, its first open weight large language models since GPT-2 in 2019, comprising two mixture of experts architectures with 120B and 20B parameters. We evaluated both variants against six contemporary open source large language models ranging from 14.7B to 235B parameters, representing both dense and sparse designs, across ten benchmarks covering general knowledge, mathematical reasoning, code generation, multilingual understanding, and conversational ability. All models were tested in unquantised form under standardised inference settings, with statistical validation using McNemars test and effect size analysis. Results show that gpt-oss-20B consistently outperforms gpt-oss-120B on several benchmarks, such as HumanEval and MMLU, despite requiring substantially less memory and energy per response. Both models demonstrate mid-tier overall performance within the current open source landscape, with relative strength in code generation and notable weaknesses in multilingual tasks. These findings provide empirical evidence that scaling in sparse architectures may not yield proportional performance gains, underscoring the need for further investigation into optimisation strategies and informing more efficient model selection for future open source deployments. More details and evaluation scripts are available at the \href{https://ai-agent-lab.github.io/gpt-oss}{Project Webpage}.

Method: CARE uses a context assessor that encodes compact memory token embeddings and employs grounded/adversarial soft prompting to detect unreliable context and provide guidance signals for reasoning.

Result: CARE achieves an average 5.0% performance gain on QA and fact-checking benchmarks by effectively mitigating context-memory conflicts.

Conclusion: CARE establishes a promising direction for developing trustworthy and adaptive RAG systems that can handle conflicts between external context and parametric knowledge.

Abstract: Retrieval-augmented generation (RAG) enhances the capabilities of large language models (LLMs) by incorporating external knowledge into their input prompts. However, when the retrieved context contradicts the LLM’s parametric knowledge, it often fails to resolve the conflict between incorrect external context and correct parametric knowledge, known as context-memory conflict. To tackle this problem, we introduce Conflict-Aware REtrieval-Augmented Generation (CARE), consisting of a context assessor and a base LLM. The context assessor encodes compact memory token embeddings from raw context tokens. Through grounded/adversarial soft prompting, the context assessor is trained to discern unreliable context and capture a guidance signal that directs reasoning toward the more reliable knowledge source. Extensive experiments show that CARE effectively mitigates context-memory conflicts, leading to an average performance gain of 5.0% on QA and fact-checking benchmarks, establishing a promising direction for trustworthy and adaptive RAG systems.

Method: Used training data influence scores to sort training examples by difficulty, creating a curriculum that presents data from simpler to more complex examples based on how much each example affects the model’s output.

Result: Models trained with this curriculum learning approach outperformed randomly trained models by over 10 percentage points in benchmark evaluations.

Conclusion: Curriculum learning is effective for language model pre-training when using model-centric difficulty metrics like training data influence, rather than conventional human-centered difficulty measures.

Abstract: Curriculum learning, a training technique where data is presented to the model in order of example difficulty (e.g., from simpler to more complex documents), has shown limited success for pre-training language models. In this work, we investigate whether curriculum learning becomes competitive if we replace conventional human-centered difficulty metrics with one that more closely corresponds to example difficulty as observed during model training. Specifically, we experiment with sorting training examples by their \textit{training data influence}, a score which estimates the effect of individual training examples on the model’s output. Models trained on our curricula are able to outperform ones trained in random order by over 10 percentage points in benchmarks, confirming that curriculum learning is beneficial for language model pre-training, as long as a more model-centric notion of difficulty is adopted.

Method: Constructed a dialogue world model using POMDP to model emotion, sentiment, and intention as user beliefs, solved by maximizing information bottleneck. Applied model-based reinforcement learning framework with joint training of policy, critic, and dialogue world model.

Result: Achieved state-of-the-art performance on emotion classification and sentiment identification. Dialogue quality was enhanced through joint training. The approach maintains good exploration-exploitation balance and transfers well to out-of-domain scenarios like empathetic dialogues.

Conclusion: The dialogue world model framework DreamCUB successfully extends world modeling to natural language tasks, demonstrating strong performance in user state prediction and dialogue quality improvement with good generalization capabilities.

Abstract: World models have been widely utilized in robotics, gaming, and auto-driving. However, their applications on natural language tasks are relatively limited. In this paper, we construct the dialogue world model, which could predict the user’s emotion, sentiment, and intention, and future utterances. By defining a POMDP, we argue emotion, sentiment and intention can be modeled as the user belief and solved by maximizing the information bottleneck. By this user belief modeling, we apply the model-based reinforcement learning framework to the dialogue system, and propose a framework called DreamCUB. Experiments show that the pretrained dialogue world model can achieve state-of-the-art performances on emotion classification and sentiment identification, while dialogue quality is also enhanced by joint training of the policy, critic and dialogue world model. Further analysis shows that this manner holds a reasonable exploration-exploitation balance and also transfers well to out-of-domain scenarios such as empathetic dialogues.

Method: Uses LLM agents to conduct multi-turn interactions, invoke knowledge tools for deeper knowledge in question generation, and plan query strategies for difficulty adjustment. Implemented as JudgeAgent framework with knowledge-driven synthesis and difficulty scoring.

Result: Extensive experiments validate JudgeAgent’s effectiveness in providing valuable suggestions and accurately identifying knowledge/capability boundaries of target models.

Conclusion: Agent-as-Interviewer paradigm enables more comprehensive evaluation of LLMs’ knowledge boundaries and capabilities through dynamic multi-turn interactions.

Abstract: Current evaluation paradigms for large language models (LLMs) suffer from overestimated or biased evaluations and mismatched question difficulty, leading to incomplete evaluations of knowledge and capability boundaries, which hinder their effective application and optimization. To address these challenges, we propose Agent-as-Interviewer, a dynamic evaluation paradigm that employs LLM agents to conduct multi-turn interactions for evaluation. Unlike current benchmarking or dynamic interaction paradigms, Agent-as-Interviewer utilizes agents to invoke knowledge tools for wider and deeper knowledge in the dynamic multi-turn question generation, achieving more comprehensive evaluations of LLM’s knowledge boundaries. It also leverages agents to plan query strategies for adjustment of the question difficulty levels, enhancing the difficulty control to match the actual capabilities of target LLMs. Based on this paradigm, we develop JudgeAgent, a knowledge-wise dynamic evaluation framework that employs knowledge-driven synthesis as the agent’s tool and uses difficulty scoring as strategy guidance, thereby finally providing valuable suggestions to help targets optimize themselves. Extensive experiments validate the effectiveness of JudgeAgent’s suggestions, demonstrating that Agent-as-Interviewer can accurately identify the knowledge and capability boundaries of target models. The source code is available on https://github.com/DataArcTech/JudgeAgent.

Method: Proposes Co-Modality-based RAG (CMRAG) with Unified Encoding Model (UEM) for shared embedding space via triplet training, and Unified Co-Modality-informed Retrieval (UCMR) for cross-modal similarity fusion.

Result: CMRAG consistently outperforms single-modality RAG methods across multiple visual document question-answering benchmarks.

Conclusion: Unified integration of co-modality information effectively improves performance in complex visual document QA systems.

Abstract: Retrieval-Augmented Generation (RAG) has become a core paradigm in document question answering tasks. However, existing methods have limitations when dealing with multimodal documents: one category of methods relies on layout analysis and text extraction, which can only utilize explicit text information and struggle to capture images or unstructured content; the other category treats document segmentation as visual input and directly passes it to visual language models (VLMs) for processing, yet it ignores the semantic advantages of text, leading to suboptimal retrieval and generation results. To address these research gaps, we propose the Co-Modality-based RAG (CMRAG) framework, which can simultaneously leverage texts and images for more accurate retrieval and generation. Our framework includes two key components: (1) a Unified Encoding Model (UEM) that projects queries, parsed text, and images into a shared embedding space via triplet-based training, and (2) a Unified Co-Modality-informed Retrieval (UCMR) method that statistically normalizes similarity scores to effectively fuse cross-modal signals. To support research in this direction, we further construct and release a large-scale triplet dataset of (query, text, image) examples. Experiments demonstrate that our proposed framework consistently outperforms single-modality–based RAG in multiple visual document question-answering (VDQA) benchmarks. The findings of this paper show that integrating co-modality information into the RAG framework in a unified manner is an effective approach to improving the performance of complex VDQA systems.

Method: MoT explores problems horizontally and vertically using column-cell communication, reduces redundancy in thought nodes, and employs fact-correction with knowledge graph triples and original text to construct knowledge units.

Result: Extensive experiments in 24-point game, question answering, and proposition writing show MoT outperforms state-of-the-art methods with reasoning time only 14.4% of baseline.

Conclusion: MoT provides an efficient and accurate reasoning framework that enhances LLM capabilities through multi-strategy deep thinking and knowledge correction mechanisms.

Abstract: Large Language Models (LLMs) face significant accuracy degradation due to insufficient reasoning ability when dealing with complex and abstract tasks. Thought structures such as Chain of Thought (CoT) and Tree of Thought (ToT) focus on enhancing the reasoning capability of LLMs. However, they suffer from inherent drawbacks such as redundancy within the same layer of the tree structure and the singularity of the paths in the chain structure. Some studies have utilized Retrieval-Augmented Generation (RAG) methods to enhance CoT and ToT in mitigating hallucinations in LLMs, yet the fundamental shortcomings of the thought structures still persist. Furthermore, when dealing with multi-entity and multi-hop information, the retrieved verification knowledge often contains large amounts of fragmented, superficial, or even erroneous data, misleading the reasoning process of LLMs. To address these issues, we propose the Matrix of Thought (MoT), a novel and efficient thought structure for LLMs. MoT explores problems in both horizontal and vertical dimensions through a “column-cell communication” mechanism, enabling LLMs to actively engage in multi-strategy and deep thinking while reducing redundancy in the thought nodes within the column cells, thereby enhancing the reasoning capability of LLMs. Additionally, through a fact-correction mechanism, it leverages the knowledge graph triples retrieved by RAG and the original text to construct knowledge units and correct erroneous answers. To validate the effectiveness of this method, we conducted extensive experiments in three tasks: 24-point game, question answering evaluation, and proposition writing.The results demonstrate that our framework outperforms state-of-the-art methods, with reasoning time only 14.4% of that of the baseline method, proving its efficiency and accuracy. The code for framework is available at https://github.com/lyfiter/mtqa.

Method: Treats core music theory rules (beats, intervals) as programmatic functions to systematically synthesize a vast corpus of verifiable sheet music reasoning problems in both textual and visual modalities.

Result: Models show significant improvements on SSMR-Bench and established human-crafted benchmarks (MusicTheoryBench, MMMU music subset) when trained with synthetic data. Enhanced reasoning ability also facilitates music composition.

Conclusion: The approach successfully addresses the data scarcity problem in sheet music reasoning, demonstrating that synthetic data generation and reasoning training significantly improve LLMs/MLLMs’ sheet music interpretation and composition capabilities.

Abstract: Enhancing the ability of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) to interpret sheet music is a crucial step toward building AI musicians. However, current research lacks both evaluation benchmarks and training data for sheet music reasoning. Inspired by mathematics, where simple operations yield infinite verifiable problems, we introduce a novel approach that treats core music theory rules, such as those governing beats and intervals, as programmatic functions to systematically synthesize a vast and diverse corpus of sheet music reasoning problems. This approach allows us to introduce a data synthesis framework that generates verifiable sheet music questions in both textual and visual modalities, leading to the Synthetic Sheet Music Reasoning Benchmark (SSMR-Bench) and a complementary training set. Evaluation results on SSMR-Bench highlight the key role reasoning plays in interpreting sheet music, while also pointing out the ongoing challenges in understanding sheet music in a visual format. By leveraging synthetic data for RLVR, all models show significant improvements on the SSMR-Bench. Additionally, they also demonstrate considerable advancements on previously established human-crafted benchmarks, such as MusicTheoryBench and the music subset of MMMU. Finally, our results show that the enhanced reasoning ability can also facilitate music composition.

Method: Systematic data generation using model-based exploration and iterative, long-to-short query evolution to create challenging query-answer pairs requiring multi-step reasoning and complex web navigation. The model is developed through supervised fine-tuning followed by reinforcement learning, supporting 128K context length and up to 100 tool calling turns.

Result: WebExplorer-8B achieves state-of-the-art performance at its scale across diverse information-seeking benchmarks. As an 8B model, it effectively searches over an average of 16 turns after RL training, outperforming WebSailor-72B on BrowseComp-en/zh and achieving best performance among models up to 100B parameters on WebWalkerQA and FRAMES. It also shows strong generalization on HLE benchmark.

Conclusion: The approach provides a practical path toward long-horizon web agents, demonstrating that systematic data generation can enable smaller models to achieve competitive performance against much larger models in complex web navigation tasks.

Abstract: The paradigm of Large Language Models (LLMs) has increasingly shifted toward agentic applications, where web browsing capabilities are fundamental for retrieving information from diverse online sources. However, existing open-source web agents either demonstrate limited information-seeking abilities on complex tasks or lack transparent implementations. In this work, we identify that the key challenge lies in the scarcity of challenging data for information seeking. To address this limitation, we introduce WebExplorer: a systematic data generation approach using model-based exploration and iterative, long-to-short query evolution. This method creates challenging query-answer pairs that require multi-step reasoning and complex web navigation. By leveraging our curated high-quality dataset, we successfully develop advanced web agent WebExplorer-8B through supervised fine-tuning followed by reinforcement learning. Our model supports 128K context length and up to 100 tool calling turns, enabling long-horizon problem solving. Across diverse information-seeking benchmarks, WebExplorer-8B achieves the state-of-the-art performance at its scale. Notably, as an 8B-sized model, WebExplorer-8B is able to effectively search over an average of 16 turns after RL training, achieving higher accuracy than WebSailor-72B on BrowseComp-en/zh and attaining the best performance among models up to 100B parameters on WebWalkerQA and FRAMES. Beyond these information-seeking tasks, our model also achieves strong generalization on the HLE benchmark even though it is only trained on knowledge-intensive QA data. These results highlight our approach as a practical path toward long-horizon web agents.

Method: Summarizing key cognitive science theories about analogical reasoning processes and relating them to concepts in natural language processing, showing their relevance beyond just analogy solving.

Result: Demonstrates that cognitive processes underlying analogical reasoning are relevant for several major challenges in NLP research and can guide better optimization of relational understanding in text.

Conclusion: Cognitive perspectives on analogical reasoning can help NLP researchers move beyond entity-level similarity and improve relational understanding in language models for various NLP challenges.

Abstract: Analogical reasoning is an essential aspect of human cognition. In this paper, we summarize key theory about the processes underlying analogical reasoning from the cognitive science literature and relate it to current research in natural language processing. While these processes can be easily linked to concepts in NLP, they are generally not viewed through a cognitive lens. Furthermore, we show how these notions are relevant for several major challenges in NLP research, not directly related to analogy solving. This may guide researchers to better optimize relational understanding in text, as opposed to relying heavily on entity-level similarity.

Method: Evaluated LLMs on three probabilistic tasks (mode identification, maximum likelihood estimation, sample generation) using explicit discrete probability distributions and prompting for joint/conditional distribution queries.

Result: Larger models show stronger inference and surprising sample generation capabilities, but performance degrades over 60% with longer context and models are sensitive to probability notation variations.

Conclusion: LLMs possess probabilistic reasoning abilities with size-dependent performance, but need improvements in notation robustness and context handling for reliable probabilistic inference.

Abstract: Despite widespread success in language understanding and generation, large language models (LLMs) exhibit unclear and often inconsistent behavior when faced with tasks that require probabilistic reasoning. In this work, we present the first comprehensive study of the reasoning capabilities of LLMs over explicit discrete probability distributions. Given observations from a probability distribution, we evaluate models on three carefully designed tasks, mode identification, maximum likelihood estimation, and sample generation, by prompting them to provide responses to queries about either the joint distribution or its conditionals. These tasks thus probe a range of probabilistic skills, including frequency analysis, marginalization, and generative behavior. Through comprehensive empirical evaluations, we demonstrate that there exists a clear performance gap between smaller and larger models, with the latter demonstrating stronger inference and surprising capabilities in sample generation. Furthermore, our investigations reveal notable limitations, including sensitivity to variations in the notation utilized to represent probabilistic outcomes and performance degradation of over 60% as context length increases. Together, our results provide a detailed understanding of the probabilistic reasoning abilities of LLMs and identify key directions for future improvement.

Method: Token-Aware Phase Attention (TAPA) incorporates a learnable phase function into the attention mechanism to preserve token interactions over long distances.

Result: TAPA extends to longer contexts with direct and light fine-tuning, extrapolates to unseen lengths, and achieves significantly lower perplexity on long-context than RoPE families.

Conclusion: TAPA provides an effective positional encoding method that overcomes RoPE’s limitations for long-context modeling with minimal fine-tuning requirements.

Abstract: We prove under practical assumptions that Rotary Positional Embedding (RoPE) introduces an intrinsic distance-dependent bias in attention scores that limits RoPE’s ability to model long-context. RoPE extension methods may alleviate this issue, but they typically require post-hoc adjustments after pretraining, such as rescaling or hyperparameters retuning. This paper introduces Token-Aware Phase Attention (TAPA), a new positional encoding method that incorporates a learnable phase function into the attention mechanism. TAPA preserves token interactions over long range, extends to longer contexts with direct and light fine-tuning, extrapolates to unseen lengths, and attains significantly lower perplexity on long-context than RoPE families.

Method: Proposed DivLogicEval benchmark with natural sentences composed of diverse statements in counterintuitive ways, and introduced a new evaluation metric that mitigates bias and randomness in LLMs.

Result: Experiments demonstrated the extent of logical reasoning required for DivLogicEval questions and compared performance of different popular LLMs in logical reasoning tasks.

Conclusion: DivLogicEval provides a more reliable benchmark for evaluating logical reasoning in LLMs by addressing distributional biases and introducing better evaluation metrics.

Abstract: Logic reasoning in natural language has been recognized as an important measure of human intelligence for Large Language Models (LLMs). Popular benchmarks may entangle multiple reasoning skills and thus provide unfaithful evaluations on the logic reasoning skill. Meanwhile, existing logic reasoning benchmarks are limited in language diversity and their distributions are deviated from the distribution of an ideal logic reasoning benchmark, which may lead to biased evaluation results. This paper thereby proposes a new classical logic benchmark DivLogicEval, consisting of natural sentences composed of diverse statements in a counterintuitive way. To ensure a more reliable evaluation, we also introduce a new evaluation metric that mitigates the influence of bias and randomness inherent in LLMs. Through experiments, we demonstrate the extent to which logical reasoning is required to answer the questions in DivLogicEval and compare the performance of different popular LLMs in conducting logical reasoning.

Method: Use a short warm-start fine-tune, extract a task-aware steering vector from KL divergence gradient between warm-started and pre-trained models, then use this vector to guide decoding to steer output distribution toward task distribution.

Result: Across three tasks and nine benchmarks, SVD improved multiple-choice accuracy by up to 5 points and open-ended truthfulness by 2 points, with 1-2 point gains on commonsense datasets, without adding trainable parameters beyond PEFT adapters.

Conclusion: SVD offers a lightweight, theoretically grounded path to stronger task adaptation for large language models by directly aligning output distributions during decoding rather than indirectly through weight updates.

Abstract: Adapting billion-parameter language models to a downstream task is still costly, even with parameter-efficient fine-tuning (PEFT). We re-cast task adaptation as output-distribution alignment: the objective is to steer the output distribution toward the task distribution directly during decoding rather than indirectly through weight updates. Building on this view, we introduce Steering Vector Decoding (SVD), a lightweight, PEFT-compatible, and theoretically grounded method. We start with a short warm-start fine-tune and extract a task-aware steering vector from the Kullback-Leibler (KL) divergence gradient between the output distribution of the warm-started and pre-trained models. This steering vector is then used to guide the decoding process to steer the model’s output distribution towards the task distribution. We theoretically prove that SVD is first-order equivalent to the gradient step of full fine-tuning and derive a globally optimal solution for the strength of the steering vector. Across three tasks and nine benchmarks, SVD paired with four standard PEFT methods improves multiple-choice accuracy by up to 5 points and open-ended truthfulness by 2 points, with similar gains (1-2 points) on commonsense datasets without adding trainable parameters beyond the PEFT adapter. SVD thus offers a lightweight, theoretically grounded path to stronger task adaptation for large language models.

Method: ZeroRepo uses RPG - a structured graph representation of capabilities, file structures, data flows, and functions - in three stages: proposal planning, implementation construction, and graph-guided code generation with test validation.

Result: On RepoCraft benchmark (6 projects, 1,052 tasks), ZeroRepo generated nearly 36K code lines and 445K tokens (3.9x larger than Claude Code), with 81.5% coverage and 69.7% test accuracy, improving over Claude Code by 27.3 and 35.8 points.

Conclusion: RPG effectively models complex dependencies, enables sophisticated planning through near-linear scaling, and improves agent understanding of repositories, accelerating localization.

Abstract: Large language models excel at generating individual functions or single files of code, yet generating complete repositories from scratch remains a fundamental challenge. This capability is key to building coherent software systems from high-level specifications and realizing the full potential of automated code generation. The process requires planning at two levels: deciding what features and modules to build (proposal stage) and defining their implementation details (implementation stage). Current approaches rely on natural language planning, which often produces unclear specifications, misaligned components, and brittle designs due to its inherent ambiguity and lack of structure. To address these limitations, we introduce the Repository Planning Graph (RPG), a structured representation that encodes capabilities, file structures, data flows, and functions in a unified graph. By replacing free-form natural language with an explicit blueprint, RPG enables consistent long-horizon planning for repository generation. Building on RPG, we develop ZeroRepo, a graph-driven framework that operates in three stages: proposal-level planning, implementation-level construction, and graph-guided code generation with test validation. To evaluate, we construct RepoCraft, a benchmark of six real-world projects with 1,052 tasks. On RepoCraft, ZeroRepo produces nearly 36K Code Lines and 445K Code Tokens, on average 3.9$\times$ larger than the strongest baseline (Claude Code), and 68$\times$ larger than other baselines. It achieves 81.5% coverage and 69.7% test accuracy, improving over Claude Code by 27.3 and 35.8 points. Further analysis shows that RPG models complex dependencies, enables more sophisticated planning through near-linear scaling, and improves agent understanding of repositories, thus accelerating localization.

Method: Proposes QWHA method that integrates FT-based adapters using Walsh-Hadamard Transform as the transform kernel, with novel adapter initialization scheme including adaptive parameter selection and value refinement.

Result: QWHA consistently outperforms baselines in low-bit quantization accuracy and achieves significant training speedups over existing FT-based adapters.

Conclusion: QWHA effectively mitigates quantization errors while facilitating fine-tuning, with substantially reduced computational cost compared to existing approaches.

Abstract: The demand for efficient deployment of large language models (LLMs) has driven interest in quantization, which reduces inference cost, and parameter-efficient fine-tuning (PEFT), which lowers training overhead. This motivated the development of quantization-aware PEFT to produce accurate yet efficient quantized models. In this setting, reducing quantization error prior to fine-tuning is crucial for achieving high model accuracy. However, existing methods that rely on low-rank adaptation suffer from limited representational capacity. Recent Fourier-related transform (FT)-based adapters offer greater representational power than low-rank adapters, but their direct integration into quantized models often results in ineffective error reduction and increased computational overhead. To overcome these limitations, we propose QWHA, a method that integrates FT-based adapters into quantized models by employing the Walsh-Hadamard Transform (WHT) as the transform kernel, together with a novel adapter initialization scheme incorporating adaptive parameter selection and value refinement. We demonstrate that QWHA effectively mitigates quantization errors while facilitating fine-tuning, and that its design substantially reduces computational cost. Experimental results show that QWHA consistently outperforms baselines in low-bit quantization accuracy and achieves significant training speedups over existing FT-based adapters. The code is available at https://github.com/vantaa89/qwha.

Method: Proposes Semantic Reformulation Entropy (SRE) with input-side semantic reformulations for faithful paraphrases and progressive energy-based hybrid clustering for stable semantic grouping.

Result: Experiments on SQuAD and TriviaQA show SRE outperforms strong baselines, providing more robust and generalizable hallucination detection.

Conclusion: Combining input diversification with multi-signal clustering substantially enhances semantic-level uncertainty estimation in LLMs.

Abstract: Reliable question answering with large language models (LLMs) is challenged by hallucinations, fluent but factually incorrect outputs arising from epistemic uncertainty. Existing entropy-based semantic-level uncertainty estimation methods are limited by sampling noise and unstable clustering of variable-length answers. We propose Semantic Reformulation Entropy (SRE), which improves uncertainty estimation in two ways. First, input-side semantic reformulations produce faithful paraphrases, expand the estimation space, and reduce biases from superficial decoder tendencies. Second, progressive, energy-based hybrid clustering stabilizes semantic grouping. Experiments on SQuAD and TriviaQA show that SRE outperforms strong baselines, providing more robust and generalizable hallucination detection. These results demonstrate that combining input diversification with multi-signal clustering substantially enhances semantic-level uncertainty estimation.

Method: Hierarchical Collaborative Low-Rank Adaptation (HiCoLoRA) with: 1) hierarchical LoRA architecture for dynamic layer-specific processing, 2) Spectral Joint Domain-Slot Clustering to identify transferable associations, 3) Adaptive Linear Fusion Mechanism, and 4) Semantic-Enhanced SVD Initialization (SemSVD-Init) to preserve pre-trained knowledge.

Result: Outperforms baselines on multi-domain datasets MultiWOZ and SGD, achieving state-of-the-art performance in zero-shot dialog state tracking.

Conclusion: HiCoLoRA effectively addresses semantic misalignment challenges in zero-shot DST through hierarchical adaptation, spectral clustering, and knowledge-preserving initialization, demonstrating superior generalization to new domains without costly data annotation.

Abstract: Zero-shot Dialog State Tracking (zs-DST) is essential for enabling Task-Oriented Dialog Systems (TODs) to generalize to new domains without costly data annotation. A central challenge lies in the semantic misalignment between dynamic dialog contexts and static prompts, leading to inflexible cross-layer coordination, domain interference, and catastrophic forgetting. To tackle this, we propose Hierarchical Collaborative Low-Rank Adaptation (HiCoLoRA), a framework that enhances zero-shot slot inference through robust prompt alignment. It features a hierarchical LoRA architecture for dynamic layer-specific processing (combining lower-layer heuristic grouping and higher-layer full interaction), integrates Spectral Joint Domain-Slot Clustering to identify transferable associations (feeding an Adaptive Linear Fusion Mechanism), and employs Semantic-Enhanced SVD Initialization (SemSVD-Init) to preserve pre-trained knowledge. Experiments on multi-domain datasets MultiWOZ and SGD show that HiCoLoRA outperforms baselines, achieving SOTA in zs-DST. Code is available at https://github.com/carsonz/HiCoLoRA.

Method: Thinking augmented Pre-Training (TPT) - a universal methodology that augments text data with automatically generated thinking trajectories to increase training volume and make high-quality tokens more learnable.

Result: TPT enhances data efficiency by 3x, improves post-training performance by over 10% on reasoning benchmarks for a 3B parameter model, and works across diverse training configurations up to 100B tokens.

Conclusion: TPT is an effective approach that substantially improves LLM performance across various model sizes and families by making complex tokens more learnable through reasoning augmentation.

Abstract: This paper introduces a simple and scalable approach to improve the data efficiency of large language model (LLM) training by augmenting existing text data with thinking trajectories. The compute for pre-training LLMs has been growing at an unprecedented rate, while the availability of high-quality data remains limited. Consequently, maximizing the utility of available data constitutes a significant research challenge. A primary impediment is that certain high-quality tokens are difficult to learn given a fixed model capacity, as the underlying rationale for a single token can be exceptionally complex and deep. To address this issue, we propose Thinking augmented Pre-Training (TPT), a universal methodology that augments text with automatically generated thinking trajectories. Such augmentation effectively increases the volume of the training data and makes high-quality tokens more learnable through step-by-step reasoning and decomposition. We apply TPT across diverse training configurations up to $100$B tokens, encompassing pre-training with both constrained and abundant data, as well as mid-training from strong open-source checkpoints. Experimental results indicate that our method substantially improves the performance of LLMs across various model sizes and families. Notably, TPT enhances the data efficiency of LLM pre-training by a factor of $3$. For a $3$B parameter model, it improves the post-training performance by over $10%$ on several challenging reasoning benchmarks.

Method: Uses collaborative intelligence between specialized agents: summary generators and reviewers for creating/evaluating summaries, and quiz generators and reviewers for creating comprehension questions as quality checks. Includes an examinee agent to validate if summaries contain information needed to answer quiz questions, enabling iterative refinement through adversarial feedback.

Result: Significantly outperforms state-of-the-art methods on three benchmarks across ROUGE, BERTScore, LLM-as-a-Judge, and human evaluations. Comprehensive analyses show effectiveness of multi-agent collaboration, agent configurations, and quizzing mechanism.

Conclusion: Establishes a new approach for long document summarization using adversarial agentic collaboration to improve summarization quality through multifaceted feedback mechanisms.

Abstract: Long document summarization remains a significant challenge for current large language models (LLMs), as existing approaches commonly struggle with information loss, factual inconsistencies, and coherence issues when processing excessively long documents. We propose SummQ, a novel adversarial multi-agent framework that addresses these limitations through collaborative intelligence between specialized agents operating in two complementary domains: summarization and quizzing. Our approach employs summary generators and reviewers that work collaboratively to create and evaluate comprehensive summaries, while quiz generators and reviewers create comprehension questions that serve as continuous quality checks for the summarization process. This adversarial dynamic, enhanced by an examinee agent that validates whether the generated summary contains the information needed to answer the quiz questions, enables iterative refinement through multifaceted feedback mechanisms. We evaluate SummQ on three widely used long document summarization benchmarks. Experimental results demonstrate that our framework significantly outperforms existing state-of-the-art methods across ROUGE and BERTScore metrics, as well as in LLM-as-a-Judge and human evaluations. Our comprehensive analyses reveal the effectiveness of the multi-agent collaboration dynamics, the influence of different agent configurations, and the impact of the quizzing mechanism. This work establishes a new approach for long document summarization that uses adversarial agentic collaboration to improve summarization quality.

Method: Used harmonized dependency treebanks and mixed-effects modeling across multiple languages to evaluate sentence length, dependency length, and Intervener Complexity as predictors of memory load.

Result: All three factors (sentence length, dependency length, Intervener Complexity) are positively associated with memory load, with sentence length having the broadest influence and Intervener Complexity offering explanatory power beyond linear distance.

Conclusion: The findings reconcile linear and hierarchical perspectives by treating dependency length as a surface signature while identifying intervening heads as a more proximate indicator of integration demands, providing a principled path for evaluating memory load theories.

Abstract: This study examines whether sentence-level memory load in comprehension is better explained by linear proximity between syntactically related words or by the structural density of the intervening material. Building on locality-based accounts and cross-linguistic evidence for dependency length minimization, the work advances Intervener Complexity-the number of intervening heads between a head and its dependent-as a structurally grounded lens that refines linear distance measures. Using harmonized dependency treebanks and a mixed-effects framework across multiple languages, the analysis jointly evaluates sentence length, dependency length, and Intervener Complexity as predictors of the Memory-load measure. Studies in Psycholinguistics have reported the contributions of feature interference and misbinding to memory load during processing. For this study, I operationalized sentence-level memory load as the linear sum of feature misbinding and feature interference for tractability; current evidence does not establish that their cognitive contributions combine additively. All three factors are positively associated with memory load, with sentence length exerting the broadest influence and Intervener Complexity offering explanatory power beyond linear distance. Conceptually, the findings reconcile linear and hierarchical perspectives on locality by treating dependency length as an important surface signature while identifying intervening heads as a more proximate indicator of integration and maintenance demands. Methodologically, the study illustrates how UD-based graph measures and cross-linguistic mixed-effects modelling can disentangle linear and structural contributions to processing efficiency, providing a principled path for evaluating competing theories of memory load in sentence comprehension.

Method: Used synthetic training datasets to analyze syntactic-domain correlations, developed an evaluation framework to detect this phenomenon in trained models, and conducted case studies on safety finetuning implications.

Result: Found that syntactic-domain correlations lower performance on entity knowledge tasks (mean 0.51 +/- 0.06) and can be used to bypass refusals in both open (OLMo-2-7B, Llama-4-Maverick) and closed (GPT-4o) models.

Conclusion: There is a need to explicitly test for syntactic-domain correlations and ensure syntactic diversity in training data within domains to prevent spurious correlations that can compromise model performance and safety.

Abstract: For an LLM to correctly respond to an instruction it must understand both the semantics and the domain (i.e., subject area) of a given task-instruction pair. However, syntax can also convey implicit information Recent work shows that syntactic templates – frequent sequences of Part-of-Speech (PoS) tags – are prevalent in training data and often appear in model outputs. In this work we characterize syntactic templates, domain, and semantics in task-instruction pairs. We identify cases of spurious correlations between syntax and domain, where models learn to associate a domain with syntax during training; this can sometimes override prompt semantics. Using a synthetic training dataset, we find that the syntactic-domain correlation can lower performance (mean 0.51 +/- 0.06) on entity knowledge tasks in OLMo-2 models (1B-13B). We introduce an evaluation framework to detect this phenomenon in trained models, and show that it occurs on a subset of the FlanV2 dataset in open (OLMo-2-7B; Llama-4-Maverick), and closed (GPT-4o) models. Finally, we present a case study on the implications for safety finetuning, showing that unintended syntactic-domain correlations can be used to bypass refusals in OLMo-2-7B Instruct and GPT-4o. Our findings highlight two needs: (1) to explicitly test for syntactic-domain correlations, and (2) to ensure syntactic diversity in training data, specifically within domains, to prevent such spurious correlations.

Method: The authors fine-tuned ChatBioGPT from BioGPT on medical datasets, then developed GEP - a greedy coordinate gradient-based method specifically designed for PII extraction from SLMs, testing it against template-based approaches.

Result: GEP demonstrated up to 60x more PII leakage compared to template-based methods, and maintained a 4.53% leakage rate even in free-style insertion scenarios with varied syntactic expressions.

Conclusion: SLMs are vulnerable to sophisticated PII extraction attacks, and the proposed GEP method effectively reveals these vulnerabilities, highlighting the need for better privacy protection in SLM deployments.

Abstract: Small language models (SLMs) become unprecedentedly appealing due to their approximately equivalent performance compared to large language models (LLMs) in certain fields with less energy and time consumption during training and inference. However, the personally identifiable information (PII) leakage of SLMs for downstream tasks has yet to be explored. In this study, we investigate the PII leakage of the chatbot based on SLM. We first finetune a new chatbot, i.e., ChatBioGPT based on the backbone of BioGPT using medical datasets Alpaca and HealthCareMagic. It shows a matchable performance in BERTscore compared with previous studies of ChatDoctor and ChatGPT. Based on this model, we prove that the previous template-based PII attacking methods cannot effectively extract the PII in the dataset for leakage detection under the SLM condition. We then propose GEP, which is a greedy coordinate gradient-based (GCG) method specifically designed for PII extraction. We conduct experimental studies of GEP and the results show an increment of up to 60$\times$ more leakage compared with the previous template-based methods. We further expand the capability of GEP in the case of a more complicated and realistic situation by conducting free-style insertion where the inserted PII in the dataset is in the form of various syntactic expressions instead of fixed templates, and GEP is still able to reveal a PII leakage rate of up to 4.53%.

Method: Used difference-in-means directions, activation additions, and subspace geometry analysis across multiple models and datasets to examine sycophantic behaviors.

Result: Found that sycophantic agreement, sycophantic praise, and genuine agreement are encoded along distinct linear directions; each behavior can be independently amplified or suppressed; and the representational structure is consistent across model families and scales.

Conclusion: Sycophantic behaviors correspond to distinct, independently steerable representations rather than a single unified mechanism.

Abstract: Large language models (LLMs) often exhibit sycophantic behaviors – such as excessive agreement with or flattery of the user – but it is unclear whether these behaviors arise from a single mechanism or multiple distinct processes. We decompose sycophancy into sycophantic agreement and sycophantic praise, contrasting both with genuine agreement. Using difference-in-means directions, activation additions, and subspace geometry across multiple models and datasets, we show that: (1) the three behaviors are encoded along distinct linear directions in latent space; (2) each behavior can be independently amplified or suppressed without affecting the others; and (3) their representational structure is consistent across model families and scales. These results suggest that sycophantic behaviors correspond to distinct, independently steerable representations.

Method: Built on Stable-Diffusion backbone with lip-habit-modulated mechanism that jointly models phonemic audio-visual synchronization and speaker-specific orofacial dynamics. Uses occlusion-aware training strategy by explicitly exposing occlusion objects to inpainting process. Employs hybrid Mamba-Transformer architecture for training efficiency.

Result: Achieves superior performance in lip habit resemblance and occlusion robustness. Surpasses other methods in audio-lip sync, video quality, and resolution consistency. Eliminates need for cost-intensive priors and exhibits superior training efficiency.

Conclusion: StableDub expands the applicability of visual dubbing methods by addressing key limitations in lip habit modeling and occlusion handling, making it more practical for real-world deployment.

Abstract: The visual dubbing task aims to generate mouth movements synchronized with the driving audio, which has seen significant progress in recent years. However, two critical deficiencies hinder their wide application: (1) Audio-only driving paradigms inadequately capture speaker-specific lip habits, which fail to generate lip movements similar to the target avatar; (2) Conventional blind-inpainting approaches frequently produce visual artifacts when handling obstructions (e.g., microphones, hands), limiting practical deployment. In this paper, we propose StableDub, a novel and concise framework integrating lip-habit-aware modeling with occlusion-robust synthesis. Specifically, building upon the Stable-Diffusion backbone, we develop a lip-habit-modulated mechanism that jointly models phonemic audio-visual synchronization and speaker-specific orofacial dynamics. To achieve plausible lip geometries and object appearances under occlusion, we introduce the occlusion-aware training strategy by explicitly exposing the occlusion objects to the inpainting process. By incorporating the proposed designs, the model eliminates the necessity for cost-intensive priors in previous methods, thereby exhibiting superior training efficiency on the computationally intensive diffusion-based backbone. To further optimize training efficiency from the perspective of model architecture, we introduce a hybrid Mamba-Transformer architecture, which demonstrates the enhanced applicability in low-resource research scenarios. Extensive experimental results demonstrate that StableDub achieves superior performance in lip habit resemblance and occlusion robustness. Our method also surpasses other methods in audio-lip sync, video quality, and resolution consistency. We expand the applicability of visual dubbing methods from comprehensive aspects, and demo videos can be found at https://stabledub.github.io.

Method: Proposes Random DPO framework that uses random contrastive sampling to construct training pairs for Direct Preference Optimization, eliminating the need for reward models or human preference annotations.

Result: Experiments show the method improves clinical performance metrics by up to 5% when supplementing three state-of-the-art models, without requiring additional training data.

Conclusion: The Random DPO framework effectively enhances radiography report generation accuracy in a model-agnostic way, making it suitable for clinical deployment without extra data requirements.

Abstract: Radiography Report Generation (RRG) has gained significant attention in medical image analysis as a promising tool for alleviating the growing workload of radiologists. However, despite numerous advancements, existing methods have yet to achieve the quality required for deployment in real-world clinical settings. Meanwhile, large Visual Language Models (VLMs) have demonstrated remarkable progress in the general domain by adopting training strategies originally designed for Large Language Models (LLMs), such as alignment techniques. In this paper, we introduce a model-agnostic framework to enhance RRG accuracy using Direct Preference Optimization (DPO). Our approach leverages random contrastive sampling to construct training pairs, eliminating the need for reward models or human preference annotations. Experiments on supplementing three state-of-the-art models with our Random DPO show that our method improves clinical performance metrics by up to 5%, without requiring any additional training data.

Method: Proposes Vector Field Rectification with Sample Deviation to incorporate source video information into denoising, Structure-and-Motion-Preserving Initialization for motion-aware noise generation, and Deviation Caching to minimize computational overhead.

Result: The method achieves superior editing quality and consistency compared to existing approaches, offering a lightweight plug-and-play solution for video editing.

Conclusion: IF-V2V provides an efficient inversion-free approach for video editing that effectively transfers image editing capabilities to videos while maintaining consistency and quality.

Abstract: Although image editing techniques have advanced significantly, video editing, which aims to manipulate videos according to user intent, remains an emerging challenge. Most existing image-conditioned video editing methods either require inversion with model-specific design or need extensive optimization, limiting their capability of leveraging up-to-date image-to-video (I2V) models to transfer the editing capability of image editing models to the video domain. To this end, we propose IF-V2V, an Inversion-Free method that can adapt off-the-shelf flow-matching-based I2V models for video editing without significant computational overhead. To circumvent inversion, we devise Vector Field Rectification with Sample Deviation to incorporate information from the source video into the denoising process by introducing a deviation term into the denoising vector field. To further ensure consistency with the source video in a model-agnostic way, we introduce Structure-and-Motion-Preserving Initialization to generate motion-aware temporally correlated noise with structural information embedded. We also present a Deviation Caching mechanism to minimize the additional computational cost for denoising vector rectification without significantly impacting editing quality. Evaluations demonstrate that our method achieves superior editing quality and consistency over existing approaches, offering a lightweight plug-and-play solution to realize visual creativity.

Method: Analyzed a large balanced dataset (168 ASC, 157 non-autistic participants) using multimodal analysis of facial expressions, voice prosody, head motion, HRV, and gaze behavior. Introduced novel statistical descriptors for gaze variability and used late fusion for classification.

Result: Improved gaze-based classification from 64% to 69% using novel gaze descriptors. Achieved 74% accuracy with multimodal late fusion, demonstrating effective integration of behavioral markers across modalities.

Conclusion: The findings highlight the potential for scalable, video-based screening tools to support autism assessment through improved multimodal analysis.

Abstract: Due to the complex and resource-intensive nature of diagnosing Autism Spectrum Condition (ASC), several computer-aided diagnostic support methods have been proposed to detect autism by analyzing behavioral cues in patient video data. While these models show promising results on some datasets, they struggle with poor gaze feature performance and lack of real-world generalizability. To tackle these challenges, we analyze a standardized video dataset comprising 168 participants with ASC (46% female) and 157 non-autistic participants (46% female), making it, to our knowledge, the largest and most balanced dataset available. We conduct a multimodal analysis of facial expressions, voice prosody, head motion, heart rate variability (HRV), and gaze behavior. To address the limitations of prior gaze models, we introduce novel statistical descriptors that quantify variability in eye gaze angles, improving gaze-based classification accuracy from 64% to 69% and aligning computational findings with clinical research on gaze aversion in ASC. Using late fusion, we achieve a classification accuracy of 74%, demonstrating the effectiveness of integrating behavioral markers across multiple modalities. Our findings highlight the potential for scalable, video-based screening tools to support autism assessment.

Method: Partitions KV cache into fixed-size chunks and uses a recurrent gating module to summarize and filter historical context based on learned utility scores, preserving recent details while pruning stale memory while maintaining causality.

Result: Achieves up to 1.21x inference speedup and 36% KV memory reduction with minimal impact on task success, seamlessly integrating into existing autoregressive and hybrid VLA stacks.

Conclusion: KV-Efficient VLA enables scalable inference for VLA models without requiring modifications to training pipelines or downstream control logic, addressing critical inference inefficiencies for real-world deployment.

Abstract: Vision-Language-Action (VLA) models promise unified robotic perception and control, yet their scalability is constrained by the quadratic cost of attention and the unbounded growth of key-value (KV) memory during long-horizon inference. While recent methods improve generalization through scaling backbone architectures, they often neglect the inference inefficiencies critical to real-time deployment. In this work, we present KV-Efficient VLA, a model-agnostic memory compression framework that addresses these limitations by introducing a lightweight, training-friendly mechanism to selectively retain high-utility context. Our method partitions the KV cache into fixed size chunks and employs a recurrent gating module to summarize and filter historical context according to learned utility scores. This design preserves recent fine-grained detail while aggressively pruning stale, low-relevance memory, all while maintaining causality. Theoretically, KV-Efficient VLA yields up to 1.21x inference speedup and 36% KV memory reduction, with minimal impact on task success. Our method integrates seamlessly into existing autoregressive and hybrid VLA stacks, enabling scalable inference without modifying training pipelines or downstream control logic.

Method: Created synthetic dataset by perturbing findings and locations in ground truth reports, then trained multi-label cross-modal contrastive regression network on real/fake findings-location pairs with images.

Result: Achieved high accuracy in finding veracity prediction and localization on multiple X-ray datasets, with 0.997 concordance correlation coefficient with ground truth-based verification for SOTA report generators.

Conclusion: The model shows robustness and effectiveness for error detection in radiology reports, pointing to its utility in clinical inference workflows.

Abstract: With the emergence of large-scale vision language models (VLM), it is now possible to produce realistic-looking radiology reports for chest X-ray images. However, their clinical translation has been hampered by the factual errors and hallucinations in the produced descriptions during inference. In this paper, we present a novel phrase-grounded fact-checking model (FC model) that detects errors in findings and their indicated locations in automatically generated chest radiology reports. Specifically, we simulate the errors in reports through a large synthetic dataset derived by perturbing findings and their locations in ground truth reports to form real and fake findings-location pairs with images. A new multi-label cross-modal contrastive regression network is then trained on this dataset. We present results demonstrating the robustness of our method in terms of accuracy of finding veracity prediction and localization on multiple X-ray datasets. We also show its effectiveness for error detection in reports of SOTA report generators on multiple datasets achieving a concordance correlation coefficient of 0.997 with ground truth-based verification, thus pointing to its utility during clinical inference in radiology workflows.

Method: The MDF-MLLM integrates skip features from four U-Net encoder layers into cross-attention blocks within a LLaMA 3.2 11B MLLM, using patch-wise projection, scaled cross-attention, and FiLM-based U-Net modulation for feature fusion.

Result: MDF-MLLM achieved 94% accuracy on dual-type disease classification, representing a 56% improvement over baseline (60%). Recall and F1-scores improved by up to 67% and 35% respectively, with particular gains for inherited diseases with rich clinical text.

Conclusion: MDF-MLLM provides a generalizable, interpretable framework for fundus image classification that outperforms traditional MLLMs through multi-scale feature fusion, showing promise for clinical decision support systems.

Abstract: This study aimed to enhance disease classification accuracy from retinal fundus images by integrating fine-grained image features and global textual context using a novel multimodal deep learning architecture. Existing multimodal large language models (MLLMs) often struggle to capture low-level spatial details critical for diagnosing retinal diseases such as glaucoma, diabetic retinopathy, and retinitis pigmentosa. This model development and validation study was conducted on 1,305 fundus image-text pairs compiled from three public datasets (FIVES, HRF, and StoneRounds), covering acquired and inherited retinal diseases, and evaluated using classification accuracy and F1-score. The MDF-MLLM integrates skip features from four U-Net encoder layers into cross-attention blocks within a LLaMA 3.2 11B MLLM. Vision features are patch-wise projected and fused using scaled cross-attention and FiLM-based U-Net modulation. Baseline MLLM achieved 60% accuracy on the dual-type disease classification task. MDF-MLLM, with both U-Net and MLLM components fully fine-tuned during training, achieved a significantly higher accuracy of 94%, representing a 56% improvement. Recall and F1-scores improved by as much as 67% and 35% over baseline, respectively. Ablation studies confirmed that the multi-depth fusion approach contributed to substantial gains in spatial reasoning and classification, particularly for inherited diseases with rich clinical text. MDF-MLLM presents a generalizable, interpretable, and modular framework for fundus image classification, outperforming traditional MLLM baselines through multi-scale feature fusion. The architecture holds promise for real-world deployment in clinical decision support systems. Future work will explore synchronized training techniques, a larger pool of diseases for more generalizability, and extending the model for segmentation tasks.

Method: Three-step approach: 1) LLM decouples unsafe prompts into pseudo-safe and harmful components, 2) LLM rewrites harmful prompts into adversarial prompts to bypass filters, 3) Visual language model generates captions for iterative refinement to maintain semantic consistency.

Result: The method successfully bypasses T2I model safety filters to generate NSFW content while maintaining semantic alignment with original unsafe prompts through multimodal feedback loops.

Conclusion: MPDA demonstrates that multimodal approaches combining text and image modalities can effectively bypass T2I safety filters, revealing new vulnerabilities in current content moderation systems that require enhanced multimodal defense mechanisms.

Abstract: Text-to-image (T2I) models have been widely applied in generating high-fidelity images across various domains. However, these models may also be abused to produce Not-Safe-for-Work (NSFW) content via jailbreak attacks. Existing jailbreak methods primarily manipulate the textual prompt, leaving potential vulnerabilities in image-based inputs largely unexplored. Moreover, text-based methods face challenges in bypassing the model’s safety filters. In response to these limitations, we propose the Multimodal Prompt Decoupling Attack (MPDA), which utilizes image modality to separate the harmful semantic components of the original unsafe prompt. MPDA follows three core steps: firstly, a large language model (LLM) decouples unsafe prompts into pseudo-safe prompts and harmful prompts. The former are seemingly harmless sub-prompts that can bypass filters, while the latter are sub-prompts with unsafe semantics that trigger filters. Subsequently, the LLM rewrites the harmful prompts into natural adversarial prompts to bypass safety filters, which guide the T2I model to modify the base image into an NSFW output. Finally, to ensure semantic consistency between the generated NSFW images and the original unsafe prompts, the visual language model generates image captions, providing a new pathway to guide the LLM in iterative rewriting and refining the generated content.

Method: Integrates three components: Image2Cloud for 3D point cloud reconstruction from images, ObjMotionNet for encoding object trajectories into optical flow features, and Spatial Triple-Attention Transformer for lighting control. Uses a three-stage training strategy with the VLD dataset.

Result: Outperforms existing methods in control precision and visual coherence, demonstrating superior performance in joint control scenarios.

Conclusion: VidCRAFT3 provides a unified framework for flexible and precise joint control in image-to-video generation, addressing key limitations of previous approaches through integrated 3D reconstruction and attention mechanisms.

Abstract: Controllable image-to-video (I2V) generation transforms a reference image into a coherent video guided by user-specified control signals. In content creation workflows, precise and simultaneous control over camera motion, object motion, and lighting direction enhances both accuracy and flexibility. However, existing approaches typically treat these control signals separately, largely due to the scarcity of datasets with high-quality joint annotations and mismatched control spaces across modalities. We present VidCRAFT3, a unified and flexible I2V framework that supports both independent and joint control over camera motion, object motion, and lighting direction by integrating three core components. Image2Cloud reconstructs a 3D point cloud from the reference image to enable precise camera motion control. ObjMotionNet encodes sparse object trajectories into multi-scale optical flow features to guide object motion. The Spatial Triple-Attention Transformer integrates lighting direction embeddings via parallel cross-attention. To address the scarcity of jointly annotated data, we curate the VideoLightingDirection (VLD) dataset of synthetic static-scene video clips with per-frame lighting-direction labels, and adopt a three-stage training strategy that enables robust learning without fully joint annotations. Extensive experiments show that VidCRAFT3 outperforms existing methods in control precision and visual coherence. Code and data will be released. Project page: https://sixiaozheng.github.io/VidCRAFT3/.

Method: Proposes a mutual learning framework with three intertwined tasks: salient object detection, foreground contour detection, and edge detection. Uses a novel Mutual Learning Module (MLM) with multiple network branches trained in mutual learning manner.

Result: Extensive experiments on seven challenging datasets demonstrate state-of-the-art results in both salient object detection and edge detection.

Conclusion: The proposed multi-task learning approach with mutual learning effectively improves salient object detection performance by addressing incomplete predictions and boundary accuracy issues.

Abstract: Though deep learning techniques have made great progress in salient object detection recently, the predicted saliency maps still suffer from incomplete predictions due to the internal complexity of objects and inaccurate boundaries caused by strides in convolution and pooling operations. To alleviate these issues, we propose to train saliency detection networks by exploiting the supervision from not only salient object detection, but also foreground contour detection and edge detection. First, we leverage salient object detection and foreground contour detection tasks in an intertwined manner to generate saliency maps with uniform highlight. Second, the foreground contour and edge detection tasks guide each other simultaneously, thereby leading to precise foreground contour prediction and reducing the local noises for edge prediction. In addition, we develop a novel mutual learning module (MLM) which serves as the building block of our method. Each MLM consists of multiple network branches trained in a mutual learning manner, which improves the performance by a large margin. Extensive experiments on seven challenging datasets demonstrate that the proposed method has delivered state-of-the-art results in both salient object detection and edge detection.

Method: Developed a deep learning model trained on shipwreck data from Great Lakes and Irish coasts, enhanced with synthetic data generation. The tool automatically preprocesses bathymetry data, performs inference, thresholds outputs, and produces segmentation masks or bounding boxes.

Result: Demonstrated superior segmentation performance compared to deep learning-based ArcGIS toolkit and classical inverse sinkhole detection methods.

Conclusion: ShipwreckFinder provides an effective open-source solution for automated shipwreck detection, making maritime archaeology more accessible and efficient.

Abstract: In this paper, we introduce ShipwreckFinder, an open-source QGIS plugin that detects shipwrecks from multibeam sonar data. Shipwrecks are an important historical marker of maritime history, and can be discovered through manual inspection of bathymetric data. However, this is a time-consuming process and often requires expert analysis. Our proposed tool allows users to automatically preprocess bathymetry data, perform deep learning inference, threshold model outputs, and produce either pixel-wise segmentation masks or bounding boxes of predicted shipwrecks. The backbone of this open-source tool is a deep learning model, which is trained on a variety of shipwreck data from the Great Lakes and the coasts of Ireland. Additionally, we employ synthetic data generation in order to increase the size and diversity of our dataset. We demonstrate superior segmentation performance with our open-source tool and training pipeline as compared to a deep learning-based ArcGIS toolkit and a more classical inverse sinkhole detection method. The open-source tool can be found at https://github.com/umfieldrobotics/ShipwreckFinderQGISPlugin.

Method: Uses multimodal joint representation to integrate multiple modalities into the same latent space, enabling fair relevance scoring across different modalities at one scale.

Result: MAJORScore increases relevance scoring by 26.03%-64.29% for consistent modalities and decreases by 13.28%-20.54% for inconsistent modalities compared to existing methods.

Conclusion: MAJORScore provides a more reliable metric for evaluating similarity on large-scale multimodal datasets and multimodal model performance evaluation.

Abstract: The multimodal relevance metric is usually borrowed from the embedding ability of pretrained contrastive learning models for bimodal data, which is used to evaluate the correlation between cross-modal data (e.g., CLIP). However, the commonly used evaluation metrics are only suitable for the associated analysis between two modalities, which greatly limits the evaluation of multimodal similarity. Herein, we propose MAJORScore, a brand-new evaluation metric for the relevance of multiple modalities ($N$ modalities, $N\ge3$) via multimodal joint representation for the first time. The ability of multimodal joint representation to integrate multiple modalities into the same latent space can accurately represent different modalities at one scale, providing support for fair relevance scoring. Extensive experiments have shown that MAJORScore increases by 26.03%-64.29% for consistent modality and decreases by 13.28%-20.54% for inconsistence compared to existing methods. MAJORScore serves as a more reliable metric for evaluating similarity on large-scale multimodal datasets and multimodal model performance evaluation.

Method: Builds on a lightweight sparse-convolutional backbone with two dedicated heads: one for 3D object detection and one for layout estimation using a novel parametric wall representation. Works with multi-view images without ground-truth camera poses or depth supervision.

Result: Achieves state-of-the-art performance across three challenging benchmarks: using ground-truth point clouds, posed images, and unposed images. Performs on par with specialized 3D object detection methods while significantly advancing layout estimation.

Conclusion: TUN3D sets a new benchmark in holistic indoor scene understanding by enabling joint layout estimation and 3D object detection from multi-view images without requiring camera poses or depth supervision.

Abstract: Layout estimation and 3D object detection are two fundamental tasks in indoor scene understanding. When combined, they enable the creation of a compact yet semantically rich spatial representation of a scene. Existing approaches typically rely on point cloud input, which poses a major limitation since most consumer cameras lack depth sensors and visual-only data remains far more common. We address this issue with TUN3D, the first method that tackles joint layout estimation and 3D object detection in real scans, given multi-view images as input, and does not require ground-truth camera poses or depth supervision. Our approach builds on a lightweight sparse-convolutional backbone and employs two dedicated heads: one for 3D object detection and one for layout estimation, leveraging a novel and effective parametric wall representation. Extensive experiments show that TUN3D achieves state-of-the-art performance across three challenging scene understanding benchmarks: (i) using ground-truth point clouds, (ii) using posed images, and (iii) using unposed images. While performing on par with specialized 3D object detection methods, TUN3D significantly advances layout estimation, setting a new benchmark in holistic indoor scene understanding. Code is available at https://github.com/col14m/tun3d .

Method: Developed a cloud-based AI platform that processes and analyzes point cloud data of scaffolding structures. The system compares recent point cloud data with certified reference data to detect structural modifications and deviations from design rules.

Result: The proposed system enables automated monitoring of scaffolding structures, detecting alterations that may compromise integrity and stability.

Conclusion: AI and digitization can enhance scaffolding inspection accuracy, reduce manual inspection time and effort, and contribute to improved safety on construction sites through automated structural monitoring.

Abstract: In the construction industry, safety assessment is vital to ensure both the reliability of assets and the safety of workers. Scaffolding, a key structural support asset requires regular inspection to detect and identify alterations from the design rules that may compromise the integrity and stability. At present, inspections are primarily visual and are conducted by site manager or accredited personnel to identify deviations. However, visual inspection is time-intensive and can be susceptible to human errors, which can lead to unsafe conditions. This paper explores the use of Artificial Intelligence (AI) and digitization to enhance the accuracy of scaffolding inspection and contribute to the safety improvement. A cloud-based AI platform is developed to process and analyse the point cloud data of scaffolding structure. The proposed system detects structural modifications through comparison and evaluation of certified reference data with the recent point cloud data. This approach may enable automated monitoring of scaffolding, reducing the time and effort required for manual inspections while enhancing the safety on a construction site.

Method: Proposes skeletonisation scale-spaces that leverage sparsification of the medial axis for hierarchical shape simplification. The framework satisfies scale-space properties including hierarchical architecture, controllable simplification, and geometric equivariance. Also introduces densification for inverse progression from coarse to fine scales.

Result: Proof-of-concept experiments demonstrate effectiveness for robust skeletonisation, shape compression, and stiffness enhancement for additive manufacturing. The framework produces hierarchical simplifications and can create overcomplete shape representations.

Conclusion: The skeletonisation scale-space framework provides a theoretically grounded approach that overcomes noise sensitivity in medial axis computation while enabling hierarchical shape analysis with practical applications in multiple domains.

Abstract: The Hamilton-Jacobi skeleton, also known as the medial axis, is a powerful shape descriptor that represents binary objects in terms of the centres of maximal inscribed discs. Despite its broad applicability, the medial axis suffers from sensitivity to noise: minor boundary variations can lead to disproportionately large and undesirable expansions of the skeleton. Classical pruning methods mitigate this shortcoming by systematically removing extraneous skeletal branches. This sequential simplification of skeletons resembles the principle of sparsification scale-spaces that embed images into a family of reconstructions from increasingly sparse pixel representations. We combine both worlds by introducing skeletonisation scale-spaces: They leverage sparsification of the medial axis to achieve hierarchical simplification of shapes. Unlike conventional pruning, our framework inherently satisfies key scale-space properties such as hierarchical architecture, controllable simplification, and equivariance to geometric transformations. We provide a rigorous theoretical foundation in both continuous and discrete formulations and extend the concept further with densification. This allows inverse progression from coarse to fine scales and can even reach beyond the original skeleton to produce overcomplete shape representations with relevancy for practical applications. Through proof-of-concept experiments, we demonstrate the effectiveness of our framework for practical tasks including robust skeletonisation, shape compression, and stiffness enhancement for additive manufacturing.

Method: Three-component framework: image evaluator for dataset construction, supervised prompt rewriter for revised prompts, and DPO-trained ranker for optimal prompt selection. Uses extended Grounded SAM for improved evaluation.

Result: Created first counterfactual size text-image dataset. Achieved 114% improvement over backbone model. Outperformed state-of-the-art baselines and ChatGPT-4o.

Conclusion: Establishes a foundation for future research on counterfactual controllability in text-to-image generation.

Abstract: Text-to-image generation has advanced rapidly with large-scale multimodal training, yet fine-grained controllability remains a critical challenge. Counterfactual controllability, defined as the capacity to deliberately generate images that contradict common-sense patterns, remains a major challenge but plays a crucial role in enabling creativity and exploratory applications. In this work, we address this gap with a focus on counterfactual size (e.g., generating a tiny walrus beside a giant button) and propose an automatic prompt engineering framework that adapts base prompts into revised prompts for counterfactual images. The framework comprises three components: an image evaluator that guides dataset construction by identifying successful image generations, a supervised prompt rewriter that produces revised prompts, and a DPO-trained ranker that selects the optimal revised prompt. We construct the first counterfactual size text-image dataset and enhance the image evaluator by extending Grounded SAM with refinements, achieving a 114 percent improvement over its backbone. Experiments demonstrate that our method outperforms state-of-the-art baselines and ChatGPT-4o, establishing a foundation for future research on counterfactual controllability.

Method: Tested powerful visual foundational models (DINOv2, DINOv3, PE-Core) on SAR data, then performed self-supervised finetuning of SSL models with SAR data to create AFRL-DINOv2 models, analyzing different backbones and downstream adaptation recipes.

Result: Self-supervised finetuning with SAR data produced state-of-the-art SAR foundation models (AFRL-DINOv2) that significantly outperformed the best existing SAR-domain model SARATR-X, with analysis of performance trade-offs across different backbones and adaptation methods.

Conclusion: While self-supervised finetuning is a viable path forward for SAR foundation models and achieves new state-of-the-art results, there is still significant room for improvement in SAR foundation model development.

Abstract: In this work we investigate the viability of foundational AI/ML models for Synthetic Aperture Radar (SAR) object recognition tasks. We are inspired by the tremendous progress being made in the wider community, particularly in the natural image domain where frontier labs are training huge models on web-scale datasets with unprecedented computing budgets. It has become clear that these models, often trained with Self-Supervised Learning (SSL), will transform how we develop AI/ML solutions for object recognition tasks - they can be adapted downstream with very limited labeled data, they are more robust to many forms of distribution shift, and their features are highly transferable out-of-the-box. For these reasons and more, we are motivated to apply this technology to the SAR domain. In our experiments we first run tests with today’s most powerful visual foundational models, including DINOv2, DINOv3 and PE-Core and observe their shortcomings at extracting semantically-interesting discriminative SAR target features when used off-the-shelf. We then show that Self-Supervised finetuning of publicly available SSL models with SAR data is a viable path forward by training several AFRL-DINOv2s and setting a new state-of-the-art for SAR foundation models, significantly outperforming today’s best SAR-domain model SARATR-X. Our experiments further analyze the performance trade-off of using different backbones with different downstream task-adaptation recipes, and we monitor each model’s ability to overcome challenges within the downstream environments (e.g., extended operating conditions and low amounts of labeled data). We hope this work will inform and inspire future SAR foundation model builders, because despite our positive results, we still have a long way to go.

Method: Used two custom deep neural network architectures (O-Net and Theta-Net) to super-resolve images from non-fluorescent phase-modulated microscopy (Zernike PCM and DIC), tested on custom-fabricated nanoscale targets calibrated via atomic force microscopy.

Result: Both O-Net and Theta-Net performed well but were complementary: O-Net performed better with high signal-to-noise ratio images, while Theta-Net was preferred for low signal-to-noise ratio images.

Conclusion: Model architecture and source image signal-to-noise ratio significantly impact super-resolution performance, even when using the same training data and epochs, highlighting the importance of choosing appropriate DNN models for non-fluorescent optical nanoscopy.

Abstract: The field of optical microscopy spans across numerous industries and research domains, ranging from education to healthcare, quality inspection and analysis. Nonetheless, a key limitation often cited by optical microscopists refers to the limit of its lateral resolution (typically defined as ~200nm), with potential circumventions involving either costly external modules (e.g. confocal scan heads, etc) and/or specialized techniques [e.g. super-resolution (SR) fluorescent microscopy]. Addressing these challenges in a normal (non-specialist) context thus remains an aspect outside the scope of most microscope users & facilities. This study thus seeks to evaluate an alternative & economical approach to achieving SR optical microscopy, involving non-fluorescent phase-modulated microscopical modalities such as Zernike phase contrast (PCM) and differential interference contrast (DIC) microscopy. Two in silico deep neural network (DNN) architectures which we developed previously (termed O-Net and Theta-Net) are assessed on their abilities to resolve a custom-fabricated test target containing nanoscale features calibrated via atomic force microscopy (AFM). The results of our study demonstrate that although both O-Net and Theta-Net seemingly performed well when super-resolving these images, they were complementary (rather than competing) approaches to be considered for image SR, particularly under different image signal-to-noise ratios (SNRs). High image SNRs favoured the application of O-Net models, while low SNRs inclined preferentially towards Theta-Net models. These findings demonstrate the importance of model architectures (in conjunction with the source image SNR) on model performance and the SR quality of the generated images where DNN models are utilized for non-fluorescent optical nanoscopy, even where the same training dataset & number of epochs are being used.

Method: Proposes Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigation (DMTF-AVN) with multi-target architecture and refined Transformer mechanism to filter and selectively fuse cross-modal information.

Result: Achieves state-of-the-art performance on Replica and Matterport3D datasets, outperforming existing methods in success rate (SR), path efficiency (SPL), and scene adaptation (SNA).

Conclusion: The model exhibits strong scalability and generalizability, paving the way for advanced multimodal fusion strategies in robotic navigation.

Abstract: Audiovisual embodied navigation enables robots to locate audio sources by dynamically integrating visual observations from onboard sensors with the auditory signals emitted by the target. The core challenge lies in effectively leveraging multimodal cues to guide navigation. While prior works have explored basic fusion of visual and audio data, they often overlook deeper perceptual context. To address this, we propose the Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigation (DMTF-AVN). Our approach uses a multi-target architecture coupled with a refined Transformer mechanism to filter and selectively fuse cross-modal information. Extensive experiments on the Replica and Matterport3D datasets demonstrate that DMTF-AVN achieves state-of-the-art performance, outperforming existing methods in success rate (SR), path efficiency (SPL), and scene adaptation (SNA). Furthermore, the model exhibits strong scalability and generalizability, paving the way for advanced multimodal fusion strategies in robotic navigation. The code and videos are available at https://github.com/zzzmmm-svg/DMTF.

Method: Supervised sparse autoencoder training with systematic concept labeling to promote one-to-one concept-neuron mappings, mitigating feature splitting and promoting feature centralization through cross-entropy computation.

Result: Learns specialized neurons with stronger concept associations, reduces hyperparameter search by 96.67%, achieves 9.22% improvement on UnlearnCanvas benchmark, and shows 28.4% improvement in sequential unlearning for 9-object removal.

Conclusion: SAEmnesia provides an efficient and scalable approach to concept unlearning with minimal computational overhead and significant performance improvements over current methods.

Abstract: Effective concept unlearning in text-to-image diffusion models requires precise localization of concept representations within the model’s latent space. While sparse autoencoders successfully reduce neuron polysemanticity (i.e., multiple concepts per neuron) compared to the original network, individual concept representations can still be distributed across multiple latent features, requiring extensive search procedures for concept unlearning. We introduce SAEmnesia, a supervised sparse autoencoder training method that promotes one-to-one concept-neuron mappings through systematic concept labeling, mitigating feature splitting and promoting feature centralization. Our approach learns specialized neurons with significantly stronger concept associations compared to unsupervised baselines. The only computational overhead introduced by SAEmnesia is limited to cross-entropy computation during training. At inference time, this interpretable representation reduces hyperparameter search by 96.67% with respect to current approaches. On the UnlearnCanvas benchmark, SAEmnesia achieves a 9.22% improvement over the state-of-the-art. In sequential unlearning tasks, we demonstrate superior scalability with a 28.4% improvement in unlearning accuracy for 9-object removal.

Method: The paper introduces a lightweight mechanism that extracts intra-class diversity by forming per-class clusters, which are then used for final sampling to create representative coresets.

Result: Extensive classification experiments on a biomedical imaging dataset show the proposed scheme outperforms random sampling on several performance metrics under uniform conditions.

Conclusion: The intelligent coreset selection method effectively addresses the limitations of random sampling by capturing intra-class diversity, providing a more representative subset for efficient deep learning model training.

Abstract: Deep Learning models have transformed various domains, including the healthcare sector, particularly biomedical image classification by learning intricate features and enabling accurate diagnostics pertaining to complex diseases. Recent studies have adopted two different approaches to train DL models: training from scratch and transfer learning. Both approaches demand substantial computational time and resources due to the involvement of massive datasets in model training. These computational demands are further increased due to the design-space exploration required for selecting optimal hyperparameters, which typically necessitates several training rounds. With the growing sizes of datasets, exploring solutions to this problem has recently gained the research community’s attention. A plausible solution is to select a subset of the dataset for training and hyperparameter search. This subset, referred to as the corset, must be a representative set of the original dataset. A straightforward approach to selecting the coreset could be employing random sampling, albeit at the cost of compromising the representativeness of the original dataset. A critical limitation of random sampling is the bias towards the dominant classes in an imbalanced dataset. Even if the dataset has inter-class balance, this random sampling will not capture intra-class diversity. This study addresses this issue by introducing an intelligent, lightweight mechanism for coreset selection. Specifically, it proposes a method to extract intra-class diversity, forming per-class clusters that are utilized for the final sampling. We demonstrate the efficacy of the proposed methodology by conducting extensive classification experiments on a well-known biomedical imaging dataset. Results demonstrate that the proposed scheme outperforms the random sampling approach on several performance metrics for uniform conditions.

Method: Developed LongiMam - an end-to-end deep learning model combining convolutional and recurrent neural networks to integrate current and up to four prior mammograms, capturing spatial and temporal patterns.

Result: LongiMam consistently improved prediction when prior mammograms were included, with current+prior visits outperforming single-visit models. Model performed best in women with observed mammographic density changes over time and was effective across key risk groups.

Conclusion: Longitudinal modeling enhances breast cancer prediction and supports using repeated mammograms to refine risk stratification in screening programs. LongiMam is available as open-source software.

Abstract: Risk-adapted breast cancer screening requires robust models that leverage longitudinal imaging data. Most current deep learning models use single or limited prior mammograms and lack adaptation for real-world settings marked by imbalanced outcome distribution and heterogeneous follow-up. We developed LongiMam, an end-to-end deep learning model that integrates both current and up to four prior mammograms. LongiMam combines a convolutional and a recurrent neural network to capture spatial and temporal patterns predictive of breast cancer. The model was trained and evaluated using a large, population-based screening dataset with disproportionate case-to-control ratio typical of clinical screening. Across several scenarios that varied in the number and composition of prior exams, LongiMam consistently improved prediction when prior mammograms were included. The addition of prior and current visits outperformed single-visit models, while priors alone performed less well, highlighting the importance of combining historical and recent information. Subgroup analyses confirmed the model’s efficacy across key risk groups, including women with dense breasts and those aged 55 years or older. Moreover, the model performed best in women with observed changes in mammographic density over time. These findings demonstrate that longitudinal modeling enhances breast cancer prediction and support the use of repeated mammograms to refine risk stratification in screening programs. LongiMam is publicly available as open-source software.

Method: Uses hierarchical feature fusion strategy and joint multi-modal, multi-task training to create unified embeddings that support any-to-any cross-modal retrieval and prompt-aware generation.

Result: Sets new SOTA on MMEB-v2 video benchmark, achieves superior audio/video-to-audio retrieval, and significantly outperforms existing models in multimodal QA. Ablation studies confirm joint training benefits.

Conclusion: WAVE enables broad cross-modal applications and opens possibilities for versatile audio-visual learning with its unified representation space and prompt-aware capabilities.

Abstract: While embeddings from multimodal large language models (LLMs) excel as general-purpose representations, their application to dynamic modalities like audio and video remains underexplored. We introduce WAVE (\textbf{u}nified & \textbf{v}ersatile \textbf{a}udio-\textbf{v}isual \textbf{e}mbeddings), the first LLM-based embedding that creates a unified representation space for text, audio, and video modalities. WAVE employs a novel hierarchical feature fusion strategy and a joint multi-modal, multi-task training approach to enable two key capabilities: any-to-any cross-modal retrieval and the generation of prompt-aware embeddings tailored to user instructions. Experimentally, WAVE sets a new state-of-the-art on the MMEB-v2 video benchmark and achieves superior results in audio and video-to-audio retrieval. Its prompt-aware nature also yields remarkable performance in multimodal question answering, significantly outperforming existing embedding models. Ablation studies validate our joint training strategy, demonstrating improved performance across all modalities. With a newly introduced benchmark for versatile audio-visual learning, WAVE opens up broad possibilities for cross-modal, any-to-any applications. Our code, checkpoints, and data will be released.

Method: Used correlation analysis between CNN architectures and human behavioral/fMRI data, plus developed Object2Brain framework combining GradCAM, object detection, and correlation analysis to study object class influences.

Result: CNNs struggle with higher-order social cognition tasks and show different object class sensitivities across architectures despite similar correlation trends.

Conclusion: CNN-brain correspondence is limited for complex social cognition tasks, suggesting CNNs may not adequately model higher-order human brain processing in social contexts.

Abstract: Convolutional Neural Networks (CNNs) are a popular type of computer model that have proven their worth in many computer vision tasks. Moreover, they form an interesting study object for the field of psychology, with shown correspondences between the workings of CNNs and the human brain. However, these correspondences have so far mostly been studied in the context of general visual perception. In contrast, this paper explores to what extent this correspondence also holds for a more complex brain process, namely social cognition. To this end, we assess the alignment between popular CNN architectures and both human behavioral and fMRI data for image valence appraisal through a correlation analysis. We show that for this task CNNs struggle to go beyond simple visual processing, and do not seem to reflect higher-order brain processing. Furthermore, we present Object2Brain, a novel framework that combines GradCAM and object detection at the CNN-filter level with the aforementioned correlation analysis to study the influence of different object classes on the CNN-to-human correlations. Despite similar correlation trends, different CNN architectures are shown to display different object class sensitivities.

Method: Uses Denoising Diffusion Probabilistic Models (DDPM) and Flow Matching integrated within a Separation U-Net to synthesize separated sound spectrograms from noise distribution, conditioned on mixed audio and visual information.

Result: Both DDPM and Flow Matching variants of DAVIS surpass existing methods on AVE and MUSIC datasets, demonstrating superior separation quality.

Conclusion: The generative framework proves effective for audio-visual source separation, with both diffusion-based approaches outperforming current state-of-the-art methods.

Abstract: We propose DAVIS, a Diffusion-based Audio-VIsual Separation framework that solves the audio-visual sound source separation task through generative learning. Existing methods typically frame sound separation as a mask-based regression problem, achieving significant progress. However, they face limitations in capturing the complex data distribution required for high-quality separation of sounds from diverse categories. In contrast, DAVIS circumvents these issues by leveraging potent generative modeling paradigms, specifically Denoising Diffusion Probabilistic Models (DDPM) and the more recent Flow Matching (FM), integrated within a specialized Separation U-Net architecture. Our framework operates by synthesizing the desired separated sound spectrograms directly from a noise distribution, conditioned concurrently on the mixed audio input and associated visual information. The inherent nature of its generative objective makes DAVIS particularly adept at producing high-quality sound separations for diverse sound categories. We present comparative evaluations of DAVIS, encompassing both its DDPM and Flow Matching variants, against leading methods on the standard AVE and MUSIC datasets. The results affirm that both variants surpass existing approaches in separation quality, highlighting the efficacy of our generative framework for tackling the audio-visual source separation task.

Method: Two-step process: 1) Removal step where experts identify and remove undesired concepts using concept explanations; 2) Retraining step using CBDebug to convert concept-level feedback into sample-level auxiliary labels for supervised bias mitigation and targeted augmentation.

Result: CBDebug significantly outperforms prior retraining methods across multiple CBM architectures (PIP-Net, Post-hoc CBM) and benchmarks with known spurious correlations.

Conclusion: The framework effectively reduces model reliance on undesired concepts by leveraging CBM interpretability as a bridge for converting expert feedback into actionable training signals.

Abstract: Concept Bottleneck Models (CBMs) use a set of human-interpretable concepts to predict the final task label, enabling domain experts to not only validate the CBM’s predictions, but also intervene on incorrect concepts at test time. However, these interventions fail to address systemic misalignment between the CBM and the expert’s reasoning, such as when the model learns shortcuts from biased data. To address this, we present a general interpretable debugging framework for CBMs that follows a two-step process of Removal and Retraining. In the Removal step, experts use concept explanations to identify and remove any undesired concepts. In the Retraining step, we introduce CBDebug, a novel method that leverages the interpretability of CBMs as a bridge for converting concept-level user feedback into sample-level auxiliary labels. These labels are then used to apply supervised bias mitigation and targeted augmentation, reducing the model’s reliance on undesired concepts. We evaluate our framework with both real and automated expert feedback, and find that CBDebug significantly outperforms prior retraining methods across multiple CBM architectures (PIP-Net, Post-hoc CBM) and benchmarks with known spurious correlations.

Method: Used ResNet-18 trained on ImageNette, applied magnitude-based pruning followed by fine-tuning, compared post-hoc explanations from Vanilla Gradients and Integrated Gradients across pruning levels, and applied CRAFT-based concept extraction to track semantic coherence changes.

Result: Light-to-moderate pruning improved saliency-map focus and faithfulness while retaining distinct, semantically meaningful concepts. Aggressive pruning merged heterogeneous features, reducing saliency map sparsity and concept coherence despite maintaining accuracy.

Conclusion: Pruning can shape internal representations toward more human-aligned attention patterns, but excessive pruning undermines interpretability.

Abstract: Prior works have shown that neural networks can be heavily pruned while preserving performance, but the impact of pruning on model interpretability remains unclear. In this work, we investigate how magnitude-based pruning followed by fine-tuning affects both low-level saliency maps and high-level concept representations. Using a ResNet-18 trained on ImageNette, we compare post-hoc explanations from Vanilla Gradients (VG) and Integrated Gradients (IG) across pruning levels, evaluating sparsity and faithfulness. We further apply CRAFT-based concept extraction to track changes in semantic coherence of learned concepts. Our results show that light-to-moderate pruning improves saliency-map focus and faithfulness while retaining distinct, semantically meaningful concepts. In contrast, aggressive pruning merges heterogeneous features, reducing saliency map sparsity and concept coherence despite maintaining accuracy. These findings suggest that while pruning can shape internal representations toward more human-aligned attention patterns, excessive pruning undermines interpretability.

Method: Segment images into key and non-key regions; key regions processed with image-oriented semantic encoder, non-key regions compressed via image-to-text modeling; uses model quantization and low-rank adaptation for lightweight deployment.

Result: Outperforms traditional methods in both semantic fidelity and visual quality for image transmission tasks.

Conclusion: The proposed system effectively enhances image transmission by addressing regional importance differences and optimizing resource utilization.

Abstract: The rapid development of generative artificial intelligence (AI) has introduced significant opportunities for enhancing the efficiency and accuracy of image transmission within semantic communication systems. Despite these advancements, existing methodologies often neglect the difference in importance of different regions of the image, potentially compromising the reconstruction quality of visually critical content. To address this issue, we introduce an innovative generative semantic communication system that refines semantic granularity by segmenting images into key and non-key regions. Key regions, which contain essential visual information, are processed using an image oriented semantic encoder, while non-key regions are efficiently compressed through an image-to-text modeling approach. Additionally, to mitigate the substantial storage and computational demands posed by large AI models, the proposed system employs a lightweight deployment strategy incorporating model quantization and low-rank adaptation fine-tuning techniques, significantly boosting resource utilization without sacrificing performance. Simulation results demonstrate that the proposed system outperforms traditional methods in terms of both semantic fidelity and visual quality, thereby affirming its effectiveness for image transmission tasks.

Method: Proposes a wavelet packet-based frequency degradation pyramid to provide multiscale intermediate targets with increasing bandwidth, and guides reverse diffusion to progressively complement high-frequency details. Also designs a multiscale frequency refinement network to predict high-frequency components at multiple scales.

Result: Comprehensive evaluations on popular benchmarks show FDDiff outperforms prior generative methods with higher-fidelity super-resolution results.

Conclusion: The proposed FDDiff successfully addresses hallucination problems in diffusion-based super-resolution by decomposing the high-frequency complementing process into finer-grained steps using frequency domain guidance.

Abstract: The performance of single image super-resolution depends heavily on how to generate and complement high-frequency details to low-resolution images. Recently, diffusion-based DDPM models exhibit great potential in generating high-quality details for super-resolution tasks. They tend to directly predict high-frequency information of wide bandwidth by solely utilizing the high-resolution ground truth as the target for all sampling timesteps. However, as a result, they encounter hallucination problem that they generate mismatching artifacts. To tackle this problem and achieve higher-quality super-resolution, we propose a novel Frequency Domain-guided multiscale Diffusion model (FDDiff), which decomposes the high-frequency information complementing process into finer-grained steps. In particular, a wavelet packet-based frequency degradation pyramid is developed to provide multiscale intermediate targets with increasing bandwidth. Based on these targets, FDDiff guides reverse diffusion process to progressively complement missing high-frequency details over timesteps. Moreover, a multiscale frequency refinement network is designed to predict the required high-frequency components at multiple scales within one unified network. Comprehensive evaluations on popular benchmarks are conducted, and demonstrate that FDDiff outperforms prior generative methods with higher-fidelity super-resolution results.

Method: Created labeled mmWave datasets using a testbed with specific experimental settings and signal features, and demonstrated utility through validation on downstream tasks using parameter-efficient fine-tuning to adapt ISAC models.

Result: Successfully developed and released open datasets that enable research in various human sensing applications, and demonstrated effective model adaptation through fine-tuning that reduces computational complexity while maintaining performance.

Conclusion: mmHSense datasets provide valuable resources for advancing mmWave ISAC research, supporting both signal processing and deep learning approaches while enabling efficient model adaptation across different human sensing tasks.

Abstract: This article presents mmHSense, a set of open labeled mmWave datasets to support human sensing research within Integrated Sensing and Communication (ISAC) systems. The datasets can be used to explore mmWave ISAC for various end applications such as gesture recognition, person identification, pose estimation, and localization. Moreover, the datasets can be used to develop and advance signal processing and deep learning research on mmWave ISAC. This article describes the testbed, experimental settings, and signal features for each dataset. Furthermore, the utility of the datasets is demonstrated through validation on a specific downstream task. In addition, we demonstrate the use of parameter-efficient fine-tuning to adapt ISAC models to different tasks, significantly reducing computational complexity while maintaining performance on prior tasks.

Method: HiSin exploits spectral sparsity and structural heterogeneity of projection data, progressively extracting global structure at low resolution and deferring high-resolution inference to small patches with frequency-aware patch skipping and structure-adaptive step allocation.

Result: HiSin reduces peak memory usage by up to 30.81% and inference time by up to 17.58% compared to state-of-the-art methods while maintaining inpainting accuracy.

Conclusion: The proposed HiSin framework enables efficient high-resolution sinogram inpainting by leveraging spectral properties and structural features, making diffusion models practical for CT reconstruction applications.

Abstract: High-resolution sinogram inpainting is essential for computed tomography reconstruction, as missing high-frequency projections can lead to visible artifacts and diagnostic errors. Diffusion models are well-suited for this task due to their robustness and detail-preserving capabilities, but their application to high-resolution inputs is limited by excessive memory and computational demands. To address this limitation, we propose HiSin, a novel diffusion-based framework for efficient sinogram inpainting that exploits spectral sparsity and structural heterogeneity of projection data. It progressively extracts global structure at low resolution and defers high-resolution inference to small patches, enabling memory-efficient inpainting. Considering the structural features of sinograms, we incorporate frequency-aware patch skipping and structure-adaptive step allocation to reduce redundant computation. Experimental results show that HiSin reduces peak memory usage by up to 30.81% and inference time by up to 17.58% than the state-of-the-art framework, and maintains inpainting accuracy across.

Method: Train single-image super-resolution models on high-resolution observational gridded data, then apply them to low-resolution climate projections. Use climate indicator-based assessment with observed climate indices from weather stations for evaluation.

Result: Experiments on daily mean temperature show that super-resolution models can downscale climate projections without increasing the error of climate indicators compared to original low-resolution projections.

Conclusion: Single-image super-resolution models are effective for statistically downscaling climate projections to higher resolution when high-resolution training data is unavailable.

Abstract: High-resolution climate projections are essential for local decision-making. However, available climate projections have low spatial resolution (e.g. 12.5 km), which limits their usability. We address this limitation by leveraging single-image super-resolution models to statistically downscale climate projections to 1-km resolution. Since high-resolution climate projections are unavailable for training, we train models on a high-resolution observational gridded data set and apply them to low-resolution climate projections. We propose a climate indicator-based assessment using observed climate indices computed at weather station locations to evaluate the downscaled climate projections without ground-truth high-resolution climate projections. Experiments on daily mean temperature demonstrate that single-image super-resolution models can downscale climate projections without increasing the error of climate indicators compared to low-resolution climate projections.

Method: STQE network includes: recoloring-based motion compensation for inter-frame alignment, channel-aware temporal attention for dynamic region highlighting, Gaussian-guided neighborhood feature aggregation for spatial dependencies, and Pearson correlation-based joint loss to prevent over-smoothing.

Result: Applied to G-PCC test model, achieved improvements of 0.855 dB, 0.682 dB, and 0.828 dB delta PSNR with BD-rate reductions of -25.2%, -31.6%, and -32.5% for Luma, Cb, and Cr components respectively.

Conclusion: STQE network effectively enhances visual quality of compressed dynamic point clouds by exploiting spatial-temporal correlations, demonstrating significant performance improvements over existing methods.

Abstract: Very few studies have addressed quality enhancement for compressed dynamic point clouds. In particular, the effective exploitation of spatial-temporal correlations between point cloud frames remains largely unexplored. Addressing this gap, we propose a spatial-temporal attribute quality enhancement (STQE) network that exploits both spatial and temporal correlations to improve the visual quality of G-PCC compressed dynamic point clouds. Our contributions include a recoloring-based motion compensation module that remaps reference attribute information to the current frame geometry to achieve precise inter-frame geometric alignment, a channel-aware temporal attention module that dynamically highlights relevant regions across bidirectional reference frames, a Gaussian-guided neighborhood feature aggregation module that efficiently captures spatial dependencies between geometry and color attributes, and a joint loss function based on the Pearson correlation coefficient, designed to alleviate over-smoothing effects typical of point-wise mean squared error optimization. When applied to the latest G-PCC test model, STQE achieved improvements of 0.855 dB, 0.682 dB, and 0.828 dB in delta PSNR, with Bj{\o}ntegaard Delta rate (BD-rate) reductions of -25.2%, -31.6%, and -32.5% for the Luma, Cb, and Cr components, respectively.

Method: Jailbreaking with Loss-guided Image Perturbation (JaiLIP) minimizes a joint objective combining MSE loss between clean/adversarial images and the model’s harmful-output loss.

Result: JaiLIP generates highly effective and imperceptible adversarial images that outperform existing methods in producing toxicity, as measured by Perspective API and Detoxify metrics.

Conclusion: Image-based jailbreak attacks pose practical challenges for VLMs, highlighting the need for efficient defense mechanisms to protect against such vulnerabilities.

Abstract: Vision-Language Models (VLMs) have remarkable abilities in generating multimodal reasoning tasks. However, potential misuse or safety alignment concerns of VLMs have increased significantly due to different categories of attack vectors. Among various attack vectors, recent studies have demonstrated that image-based perturbations are particularly effective in generating harmful outputs. In the literature, many existing techniques have been proposed to jailbreak VLMs, leading to unstable performance and visible perturbations. In this study, we propose Jailbreaking with Loss-guided Image Perturbation (JaiLIP), a jailbreaking attack in the image space that minimizes a joint objective combining the mean squared error (MSE) loss between clean and adversarial image with the models harmful-output loss. We evaluate our proposed method on VLMs using standard toxicity metrics from Perspective API and Detoxify. Experimental results demonstrate that our method generates highly effective and imperceptible adversarial images, outperforming existing methods in producing toxicity. Moreover, we have evaluated our method in the transportation domain to demonstrate the attacks practicality beyond toxic text generation in specific domain. Our findings emphasize the practical challenges of image-based jailbreak attacks and the need for efficient defense mechanisms for VLMs.

Method: Evaluated 19 deep-learning systems from 4 research teams against 9 expert botanists specializing in French flora, using standardized resources and assessments.

Result: State-of-the-art deep learning models achieved performance close to the most advanced human expertise in plant identification.

Conclusion: Automated species identification using deep learning has reached near-human expert levels, demonstrating significant progress in the field.

Abstract: Automated identification of plants and animals has improved considerably in the last few years, in particular thanks to the recent advances in deep learning. The next big question is how far such automated systems are from the human expertise. Indeed, even the best experts are sometimes confused and/or disagree between each others when validating visual or audio observations of living organism. A picture actually contains only a partial information that is usually not sufficient to determine the right species with certainty. Quantifying this uncertainty and comparing it to the performance of automated systems is of high interest for both computer scientists and expert naturalists. The LifeCLEF 2018 ExpertCLEF challenge presented in this paper was designed to allow this comparison between human experts and automated systems. In total, 19 deep-learning systems implemented by 4 different research teams were evaluated with regard to 9 expert botanists of the French flora. The main outcome of this work is that the performance of state-of-the-art deep learning models is now close to the most advanced human expertise. This paper presents more precisely the resources and assessments of the challenge, summarizes the approaches and systems employed by the participating research groups, and provides an analysis of the main outcomes.

Method: QuadGPT uses an autoregressive framework with sequence prediction, featuring: 1) unified tokenization to handle mixed triangle/quad topologies, and 2) specialized Reinforcement Learning fine-tuning (tDPO) for better generation quality.

Result: Extensive experiments show QuadGPT significantly outperforms previous triangle-to-quad conversion pipelines in both geometric accuracy and topological quality.

Conclusion: The work establishes a new benchmark for native quad-mesh generation and demonstrates the effectiveness of combining large-scale autoregressive models with topology-aware RL refinement for structured 3D asset creation.

Abstract: The generation of quadrilateral-dominant meshes is a cornerstone of professional 3D content creation. However, existing generative models generate quad meshes by first generating triangle meshes and then merging triangles into quadrilaterals with some specific rules, which typically produces quad meshes with poor topology. In this paper, we introduce QuadGPT, the first autoregressive framework for generating quadrilateral meshes in an end-to-end manner. QuadGPT formulates this as a sequence prediction paradigm, distinguished by two key innovations: a unified tokenization method to handle mixed topologies of triangles and quadrilaterals, and a specialized Reinforcement Learning fine-tuning method tDPO for better generation quality. Extensive experiments demonstrate that QuadGPT significantly surpasses previous triangle-to-quad conversion pipelines in both geometric accuracy and topological quality. Our work establishes a new benchmark for native quad-mesh generation and showcases the power of combining large-scale autoregressive models with topology-aware RL refinement for creating structured 3D assets.

Method: Trains concept-specific LoRA adapters and dynamically composes them at inference, with bi-level orthogonality constraints at feature and parameter levels to prevent interference between adapters.

Result: Outperforms state-of-the-art baselines on ErasureBench-H and standard datasets, achieving higher multi-concept erasure fidelity with minimal collateral degradation.

Conclusion: DyME provides a scalable, flexible solution for multi-concept erasure that matches real-world usage patterns while maintaining model quality.

Abstract: Text-to-image diffusion models (DMs) inadvertently reproduce copyrighted styles and protected visual concepts, raising legal and ethical concerns. Concept erasure has emerged as a safeguard, aiming to selectively suppress such concepts through fine-tuning. However, existing methods do not scale to practical settings where providers must erase multiple and possibly conflicting concepts. The core bottleneck is their reliance on static erasure: a single checkpoint is fine-tuned to remove all target concepts, regardless of the actual erasure needs at inference. This rigid design mismatches real-world usage, where requests vary per generation, leading to degraded erasure success and reduced fidelity for non-target content. We propose DyME, an on-demand erasure framework that trains lightweight, concept-specific LoRA adapters and dynamically composes only those needed at inference. This modular design enables flexible multi-concept erasure, but naive composition causes interference among adapters, especially when many or semantically related concepts are suppressed. To overcome this, we introduce bi-level orthogonality constraints at both the feature and parameter levels, disentangling representation shifts and enforcing orthogonal adapter subspaces. We further develop ErasureBench-H, a new hierarchical benchmark with brand-series-character structure, enabling principled evaluation across semantic granularities and erasure set sizes. Experiments on ErasureBench-H and standard datasets (e.g., CIFAR-100, Imagenette) demonstrate that DyME consistently outperforms state-of-the-art baselines, achieving higher multi-concept erasure fidelity with minimal collateral degradation.

Method: Multi-round direct preference optimisation (MrDPO) with periodic reference policy refresh via bootstrapping from re-initialized lightweight adapters, plus supervised fine-tuning using generated high-quality video captions.

Result: Achieves SOTA results on Video-MME, WorldSense, AVUT, Video-Holmes, DailyOmni, MLVU, and LVBench benchmarks; 72B model surpasses all open-source systems; produces more detailed and accurate captions than GPT-4o and Gemini-1.5 Pro.

Conclusion: MrDPO effectively addresses reference staleness in DPO, enabling continual improvement and strong performance transfer from captioning to complex video-QA tasks, with released open-source code, models, and data.

Abstract: We present video-SALMONN 2, a family of audio-visual large language models that set new state-of-the-art (SOTA) results in video description and question answering (QA). Our core contribution is multi-round direct preference optimisation (MrDPO), paired with a caption-quality objective that jointly rewards completeness and factual accuracy. Unlike standard DPO with a fixed reference policy, MrDPO periodically refreshes the reference by bootstrapping from a newly re-initialised lightweight adapter trained on the latest preferences, avoiding reference staleness and enabling continual improvement. This strategy produces captions that are consistently more detailed and accurate than those from proprietary systems such as GPT-4o and Gemini-1.5 Pro. We further distil these gains by using our model to generate a high-quality video-caption corpus for supervised fine-tuning of new models, transferring benefits beyond captioning to strong performance on complex video-QA tasks. Across widely used audio-visual and visual-only understanding benchmarks (including Video-MME, WorldSense, AVUT, Video-Holmes, DailyOmni, MLVU, and LVBench), our 3B and 7B models achieve SOTA results at comparable scales, while the 72B model surpasses all other open-source systems. Our source code, models, and data are released at \href{https://github.com/bytedance/video-SALMONN-2}{https://github.com/bytedance/video-SALMONN-2}.

Method: Training uses generator-evaluator interplay: generator produces responses conditioned on target ratings, and mismatched responses are discarded. Specialized MLLM architecture for video-text evaluation.

Result: VideoJudge-7B outperforms larger MLLM baselines (Qwen2.5-VL 32B/72B) on 3 out of 4 benchmarks. LLM judges perform worse than MLLM judges, and chain-of-thought reasoning doesn’t help, confirming video input importance.

Conclusion: VideoJudge demonstrates specialized MLLM judges can effectively evaluate video understanding tasks, with video inputs being essential for accurate assessment.

Abstract: Precisely evaluating video understanding models remains challenging: commonly used metrics such as BLEU, ROUGE, and BERTScore fail to capture the fineness of human judgment, while obtaining such judgments through manual evaluation is costly. Recent work has explored using large language models (LLMs) or multimodal LLMs (MLLMs) as evaluators, but their extension to video understanding remains relatively unexplored. In this work, we introduce VideoJudge, a 3B and 7B-sized MLLM judge specialized to evaluate outputs from video understanding models (\textit{i.e.}, text responses conditioned on videos). To train VideoJudge, our recipe builds on the interplay between a generator and an evaluator: the generator is prompted to produce responses conditioned on a target rating, and responses not matching the evaluator’s rating are discarded. Across three out of four meta-evaluation benchmarks, VideoJudge-7B outperforms larger MLLM judge baselines such as Qwen2.5-VL (32B and 72B). Notably, we find that LLM judges (Qwen3) models perform worse than MLLM judges (Qwen2.5-VL) and long chain-of-thought reasoning does not improve performance, indicating that providing video inputs is crucial for evaluation of video understanding tasks.

Method: End-to-end compression using a simple bottleneck network followed by multi-stage residual vector quantization (RVQ), transmitting only per-pixel code indices instead of raw features.

Result: Achieves 273x compression at 30 bpp to 1365x compression at 6 bpp on DAIR-V2X dataset. At 18 bpp (455x), matches or outperforms raw-feature collaborative perception.

Conclusion: ReVQom enables efficient and accurate multi-agent collaborative perception with ultra-low-bandwidth operation, making practical V2X deployment more feasible.

Abstract: Multi-agent collaborative perception (CP) improves scene understanding by sharing information across connected agents such as autonomous vehicles, unmanned aerial vehicles, and robots. Communication bandwidth, however, constrains scalability. We present ReVQom, a learned feature codec that preserves spatial identity while compressing intermediate features. ReVQom is an end-to-end method that compresses feature dimensions via a simple bottleneck network followed by multi-stage residual vector quantization (RVQ). This allows only per-pixel code indices to be transmitted, reducing payloads from 8192 bits per pixel (bpp) of uncompressed 32-bit float features to 6-30 bpp per agent with minimal accuracy loss. On DAIR-V2X real-world CP dataset, ReVQom achieves 273x compression at 30 bpp to 1365x compression at 6 bpp. At 18 bpp (455x), ReVQom matches or outperforms raw-feature CP, and at 6-12 bpp it enables ultra-low-bandwidth operation with graceful degradation. ReVQom allows efficient and accurate multi-agent collaborative perception with a step toward practical V2X deployment.

Method: Analyzed 1,006 images from ImageFX, DALL-E V3, and Grok for 56 Saudi professions using neutral prompts. Two Saudi annotators evaluated images on five dimensions (gender, clothing, background, activities, age), with a third researcher adjudicating disagreements.

Result: Strong gender imbalance found: ImageFX 85% male, Grok 86.6% male, DALL-E V3 96% male. Cultural inaccuracies in clothing, settings, and activities were frequent. Counter-stereotypical images often resulted from cultural misinterpretations rather than progressive portrayals.

Conclusion: Current models mirror societal biases from training data and offer limited reflection of Saudi labor market dynamics. Need for more diverse training data, fairer algorithms, and culturally sensitive evaluation frameworks.

Abstract: This study investigates the extent to which contemporary Text-to-Image artificial intelligence (AI) models perpetuate gender stereotypes and cultural inaccuracies when generating depictions of professionals in Saudi Arabia. We analyzed 1,006 images produced by ImageFX, DALL-E V3, and Grok for 56 diverse Saudi professions using neutral prompts. Two trained Saudi annotators evaluated each image on five dimensions: perceived gender, clothing and appearance, background and setting, activities and interactions, and age. A third senior researcher adjudicated whenever the two primary raters disagreed, yielding 10,100 individual judgements. The results reveal a strong gender imbalance, with ImageFX outputs being 85% male, Grok 86.6% male, and DALL-E V3 96% male, indicating that DALL-E V3 exhibited the strongest overall gender stereotyping. This imbalance was most evident in leadership and technical roles. Moreover, cultural inaccuracies in clothing, settings, and depicted activities were frequently observed across all three models. Counter-stereotypical images often arise from cultural misinterpretations rather than genuinely progressive portrayals. We conclude that current models mirror societal biases embedded in their training data, generated by humans, offering only a limited reflection of the Saudi labour market’s gender dynamics and cultural nuances. These findings underscore the urgent need for more diverse training data, fairer algorithms, and culturally sensitive evaluation frameworks to ensure equitable and authentic visual outputs.

Method: Three targeted pretraining tasks: Caption (enhances video detail perception), Visual Question Answering (deepens understanding of issue definitions), and Chain-of-Thought (enhances reasoning capability).

Result: Significant performance improvements in both zero-shot and supervised fine-tuning settings, with strong generalization to emergent, previously unseen issues.

Conclusion: The proposed reasoning-enhanced MLLM pretraining paradigm effectively addresses distribution gaps and complex issue definitions, enabling unified inappropriate content detection with strong generalization capabilities.

Abstract: Short video platforms are evolving rapidly, making the identification of inappropriate content increasingly critical. Existing approaches typically train separate and small classification models for each type of issue, which requires extensive human-labeled data and lacks cross-issue generalization. We propose a reasoning-enhanced multimodal large language model (MLLM) pretraining paradigm for unified inappropriate content detection. To address the distribution gap between short video content and the original pretraining data of MLLMs, as well as the complex issue definitions, we introduce three targeted pretraining tasks: (1) \textit{Caption}, to enhance the MLLM’s perception of video details; (2) \textit{Visual Question Answering (VQA)}, to deepen the MLLM’s understanding of issue definitions and annotation guidelines; (3) \textit{Chain-of-Thought (CoT)}, to enhance the MLLM’s reasoning capability. Experimental results show that our pretraining approach significantly improves the MLLM’s performance in both zero-shot and supervised fine-tuning (SFT) settings. In addition, our pretrained model demonstrates strong generalization capabilities to emergent, previously unseen issues.

Method: Proposed GUI-Cursor model uses interactive search: at each step, the model identifies target objects, evaluates spatial relations, and moves cursor closer based on movement history, trained with multi-step online reinforcement learning.

Result: GUI-Cursor based on Qwen2.5-VL-7B improves GUI grounding accuracy, achieving state-of-the-art results on ScreenSpot-v2 (88.8% → 93.9%) and ScreenSpot-Pro (26.8% → 56.5%), solving problems within two steps for 95% of instances.

Conclusion: Interactive search approach with visual cursor feedback effectively addresses GUI grounding challenges, enabling adaptive step-wise problem solving and significantly outperforming coordinate prediction methods.

Abstract: Graphical User Interface (GUI) grounding is commonly framed as a coordinate prediction task – given a natural language instruction, generate on-screen coordinates for actions such as clicks and keystrokes. However, recent Vision Language Models (VLMs) often fail to predict accurate numeric coordinates when processing high-resolution GUI images with complex layouts. To address this issue, we reframe GUI grounding as an \emph{interactive search task}, where the VLM generates actions to move a cursor in the GUI to locate UI elements. At each step, the model determines the target object, evaluates the spatial relations between the cursor and the target, and moves the cursor closer to the target conditioned on the movement history. In this interactive process, the rendered cursor provides visual feedback to help the model align its predictions with the corresponding on-screen locations. We train our GUI grounding model, GUI-Cursor, using multi-step online reinforcement learning with a dense trajectory-based reward function. Our experimental results show that GUI-Cursor, based on Qwen2.5-VL-7B, improves the GUI grounding accuracy and achieves state-of-the-art results on ScreenSpot-v2 ($88.8% \rightarrow 93.9%$) and ScreenSpot-Pro ($26.8% \rightarrow 56.5%$). Moreover, we observe that GUI-Cursor learns to solve the problem within two steps for 95% of instances and can adaptively conduct more steps on more difficult examples.

Method: Proposes X-CoT framework that uses LLM Chain-of-Thought reasoning instead of embedding models. Expands benchmarks with additional video annotations for better semantic understanding and reduced data bias. Devises a retrieval CoT consisting of pairwise comparison steps for detailed reasoning and complete ranking.

Result: X-CoT empirically improves retrieval performance and produces detailed rationales for ranking decisions. It also facilitates model behavior analysis and data quality assessment.

Conclusion: X-CoT provides an explainable alternative to traditional text-to-video retrieval systems, addressing both performance and interpretability issues while enabling better analysis of model behavior and data quality.

Abstract: Prevalent text-to-video retrieval systems mainly adopt embedding models for feature extraction and compute cosine similarities for ranking. However, this design presents two limitations. Low-quality text-video data pairs could compromise the retrieval, yet are hard to identify and examine. Cosine similarity alone provides no explanation for the ranking results, limiting the interpretability. We ask that can we interpret the ranking results, so as to assess the retrieval models and examine the text-video data? This work proposes X-CoT, an explainable retrieval framework upon LLM CoT reasoning in place of the embedding model-based similarity ranking. We first expand the existing benchmarks with additional video annotations to support semantic understanding and reduce data bias. We also devise a retrieval CoT consisting of pairwise comparison steps, yielding detailed reasoning and complete ranking. X-CoT empirically improves the retrieval performance and produces detailed rationales. It also facilitates the model behavior and data quality analysis. Code and data are available at: https://github.com/PrasannaPulakurthi/X-CoT.

Method: Integrates background masking, patch-based analysis strategy, and efficient domain adaptation to overcome limitations of existing approaches.

Result: The method enables reliable detection of fine-grained defects in surgical instrument imagery by addressing domain shift and improving sensitivity to small defects.

Conclusion: The proposed versatile method successfully adapts unsupervised defect detection for surgical instruments, overcoming the limitations of existing approaches and enabling reliable defect detection in this specialized domain.

Abstract: Ensuring the safety of surgical instruments requires reliable detection of visual defects. However, manual inspection is prone to error, and existing automated defect detection methods, typically trained on natural/industrial images, fail to transfer effectively to the surgical domain. We demonstrate that simply applying or fine-tuning these approaches leads to issues: false positive detections arising from textured backgrounds, poor sensitivity to small, subtle defects, and inadequate capture of instrument-specific features due to domain shift. To address these challenges, we propose a versatile method that adapts unsupervised defect detection methods specifically for surgical instruments. By integrating background masking, a patch-based analysis strategy, and efficient domain adaptation, our method overcomes these limitations, enabling the reliable detection of fine-grained defects in surgical instrument imagery.

Method: Introduces Linear SEParability (LSEP) regularization that directly incorporates linear probing into network learning dynamics, promoting linear separability of intermediate layer representations without auxiliary encoders.

Result: Achieves substantial improvements in training efficiency and generation quality, with FID of 1.46 on 256×256 ImageNet dataset using flow-based transformer architectures like SiTs.

Conclusion: LSEP provides effective alternative to alignment-based methods, eliminating dependency on external encoders while maintaining or improving representation quality and generation performance.

Abstract: Efficient training strategies for large-scale diffusion models have recently emphasized the importance of improving discriminative feature representations in these models. A central line of work in this direction is representation alignment with features obtained from powerful external encoders, which improves the representation quality as assessed through linear probing. Alignment-based approaches show promise but depend on large pretrained encoders, which are computationally expensive to obtain. In this work, we propose an alternative regularization for training, based on promoting the Linear SEParability (LSEP) of intermediate layer representations. LSEP eliminates the need for an auxiliary encoder and representation alignment, while incorporating linear probing directly into the network’s learning dynamics rather than treating it as a simple post-hoc evaluation tool. Our results demonstrate substantial improvements in both training efficiency and generation quality on flow-based transformer architectures such as SiTs, achieving an FID of 1.46 on $256 \times 256$ ImageNet dataset.

Method: Integrates semivariogram - a geostatistical tool - into contrastive learning by relating feature space distance to geographical distance, capturing spatial correlation patterns to identify hard negatives and false negatives based on expected visual dissimilarity at given spatial distances.

Result: Evaluation on OSV5M dataset shows improved geo-localization performance, especially at finer granularity, demonstrating that explicit spatial prior modeling enhances accuracy.

Conclusion: Modeling spatial priors through semivariogram-based regularization significantly improves contrastive learning for image-based geo-localization by better handling spatial dependencies and addressing false/hard negative issues.

Abstract: Accurate and robust image-based geo-localization at a global scale is challenging due to diverse environments, visually ambiguous scenes, and the lack of distinctive landmarks in many regions. While contrastive learning methods show promising performance by aligning features between street-view images and corresponding locations, they neglect the underlying spatial dependency in the geographic space. As a result, they fail to address the issue of false negatives – image pairs that are both visually and geographically similar but labeled as negatives, and struggle to effectively distinguish hard negatives, which are visually similar but geographically distant. To address this issue, we propose a novel spatially regularized contrastive learning strategy that integrates a semivariogram, which is a geostatistical tool for modeling how spatial correlation changes with distance. We fit the semivariogram by relating the distance of images in feature space to their geographical distance, capturing the expected visual content in a spatial correlation. With the fitted semivariogram, we define the expected visual dissimilarity at a given spatial distance as reference to identify hard negatives and false negatives. We integrate this strategy into GeoCLIP and evaluate it on the OSV5M dataset, demonstrating that explicitly modeling spatial priors improves image-based geo-localization performance, particularly at finer granularity.

Method: Uses a Thinker-Actor dual-transformer architecture: Thinker module (pretrained large language-speech model) perceives and reasons over streaming inputs, while Actor module (chunk-wise autoregressive diffusion model) cross-attends to Thinker’s hidden states to generate synchronized multimodal responses with interleaved text/audio tokens and video latents.

Result: X-Streamer runs in real time on two A100 GPUs, sustaining hours-long consistent video chat experiences from arbitrary portraits with fine-grained cross-modality alignment and long-horizon stability.

Conclusion: The framework paves the way toward unified world modeling of interactive digital humans by enabling persistent, intelligent audiovisual interactions through a single unified architecture.

Abstract: We introduce X-Streamer, an end-to-end multimodal human world modeling framework for building digital human agents capable of infinite interactions across text, speech, and video within a single unified architecture. Starting from a single portrait, X-Streamer enables real-time, open-ended video calls driven by streaming multimodal inputs. At its core is a Thinker-Actor dual-transformer architecture that unifies multimodal understanding and generation, turning a static portrait into persistent and intelligent audiovisual interactions. The Thinker module perceives and reasons over streaming user inputs, while its hidden states are translated by the Actor into synchronized multimodal streams in real time. Concretely, the Thinker leverages a pretrained large language-speech model, while the Actor employs a chunk-wise autoregressive diffusion model that cross-attends to the Thinker’s hidden states to produce time-aligned multimodal responses with interleaved discrete text and audio tokens and continuous video latents. To ensure long-horizon stability, we design inter- and intra-chunk attentions with time-aligned multimodal positional embeddings for fine-grained cross-modality alignment and context retention, further reinforced by chunk-wise diffusion forcing and global identity referencing. X-Streamer runs in real time on two A100 GPUs, sustaining hours-long consistent video chat experiences from arbitrary portraits and paving the way toward unified world modeling of interactive digital humans.

Method: Formulate motion forecasting as conditional generation of dense trajectory grids using an architecture similar to modern video generators but outputting motion trajectories instead of pixels.

Result: The approach captures scene-wide dynamics and uncertainty, yielding more accurate and diverse predictions than prior regressors and generators, and shows effectiveness in robotics applications and real-world physics datasets.

Conclusion: Directly modeling motion trajectories rather than generating pixels enables more effective motion forecasting from single images, overcoming limitations of current video generators even when fine-tuned on physical scenarios.

Abstract: We consider the problem of forecasting motion from a single image, i.e., predicting how objects in the world are likely to move, without the ability to observe other parameters such as the object velocities or the forces applied to them. We formulate this task as conditional generation of dense trajectory grids with a model that closely follows the architecture of modern video generators but outputs motion trajectories instead of pixels. This approach captures scene-wide dynamics and uncertainty, yielding more accurate and diverse predictions than prior regressors and generators. We extensively evaluate our method on simulated data, demonstrate its effectiveness on downstream applications such as robotics, and show promising accuracy on real-world intuitive physics datasets. Although recent state-of-the-art video generators are often regarded as world models, we show that they struggle with forecasting motion from a single image, even in simple physical scenarios such as falling blocks or mechanical object interactions, despite fine-tuning on such data. We show that this limitation arises from the overhead of generating pixels rather than directly modeling motion.

Method: Evaluated both architectures on UCF Sports dataset using multiple metrics: classification accuracy, clustering performance (Silhouette score), intra-class consistency, and inter-class discrimination.

Result: DINOv3 achieved superior clustering (Silhouette: 0.31 vs 0.21) and discrimination (6.16x separation ratio) for pose-identifiable actions, while V-JEPA2 showed lower performance variance (0.094 vs 0.288) and balanced performance across all action types.

Conclusion: Spatial processing excels at static pose recognition but degrades on motion-dependent actions, while temporal modeling provides consistent reliability across diverse actions, informing architectural choices based on task requirements.

Abstract: This study presents a comprehensive comparative analysis of two prominent self-supervised learning architectures for video action recognition: DINOv3, which processes frames independently through spatial feature extraction, and V-JEPA2, which employs joint temporal modeling across video sequences. We evaluate both approaches on the UCF Sports dataset, examining feature quality through multiple dimensions including classification accuracy, clustering performance, intra-class consistency, and inter-class discrimination. Our analysis reveals fundamental architectural trade-offs: DINOv3 achieves superior clustering performance (Silhouette score: 0.31 vs 0.21) and demonstrates exceptional discrimination capability (6.16x separation ratio) particularly for pose-identifiable actions, while V-JEPA2 exhibits consistent reliability across all action types with significantly lower performance variance (0.094 vs 0.288). Through action-specific evaluation, we identify that DINOv3’s spatial processing architecture excels at static pose recognition but shows degraded performance on motion-dependent actions, whereas V-JEPA2’s temporal modeling provides balanced representation quality across diverse action categories. These findings contribute to the understanding of architectural design choices in video analysis systems and provide empirical guidance for selecting appropriate feature extraction methods based on task requirements and reliability constraints.

Method: Uses dual-architecture approach: CNN-LSTM with ResNet50 backbone for satellite imagery (xBD dataset) and Vision Transformer (ViT) for UAV pictures (RescueNet dataset), enhanced with external semantic knowledge from ConceptNet and WordNet.

Result: VLCE significantly outperforms baseline models (LLaVA and QwenVL), achieving up to 95.33% on InfoMetIC for caption informativeness while maintaining competitive semantic alignment measured by CLIPScore.

Conclusion: The dual-architecture system shows significant potential for improving disaster damage assessment by automating the production of actionable, information-dense descriptions from aerial imagery.

Abstract: Immediate damage assessment is essential after natural catastrophes; yet, conventional hand evaluation techniques are sluggish and perilous. Although satellite and unmanned aerial vehicle (UAV) photos offer extensive perspectives of impacted regions, current computer vision methodologies generally yield just classification labels or segmentation masks, so constraining their capacity to deliver a thorough situational comprehension. We introduce the Vision Language Caption Enhancer (VLCE), a multimodal system designed to produce comprehensive, contextually-informed explanations of disaster imagery. VLCE employs a dual-architecture approach: a CNN-LSTM model with a ResNet50 backbone pretrained on EuroSat satellite imagery for the xBD dataset, and a Vision Transformer (ViT) model pretrained on UAV pictures for the RescueNet dataset. Both systems utilize external semantic knowledge from ConceptNet and WordNet to expand vocabulary coverage and improve description accuracy. We assess VLCE in comparison to leading vision-language models (LLaVA and QwenVL) utilizing CLIPScore for semantic alignment and InfoMetIC for caption informativeness. Experimental findings indicate that VLCE markedly surpasses baseline models, attaining a maximum of 95.33% on InfoMetIC while preserving competitive semantic alignment. Our dual-architecture system demonstrates significant potential for improving disaster damage assessment by automating the production of actionable, information-dense descriptions from satellite and drone photos.

Method: Used multiple representational similarity metrics (RSA, Soft Matching, Linear Predictivity) and integrated them using Similarity Network Fusion (SNF) to create composite signatures for clustering analysis.

Result: Geometry and tuning metrics strongly separate model families, while linear decodability shows weaker separation. SNF achieved substantially sharper family discrimination and revealed clustering patterns: supervised ResNets/ViTs form distinct clusters, self-supervised models group together, and hybrid architectures cluster with masked autoencoders.

Conclusion: Computational strategies shaped by both architecture and training objective define representational structure beyond surface design categories, with convergence between architectural modernization and reconstruction-based training.

Abstract: Large vision models differ widely in architecture and training paradigm, yet we lack principled methods to determine which aspects of their representations are shared across families and which reflect distinctive computational strategies. We leverage a suite of representational similarity metrics, each capturing a different facet-geometry, unit tuning, or linear decodability-and assess family separability using multiple complementary measures. Metrics preserving geometry or tuning (e.g., RSA, Soft Matching) yield strong family discrimination, whereas flexible mappings such as Linear Predictivity show weaker separation. These findings indicate that geometry and tuning carry family-specific signatures, while linearly decodable information is more broadly shared. To integrate these complementary facets, we adapt Similarity Network Fusion (SNF), a method inspired by multi-omics integration. SNF achieves substantially sharper family separation than any individual metric and produces robust composite signatures. Clustering of the fused similarity matrix recovers both expected and surprising patterns: supervised ResNets and ViTs form distinct clusters, yet all self-supervised models group together across architectural boundaries. Hybrid architectures (ConvNeXt, Swin) cluster with masked autoencoders, suggesting convergence between architectural modernization and reconstruction-based training. This biology-inspired framework provides a principled typology of vision models, showing that emergent computational strategies-shaped jointly by architecture and training objective-define representational structure beyond surface design categories.

Method: Augments frozen video foundation models with a trainable geometric branch that enables joint modeling of video latents and implicit 3D fields in a single forward pass, using cross-branch supervision where geometry cues guide video generation and video priors regularize 3D prediction.

Result: Outperforms recent geometry-consistent baselines in multi-view coherence and style consistency, effectively bridging video imagination and 3D perception without requiring per-scene optimization or fine-tuning.

Conclusion: FantasyWorld successfully integrates video imagination with 3D perception through unified backbone and cross-branch information exchange, producing consistent and generalizable 3D-aware video representations suitable for downstream 3D tasks.

Abstract: High-quality 3D world models are pivotal for embodied intelligence and Artificial General Intelligence (AGI), underpinning applications such as AR/VR content creation and robotic navigation. Despite the established strong imaginative priors, current video foundation models lack explicit 3D grounding capabilities, thus being limited in both spatial consistency and their utility for downstream 3D reasoning tasks. In this work, we present FantasyWorld, a geometry-enhanced framework that augments frozen video foundation models with a trainable geometric branch, enabling joint modeling of video latents and an implicit 3D field in a single forward pass. Our approach introduces cross-branch supervision, where geometry cues guide video generation and video priors regularize 3D prediction, thus yielding consistent and generalizable 3D-aware video representations. Notably, the resulting latents from the geometric branch can potentially serve as versatile representations for downstream 3D tasks such as novel view synthesis and navigation, without requiring per-scene optimization or fine-tuning. Extensive experiments show that FantasyWorld effectively bridges video imagination and 3D perception, outperforming recent geometry-consistent baselines in multi-view coherence and style consistency. Ablation studies further confirm that these gains stem from the unified backbone and cross-branch information exchange.

Method: Built on convolutional vision transformer with: (i) component-wise convolution for joint scalar/vector processing, (ii) inter-field cross-attention for information propagation between fields, (iii) axial attentions to reduce computational burden while maintaining expressivity. Pretrained on diverse PDE datasets with evaluation using full fine-tuning and LoRA adapters.

Result: MORPH outperforms models trained from scratch in both zero-shot and full-shot generalization, matching or surpassing strong baselines and state-of-the-art models across extensive evaluations.

Conclusion: MORPH presents a powerful and flexible foundation model for scientific machine learning that effectively handles heterogeneous PDE data and enables efficient transfer learning across diverse prediction tasks.

Abstract: We introduce MORPH, a shape-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data dimensionality (1D–3D) at different resolutions, multiple fields with mixed scalar and vector components. The architecture combines (i) component-wise convolution, which jointly processes scalar and vector channels to capture local interactions, (ii) inter-field cross-attention, which models and selectively propagates information between different physical fields, (iii) axial attentions, which factorizes full spatiotemporal self-attention along individual spatial and temporal axes to reduce computational burden while retaining expressivity. We pretrain multiple model variants on a diverse collection of heterogeneous PDE datasets and evaluate transfer to a range of downstream prediction tasks. Using both full-model fine-tuning and parameter-efficient low-rank adapters (LoRA), MORPH outperforms models trained from scratch in both zero-shot and full-shot generalization. Across extensive evaluations, MORPH matches or surpasses strong baselines and recent state-of-the-art models. Collectively, these capabilities present a flexible and powerful backbone for learning from heterogeneous and multimodal nature of scientific observations, charting a path toward scalable and data-efficient scientific machine learning.

Method: Replaces YOLOv8’s CSPDarknet with MobileNetV4 backbone and introduces SlideLoss - a novel loss function that dynamically emphasizes under-represented and occluded samples.

Result: Achieves competitive mAP and superior precision with only 6.7 GFLOPs, reducing computational overhead by 1.5% while sustaining high accuracy on FLIR ADAS V2 dataset.

Conclusion: MS-YOLO effectively balances high detection quality with minimal computational costs, making it suitable for real-time edge deployment in urban environments.

Abstract: Infrared imaging has emerged as a robust solution for urban object detection under low-light and adverse weather conditions, offering significant advantages over traditional visible-light cameras. However, challenges such as class imbalance, thermal noise, and computational constraints can significantly hinder model performance in practical settings. To address these issues, we evaluate multiple YOLO variants on the FLIR ADAS V2 dataset, ultimately selecting YOLOv8 as our baseline due to its balanced accuracy and efficiency. Building on this foundation, we present \texttt{MS-YOLO} (\textbf{M}obileNetv4 and \textbf{S}lideLoss based on YOLO), which replaces YOLOv8’s CSPDarknet backbone with the more efficient MobileNetV4, reducing computational overhead by \textbf{1.5%} while sustaining high accuracy. In addition, we introduce \emph{SlideLoss}, a novel loss function that dynamically emphasizes under-represented and occluded samples, boosting precision without sacrificing recall. Experiments on the FLIR ADAS V2 benchmark show that \texttt{MS-YOLO} attains competitive mAP and superior precision while operating at only \textbf{6.7 GFLOPs}. These results demonstrate that \texttt{MS-YOLO} effectively addresses the dual challenge of maintaining high detection quality while minimizing computational costs, making it well-suited for real-time edge deployment in urban environments.

Method: Motion-Aware Transformer (MATR) explicitly predicts object movements across frames to update track queries in advance, reducing query collisions and enabling more consistent training for both detection and association.

Result: MATR achieves significant improvements: 71.3 HOTA on DanceTrack (9+ points over MOTR), 72.2 HOTA on SportsMOT, and 54.7 mTETA/41.6 mHOTA on BDD100k, all state-of-the-art results without external datasets.

Conclusion: Explicitly modeling motion within end-to-end Transformers provides a simple yet highly effective approach to advance multi-object tracking performance across diverse datasets.

Abstract: Multi-object tracking (MOT) in videos remains challenging due to complex object motions and crowded scenes. Recent DETR-based frameworks offer end-to-end solutions but typically process detection and tracking queries jointly within a single Transformer Decoder layer, leading to conflicts and degraded association accuracy. We introduce the Motion-Aware Transformer (MATR), which explicitly predicts object movements across frames to update track queries in advance. By reducing query collisions, MATR enables more consistent training and improves both detection and association. Extensive experiments on DanceTrack, SportsMOT, and BDD100k show that MATR delivers significant gains across standard metrics. On DanceTrack, MATR improves HOTA by more than 9 points over MOTR without additional data and reaches a new state-of-the-art score of 71.3 with supplementary data. MATR also achieves state-of-the-art results on SportsMOT (72.2 HOTA) and BDD100k (54.7 mTETA, 41.6 mHOTA) without relying on external datasets. These results demonstrate that explicitly modeling motion within end-to-end Transformers offers a simple yet highly effective approach to advancing multi-object tracking.

Method: Proposes two components: 1) rotation-bounded Lie relative bias for geometry-consistent alignment using in-plane angle prediction, and 2) differential group displacement that computes angular differences between frames with temporal decay and attention masks.

Result: Extensive experiments show effectiveness on publicly available benchmarks.

Conclusion: Lie groups provide a principled way to enforce spatial and temporal consistency in video deraining, making DeLiVR an efficient and robust solution.

Abstract: Videos captured in the wild often suffer from rain streaks, blur, and noise. In addition, even slight changes in camera pose can amplify cross-frame mismatches and temporal artifacts. Existing methods rely on optical flow or heuristic alignment, which are computationally expensive and less robust. To address these challenges, Lie groups provide a principled way to represent continuous geometric transformations, making them well-suited for enforcing spatial and temporal consistency in video modeling. Building on this insight, we propose DeLiVR, an efficient video deraining method that injects spatiotemporal Lie-group differential biases directly into attention scores of the network. Specifically, the method introduces two complementary components. First, a rotation-bounded Lie relative bias predicts the in-plane angle of each frame using a compact prediction module, where normalized coordinates are rotated and compared with base coordinates to achieve geometry-consistent alignment before feature aggregation. Second, a differential group displacement computes angular differences between adjacent frames to estimate a velocity. This bias computation combines temporal decay and attention masks to focus on inter-frame relationships while precisely matching the direction of rain streaks. Extensive experimental results demonstrate the effectiveness of our method on publicly available benchmarks.

Method: Two-stage approach: given initial screen image and user action, first predicts abstract layout of next UI state, then synthesizes visually consistent image based on predicted layout to simulate realistic UI transitions.

Result: UISim outperforms end-to-end UI generation baselines in generating realistic and coherent subsequent UI states, demonstrating high fidelity and effectiveness for UI simulation.

Conclusion: UISim provides practical benefits for UI testing, rapid prototyping, and synthetic data generation, while enabling advanced applications like UI navigation task planning for AI agents, streamlining UI development and enhancing AI training.

Abstract: Developing and testing user interfaces (UIs) and training AI agents to interact with them are challenging due to the dynamic and diverse nature of real-world mobile environments. Existing methods often rely on cumbersome physical devices or limited static analysis of screenshots, which hinders scalable testing and the development of intelligent UI agents. We introduce UISim, a novel image-based UI simulator that offers a dynamic and interactive platform for exploring mobile phone environments purely from screen images. Our system employs a two-stage method: given an initial phone screen image and a user action, it first predicts the abstract layout of the next UI state, then synthesizes a new, visually consistent image based on this predicted layout. This approach enables the realistic simulation of UI transitions. UISim provides immediate practical benefits for UI testing, rapid prototyping, and synthetic data generation. Furthermore, its interactive capabilities pave the way for advanced applications, such as UI navigation task planning for AI agents. Our experimental results show that UISim outperforms end-to-end UI generation baselines in generating realistic and coherent subsequent UI states, highlighting its fidelity and potential to streamline UI development and enhance AI agent training.

Method: Proposed LFA-Net with LiteFusion-Attention module that incorporates residual learning connections, Vision Mamba-inspired dynamics, and modulation-based attention for efficient local and global context capture.

Result: Achieved outstanding performance on DRIVE, STARE, and CHASE_DB datasets with dice scores of 83.28%, 87.44%, 84.50% and Jaccard indices of 72.85%, 79.31%, 74.70% respectively, using only 0.11M parameters, 0.42MB memory, and 4.46 GFLOPs.

Conclusion: LFA-Net provides high-performance retinal vessel segmentation with minimal computational requirements, making it ideal for real-world clinical applications with limited resources.

Abstract: Lightweight retinal vessel segmentation is important for the early diagnosis of vision-threatening and systemic diseases, especially in a real-world clinical environment with limited computational resources. Although segmentation methods based on deep learning are improving, existing models are still facing challenges of small vessel segmentation and high computational costs. To address these challenges, we proposed a new vascular segmentation network, LFA-Net, which incorporates a newly designed attention module, LiteFusion-Attention. This attention module incorporates residual learning connections, Vision Mamba-inspired dynamics, and modulation-based attention, enabling the model to capture local and global context efficiently and in a lightweight manner. LFA-Net offers high performance with 0.11 million parameters, 0.42 MB memory size, and 4.46 GFLOPs, which make it ideal for resource-constrained environments. We validated our proposed model on DRIVE, STARE, and CHASE_DB with outstanding performance in terms of dice scores of 83.28, 87.44, and 84.50% and Jaccard indices of 72.85, 79.31, and 74.70%, respectively. The code of LFA-Net is available online https://github.com/Mehwish4593/LFA-Net.

Method: Proposes a framework with visual context encoding module using multi-scale scene information, emotion semantic encoding module using LLM-generated emotion lexicons refined through structured emotion tree, and similarity-aware interaction to align visual and semantic information.

Result: Achieves competitive performance compared to state-of-the-art methods on three widely adopted GER datasets.

Conclusion: The proposed integration of visual scene context and label-guided semantic information effectively enhances group-level emotion recognition performance.

Abstract: Group-level emotion recognition (GER) aims to identify holistic emotions within a scene involving multiple individuals. Current existed methods underestimate the importance of visual scene contextual information in modeling individual relationships. Furthermore, they overlook the crucial role of semantic information from emotional labels for complete understanding of emotions. To address this limitation, we propose a novel framework that incorporates visual scene context and label-guided semantic information to improve GER performance. It involves the visual context encoding module that leverages multi-scale scene information to diversely encode individual relationships. Complementarily, the emotion semantic encoding module utilizes group-level emotion labels to prompt a large language model to generate nuanced emotion lexicons. These lexicons, in conjunction with the emotion labels, are then subsequently refined into comprehensive semantic representations through the utilization of a structured emotion tree. Finally, similarity-aware interaction is proposed to align and integrate visual and semantic information, thereby generating enhanced group-level emotion representations and subsequently improving the performance of GER. Experiments on three widely adopted GER datasets demonstrate that our proposed method achieves competitive performance compared to state-of-the-art methods.

Method: KG-SAM integrates: (i) medical knowledge graph for anatomical relationships, (ii) energy-based Conditional Random Field for anatomically consistent predictions, and (iii) uncertainty-aware fusion module for enhanced reliability.

Result: Achieved 82.69% average Dice score on prostate segmentation, 78.05% on abdominal MRI segmentation, and 79.68% on abdominal CT segmentation across multi-center datasets.

Conclusion: KG-SAM establishes a robust and generalizable framework that significantly advances medical image segmentation by synergistically combining anatomical priors with uncertainty-aware boundary refinement.

Abstract: While the Segment Anything Model (SAM) has achieved remarkable success in image segmentation, its direct application to medical imaging remains hindered by fundamental challenges, including ambiguous boundaries, insufficient modeling of anatomical relationships, and the absence of uncertainty quantification. To address these limitations, we introduce KG-SAM, a knowledge-guided framework that synergistically integrates anatomical priors with boundary refinement and uncertainty estimation. Specifically, KG-SAM incorporates (i) a medical knowledge graph to encode fine-grained anatomical relationships, (ii) an energy-based Conditional Random Field (CRF) to enforce anatomically consistent predictions, and (iii) an uncertainty-aware fusion module to enhance reliability in high-stakes clinical scenarios. Extensive experiments across multi-center medical datasets demonstrate the effectiveness of our approach: KG-SAM achieves an average Dice score of 82.69% on prostate segmentation and delivers substantial gains in abdominal segmentation, reaching 78.05% on MRI and 79.68% on CT. These results establish KG-SAM as a robust and generalizable framework for advancing medical image segmentation.

Method: Fine-tunes a video diffusion transformer to handle various vision tasks by representing tasks as visual sentences, where context sequences define both the task and expected output modality. Tasks can switch between understanding and generation by reversing visual sentence order.

Result: UniVid generalizes well in cross-modal inference (images and videos) and cross-source tasks (natural to annotated data) despite being trained only on natural video data. It demonstrates the potential of pre-trained video generation models as scalable unified vision foundations.

Conclusion: Pre-trained video generation models can serve as scalable and unified foundations for vision modeling, enabling generalization across modalities and sources without multi-source pre-training.

Abstract: Large language models, trained on extensive corpora, successfully unify diverse linguistic tasks within a single generative framework. Inspired by this, recent works like Large Vision Model (LVM) extend this paradigm to vision by organizing tasks into sequential visual sentences, where visual prompts serve as the context to guide outputs. However, such modeling requires task-specific pre-training across modalities and sources, which is costly and limits scalability to unseen tasks. Given that pre-trained video generation models inherently capture temporal sequence dependencies, we explore a more unified and scalable alternative: can a pre-trained video generation model adapt to diverse image and video tasks? To answer this, we propose UniVid, a framework that fine-tunes a video diffusion transformer to handle various vision tasks without task-specific modifications. Tasks are represented as visual sentences, where the context sequence defines both the task and the expected output modality. We evaluate the generalization of UniVid from two perspectives: (1) cross-modal inference with contexts composed of both images and videos, extending beyond LVM’s uni-modal setting; (2) cross-source tasks from natural to annotated data, without multi-source pre-training. Despite being trained solely on natural video data, UniVid generalizes well in both settings. Notably, understanding and generation tasks can easily switch by simply reversing the visual sentence order in this paradigm. These findings highlight the potential of pre-trained video generation models to serve as a scalable and unified foundation for vision modeling. Our code will be released at https://github.com/CUC-MIPG/UniVid.

Method: 2D reduction strategy for structured token layouts, spatial-aware merging algorithm that maintains relative positions, and max-magnitude-per-dimension token representation to preserve salient features.

Result: 1.25x speedup on SAM-H with only 0.7% mIOU drop on COCO off-the-shelf, and 1.15x speedup on DeiT-B with no top-1 accuracy drop on ImageNet after one epoch fine-tuning.

Conclusion: The method achieves state-of-the-art performance on both spatial and non-spatial architectures across various vision tasks while maintaining compatibility with spatial designs.

Abstract: Many modern ViT backbones adopt spatial architectural designs, such as window attention, decomposed relative positional embeddings in SAM, and RoPE in DINOv3. Such architectures impose new challenges on token reduction, as the vast majority of existing methods fail to preserve the spatial structure these architectures depend on. In this paper, we introduce a simple yet effective token merging method that maintains spatial integrity, enabling seamless compatibility with spatial architectures. We reconcile two seemingly conflicting requirements: (i)exploiting the uneven information distribution across the spatial layout while (ii)preserving the spatial structure post-merging. Our approach employs (i)a 2D reduction strategy to enforce structured token layouts, (ii)a spatial-aware merging algorithm that maintains relative token positions, and (iii)a novel max-magnitude-per-dimension token representation that preserves salient features. Our method demonstrates strong performance both off-the-shelf and with fine-tuning, achieving state-of-the-art results on spatial and non-spatial architectures across various vision tasks. Specifically, we achieve 1.25x speedup on SAM-H with only 0.7% mIOU drop evaluated on COCO off-the-shelf, and 1.15x speedup on DeiT-B with no top-1 accuracy drop on ImageNet within just one epoch of fine-tuning.

Method: Uses MDD-adapted feature extractor for aligned image-text pair retrieval, Graph-Structured Taylor Adaptive Scorer (GSTAS) for cross-sample relations and query-aligned signal propagation, and in-context learning to inject task-aware knowledge into LVLMs.

Result: Outperforms strong baselines on four forgery types without requiring LVLM fine-tuning.

Conclusion: GASP-ICL effectively enhances LVLMs for robust multimodal deepfake detection through training-free adaptation and precise demonstration selection.

Abstract: Multimodal deepfake detection (MDD) aims to uncover manipulations across visual, textual, and auditory modalities, thereby reinforcing the reliability of modern information systems. Although large vision-language models (LVLMs) exhibit strong multimodal reasoning, their effectiveness in MDD is limited by challenges in capturing subtle forgery cues, resolving cross-modal inconsistencies, and performing task-aligned retrieval. To this end, we propose Guided Adaptive Scorer and Propagation In-Context Learning (GASP-ICL), a training-free framework for MDD. GASP-ICL employs a pipeline to preserve semantic relevance while injecting task-aware knowledge into LVLMs. We leverage an MDD-adapted feature extractor to retrieve aligned image-text pairs and build a candidate set. We further design the Graph-Structured Taylor Adaptive Scorer (GSTAS) to capture cross-sample relations and propagate query-aligned signals, producing discriminative exemplars. This enables precise selection of semantically aligned, task-relevant demonstrations, enhancing LVLMs for robust MDD. Experiments on four forgery types show that GASP-ICL surpasses strong baselines, delivering gains without LVLM fine-tuning.

Method: ProDA uses Spatio-temporal Scene Graphs (SSGs) and introduces Dynamic Prompt Module (DPM) to guide a Graph Parsing Neural Network (GPNN) in generating action-specific representations. It features a video-adapted GPNN that aggregates information using dynamic weights.

Result: Experiments in video action recognition demonstrate the effectiveness of ProDA when compared with state-of-the-art methods.

Conclusion: The proposed ProDA framework successfully disentangles specified actions from multi-action scenes and achieves competitive performance in action recognition tasks.

Abstract: Action recognition is a fundamental task in video understanding. Existing methods typically extract unified features to process all actions in one video, which makes it challenging to model the interactions between different objects in multi-action scenarios. To alleviate this issue, we explore disentangling any specified actions from complex scenes as an effective solution. In this paper, we propose Prompt-guided Disentangled Representation for Action Recognition (ProDA), a novel framework that disentangles any specified actions from a multi-action scene. ProDA leverages Spatio-temporal Scene Graphs (SSGs) and introduces Dynamic Prompt Module (DPM) to guide a Graph Parsing Neural Network (GPNN) in generating action-specific representations. Furthermore, we design a video-adapted GPNN that aggregates information using dynamic weights. Experiments in video action recognition demonstrate the effectiveness of our approach when compared with the state-of-the-art methods. Our code can be found in https://github.com/iamsnaping/ProDA.git

Method: Uses watermarked, stability-enhanced stable diffusion techniques combined with Digital Attention Analysis Module (DAAM) to identify hateful elements in images and generate hate attention maps for blurring hateful regions.

Result: Created a multimodal dataset for hate identification and developed DeHater model that sets new standards in AI-driven image hate detection with textual prompts.

Conclusion: The approach contributes to developing more ethical AI applications in social media by providing effective tools for multimodal hate detection and content moderation.

Abstract: The rise in harmful online content not only distorts public discourse but also poses significant challenges to maintaining a healthy digital environment. In response to this, we introduce a multimodal dataset uniquely crafted for identifying hate in digital content. Central to our methodology is the innovative application of watermarked, stability-enhanced, stable diffusion techniques combined with the Digital Attention Analysis Module (DAAM). This combination is instrumental in pinpointing the hateful elements within images, thereby generating detailed hate attention maps, which are used to blur these regions from the image, thereby removing the hateful sections of the image. We release this data set as a part of the dehate shared task. This paper also describes the details of the shared task. Furthermore, we present DeHater, a vision-language model designed for multimodal dehatification tasks. Our approach sets a new standard in AI-driven image hate detection given textual prompts, contributing to the development of more ethical AI applications in social media.

Method: Two-stage training: supervised fine-tuning with annotated trajectories and image-aware RL optimization. Uses a novel trajectory construction method integrating object-level and image-level annotations, and designs an image-aware RL policy with dual reward functions for objects and images.

Result: Achieves SOTA performance with 64.82% on cross-image reasoning tasks, exceeding previous best method by 1%.

Conclusion: MIRG-RL effectively addresses cross-image reasoning challenges and demonstrates superior performance in multi-image grounding benchmarks.

Abstract: Multi-image reasoning and grounding require understanding complex cross-image relationships at both object levels and image levels. Current Large Visual Language Models (LVLMs) face two critical challenges: the lack of cross-image reasoning capabilities and insufficient cross-image reference reward modeling. To address these issues, we propose a unified framework - Multi-Image Reasoning and Grounding with Reinforcement Learning (MIRG-RL). Specifically, our two-stage training paradigm combines supervised fine-tuning with annotated trajectories and image-aware reinforcement learning optimization, progressively developing multi-image reasoning capabilities. Furthermore, we innovatively propose a method for constructing the trajectory data, which integrates object-level and image-level annotation information, and use this method to generate a lightweight reasoning-enhanced dataset. To effectively resolve cross-image ambiguities, we design an image-aware RL policy with dual reward functions for objects and images. Experiments demonstrate that MIRG-RL achieves state-of-the-art (SOTA) performance in multi-image grounding benchmarks, attaining 64.82% on cross-image reasoning tasks - exceeding the previous best method by 1%. The code and dataset have been released at https://github.com/ZEUS2035/MIRG-RL.

Method: Hybrid framework combining intra-chunk diffusion denoising with inter-chunk autoregressive generation, featuring action-guided variable-length chunking and Context-aware Mixture-of-Experts (CMoE) that adaptively activates specialized experts for each chunk.

Result: Achieves stable and consistent long-horizon generation over extended rollouts, demonstrating improved temporal consistency and visual quality compared to existing methods.

Conclusion: LongScape successfully addresses the limitations of current video generation approaches by providing a flexible framework that maintains both visual detail and temporal consistency for long-horizon embodied manipulation scenarios.

Abstract: Video-based world models hold significant potential for generating high-quality embodied manipulation data. However, current video generation methods struggle to achieve stable long-horizon generation: classical diffusion-based approaches often suffer from temporal inconsistency and visual drift over multiple rollouts, while autoregressive methods tend to compromise on visual detail. To solve this, we introduce LongScape, a hybrid framework that adaptively combines intra-chunk diffusion denoising with inter-chunk autoregressive causal generation. Our core innovation is an action-guided, variable-length chunking mechanism that partitions video based on the semantic context of robotic actions. This ensures each chunk represents a complete, coherent action, enabling the model to flexibly generate diverse dynamics. We further introduce a Context-aware Mixture-of-Experts (CMoE) framework that adaptively activates specialized experts for each chunk during generation, guaranteeing high visual quality and seamless chunk transitions. Extensive experimental results demonstrate that our method achieves stable and consistent long-horizon generation over extended rollouts. Our code is available at: https://github.com/tsinghua-fib-lab/Longscape.

Method: Proposes MoWM framework that uses motion-aware representations from latent models as high-level priors to guide extraction of fine-grained visual features from pixel space models, highlighting informative visual details for action decoding.

Result: Extensive evaluations on CALVIN benchmark demonstrate state-of-the-art task success rates and superior generalization compared to existing methods.

Conclusion: The hybrid approach effectively combines motion awareness with fine-grained visual details, providing valuable insights for future embodied planning research.

Abstract: Embodied action planning is a core challenge in robotics, requiring models to generate precise actions from visual observations and language instructions. While video generation world models are promising, their reliance on pixel-level reconstruction often introduces visual redundancies that hinder action decoding and generalization. Latent world models offer a compact, motion-aware representation, but overlook the fine-grained details critical for precise manipulation. To overcome these limitations, we propose MoWM, a mixture-of-world-model framework that fuses representations from hybrid world models for embodied action planning. Our approach uses motion-aware representations from a latent model as a high-level prior, which guides the extraction of fine-grained visual features from the pixel space model. This design allows MoWM to highlight the informative visual details needed for action decoding. Extensive evaluations on the CALVIN benchmark demonstrate that our method achieves state-of-the-art task success rates and superior generalization. We also provide a comprehensive analysis of the strengths of each feature space, offering valuable insights for future research in embodied planning. The code is available at: https://github.com/tsinghua-fib-lab/MoWM.

Method: 1) Foreground-background separation guidance using LLM to convert prompts into foreground/background components. 2) Inter-frame Spatial-Temporal Decoupled 3D-RoPE (STD-RoPE) that modifies foreground tokens’ position embedding to eliminate cross-frame spatial discrepancies and strengthen cross-frame attention. 3) 3D-aware trajectory control through position embedding density regulation.

Result: Extensive experiments show DiTraj outperforms previous methods in both video quality and trajectory controllability.

Conclusion: DiTraj provides an effective training-free framework for trajectory control in DiT-based video generation, achieving superior performance without requiring additional training resources.

Abstract: Diffusion Transformers (DiT)-based video generation models with 3D full attention exhibit strong generative capabilities. Trajectory control represents a user-friendly task in the field of controllable video generation. However, existing methods either require substantial training resources or are specifically designed for U-Net, do not take advantage of the superior performance of DiT. To address these issues, we propose DiTraj, a simple but effective training-free framework for trajectory control in text-to-video generation, tailored for DiT. Specifically, first, to inject the object’s trajectory, we propose foreground-background separation guidance: we use the Large Language Model (LLM) to convert user-provided prompts into foreground and background prompts, which respectively guide the generation of foreground and background regions in the video. Then, we analyze 3D full attention and explore the tight correlation between inter-token attention scores and position embedding. Based on this, we propose inter-frame Spatial-Temporal Decoupled 3D-RoPE (STD-RoPE). By modifying only foreground tokens’ position embedding, STD-RoPE eliminates their cross-frame spatial discrepancies, strengthening cross-frame attention among them and thus enhancing trajectory control. Additionally, we achieve 3D-aware trajectory control by regulating the density of position embedding. Extensive experiments demonstrate that our method outperforms previous methods in both video quality and trajectory controllability.

Method: Evaluated cosine similarity, maximal marginal relevance, and hybrid method integrating dense embeddings with lexical overlap and re-ranking. Adapted EfficientRAG pipeline with token labeling and iterative refinement for query optimization.

Result: Hybrid approach substantially outperformed baseline methods with 50% relative improvement in exact match and 47% in F1 score compared to cosine similarity. Improved entity recall and evidence complementarity.

Conclusion: Hybrid retrieval-augmented generation provides practical zero-shot solution for multi-hop QA, balancing accuracy, efficiency, and interpretability, though limited in handling distractors and temporal reasoning.

Abstract: Transformer-based models have advanced the field of question answering, but multi-hop reasoning, where answers require combining evidence across multiple passages, remains difficult. This paper presents a comprehensive evaluation of retrieval strategies for multi-hop question answering within a retrieval-augmented generation framework. We compare cosine similarity, maximal marginal relevance, and a hybrid method that integrates dense embeddings with lexical overlap and re-ranking. To further improve retrieval, we adapt the EfficientRAG pipeline for query optimization, introducing token labeling and iterative refinement while maintaining efficiency. Experiments on the HotpotQA dataset show that the hybrid approach substantially outperforms baseline methods, achieving a relative improvement of 50 percent in exact match and 47 percent in F1 score compared to cosine similarity. Error analysis reveals that hybrid retrieval improves entity recall and evidence complementarity, while remaining limited in handling distractors and temporal reasoning. Overall, the results suggest that hybrid retrieval-augmented generation provides a practical zero-shot solution for multi-hop question answering, balancing accuracy, efficiency, and interpretability.

Method: Proposed HDR-4DGS uses Gaussian Splatting with an innovative dynamic tone-mapping module that connects HDR and LDR domains while maintaining temporal radiance coherence by adapting tone-mapping functions according to evolving radiance distributions.

Result: HDR-4DGS achieves temporal radiance consistency and spatially accurate color translation, enabling photorealistic HDR renderings from arbitrary viewpoints and time instances, surpassing state-of-the-art methods in both quantitative performance and visual fidelity.

Conclusion: The proposed HDR-4DGS framework successfully addresses the challenging problem of HDR Dynamic Novel View Synthesis by jointly modeling temporal radiance variations and 3D translation between LDR and HDR domains.

Abstract: High Dynamic Range Novel View Synthesis (HDR NVS) seeks to learn an HDR 3D model from Low Dynamic Range (LDR) training images captured under conventional imaging conditions. Current methods primarily focus on static scenes, implicitly assuming all scene elements remain stationary and non-living. However, real-world scenarios frequently feature dynamic elements, such as moving objects, varying lighting conditions, and other temporal events, thereby presenting a significantly more challenging scenario. To address this gap, we propose a more realistic problem named HDR Dynamic Novel View Synthesis (HDR DNVS), where the additional dimension ``Dynamic’’ emphasizes the necessity of jointly modeling temporal radiance variations alongside sophisticated 3D translation between LDR and HDR. To tackle this complex, intertwined challenge, we introduce HDR-4DGS, a Gaussian Splatting-based architecture featured with an innovative dynamic tone-mapping module that explicitly connects HDR and LDR domains, maintaining temporal radiance coherence by dynamically adapting tone-mapping functions according to the evolving radiance distributions across the temporal dimension. As a result, HDR-4DGS achieves both temporal radiance consistency and spatially accurate color translation, enabling photorealistic HDR renderings from arbitrary viewpoints and time instances. Extensive experiments demonstrate that HDR-4DGS surpasses existing state-of-the-art methods in both quantitative performance and visual fidelity. Source code will be released.

Method: Proposes SRHand with geometric-aware implicit image function (GIIF) that learns hand priors to upsample images while jointly optimizing implicit image function and explicit 3D hand meshes.

Result: Significantly outperforms state-of-the-art methods on InterHand2.6M and Goliath datasets, achieving fine details like wrinkles and nails while preserving multi-view and pose consistency.

Conclusion: SRHand successfully reconstructs detailed 3D hand avatars from low-resolution images by combining implicit image representation with explicit hand geometry optimization.

Abstract: Reconstructing detailed hand avatars plays a crucial role in various applications. While prior works have focused on capturing high-fidelity hand geometry, they heavily rely on high-resolution multi-view image inputs and struggle to generalize on low-resolution images. Multi-view image super-resolution methods have been proposed to enforce 3D view consistency. These methods, however, are limited to static objects/scenes with fixed resolutions and are not applicable to articulated deformable hands. In this paper, we propose SRHand (Super-Resolution Hand), the method for reconstructing detailed 3D geometry as well as textured images of hands from low-resolution images. SRHand leverages the advantages of implicit image representation with explicit hand meshes. Specifically, we introduce a geometric-aware implicit image function (GIIF) that learns detailed hand prior by upsampling the coarse input images. By jointly optimizing the implicit image function and explicit 3D hand shapes, our method preserves multi-view and pose consistency among upsampled hand images, and achieves fine-detailed 3D reconstruction (wrinkles, nails). In experiments using the InterHand2.6M and Goliath datasets, our method significantly outperforms state-of-the-art image upsampling methods adapted to hand datasets, and 3D hand reconstruction methods, quantitatively and qualitatively. Project page: https://yunminjin2.github.io/projects/srhand

Method: Position paper analyzing current research practices, highlighting reliance on old low-resolution datasets like ImageNet, and examining how modern smartphone photography algorithms resemble fake image generators.

Result: The analysis reveals fundamental flaws in current fake detection approaches, showing that the distinction between “real” and “fake” is becoming increasingly blurred due to similar underlying technologies in both image generation and modern photography.

Conclusion: The paper calls for rethinking the concept of “real” images, developing clear technical definitions, creating new benchmark datasets, and questioning whether fake image detection remains a valid research objective given technological convergence.

Abstract: The wide availability and low usability barrier of modern image generation models has triggered the reasonable fear of criminal misconduct and negative social implications. The machine learning community has been engaging this problem with an extensive series of publications proposing algorithmic solutions for the detection of “fake”, e.g. entirely generated or partially manipulated images. While there is undoubtedly some progress towards technical solutions of the problem, we argue that current and prior work is focusing too much on generative algorithms and “fake” data-samples, neglecting a clear definition and data collection of “real” images. The fundamental question “what is a real image?” might appear to be quite philosophical, but our analysis shows that the development and evaluation of basically all current “fake”-detection methods is relying on only a few, quite old low-resolution datasets of “real” images like ImageNet. However, the technology for the acquisition of “real” images, aka taking photos, has drastically evolved over the last decade: Today, over 90% of all photographs are produced by smartphones which typically use algorithms to compute an image from multiple inputs (over time) from multiple sensors. Based on the fact that these image formation algorithms are typically neural network architectures which are closely related to “fake”-image generators, we state the position that today, we need to re-think the concept of “real” images. The purpose of this position paper is to raise the awareness of the current shortcomings in this active field of research and to trigger an open discussion whether the detection of “fake” images is a sound objective at all. At the very least, we need a clear technical definition of “real” images and new benchmark datasets.

Method: Proposes Aes-R1 framework with AesCoT pipeline for constructing chain-of-thought aesthetic reasoning data, and RAPO (Relative-Absolute Policy Optimization) RL algorithm that jointly optimizes absolute score regression and relative ranking order.

Result: Improves backbone’s average PLCC/SRCC by 47.9%/34.8%, surpassing state-of-the-art baselines of similar size, with robust generalization under limited supervision and out-of-distribution scenarios.

Conclusion: Aes-R1 enables MLLMs to generate grounded explanations alongside faithful scores, enhancing aesthetic scoring and reasoning in a unified framework.

Abstract: Multimodal large language models (MLLMs) are well suited to image aesthetic assessment, as they can capture high-level aesthetic features leveraging their cross-modal understanding capacity. However, the scarcity of multimodal aesthetic reasoning data and the inherently subjective nature of aesthetic judgment make it difficult for MLLMs to generate accurate aesthetic judgments with interpretable rationales. To this end, we propose Aes-R1, a comprehensive aesthetic reasoning framework with reinforcement learning (RL). Concretely, Aes-R1 integrates a pipeline, AesCoT, to construct and filter high-quality chain-of-thought aesthetic reasoning data used for cold-start. After teaching the model to generate structured explanations prior to scoring, we then employ the Relative-Absolute Policy Optimization (RAPO), a novel RL algorithm that jointly optimizes absolute score regression and relative ranking order, improving both per-image accuracy and cross-image preference judgments. Aes-R1 enables MLLMs to generate grounded explanations alongside faithful scores, thereby enhancing aesthetic scoring and reasoning in a unified framework. Extensive experiments demonstrate that Aes-R1 improves the backbone’s average PLCC/SRCC by 47.9%/34.8%, surpassing state-of-the-art baselines of similar size. More ablation studies validate Aes-R1’s robust generalization under limited supervision and in out-of-distribution scenarios.

Method: Three-stage pipeline: 1) Enhanced text-to-3D background generation using 2D Gaussian Splatting with panoramic images, 2) 3D copy-and-paste approach with physics-aware object positioning, 3) Temporal animation using a part-augmented, motion-conditioned video diffusion model for view-consistent motion.

Result: The framework successfully generates harmonized 3D scenes with user-controlled object motion, demonstrating effective spatial alignment and view-consistent temporal animation through comprehensive evaluations.

Conclusion: Drag4D provides a unified architecture for interactive 3D scene generation with motion control, achieving high-quality results with precise object positioning and consistent motion animation in 3D environments.

Abstract: We introduce Drag4D, an interactive framework that integrates object motion control within text-driven 3D scene generation. This framework enables users to define 3D trajectories for the 3D objects generated from a single image, seamlessly integrating them into a high-quality 3D background. Our Drag4D pipeline consists of three stages. First, we enhance text-to-3D background generation by applying 2D Gaussian Splatting with panoramic images and inpainted novel views, resulting in dense and visually complete 3D reconstructions. In the second stage, given a reference image of the target object, we introduce a 3D copy-and-paste approach: the target instance is extracted in a full 3D mesh using an off-the-shelf image-to-3D model and seamlessly composited into the generated 3D scene. The object mesh is then positioned within the 3D scene via our physics-aware object position learning, ensuring precise spatial alignment. Lastly, the spatially aligned object is temporally animated along a user-defined 3D trajectory. To mitigate motion hallucination and ensure view-consistent temporal alignment, we develop a part-augmented, motion-conditioned video diffusion model that processes multiview image pairs together with their projected 2D trajectories. We demonstrate the effectiveness of our unified architecture through evaluations at each stage and in the final results, showcasing the harmonized alignment of user-controlled object motion within a high-quality 3D background.

Method: Builds on pre-trained video backbone with two key components: (1) Motion-aware Loss emphasizing learning at high-motion regions, (2) Audio Sync Guidance using visually aligned off-sync model without audio layers to better exploit audio cues at inference.

Result: Outperforms existing methods on AVSync15 and The Greatest Hits datasets in both synchronization accuracy and visual quality, generating 380x640 resolution, 24fps videos.

Conclusion: Syncphony effectively addresses the synchronization challenge in audio-to-video generation through motion-aware training and audio guidance, achieving superior temporal alignment while maintaining visual quality.

Abstract: Text-to-video and image-to-video generation have made rapid progress in visual quality, but they remain limited in controlling the precise timing of motion. In contrast, audio provides temporal cues aligned with video motion, making it a promising condition for temporally controlled video generation. However, existing audio-to-video (A2V) models struggle with fine-grained synchronization due to indirect conditioning mechanisms or limited temporal modeling capacity. We present Syncphony, which generates 380x640 resolution, 24fps videos synchronized with diverse audio inputs. Our approach builds upon a pre-trained video backbone and incorporates two key components to improve synchronization: (1) Motion-aware Loss, which emphasizes learning at high-motion regions; (2) Audio Sync Guidance, which guides the full model using a visually aligned off-sync model without audio layers to better exploit audio cues at inference while maintaining visual quality. To evaluate synchronization, we propose CycleSync, a video-to-audio-based metric that measures the amount of motion cues in the generated video to reconstruct the original audio. Experiments on AVSync15 and The Greatest Hits datasets demonstrate that Syncphony outperforms existing methods in both synchronization accuracy and visual quality. Project page is available at: https://jibin86.github.io/syncphony_project_page

Method: Uses SAM2 visual foundation model for multi-scale feature extraction, multi-layer adapters for fine-tuning, Text Fusion Attention Module (TFAM) for visual-text alignment, and Vision-Semantic Fusion Decoder (V-SFD) with cross-attention for final change masks.

Result: LG-CD consistently outperforms state-of-the-art methods on three datasets (LEVIR-CD, WHU-CD, SYSU-CD), demonstrating improved accuracy and robustness in change detection.

Conclusion: The approach provides new insights for generalized change detection by leveraging multimodal information, showing that language guidance significantly enhances remote sensing change detection performance.

Abstract: Remote Sensing Change Detection (RSCD) typically identifies changes in land cover or surface conditions by analyzing multi-temporal images. Currently, most deep learning-based methods primarily focus on learning unimodal visual information, while neglecting the rich semantic information provided by multimodal data such as text. To address this limitation, we propose a novel Language-Guided Change Detection model (LG-CD). This model leverages natural language prompts to direct the network’s attention to regions of interest, significantly improving the accuracy and robustness of change detection. Specifically, LG-CD utilizes a visual foundational model (SAM2) as a feature extractor to capture multi-scale pyramid features from high-resolution to low-resolution across bi-temporal remote sensing images. Subsequently, multi-layer adapters are employed to fine-tune the model for downstream tasks, ensuring its effectiveness in remote sensing change detection. Additionally, we design a Text Fusion Attention Module (TFAM) to align visual and textual information, enabling the model to focus on target change regions using text prompts. Finally, a Vision-Semantic Fusion Decoder (V-SFD) is implemented, which deeply integrates visual and semantic information through a cross-attention mechanism to produce highly accurate change detection masks. Our experiments on three datasets (LEVIR-CD, WHU-CD, and SYSU-CD) demonstrate that LG-CD consistently outperforms state-of-the-art change detection methods. Furthermore, our approach provides new insights into achieving generalized change detection by leveraging multimodal information.

Method: Proposes a diffusion-based framework with two innovations: Point-Cloud Deterministic Drag for enhanced latent-space layout control via 3D feature mapping, and Drag-Text Guided Denoising that dynamically balances drag and text conditions during denoising.

Result: Achieves high-fidelity joint editing while matching or surpassing specialized text-only or drag-only approaches, supporting flexible editing modes (text-only, drag-only, or combined).

Conclusion: Establishes a versatile and generalizable solution for controllable image manipulation that integrates the strengths of both text and drag editing paradigms.

Abstract: This paper explores image editing under the joint control of text and drag interactions. While recent advances in text-driven and drag-driven editing have achieved remarkable progress, they suffer from complementary limitations: text-driven methods excel in texture manipulation but lack precise spatial control, whereas drag-driven approaches primarily modify shape and structure without fine-grained texture guidance. To address these limitations, we propose a unified diffusion-based framework for joint drag-text image editing, integrating the strengths of both paradigms. Our framework introduces two key innovations: (1) Point-Cloud Deterministic Drag, which enhances latent-space layout control through 3D feature mapping, and (2) Drag-Text Guided Denoising, dynamically balancing the influence of drag and text conditions during denoising. Notably, our model supports flexible editing modes - operating with text-only, drag-only, or combined conditions - while maintaining strong performance in each setting. Extensive quantitative and qualitative experiments demonstrate that our method not only achieves high-fidelity joint editing but also matches or surpasses the performance of specialized text-only or drag-only approaches, establishing a versatile and generalizable solution for controllable image manipulation. Code will be made publicly available to reproduce all results presented in this work.

Method: Train CNN-based representation extractor through contrastive learning on unannotated data, then sideload it to frozen YOLO11n backbone during fine-tuning. Extensive experiments compare fusion methods and granularity.

Result: Proposed sideload-CL-adaptation model improves detection performance by 3.8% to 9.5% in terms of mAP50 on the NVD dataset.

Conclusion: The framework effectively leverages unannotated data to enhance vehicle detection performance in challenging Nordic conditions with snow coverage.

Abstract: Aside from common challenges in remote sensing like small, sparse targets and computation cost limitations, detecting vehicles from UAV images in the Nordic regions faces strong visibility challenges and domain shifts caused by diverse levels of snow coverage. Although annotated data are expensive, unannotated data is cheaper to obtain by simply flying the drones. In this work, we proposed a sideload-CL-adaptation framework that enables the use of unannotated data to improve vehicle detection using lightweight models. Specifically, we propose to train a CNN-based representation extractor through contrastive learning on the unannotated data in the pretraining stage, and then sideload it to a frozen YOLO11n backbone in the fine-tuning stage. To find a robust sideload-CL-adaptation, we conducted extensive experiments to compare various fusion methods and granularity. Our proposed sideload-CL-adaptation model improves the detection performance by 3.8% to 9.5% in terms of mAP50 on the NVD dataset.

Method: Proposes SSLCounter framework that learns implicit scene representations through neural volumetric rendering, enabling reconstruction of continuous geometry and view-dependent appearance via differential neural rendering.

Result: Achieves state-of-the-art performance and demonstrates competitive results using only 70% of training data, showing superior data efficiency across multiple benchmarks.

Conclusion: SSLCounter provides an effective self-supervised alternative to fully supervised approaches for multi-view counting, offering better data efficiency and seamless integration into existing frameworks.

Abstract: Multi-view counting (MVC) methods have attracted significant research attention and stimulated remarkable progress in recent years. Despite their success, most MVC methods have focused on improving performance by following the fully supervised learning (FSL) paradigm, which often requires large amounts of annotated data. In this work, we propose SSLCounter, a novel self-supervised learning (SSL) framework for MVC that leverages neural volumetric rendering to alleviate the reliance on large-scale annotated datasets. SSLCounter learns an implicit representation w.r.t. the scene, enabling the reconstruction of continuous geometry shape and the complex, view-dependent appearance of their 2D projections via differential neural rendering. Owing to its inherent flexibility, the key idea of our method can be seamlessly integrated into exsiting frameworks. Notably, extensive experiments demonstrate that SSLCounter not only demonstrates state-of-the-art performances but also delivers competitive performance with only using 70% proportion of training data, showcasing its superior data efficiency across multiple MVC benchmarks.

Method: Used a controlled synthetic dataset to evaluate state-of-the-art vision models and VLMs across three tasks: spatial localization, spatial reasoning, and downstream retrieval tasks.

Result: Found a stable trade-off: detectors provide precise bounding boxes with limited relational reasoning, while VLMs offer coarse layout cues and fluent captions but struggle with fine-grained spatial context.

Conclusion: Highlights the gap between localization and true spatial understanding, pointing toward the need for spatially-aware foundation models in the community.

Abstract: Spatial understanding is a critical capability for vision foundation models. While recent advances in large vision models or vision-language models (VLMs) have expanded recognition capabilities, most benchmarks emphasize localization accuracy rather than whether models capture how objects are arranged and related within a scene. This gap is consequential; effective scene understanding requires not only identifying objects, but reasoning about their relative positions, groupings, and depth. In this paper, we present a systematic benchmark for object-centric spatial reasoning in foundation models. Using a controlled synthetic dataset, we evaluate state-of-the-art vision models (e.g., GroundingDINO, Florence-2, OWLv2) and large VLMs (e.g., InternVL, LLaVA, GPT-4o) across three tasks: spatial localization, spatial reasoning, and downstream retrieval tasks. We find a stable trade-off: detectors such as GroundingDINO and OWLv2 deliver precise boxes with limited relational reasoning, while VLMs like SmolVLM and GPT-4o provide coarse layout cues and fluent captions but struggle with fine-grained spatial context. Our study highlights the gap between localization and true spatial understanding, and pointing toward the need for spatially-aware foundation models in the community.

Method: PAtch-based k-Nearest neighbor visual In-Context Learning (PANICL) leverages multiple in-context pairs to smooth assignment scores across pairs without requiring additional training.

Result: Extensive experiments show consistent improvements on foreground segmentation, object detection, colorization, multi-object segmentation, and keypoint detection. PANICL also demonstrates strong robustness to domain shifts and generalizes well to other VICL models.

Conclusion: PANICL is a versatile and broadly applicable framework that effectively mitigates bias in visual in-context learning through multi-patch utilization.

Abstract: Visual In-Context Learning (VICL) uses input-output image pairs, referred to as in-context pairs (or examples), as prompts alongside query images to guide models in performing diverse vision tasks. However, VICL often suffers from over-reliance on a single in-context pair, which can lead to biased and unstable predictions. We introduce PAtch-based $k$-Nearest neighbor visual In-Context Learning (PANICL), a general training-free framework that mitigates this issue by leveraging multiple in-context pairs. PANICL smooths assignment scores across pairs, reducing bias without requiring additional training. Extensive experiments on a variety of tasks, including foreground segmentation, single object detection, colorization, multi-object segmentation, and keypoint detection, demonstrate consistent improvements over strong baselines. Moreover, PANICL exhibits strong robustness to domain shifts, including dataset-level shift (e.g., from COCO to Pascal) and label-space shift (e.g., FSS-1000), and generalizes well to other VICL models such as SegGPT, Painter, and LVM, highlighting its versatility and broad applicability.

Method: Proposes a token-scaler-based fine-tuning mechanism with novel optimization loss on Depth-Anything v2 for better depth prediction, and a depth-aware matching process that integrates spatial relationships within LoFTR for handling challenging materials and lighting.

Result: Achieves 14.41% improvement in depth prediction on REAL275 compared to Depth-Anything v2, and surpasses state-of-the-art methods on REAL275, ClearPose, and Toyota-Light datasets with 6.1% improvement in average recall for pose estimation.

Conclusion: SingRef6D provides a robust and capable solution for 6D pose estimation in resource-limited settings, effectively handling challenging surface conditions and lighting scenarios without requiring depth sensors or dense templates.

Abstract: Recent 6D pose estimation methods demonstrate notable performance but still face some practical limitations. For instance, many of them rely heavily on sensor depth, which may fail with challenging surface conditions, such as transparent or highly reflective materials. In the meantime, RGB-based solutions provide less robust matching performance in low-light and texture-less scenes due to the lack of geometry information. Motivated by these, we propose SingRef6D, a lightweight pipeline requiring only a single RGB image as a reference, eliminating the need for costly depth sensors, multi-view image acquisition, or training view synthesis models and neural fields. This enables SingRef6D to remain robust and capable even under resource-limited settings where depth or dense templates are unavailable. Our framework incorporates two key innovations. First, we propose a token-scaler-based fine-tuning mechanism with a novel optimization loss on top of Depth-Anything v2 to enhance its ability to predict accurate depth, even for challenging surfaces. Our results show a 14.41% improvement (in $\delta_{1.05}$) on REAL275 depth prediction compared to Depth-Anything v2 (with fine-tuned head). Second, benefiting from depth availability, we introduce a depth-aware matching process that effectively integrates spatial relationships within LoFTR, enabling our system to handle matching for challenging materials and lighting conditions. Evaluations of pose estimation on the REAL275, ClearPose, and Toyota-Light datasets show that our approach surpasses state-of-the-art methods, achieving a 6.1% improvement in average recall.

Method: Proposes DynaNav with trainable hard feature selector for sparse operations and integrates feature selection with early-exit mechanism using Bayesian Optimization for optimal exit thresholds.

Result: Achieves 2.26x reduction in FLOPs, 42.3% lower inference time, and 32.8% lower memory usage compared to ViNT, while improving navigation performance across four public datasets.

Conclusion: DynaNav effectively addresses computational efficiency and interpretability challenges in visual navigation, making it suitable for resource-tight deployment scenarios.

Abstract: Visual navigation is essential for robotics and embodied AI. However, existing foundation models, particularly those with transformer decoders, suffer from high computational overhead and lack interpretability, limiting their deployment in resource-tight scenarios. To address this, we propose DynaNav, a Dynamic Visual Navigation framework that adapts feature and layer selection based on scene complexity. It employs a trainable hard feature selector for sparse operations, enhancing efficiency and interpretability. Additionally, we integrate feature selection into an early-exit mechanism, with Bayesian Optimization determining optimal exit thresholds to reduce computational cost. Extensive experiments in real-world-based datasets and simulated environments demonstrate the effectiveness of DynaNav. Compared to ViNT, DynaNav achieves a 2.26x reduction in FLOPs, 42.3% lower inference time, and 32.8% lower memory usage, while improving navigation performance across four public datasets.

Method: Run auxiliary denoising with a surrogate prompt aligned with the visual condition to extract attention masks, then use these masks during denoising of the target prompt to suppress conflicting visual guidance and strengthen text guidance.

Result: Experimental results show improved performance under loosely aligned conditions across various condition types (depth maps, edge maps, human skeletons), outperforming existing baselines.

Conclusion: SemanticControl effectively leverages misaligned but semantically relevant visual conditions without requiring training, enabling better text-to-image generation for uncommon scenes where precisely aligned visual conditions are unavailable.

Abstract: ControlNet has enabled detailed spatial control in text-to-image diffusion models by incorporating additional visual conditions such as depth or edge maps. However, its effectiveness heavily depends on the availability of visual conditions that are precisely aligned with the generation goal specified by text prompt-a requirement that often fails in practice, especially for uncommon or imaginative scenes. For example, generating an image of a cat cooking in a specific pose may be infeasible due to the lack of suitable visual conditions. In contrast, structurally similar cues can often be found in more common settings-for instance, poses of humans cooking are widely available and can serve as rough visual guides. Unfortunately, existing ControlNet models struggle to use such loosely aligned visual conditions, often resulting in low text fidelity or visual artifacts. To address this limitation, we propose SemanticControl, a training-free method for effectively leveraging misaligned but semantically relevant visual conditions. Our approach adaptively suppresses the influence of the visual condition where it conflicts with the prompt, while strengthening guidance from the text. The key idea is to first run an auxiliary denoising process using a surrogate prompt aligned with the visual condition (e.g., “a human playing guitar” for a human pose condition) to extract informative attention masks, and then utilize these masks during the denoising of the actual target prompt (e.g., cat playing guitar). Experimental results demonstrate that our method improves performance under loosely aligned conditions across various conditions, including depth maps, edge maps, and human skeletons, outperforming existing baselines. Our code is available at https://mung3477.github.io/semantic-control.

Method: Proposed an Emotion Statement Judgment task and developed an automated pipeline to efficiently construct emotion-centric statements with minimal human effort, enabling systematic evaluation of MLLMs’ visual emotion perception capabilities.

Result: MLLMs show stronger performance in emotion interpretation and context-based emotion judgment, but have relative limitations in comprehending perception subjectivity. Even top-performing models like GPT4o demonstrate significant performance gaps compared to humans.

Conclusion: The study provides a fundamental evaluation framework that contributes to advancing emotional intelligence in MLLMs by identifying key areas for future improvement in visual emotion perception capabilities.

Abstract: Recently, Multimodal Large Language Models (MLLMs) have achieved exceptional performance across diverse tasks, continually surpassing previous expectations regarding their capabilities. Nevertheless, their proficiency in perceiving emotions from images remains debated, with studies yielding divergent results in zero-shot scenarios. We argue that this inconsistency stems partly from constraints in existing evaluation methods, including the oversight of plausible responses, limited emotional taxonomies, neglect of contextual factors, and labor-intensive annotations. To facilitate customized visual emotion evaluation for MLLMs, we propose an Emotion Statement Judgment task that overcomes these constraints. Complementing this task, we devise an automated pipeline that efficiently constructs emotion-centric statements with minimal human effort. Through systematically evaluating prevailing MLLMs, our study showcases their stronger performance in emotion interpretation and context-based emotion judgment, while revealing relative limitations in comprehending perception subjectivity. When compared to humans, even top-performing MLLMs like GPT4o demonstrate remarkable performance gaps, underscoring key areas for future improvement. By developing a fundamental evaluation framework and conducting a comprehensive MLLM assessment, we hope this work contributes to advancing emotional intelligence in MLLMs. Project page: https://github.com/wdqqdw/MVEI.

Method: 1) Explicit positional supervision to separate attention regions and mitigate attribute leakage; 2) Mixture-of-Experts architecture to handle diverse scenarios; 3) Online reinforcement learning framework with scoring mechanism for multi-subject fidelity assessment and stable training for MoE architecture.

Result: Experiments show the framework significantly improves subject fidelity while better aligning with human preferences compared to existing methods.

Conclusion: MultiCrafter effectively addresses the limitations of existing multi-subject image generation methods by tackling attribute leakage through attention separation, enhancing scenario handling via MoE, and aligning with human preferences through reinforcement learning.

Abstract: Multi-subject image generation aims to synthesize user-provided subjects in a single image while preserving subject fidelity, ensuring prompt consistency, and aligning with human aesthetic preferences. However, existing methods, particularly those built on the In-Context-Learning paradigm, are limited by their reliance on simple reconstruction-based objectives, leading to both severe attribute leakage that compromises subject fidelity and failing to align with nuanced human preferences. To address this, we propose MultiCrafter, a framework that ensures high-fidelity, preference-aligned generation. First, we find that the root cause of attribute leakage is a significant entanglement of attention between different subjects during the generation process. Therefore, we introduce explicit positional supervision to explicitly separate attention regions for each subject, effectively mitigating attribute leakage. To enable the model to accurately plan the attention region of different subjects in diverse scenarios, we employ a Mixture-of-Experts architecture to enhance the model’s capacity, allowing different experts to focus on different scenarios. Finally, we design a novel online reinforcement learning framework to align the model with human preferences, featuring a scoring mechanism to accurately assess multi-subject fidelity and a more stable training strategy tailored for the MoE architecture. Experiments validate that our framework significantly improves subject fidelity while aligning with human preferences better.

Method: Uses encoder-decoder architecture with triplane-based dual-branch encoder for scalable part-aware representation learning, trained on over 5 million 3D shape-part pairs curated through model-in-the-loop annotation pipeline.

Result: Outperforms state-of-the-art methods by large margins across multiple benchmarks, achieving highly accurate part identification with single prompts and automatic decomposition into surface and internal structures.

Conclusion: PartSAM represents a decisive step toward foundation models for 3D part understanding, demonstrating emergent open-world capabilities through scalable architecture and diverse 3D data.

Abstract: Segmenting 3D objects into parts is a long-standing challenge in computer vision. To overcome taxonomy constraints and generalize to unseen 3D objects, recent works turn to open-world part segmentation. These approaches typically transfer supervision from 2D foundation models, such as SAM, by lifting multi-view masks into 3D. However, this indirect paradigm fails to capture intrinsic geometry, leading to surface-only understanding, uncontrolled decomposition, and limited generalization. We present PartSAM, the first promptable part segmentation model trained natively on large-scale 3D data. Following the design philosophy of SAM, PartSAM employs an encoder-decoder architecture in which a triplane-based dual-branch encoder produces spatially structured tokens for scalable part-aware representation learning. To enable large-scale supervision, we further introduce a model-in-the-loop annotation pipeline that curates over five million 3D shape-part pairs from online assets, providing diverse and fine-grained labels. This combination of scalable architecture and diverse 3D data yields emergent open-world capabilities: with a single prompt, PartSAM achieves highly accurate part identification, and in a Segment-Every-Part mode, it automatically decomposes shapes into both surface and internal structures. Extensive experiments show that PartSAM outperforms state-of-the-art methods by large margins across multiple benchmarks, marking a decisive step toward foundation models for 3D part understanding. Our code and model will be released soon.

Method: Customized and fine-tuned three pre-trained architectures (EfficientNet B0, ResNet18, MobileNetV2) with contrast-aware regression heads, plus a Siamese network model. Trained end-to-end using targeted data augmentations on CID2013 and CCID2014 datasets containing synthetic and authentic contrast distortions.

Result: Customized EfficientNet B0 achieved state-of-the-art performance with PLCC=0.9286/SRCC=0.9178 on CCID2014 and PLCC=0.9581/SRCC=0.9369 on CID2013, surpassing traditional methods and other deep baselines. Other models showed limited ability to capture perceptual contrast distortions.

Conclusion: Contrast-aware adaptation of lightweight pre-trained networks provides a high-performing, scalable solution for no-reference contrast quality assessment suitable for real-time and resource-constrained applications.

Abstract: Image contrast was a fundamental factor in visual perception and played a vital role in overall image quality. However, most no reference image quality assessment NR IQA models struggled to accurately evaluate contrast distortions under diverse real world conditions. In this study, we proposed a deep learning based framework for blind contrast quality assessment by customizing and fine-tuning three pre trained architectures, EfficientNet B0, ResNet18, and MobileNetV2, for perceptual Mean Opinion Score, along with an additional model built on a Siamese network, which indicated a limited ability to capture perceptual contrast distortions. Each model is modified with a contrast-aware regression head and trained end to end using targeted data augmentations on two benchmark datasets, CID2013 and CCID2014, containing synthetic and authentic contrast distortions. Performance is evaluated using Pearson Linear Correlation Coefficient and Spearman Rank Order Correlation Coefficient, which assess the alignment between predicted and human rated scores. Among these three models, our customized EfficientNet B0 model achieved state-of-the-art performance with PLCC = 0.9286 and SRCC = 0.9178 on CCID2014 and PLCC = 0.9581 and SRCC = 0.9369 on CID2013, surpassing traditional methods and outperforming other deep baselines. These results highlighted the models robustness and effectiveness in capturing perceptual contrast distortion. Overall, the proposed method demonstrated that contrast aware adaptation of lightweight pre trained networks can yield a high performing, scalable solution for no reference contrast quality assessment suitable for real time and resource constrained applications.

Method: Proposes a “reason first, then act” process where the model generates explicit, interpretable reasoning chains to decompose referring expressions, then uses these rationales to localize target objects through reinforcement fine-tuning.

Result: Geo-R1 consistently and substantially outperforms SFT baselines on three few-shot geospatial referring benchmarks and demonstrates strong cross-dataset generalization.

Conclusion: The reasoning-centric reinforcement fine-tuning paradigm enables more effective use of limited annotations, enhances generalization, and provides interpretability for geospatial referring tasks.

Abstract: Referring expression understanding in remote sensing poses unique challenges, as it requires reasoning over complex object-context relationships. While supervised fine-tuning (SFT) on multimodal large language models achieves strong performance with massive labeled datasets, they struggle in data-scarce scenarios, leading to poor generalization. To address this limitation, we propose Geo-R1, a reasoning-centric reinforcement fine-tuning (RFT) paradigm for few-shot geospatial referring. Geo-R1 enforces the model to first generate explicit, interpretable reasoning chains that decompose referring expressions, and then leverage these rationales to localize target objects. This “reason first, then act” process enables the model to make more effective use of limited annotations, enhances generalization, and provides interpretability. We validate Geo-R1 on three carefully designed few-shot geospatial referring benchmarks, where our model consistently and substantially outperforms SFT baselines. It also demonstrates strong cross-dataset generalization, highlighting its robustness. Code and data will be released at http://geo-r1.github.io.

Method: Created a medical sycophancy benchmark from PathVQA, SLAKE, and VQA-RAD datasets using psychologically motivated pressure templates. Proposed VIPER framework that filters non-evidentiary content and generates constrained evidence-first answers.

Result: VLMs showed significant vulnerability to sycophantic behavior with weak correlation to model accuracy or size. Imitation and expert-provided corrections were the most effective triggers. VIPER framework reduced sycophancy by an average amount while maintaining interpretability.

Conclusion: The benchmark and VIPER mitigation framework provide groundwork for robust deployment of medical VLMs in real-world clinician interactions, emphasizing the need for evidence-anchored defenses against sycophantic behavior.

Abstract: Vision language models(VLMs) are increasingly integrated into clinical workflows, but they often exhibit sycophantic behavior prioritizing alignment with user phrasing social cues or perceived authority over evidence based reasoning. This study evaluate clinical sycophancy in medical visual question answering through a novel clinically grounded benchmark. We propose a medical sycophancy dataset construct from PathVQA, SLAKE, and VQA-RAD stratified by different type organ system and modality. Using psychologically motivated pressure templates including various sycophancy. In our adversarial experiments on various VLMs, we found that these models are generally vulnerable, exhibiting significant variations in the occurrence of adversarial responses, with weak correlations to the model accuracy or size. Imitation and expert provided corrections were found to be the most effective triggers, suggesting that the models possess a bias mechanism independent of visual evidence. To address this, we propose Visual Information Purification for Evidence based Response (VIPER) a lightweight mitigation strategy that filters non evidentiary content for example social pressures and then generates constrained evidence first answers. This framework reduces sycophancy by an average amount outperforming baselines while maintaining interpretability. Our benchmark analysis and mitigation framework lay the groundwork for robust deployment of medical VLMs in real world clinician interactions emphasizing the need for evidence anchored defenses.

Method: GLARIFY analyzes gaze patterns, uses GPT-4o to create GLARIFY-Ambi dataset with chain-of-thought process for handling noisy gaze, and designs a heatmap module to incorporate gaze information into VLMs while preserving pretrained knowledge.

Result: Experiments on hold-out test set demonstrate that GLARIFY significantly outperforms baselines in handling ambiguous queries and noisy gaze patterns.

Conclusion: GLARIFY robustly aligns VLMs with human attention, paving the way for usable and intuitive interaction paradigms with visual assistants in smart glasses applications.

Abstract: With the rise in popularity of smart glasses, users’ attention has been integrated into Vision-Language Models (VLMs) to streamline multi-modal querying in daily scenarios. However, leveraging gaze data to model users’ attention may introduce ambiguity challenges: (1) users’ verbal questions become ambiguous by using pronouns or skipping context, (2) humans’ gaze patterns can be noisy and exhibit complex spatiotemporal relationships with their spoken questions. Previous works only consider single image as visual modality input, failing to capture the dynamic nature of the user’s attention. In this work, we introduce GLARIFY, a novel method to leverage spatiotemporal gaze information to enhance the model’s effectiveness in real-world applications. Initially, we analyzed hundreds of querying samples with the gaze modality to demonstrate the noisy nature of users’ gaze patterns. We then utilized GPT-4o to design an automatic data synthesis pipeline to generate the GLARIFY-Ambi dataset, which includes a dedicated chain-of-thought (CoT) process to handle noisy gaze patterns. Finally, we designed a heatmap module to incorporate gaze information into cutting-edge VLMs while preserving their pretrained knowledge. We evaluated GLARIFY using a hold-out test set. Experiments demonstrate that GLARIFY significantly outperforms baselines. By robustly aligning VLMs with human attention, GLARIFY paves the way for a usable and intuitive interaction paradigm with a visual assistant.

Method: Introduces Balanced Position Assignment (BaPA), a simple mechanism that assigns identical position embeddings to all image tokens to promote balanced integration of visual information across spatial positions.

Result: BaPA enhances spatial robustness without retraining and boosts performance across multimodal benchmarks when combined with lightweight fine-tuning, achieving more balanced attention and holistic visual understanding.

Conclusion: The spatial bias in LVLMs stems from unbalanced position embedding designs, and BaPA effectively mitigates this issue by promoting equal treatment of visual information across all spatial locations.

Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable success across a wide range of multimodal tasks, yet their robustness to spatial variations remains insufficiently understood. In this work, we present a systematic study of the spatial bias of LVLMs, focusing on how models respond when identical key visual information is placed at different locations within an image. Through a carefully designed probing dataset, we demonstrate that current LVLMs often produce inconsistent outputs under such spatial shifts, revealing a fundamental limitation in their spatial-semantic understanding. Further analysis shows that this phenomenon originates not from the vision encoder, which reliably perceives and interprets visual content across positions, but from the unbalanced design of position embeddings in the language model component. In particular, the widely adopted position embedding strategies, such as RoPE, introduce imbalance during cross-modal interaction, leading image tokens at different positions to exert unequal influence on semantic understanding. To mitigate this issue, we introduce Balanced Position Assignment (BaPA), a simple yet effective mechanism that assigns identical position embeddings to all image tokens, promoting a more balanced integration of visual information. Extensive experiments show that BaPA enhances the spatial robustness of LVLMs without retraining and further boosts their performance across diverse multimodal benchmarks when combined with lightweight fine-tuning. Further analysis of information flow reveals that BaPA yields balanced attention, enabling more holistic visual understanding.

Method: Uses an automated pipeline to construct image pairs with annotated semantic and visual correspondences from existing datasets, and designs a contrastive architecture to separate the two feature types.

Result: The approach outperforms global feature-based metrics like CLIP, DINO, and vision-language models in quantifying visual inconsistencies while enabling spatial localization of inconsistent regions.

Conclusion: This is the first method that supports both quantification and localization of inconsistencies in subject-driven generation, providing a valuable tool for advancing this task.

Abstract: We propose a novel approach for disentangling visual and semantic features from the backbones of pre-trained diffusion models, enabling visual correspondence in a manner analogous to the well-established semantic correspondence. While diffusion model backbones are known to encode semantically rich features, they must also contain visual features to support their image synthesis capabilities. However, isolating these visual features is challenging due to the absence of annotated datasets. To address this, we introduce an automated pipeline that constructs image pairs with annotated semantic and visual correspondences based on existing subject-driven image generation datasets, and design a contrastive architecture to separate the two feature types. Leveraging the disentangled representations, we propose a new metric, Visual Semantic Matching (VSM), that quantifies visual inconsistencies in subject-driven image generation. Empirical results show that our approach outperforms global feature-based metrics such as CLIP, DINO, and vision–language models in quantifying visual inconsistencies while also enabling spatial localization of inconsistent regions. To our knowledge, this is the first method that supports both quantification and localization of inconsistencies in subject-driven generation, offering a valuable tool for advancing this task. Project Page:https://abdo-eldesokey.github.io/mind-the-glitch/

Method: A two-stage ‘coarse-to-fine’ pipeline: first analyzes a downsampled image to identify task-relevant regions, then crops only these regions at full resolution for detailed processing. Uses reinforcement learning with reward components for coarse-to-fine perception.

Result: ERGO achieves higher accuracy than original models and competitive methods with greater efficiency. It surpasses Qwen2.5-VL-7B on V* benchmark by 4.7 points while using only 23% of vision tokens, achieving 3x inference speedup.

Conclusion: The proposed reasoning-driven perception approach effectively reduces computational costs while preserving accuracy by focusing on relevant image regions, demonstrating significant efficiency improvements in vision-language tasks.

Abstract: Efficient processing of high-resolution images is crucial for real-world vision-language applications. However, existing Large Vision-Language Models (LVLMs) incur substantial computational overhead due to the large number of vision tokens. With the advent of “thinking with images” models, reasoning now extends beyond text to the visual domain. This capability motivates our two-stage “coarse-to-fine” reasoning pipeline: first, a downsampled image is analyzed to identify task-relevant regions; then, only these regions are cropped at full resolution and processed in a subsequent reasoning stage. This approach reduces computational cost while preserving fine-grained visual details where necessary. A major challenge lies in inferring which regions are truly relevant to a given query. Recent related methods often fail in the first stage after input-image downsampling, due to perception-driven reasoning, where clear visual information is required for effective reasoning. To address this issue, we propose ERGO (Efficient Reasoning & Guided Observation) that performs reasoning-driven perception-leveraging multimodal context to determine where to focus. Our model can account for perceptual uncertainty, expanding the cropped region to cover visually ambiguous areas for answering questions. To this end, we develop simple yet effective reward components in a reinforcement learning framework for coarse-to-fine perception. Across multiple datasets, our approach delivers higher accuracy than the original model and competitive methods, with greater efficiency. For instance, ERGO surpasses Qwen2.5-VL-7B on the V* benchmark by 4.7 points while using only 23% of the vision tokens, achieving a 3x inference speedup. The code and models can be found at: https://github.com/nota-github/ERGO.

Method: Introduces a variational formulation with dual constraints: spatial constraints analyze gradient pattern changes across focus levels to distinguish depth edges from texture artifacts, and focal constraints enforce unimodal, monotonic focus probabilities aligned with physical focus behavior.

Result: Comprehensive experiments on four public datasets show that DualFocus consistently outperforms state-of-the-art methods in both depth accuracy and perceptual quality.

Conclusion: The proposed dual constraint framework with spatial and focal inductive biases significantly improves robustness and accuracy in challenging DFF scenarios, demonstrating superior performance over existing methods.

Abstract: Depth-from-Focus (DFF) enables precise depth estimation by analyzing focus cues across a stack of images captured at varying focal lengths. While recent learning-based approaches have advanced this field, they often struggle in complex scenes with fine textures or abrupt depth changes, where focus cues may become ambiguous or misleading. We present DualFocus, a novel DFF framework that leverages the focal stack’s unique gradient patterns induced by focus variation, jointly modeling focus changes over spatial and focal dimensions. Our approach introduces a variational formulation with dual constraints tailored to DFF: spatial constraints exploit gradient pattern changes across focus levels to distinguish true depth edges from texture artifacts, while focal constraints enforce unimodal, monotonic focus probabilities aligned with physical focus behavior. These inductive biases improve robustness and accuracy in challenging regions. Comprehensive experiments on four public datasets demonstrate that DualFocus consistently outperforms state-of-the-art methods in both depth accuracy and perceptual quality.

Method: Proposes RDcomm framework with two key innovations: 1) task entropy discrete coding that assigns task-relevant codeword-lengths to features, and 2) mutual-information-driven message selection using neural estimation to minimize redundancy.

Result: RDcomm achieves state-of-the-art accuracy on DAIR-V2X and OPV2V datasets for 3D object detection and BEV segmentation, while reducing communication volume by up to 108 times compared to previous methods.

Conclusion: The proposed rate-distortion theory provides theoretical foundation for collaborative perception, and RDcomm demonstrates practical effectiveness in balancing performance and communication efficiency through principled information-theoretic approaches.

Abstract: Collaborative perception emphasizes enhancing environmental understanding by enabling multiple agents to share visual information with limited bandwidth resources. While prior work has explored the empirical trade-off between task performance and communication volume, a significant gap remains in the theoretical foundation. To fill this gap, we draw on information theory and introduce a pragmatic rate-distortion theory for multi-agent collaboration, specifically formulated to analyze performance-communication trade-off in goal-oriented multi-agent systems. This theory concretizes two key conditions for designing optimal communication strategies: supplying pragmatically relevant information and transmitting redundancy-less messages. Guided by these two conditions, we propose RDcomm, a communication-efficient collaborative perception framework that introduces two key innovations: i) task entropy discrete coding, which assigns features with task-relevant codeword-lengths to maximize the efficiency in supplying pragmatic information; ii) mutual-information-driven message selection, which utilizes mutual information neural estimation to approach the optimal redundancy-less condition. Experiments on 3D object detection and BEV segmentation demonstrate that RDcomm achieves state-of-the-art accuracy on DAIR-V2X and OPV2V, while reducing communication volume by up to 108 times. The code will be released.

Method: Frames error discovery as structured search for minimal failure-inducing concepts, using novel acceleration techniques to make the computationally explosive problem tractable.

Result: Applied to Stable Diffusion models, discovered over 247,000 previously unknown error slices in SD1.5 alone, providing large-scale evidence linking failures to training data scarcity.

Conclusion: Establishes a new diagnostic-first methodology for deep model auditing to guide development of more robust generative AI.

Abstract: Static benchmarks have provided a valuable foundation for comparing Text-to-Image (T2I) models. However, their passive design offers limited diagnostic power, struggling to uncover the full landscape of systematic failures or isolate their root causes. We argue for a complementary paradigm: active exploration. We introduce FailureAtlas, the first framework designed to autonomously explore and map the vast failure landscape of T2I models at scale. FailureAtlas frames error discovery as a structured search for minimal, failure-inducing concepts. While it is a computationally explosive problem, we make it tractable with novel acceleration techniques. When applied to Stable Diffusion models, our method uncovers hundreds of thousands of previously unknown error slices (over 247,000 in SD1.5 alone) and provides the first large-scale evidence linking these failures to data scarcity in the training set. By providing a principled and scalable engine for deep model auditing, FailureAtlas establishes a new, diagnostic-first methodology to guide the development of more robust generative AI. The code is available at https://github.com/cure-lab/FailureAtlas

Method: Combines Qwen2.5-VL-3B-Instruct model with SmolAgent-based orchestration layer, supporting chain-of-thought reasoning, speech-vision fusion, dynamic tool invocation, and hybrid retrieval for structured scene graph generation.

Result: Achieves competitive accuracy on Video-MME benchmark and custom clinical dataset, showing improved robustness compared to state-of-the-art VLMs.

Conclusion: The framework demonstrates strong potential for applications in robot-assisted surgery, patient monitoring, and clinical decision support systems.

Abstract: Healthcare robotics requires robust multimodal perception and reasoning to ensure safety in dynamic clinical environments. Current Vision-Language Models (VLMs) demonstrate strong general-purpose capabilities but remain limited in temporal reasoning, uncertainty estimation, and structured outputs needed for robotic planning. We present a lightweight agentic multimodal framework for video-based scene understanding. Combining the Qwen2.5-VL-3B-Instruct model with a SmolAgent-based orchestration layer, it supports chain-of-thought reasoning, speech-vision fusion, and dynamic tool invocation. The framework generates structured scene graphs and leverages a hybrid retrieval module for interpretable and adaptive reasoning. Evaluations on the Video-MME benchmark and a custom clinical dataset show competitive accuracy and improved robustness compared to state-of-the-art VLMs, demonstrating its potential for applications in robot-assisted surgery, patient monitoring, and decision support.

Method: Introduces hallucination amplification: projecting captions into visual space via text-to-image model to reveal implicit hallucination signals as negative anchors, while original image provides positive anchors. Edits decoder hidden states by pulling representations toward faithful semantics and pushing away from hallucination directions.

Result: Significantly reduces hallucinations at object, attribute, and relation levels while preserving recall and caption richness. Achieves over 5% hallucination reduction on CHAIR using LLaVA-v1.5-7B. Validates strong cross-architecture generalization on diverse models.

Conclusion: The training-free, self-supervised method effectively mitigates hallucinations without side effects on hallucination-free captions, demonstrating robustness and practical plug-and-play applicability.

Abstract: Multimodal large language models (MLLMs) have achieved remarkable success across diverse vision-language tasks, yet they remain highly susceptible to hallucinations, producing content that is fluent but inconsistent with visual evidence. Such hallucinations, spanning objects, attributes, and relations, persist even in larger models, while existing mitigation approaches often require additional finetuning, handcrafted priors, or trade-offs that compromise informativeness and scalability. To address this limitation, we propose a training-free, self-supervised method for hallucination mitigation. Our approach introduces a novel hallucination amplification mechanism: a caption is projected into the visual space via a text-to-image model to reveal implicit hallucination signals, serving as a negative anchor, while the original image provides a positive anchor. Leveraging these dual anchors, we edit decoder hidden states by pulling representations toward faithful semantics and pushing them away from hallucination directions. This correction requires no human priors or additional training costs, ensuring both effectiveness and efficiency. Extensive experiments across multiple benchmarks show that our method significantly reduces hallucinations at the object, attribute, and relation levels while largely preserving recall and caption richness, e.g., achieving a hallucination reduction by over 5% using LLaVA-v1.5-7B on CHAIR. Furthermore, results on diverse architectures, including LLaVA-NEXT-7B, Cambrian-8B, and InstructBLIP-7B, validate strong cross-architecture generalization. More importantly, when applied to hallucination-free captions, our method introduces almost no side effects, underscoring its robustness and practical plug-and-play applicability. The implementation will be publicly available.

Method: Three-stage iterative process: (1) Diverse Sample Generation for exploring reasoning paths, (2) Dual Foresight Decoding to evaluate samples based on visual focus and reasoning progression, (3) Visual Focus Adjustment to precisely adjust focus toward beneficial regions for future reasoning.

Result: Consistent performance improvements of 3.1-5.8% across multiple benchmarks using Qwen2.5-VL, InternVL-2.5, and Llava-Next with controllable computational overhead.

Conclusion: CoFFT successfully enhances VLMs’ visual reasoning by creating an interdependent cycle where reasoning guides visual focus and visual focus informs subsequent reasoning, addressing limitations of current VLMs.

Abstract: Despite significant advances in Vision Language Models (VLMs), they remain constrained by the complexity and redundancy of visual input. When images contain large amounts of irrelevant information, VLMs are susceptible to interference, thus generating excessive task-irrelevant reasoning processes or even hallucinations. This limitation stems from their inability to discover and process the required regions during reasoning precisely. To address this limitation, we present the Chain of Foresight-Focus Thought (CoFFT), a novel training-free approach that enhances VLMs’ visual reasoning by emulating human visual cognition. Each Foresight-Focus Thought consists of three stages: (1) Diverse Sample Generation: generates diverse reasoning samples to explore potential reasoning paths, where each sample contains several reasoning steps; (2) Dual Foresight Decoding: rigorously evaluates these samples based on both visual focus and reasoning progression, adding the first step of optimal sample to the reasoning process; (3) Visual Focus Adjustment: precisely adjust visual focus toward regions most beneficial for future reasoning, before returning to stage (1) to generate subsequent reasoning samples until reaching the final answer. These stages function iteratively, creating an interdependent cycle where reasoning guides visual focus and visual focus informs subsequent reasoning. Empirical results across multiple benchmarks using Qwen2.5-VL, InternVL-2.5, and Llava-Next demonstrate consistent performance improvements of 3.1-5.8% with controllable increasing computational overhead.

Method: Created a new egocentric video dataset with ground-truth annotations, then benchmarked multimodal large language models (MLLMs) that jointly process images, audio, and text against conventional task-specific models.

Result: MLLMs outperformed specialized baselines even without task-specific fine-tuning, demonstrating their promise for holistic understanding of instructional interactions.

Conclusion: Multimodal large language models show strong potential for comprehensive understanding of face-to-face instructional scenes by effectively integrating verbal and nonverbal communication modalities.

Abstract: Analyzing instructional interactions between an instructor and a learner who are co-present in the same physical space is a critical problem for educational support and skill transfer. Yet such face-to-face instructional scenes have not been systematically studied in computer vision. We identify two key reasons: i) the lack of suitable datasets and ii) limited analytical techniques. To address this gap, we present a new egocentric video dataset of face-to-face instruction and provide ground-truth annotations for two fundamental tasks that serve as a first step toward a comprehensive understanding of instructional interactions: procedural step segmentation and conversation-state classification. Using this dataset, we benchmark multimodal large language models (MLLMs) against conventional task-specific models. Since face-to-face instruction involves multiple modalities (speech content and prosody, gaze and body motion, and visual context), effective understanding requires methods that handle verbal and nonverbal communication in an integrated manner. Accordingly, we evaluate recently introduced MLLMs that jointly process images, audio, and text. This evaluation quantifies the extent to which current machine learning models understand face-to-face instructional scenes. In experiments, MLLMs outperform specialized baselines even without task-specific fine-tuning, suggesting their promise for holistic understanding of instructional interactions.

Method: Proposes Spectral Cross-Attentional Network (SpecXNet) with Dual-Domain Feature Coupler (DDFC) that decomposes features into local spatial branch for texture anomalies and global spectral branch using FFT for periodic inconsistencies. Uses Dual Fourier Attention (DFA) to dynamically fuse spatial and spectral features. Built on modified XceptionNet backbone.

Result: Achieves state-of-the-art accuracy on multiple deepfake benchmarks, particularly under cross-dataset and unseen manipulation scenarios, while maintaining real-time feasibility.

Conclusion: Unified spatial-spectral learning is effective for robust and generalizable deepfake detection. Code released on GitHub for reproducibility.

Abstract: The increasing realism of content generated by GANs and diffusion models has made deepfake detection significantly more challenging. Existing approaches often focus solely on spatial or frequency-domain features, limiting their generalization to unseen manipulations. We propose the Spectral Cross-Attentional Network (SpecXNet), a dual-domain architecture for robust deepfake detection. The core \textbf{Dual-Domain Feature Coupler (DDFC)} decomposes features into a local spatial branch for capturing texture-level anomalies and a global spectral branch that employs Fast Fourier Transform to model periodic inconsistencies. This dual-domain formulation allows SpecXNet to jointly exploit localized detail and global structural coherence, which are critical for distinguishing authentic from manipulated images. We also introduce the \textbf{Dual Fourier Attention (DFA)} module, which dynamically fuses spatial and spectral features in a content-aware manner. Built atop a modified XceptionNet backbone, we embed the DDFC and DFA modules within a separable convolution block. Extensive experiments on multiple deepfake benchmarks show that SpecXNet achieves state-of-the-art accuracy, particularly under cross-dataset and unseen manipulation scenarios, while maintaining real-time feasibility. Our results highlight the effectiveness of unified spatial-spectral learning for robust and generalizable deepfake detection. To ensure reproducibility, we released the full code on \href{https://github.com/inzamamulDU/SpecXNet}{\textcolor{blue}{\textbf{GitHub}}}.

Method: Fine-tune a multiview material diffusion model conditioned on depth and normal maps, then use Gaussian material representation to model PBR material channels and reconstruct point clouds that can be rendered with PBR attributes.

Result: Extensive experiments show materials produced by MGM are more visually appealing than baseline methods and enhance material modeling for practical downstream rendering applications.

Conclusion: MGM successfully generates high-quality 3D content with PBR materials, enabling dynamic relighting and advancing practical rendering applications beyond current RGB-only approaches.

Abstract: The increasing demand for 3D assets across various industries necessitates efficient and automated methods for 3D content creation. Leveraging 3D Gaussian Splatting, recent large reconstruction models (LRMs) have demonstrated the ability to efficiently achieve high-quality 3D rendering by integrating multiview diffusion for generation and scalable transformers for reconstruction. However, existing models fail to produce the material properties of assets, which is crucial for realistic rendering in diverse lighting environments. In this paper, we introduce the Large Material Gaussian Model (MGM), a novel framework designed to generate high-quality 3D content with Physically Based Rendering (PBR) materials, ie, albedo, roughness, and metallic properties, rather than merely producing RGB textures with uncontrolled light baking. Specifically, we first fine-tune a new multiview material diffusion model conditioned on input depth and normal maps. Utilizing the generated multiview PBR images, we explore a Gaussian material representation that not only aligns with 2D Gaussian Splatting but also models each channel of the PBR materials. The reconstructed point clouds can then be rendered to acquire PBR attributes, enabling dynamic relighting by applying various ambient light maps. Extensive experiments demonstrate that the materials produced by our method not only exhibit greater visual appeal compared to baseline methods but also enhance material modeling, thereby enabling practical downstream rendering applications.

Method: Proposes self-supervised signals based on multi-view augmentations of single partial point clouds and incorporates Mamba architecture to enhance learning capability for generating higher quality point clouds.

Result: Achieves state-of-the-art performance on both synthetic and real-world datasets.

Conclusion: The proposed self-supervised method with multi-view augmentations and Mamba architecture effectively addresses limitations of existing approaches and demonstrates superior performance in point cloud completion tasks.

Abstract: Point cloud completion aims to reconstruct complete shapes from partial observations. Although current methods have achieved remarkable performance, they still have some limitations: Supervised methods heavily rely on ground truth, which limits their generalization to real-world datasets due to the synthetic-to-real domain gap. Unsupervised methods require complete point clouds to compose unpaired training data, and weakly-supervised methods need multi-view observations of the object. Existing self-supervised methods frequently produce unsatisfactory predictions due to the limited capabilities of their self-supervised signals. To overcome these challenges, we propose a novel self-supervised point cloud completion method. We design a set of novel self-supervised signals based on multi-view augmentations of the single partial point cloud. Additionally, to enhance the model’s learning ability, we first incorporate Mamba into self-supervised point cloud completion task, encouraging the model to generate point clouds with better quality. Experiments on synthetic and real-world datasets demonstrate that our method achieves state-of-the-art results.

Method: Introduces a tri-level knowledge fusion loss to transfer different levels of knowledge and a semi-supervised distillation method utilizing both labeled and unlabeled data.

Result: Achieves significant reductions in computational cost and latency while maintaining high-fidelity generation capabilities and controllability.

Conclusion: Refine-Control effectively addresses deployment challenges of conditional image generation models on edge devices through efficient knowledge distillation.

Abstract: Conditional image generation models have achieved remarkable results by leveraging text-based control to generate customized images. However, the high resource demands of these models and the scarcity of well-annotated data have hindered their deployment on edge devices, leading to enormous costs and privacy concerns, especially when user data is sent to a third party. To overcome these challenges, we propose Refine-Control, a semi-supervised distillation framework. Specifically, we improve the performance of the student model by introducing a tri-level knowledge fusion loss to transfer different levels of knowledge. To enhance generalization and alleviate dataset scarcity, we introduce a semi-supervised distillation method utilizing both labeled and unlabeled data. Our experiments reveal that Refine-Control achieves significant reductions in computational cost and latency, while maintaining high-fidelity generation capabilities and controllability, as quantified by comparative metrics.

Method: Uses joint graphs to capture class relationships, constructs loss functions based on joint graph entropy, employs Siamese structures for spatial transformation invariance, and implements both self-knowledge and teacher-knowledge distillation frameworks.

Result: Extensive experiments on ScanObject, ModelNet40, ScanntV2_cls and ModelNet-C demonstrate competitive performance.

Conclusion: JGEKD effectively handles non-IID 3D point cloud data by transferring class correlation knowledge through joint graph entropy distillation, achieving robust and competitive classification results.

Abstract: Classification tasks in 3D point clouds often assume that class events \replaced{are }{follow }independent and identically distributed (IID), although this assumption destroys the correlation between classes. This \replaced{study }{paper }proposes a classification strategy, \textbf{J}oint \textbf{G}raph \textbf{E}ntropy \textbf{K}nowledge \textbf{D}istillation (JGEKD), suitable for non-independent and identically distributed 3D point cloud data, \replaced{which }{the strategy } achieves knowledge transfer of class correlations through knowledge distillation by constructing a loss function based on joint graph entropy. First\deleted{ly}, we employ joint graphs to capture add{the }hidden relationships between classes\replaced{ and}{,} implement knowledge distillation to train our model by calculating the entropy of add{add }graph.\replaced{ Subsequently}{ Then}, to handle 3D point clouds \deleted{that is }invariant to spatial transformations, we construct \replaced{S}{s}iamese structures and develop two frameworks, self-knowledge distillation and teacher-knowledge distillation, to facilitate information transfer between different transformation forms of the same data. \replaced{In addition}{ Additionally}, we use the above framework to achieve knowledge transfer between point clouds and their corrupted forms, and increase the robustness against corruption of model. Extensive experiments on ScanObject, ModelNet40, ScanntV2_cls and ModelNet-C demonstrate that the proposed strategy can achieve competitive results.

Method: Train large multimodal models on a dataset of production-quality procedural materials and combine them with a constrained tree search inference algorithm that ensures syntactic validity while efficiently navigating the program space.

Result: MultiMat is more efficient in both unconditional and conditional graph synthesis with higher visual quality and fidelity than text-only baselines, establishing new state-of-the-art performance.

Conclusion: Multimodal program synthesis that leverages both visual and textual representations significantly improves the generation of procedural material graphs compared to text-only approaches.

Abstract: Material node graphs are programs that generate the 2D channels of procedural materials, including geometry such as roughness and displacement maps, and reflectance such as albedo and conductivity maps. They are essential in computer graphics for representing the appearance of virtual 3D objects parametrically and at arbitrary resolution. In particular, their directed acyclic graph structures and intermediate states provide an intuitive understanding and workflow for interactive appearance modeling. Creating such graphs is a challenging task and typically requires professional training. While recent neural program synthesis approaches attempt to simplify this process, they solely represent graphs as textual programs, failing to capture the inherently visual-spatial nature of node graphs that makes them accessible to humans. To address this gap, we present MultiMat, a multimodal program synthesis framework that leverages large multimodal models to process both visual and textual graph representations for improved generation of procedural material graphs. We train our models on a new dataset of production-quality procedural materials and combine them with a constrained tree search inference algorithm that ensures syntactic validity while efficiently navigating the program space. Our experimental results show that our multimodal program synthesis method is more efficient in both unconditional and conditional graph synthesis with higher visual quality and fidelity than text-only baselines, establishing new state-of-the-art performance.

Method: Combines StyleGAN’s structured latent space with DragGAN’s intuitive manipulation and PCA’s dimensionality reduction, applied to AFHQ dataset with focus on W+ latent layers.

Result: PCA integration reduces optimization time while maintaining visual quality and boosting SSIM, particularly in shallower latent spaces (W+ layers = 3). Enables cross-domain alignment between AFHQ-Dog and AFHQ-Cat models.

Conclusion: Demonstrates efficient and interpretable latent space control for image synthesis and editing applications through the integration of PCA with GAN frameworks.

Abstract: This work integrates StyleGAN, DragGAN and Principal Component Analysis (PCA) to enhance the latent space efficiency and controllability of GAN-generated images. Style-GAN provides a structured latent space, DragGAN enables intuitive image manipulation, and PCA reduces dimensionality and facilitates cross-model alignment for more streamlined and interpretable exploration of latent spaces. We apply our techniques to the Animal Faces High Quality (AFHQ) dataset, and find that our approach of integrating PCA-based dimensionality reduction with the Drag-GAN framework for image manipulation retains performance while improving optimization efficiency. Notably, introducing PCA into the latent W+ layers of DragGAN can consistently reduce the total optimization time while maintaining good visual quality and even boosting the Structural Similarity Index Measure (SSIM) of the optimized image, particularly in shallower latent spaces (W+ layers = 3). We also demonstrate capability for aligning images generated by two StyleGAN models trained on similar but distinct data domains (AFHQ-Dog and AFHQ-Cat), and show that we can control the latent space of these aligned images to manipulate the images in an intuitive and interpretable manner. Our findings highlight the possibility for efficient and interpretable latent space control for a wide range of image synthesis and editing applications.

Method: Two-stage parsing strategy: first stage performs layout analysis on downsampled images, second stage performs targeted content recognition on native-resolution crops guided by the layout. Uses comprehensive data engine for training data generation.

Result: Achieves state-of-the-art performance on multiple benchmarks, surpassing both general-purpose and domain-specific models across various recognition tasks with significantly lower computational overhead.

Conclusion: MinerU2.5 demonstrates strong document parsing ability through its efficient two-stage approach, balancing computational efficiency with high recognition accuracy for complex document elements.

Abstract: We introduce MinerU2.5, a 1.2B-parameter document parsing vision-language model that achieves state-of-the-art recognition accuracy while maintaining exceptional computational efficiency. Our approach employs a coarse-to-fine, two-stage parsing strategy that decouples global layout analysis from local content recognition. In the first stage, the model performs efficient layout analysis on downsampled images to identify structural elements, circumventing the computational overhead of processing high-resolution inputs. In the second stage, guided by the global layout, it performs targeted content recognition on native-resolution crops extracted from the original image, preserving fine-grained details in dense text, complex formulas, and tables. To support this strategy, we developed a comprehensive data engine that generates diverse, large-scale training corpora for both pretraining and fine-tuning. Ultimately, MinerU2.5 demonstrates strong document parsing ability, achieving state-of-the-art performance on multiple benchmarks, surpassing both general-purpose and domain-specific models across various recognition tasks, while maintaining significantly lower computational overhead.

Method: Two-stage alignment strategy: supervised fine-tuning to instill cognitive architecture, then Group Reward Policy Optimization to refine reasoning policy. Uses Geo-CoT380k dataset of structured rationales.

Result: RSThinker model significantly outperforms state-of-the-art models across comprehensive tasks, providing both final answers and verifiable analytical traces.

Conclusion: The framework enables transition from opaque perception to structured, verifiable reasoning in Earth Observation, with public release of dataset and model.

Abstract: Vision-Language Models (VLMs) in remote sensing often fail at complex analytical tasks, a limitation stemming from their end-to-end training paradigm that bypasses crucial reasoning steps and leads to unverifiable outputs. To address this limitation, we introduce the Perceptually-Grounded Geospatial Chain-of-Thought (Geo-CoT), a framework that models remote sensing analysis as a verifiable, multi-step process. We instill this analytical process through a two-stage alignment strategy, leveraging Geo-CoT380k, the first large-scale dataset of structured Geo-CoT rationales. This strategy first employs supervised fine-tuning (SFT) to instill the foundational cognitive architecture, then leverages Group Reward Policy Optimization (GRPO) to refine the model’s reasoning policy towards factual correctness. The resulting model, RSThinker, outputs both a final answer and its justifying, verifiable analytical trace. This capability yields dominant performance, significantly outperforming state-of-the-art models across a comprehensive range of tasks. The public release of our Geo-CoT380k dataset and RSThinker model upon publication serves as a concrete pathway from opaque perception towards structured, verifiable reasoning for Earth Observation.

Method: Leverages pre-trained 2D segmentation model to generate multi-granularity 2D masks lifted into 3D, estimates foreground probability for Gaussian points, optimizes boundaries using semantic entropy and geometric opacity, and uses Vision-Language Model to distill robust textual features from representative viewpoints.

Result: Reduces scene adaptation time from hours to minutes, outperforms established training-based frameworks on open-vocabulary 3D object selection and semantic segmentation tasks.

Conclusion: MUSplat successfully eliminates costly per-scene training while addressing monosemous limitations and achieving superior performance in open-vocabulary 3D understanding.

Abstract: Lifting 2D open-vocabulary understanding into 3D Gaussian Splatting (3DGS) scenes is a critical challenge. However, mainstream methods suffer from three key flaws: (i) their reliance on costly per-scene retraining prevents plug-and-play application; (ii) their restrictive monosemous design fails to represent complex, multi-concept semantics; and (iii) their vulnerability to cross-view semantic inconsistencies corrupts the final semantic representation. To overcome these limitations, we introduce MUSplat, a training-free framework that abandons feature optimization entirely. Leveraging a pre-trained 2D segmentation model, our pipeline generates and lifts multi-granularity 2D masks into 3D, where we estimate a foreground probability for each Gaussian point to form initial object groups. We then optimize the ambiguous boundaries of these initial groups using semantic entropy and geometric opacity. Subsequently, by interpreting the object’s appearance across its most representative viewpoints, a Vision-Language Model (VLM) distills robust textual features that reconciles visual inconsistencies, enabling open-vocabulary querying via semantic matching. By eliminating the costly per-scene training process, MUSplat reduces scene adaptation time from hours to mere minutes. On benchmark tasks for open-vocabulary 3D object selection and semantic segmentation, MUSplat outperforms established training-based frameworks while simultaneously addressing their monosemous limitations.

Method: Collected multi-temporal single-view and panoramic street-view images from 11 representative cities worldwide, and generated high-quality question-answer pairs through a hybrid pipeline of spatial clustering, rule-based generation, model-assisted prompting, and manual annotation.

Result: Gemini-2.5 Pro achieved the best overall performance, approaching human expert levels with only 1.5% average gap. Most models performed well on scene understanding tasks, with some surpassing human annotators in pixel-level change detection. However, performance dropped notably in temporal reasoning tasks, while several models reached human-level consistency in subjective perception dimensions like beauty and safety.

Conclusion: UrbanFeel provides a comprehensive benchmark for evaluating MLLMs in urban environments, revealing that while models excel at scene understanding and some subjective perception tasks, they still struggle with temporal reasoning over urban development, highlighting areas for future improvement.

Abstract: Urban development impacts over half of the global population, making human-centered understanding of its structural and perceptual changes essential for sustainable development. While Multimodal Large Language Models (MLLMs) have shown remarkable capabilities across various domains, existing benchmarks that explore their performance in urban environments remain limited, lacking systematic exploration of temporal evolution and subjective perception of urban environment that aligns with human perception. To address these limitations, we propose UrbanFeel, a comprehensive benchmark designed to evaluate the performance of MLLMs in urban development understanding and subjective environmental perception. UrbanFeel comprises 14.3K carefully constructed visual questions spanning three cognitively progressive dimensions: Static Scene Perception, Temporal Change Understanding, and Subjective Environmental Perception. We collect multi-temporal single-view and panoramic street-view images from 11 representative cities worldwide, and generate high-quality question-answer pairs through a hybrid pipeline of spatial clustering, rule-based generation, model-assisted prompting, and manual annotation. Through extensive evaluation of 20 state-of-the-art MLLMs, we observe that Gemini-2.5 Pro achieves the best overall performance, with its accuracy approaching human expert levels and narrowing the average gap to just 1.5%. Most models perform well on tasks grounded in scene understanding. In particular, some models even surpass human annotators in pixel-level change detection. However, performance drops notably in tasks requiring temporal reasoning over urban development. Additionally, in the subjective perception dimension, several models reach human-level or even higher consistency in evaluating dimension such as beautiful and safety.

Method: Dual Experts framework with frozen source-domain model (with Conv-Adapter) and pretrained vision-language model (with trainable text prompt), plus Retrieval-Augmented-Interaction pipeline that retrieves pseudo-source/complex samples, fine-tunes experts separately, and enforces learning consistency.

Result: Extensive experiments on four benchmark datasets demonstrate that the approach matches state-of-the-art performance in source-free unsupervised domain adaptation.

Conclusion: EXCL effectively addresses SFUDA challenges by leveraging dual experts and retrieval-augmentation interaction, providing a robust solution for domain adaptation without source data access.

Abstract: Source-Free Unsupervised Domain Adaptation (SFUDA) addresses the realistic challenge of adapting a source-trained model to a target domain without access to the source data, driven by concerns over privacy and cost. Existing SFUDA methods either exploit only the source model’s predictions or fine-tune large multimodal models, yet both neglect complementary insights and the latent structure of target data. In this paper, we propose the Experts Cooperative Learning (EXCL). EXCL contains the Dual Experts framework and Retrieval-Augmentation-Interaction optimization pipeline. The Dual Experts framework places a frozen source-domain model (augmented with Conv-Adapter) and a pretrained vision-language model (with a trainable text prompt) on equal footing to mine consensus knowledge from unlabeled target samples. To effectively train these plug-in modules under purely unsupervised conditions, we introduce Retrieval-Augmented-Interaction(RAIN), a three-stage pipeline that (1) collaboratively retrieves pseudo-source and complex target samples, (2) separately fine-tunes each expert on its respective sample set, and (3) enforces learning object consistency via a shared learning result. Extensive experiments on four benchmark datasets demonstrate that our approach matches state-of-the-art performance.

Method: Three key innovations: (1) One-Step Inversion-and-Editing (OSIE) pipeline that bypasses iterative processes; (2) Background Shield (BG-Shield) for selective feature modification only in edit regions; (3) Sparsified Spatial Cross-Attention (SSCA) to suppress semantic leakage and ensure precise localized edits.

Result: FlashEdit performs edits in under 0.2 seconds (150x speedup over prior methods) while maintaining superior background consistency and structural integrity.

Conclusion: FlashEdit enables high-fidelity, real-time image editing through its efficient framework design, making text-guided image editing practical for real-world applications.

Abstract: Text-guided image editing with diffusion models has achieved remarkable quality but suffers from prohibitive latency, hindering real-world applications. We introduce FlashEdit, a novel framework designed to enable high-fidelity, real-time image editing. Its efficiency stems from three key innovations: (1) a One-Step Inversion-and-Editing (OSIE) pipeline that bypasses costly iterative processes; (2) a Background Shield (BG-Shield) technique that guarantees background preservation by selectively modifying features only within the edit region; and (3) a Sparsified Spatial Cross-Attention (SSCA) mechanism that ensures precise, localized edits by suppressing semantic leakage to the background. Extensive experiments demonstrate that FlashEdit maintains superior background consistency and structural integrity, while performing edits in under 0.2 seconds, which is an over 150$\times$ speedup compared to prior multi-step methods. Our code will be made publicly available at https://github.com/JunyiWuCode/FlashEdit.

Method: Developed Neural-MedBench integrating multi-sequence MRI scans, EHRs, and clinical notes with three task families: differential diagnosis, lesion recognition, and rationale generation. Used hybrid scoring combining LLM-based graders, clinician validation, and semantic similarity metrics.

Result: Evaluation of GPT-4o, Claude-4, and MedGemma showed sharp performance drops compared to conventional datasets. Error analysis revealed reasoning failures dominate model shortcomings rather than perceptual errors.

Conclusion: Proposes a Two-Axis Evaluation Framework: breadth-oriented datasets for statistical generalization and depth-oriented benchmarks like Neural-MedBench for reasoning fidelity. Released as open diagnostic testbed for rigorous assessment of clinically trustworthy AI.

Abstract: Recent advances in vision-language models (VLMs) have achieved remarkable performance on standard medical benchmarks, yet their true clinical reasoning ability remains unclear. Existing datasets predominantly emphasize classification accuracy, creating an evaluation illusion in which models appear proficient while still failing at high-stakes diagnostic reasoning. We introduce Neural-MedBench, a compact yet reasoning-intensive benchmark specifically designed to probe the limits of multimodal clinical reasoning in neurology. Neural-MedBench integrates multi-sequence MRI scans, structured electronic health records, and clinical notes, and encompasses three core task families: differential diagnosis, lesion recognition, and rationale generation. To ensure reliable evaluation, we develop a hybrid scoring pipeline that combines LLM-based graders, clinician validation, and semantic similarity metrics. Through systematic evaluation of state-of-the-art VLMs, including GPT-4o, Claude-4, and MedGemma, we observe a sharp performance drop compared to conventional datasets. Error analysis shows that reasoning failures, rather than perceptual errors, dominate model shortcomings. Our findings highlight the necessity of a Two-Axis Evaluation Framework: breadth-oriented large datasets for statistical generalization, and depth-oriented, compact benchmarks such as Neural-MedBench for reasoning fidelity. We release Neural-MedBench at https://neuromedbench.github.io/ as an open and extensible diagnostic testbed, which guides the expansion of future benchmarks and enables rigorous yet cost-effective assessment of clinically trustworthy AI.

Method: Represents lane lines as discrete sequences with iterative generation, supports multi-modal inputs (BEV, PV, text prompts), and uses state update strategy for global continuity and consistency.

Result: Achieves state-of-the-art performance on OpenSatMap dataset, can infer occluded roads and predict missing roads from dataset annotations.

Conclusion: UniMapGen provides an efficient generative framework for large-scale map construction that overcomes limitations of traditional and satellite-based methods through discrete sequence representation and multi-modal input support.

Abstract: Large-scale map construction is foundational for critical applications such as autonomous driving and navigation systems. Traditional large-scale map construction approaches mainly rely on costly and inefficient special data collection vehicles and labor-intensive annotation processes. While existing satellite-based methods have demonstrated promising potential in enhancing the efficiency and coverage of map construction, they exhibit two major limitations: (1) inherent drawbacks of satellite data (e.g., occlusions, outdatedness) and (2) inefficient vectorization from perception-based methods, resulting in discontinuous and rough roads that require extensive post-processing. This paper presents a novel generative framework, UniMapGen, for large-scale map construction, offering three key innovations: (1) representing lane lines as \textbf{discrete sequence} and establishing an iterative strategy to generate more complete and smooth map vectors than traditional perception-based methods. (2) proposing a flexible architecture that supports \textbf{multi-modal} inputs, enabling dynamic selection among BEV, PV, and text prompt, to overcome the drawbacks of satellite data. (3) developing a \textbf{state update} strategy for global continuity and consistency of the constructed large-scale map. UniMapGen achieves state-of-the-art performance on the OpenSatMap dataset. Furthermore, UniMapGen can infer occluded roads and predict roads missing from dataset annotations. Our code will be released.

Method: Joint optimization of attributes for rendered depth and normals quality, novel roughness supervision based on multi-view photometric variation, and carefully designed loss and optimization process using 3D Gaussian Splatting.

Result: Produces reconstruction results comparable to state-of-the-art methods, delivering triangle meshes and associated material components. Validated on widely used datasets with qualitative comparisons showing effectiveness.

Conclusion: GS-2M provides a unified solution that maintains geometric details, handles reflective surfaces well, and eliminates the need for complex neural components while achieving state-of-the-art performance.

Abstract: We propose a unified solution for mesh reconstruction and material decomposition from multi-view images based on 3D Gaussian Splatting, referred to as GS-2M. Previous works handle these tasks separately and struggle to reconstruct highly reflective surfaces, often relying on priors from external models to enhance the decomposition results. Conversely, our method addresses these two problems by jointly optimizing attributes relevant to the quality of rendered depth and normals, maintaining geometric details while being resilient to reflective surfaces. Although contemporary works effectively solve these tasks together, they often employ sophisticated neural components to learn scene properties, which hinders their performance at scale. To further eliminate these neural components, we propose a novel roughness supervision strategy based on multi-view photometric variation. When combined with a carefully designed loss and optimization process, our unified framework produces reconstruction results comparable to state-of-the-art methods, delivering triangle meshes and their associated material components for downstream tasks. We validate the effectiveness of our approach with widely used datasets from previous works and qualitative comparisons with state-of-the-art surface reconstruction methods.

Method: Proposes a Spatial Reasoning Chain that decomposes scene generation into object inference, spatial interrelation reasoning, and scene graph construction. Uses an LLM-based framework enhanced with DPO algorithms to generate physically plausible 3D layouts.

Result: MesaTask-10K dataset with 10,700 synthetic tabletop scenes with manually crafted realistic layouts. MesaTask framework demonstrates superior performance compared to baselines in generating task-conforming scenes with realistic layouts.

Conclusion: The proposed MesaTask framework effectively bridges the gap between high-level task instructions and tabletop scene generation, producing physically plausible and task-aligned scenes that outperform existing methods.

Abstract: The ability of robots to interpret human instructions and execute manipulation tasks necessitates the availability of task-relevant tabletop scenes for training. However, traditional methods for creating these scenes rely on time-consuming manual layout design or purely randomized layouts, which are limited in terms of plausibility or alignment with the tasks. In this paper, we formulate a novel task, namely task-oriented tabletop scene generation, which poses significant challenges due to the substantial gap between high-level task instructions and the tabletop scenes. To support research on such a challenging task, we introduce MesaTask-10K, a large-scale dataset comprising approximately 10,700 synthetic tabletop scenes with manually crafted layouts that ensure realistic layouts and intricate inter-object relations. To bridge the gap between tasks and scenes, we propose a Spatial Reasoning Chain that decomposes the generation process into object inference, spatial interrelation reasoning, and scene graph construction for the final 3D layout. We present MesaTask, an LLM-based framework that utilizes this reasoning chain and is further enhanced with DPO algorithms to generate physically plausible tabletop scenes that align well with given task descriptions. Exhaustive experiments demonstrate the superior performance of MesaTask compared to baselines in generating task-conforming tabletop scenes with realistic layouts. Project page is at https://mesatask.github.io/

Method: Applied rule-based reinforcement learning to Document Image Classification task, testing on three out-of-distribution scenarios: different images, unseen classes, and different modalities.

Result: Reinforcement learning demonstrated better generalization capabilities compared to traditional methods across all three out-of-distribution scenarios.

Conclusion: Rule-based reinforcement learning is effective for document analysis tasks and shows superior generalization to out-of-distribution data.

Abstract: Rule-based reinforcement learning has been gaining popularity ever since DeepSeek-R1 has demonstrated its success through simple verifiable rewards. In the domain of document analysis, reinforcement learning is not as prevalent, even though many downstream tasks may benefit from the emerging properties of reinforcement learning, particularly the enhanced reason capabilities. We study the effects of rule-based reinforcement learning with the task of Document Image Classification which is one of the most commonly studied downstream tasks in document analysis. We find that reinforcement learning tends to have better generalisation capabilities to out-of-distritbution data, which we examine in three different scenarios, namely out-of-distribution images, unseen classes and different modalities. Our code is available at https://github.com/jungomi/vision-finetune.

Method: SceneSplit fragments harmful narratives into multiple individually benign scenes, uses sequential scene combination to constrain the output space to unsafe regions, employs iterative scene manipulation to bypass safety filters, and utilizes a strategy library to reuse successful attack patterns.

Result: SceneSplit achieves high Attack Success Rates: 77.2% on Luma Ray2, 84.1% on Hailuo, and 78.2% on Veo2 across 11 safety categories, significantly outperforming existing baselines.

Conclusion: Current T2V safety mechanisms are vulnerable to narrative structure exploitation attacks, providing new insights for understanding and improving T2V model safety.

Abstract: Along with the rapid advancement of numerous Text-to-Video (T2V) models, growing concerns have emerged regarding their safety risks. While recent studies have explored vulnerabilities in models like LLMs, VLMs, and Text-to-Image (T2I) models through jailbreak attacks, T2V models remain largely unexplored, leaving a significant safety gap. To address this gap, we introduce SceneSplit, a novel black-box jailbreak method that works by fragmenting a harmful narrative into multiple scenes, each individually benign. This approach manipulates the generative output space, the abstract set of all potential video outputs for a given prompt, using the combination of scenes as a powerful constraint to guide the final outcome. While each scene individually corresponds to a wide and safe space where most outcomes are benign, their sequential combination collectively restricts this space, narrowing it to an unsafe region and significantly increasing the likelihood of generating a harmful video. This core mechanism is further enhanced through iterative scene manipulation, which bypasses the safety filter within this constrained unsafe region. Additionally, a strategy library that reuses successful attack patterns further improves the attack’s overall effectiveness and robustness. To validate our method, we evaluate SceneSplit across 11 safety categories on T2V models. Our results show that it achieves a high average Attack Success Rate (ASR) of 77.2% on Luma Ray2, 84.1% on Hailuo, and 78.2% on Veo2, significantly outperforming the existing baseline. Through this work, we demonstrate that current T2V safety mechanisms are vulnerable to attacks that exploit narrative structure, providing new insights for understanding and improving the safety of T2V models.

Method: Leverages the difference between current prediction and weighted average of past predictions to steer sampling process, requiring no additional computation, training, or fine-tuning.

Result: Achieves SOTA FID of 1.61 for unguided ImageNet generation at 256×256 with only 30 steps (vs standard 250), consistently improves image quality across diverse models and sampling budgets.

Conclusion: HiGS is a plug-and-play enhancement that enables faster generation with higher fidelity in diffusion sampling.

Abstract: While diffusion models have made remarkable progress in image generation, their outputs can still appear unrealistic and lack fine details, especially when using fewer number of neural function evaluations (NFEs) or lower guidance scales. To address this issue, we propose a novel momentum-based sampling technique, termed history-guided sampling (HiGS), which enhances quality and efficiency of diffusion sampling by integrating recent model predictions into each inference step. Specifically, HiGS leverages the difference between the current prediction and a weighted average of past predictions to steer the sampling process toward more realistic outputs with better details and structure. Our approach introduces practically no additional computation and integrates seamlessly into existing diffusion frameworks, requiring neither extra training nor fine-tuning. Extensive experiments show that HiGS consistently improves image quality across diverse models and architectures and under varying sampling budgets and guidance scales. Moreover, using a pretrained SiT model, HiGS achieves a new state-of-the-art FID of 1.61 for unguided ImageNet generation at 256$\times$256 with only 30 sampling steps (instead of the standard 250). We thus present HiGS as a plug-and-play enhancement to standard diffusion sampling that enables faster generation with higher fidelity.

Method: Uses a dual-stream CNN-Transformer architecture with Paired Window Attention (PWA) for multi-scale information retrieval and Johnson-Lindenstrauss lemma-guided convolution (JLC) for robust local feature extraction. Incorporates modal interaction and Spatially Decoupled Knowledge Transfer (SDKT) via Gram matrices to inject texture priors from self-supervised networks.

Result: Achieves 26% Dice improvement on multimodal benchmarks while increasing GPU throughput by 11x and CPU throughput by 48x compared to baselines.

Conclusion: VeloxSeg successfully addresses the efficiency/robustness trade-off in lightweight 3D medical image segmentation, providing significant performance gains and computational efficiency without extra inference cost.

Abstract: Lightweight 3D medical image segmentation remains constrained by a fundamental “efficiency / robustness conflict”, particularly when processing complex anatomical structures and heterogeneous modalities. In this paper, we study how to redesign the framework based on the characteristics of high-dimensional 3D images, and explore data synergy to overcome the fragile representation of lightweight methods. Our approach, VeloxSeg, begins with a deployable and extensible dual-stream CNN-Transformer architecture composed of Paired Window Attention (PWA) and Johnson-Lindenstrauss lemma-guided convolution (JLC). For each 3D image, we invoke a “glance-and-focus” principle, where PWA rapidly retrieves multi-scale information, and JLC ensures robust local feature extraction with minimal parameters, significantly enhancing the model’s ability to operate with low computational budget. Followed by an extension of the dual-stream architecture that incorporates modal interaction into the multi-scale image-retrieval process, VeloxSeg efficiently models heterogeneous modalities. Finally, Spatially Decoupled Knowledge Transfer (SDKT) via Gram matrices injects the texture prior extracted by a self-supervised network into the segmentation network, yielding stronger representations than baselines at no extra inference cost. Experimental results on multimodal benchmarks show that VeloxSeg achieves a 26% Dice improvement, alongside increasing GPU throughput by 11x and CPU by 48x. Codes are available at https://github.com/JinPLu/VeloxSeg.

Method: Non-parametric flow-matching model built on non-local patch matching, combining diffusion model insights with classical texture optimization techniques without requiring neural network training.

Result: Experimental results show NIFTY outperforms representative methods from the literature in exemplar-based texture synthesis.

Conclusion: NIFTY provides an effective hybrid approach that leverages strengths of both diffusion models and patch-based methods while avoiding their respective limitations.

Abstract: This paper addresses the problem of exemplar-based texture synthesis. We introduce NIFTY, a hybrid framework that combines recent insights on diffusion models trained with convolutional neural networks, and classical patch-based texture optimization techniques. NIFTY is a non-parametric flow-matching model built on non-local patch matching, which avoids the need for neural network training while alleviating common shortcomings of patch-based methods, such as poor initialization or visual artifacts. Experimental results demonstrate the effectiveness of the proposed approach compared to representative methods from the literature. Code is available at https://github.com/PierrickCh/Nifty.git

Method: Three lightweight policy heads (Step-Skip, Cache-Reuse, Sparse-Attention) observe denoising state and independently decide speed-up at each timestep. Parameters are trained online via Group Relative Policy Optimization while generator remains frozen, with an adversarial discriminator to prevent reward hacking.

Result: RAPID3 achieves nearly 3x faster sampling across state-of-the-art DiT backbones including Stable Diffusion 3 and FLUX, while maintaining competitive generation quality.

Conclusion: The framework successfully enables image-wise adaptive acceleration for diffusion transformers without requiring updates to the base generator, achieving significant speed improvements with preserved quality.

Abstract: Diffusion Transformers (DiTs) excel at visual generation yet remain hampered by slow sampling. Existing training-free accelerators - step reduction, feature caching, and sparse attention - enhance inference speed but typically rely on a uniform heuristic or a manually designed adaptive strategy for all images, leaving quality on the table. Alternatively, dynamic neural networks offer per-image adaptive acceleration, but their high fine-tuning costs limit broader applicability. To address these limitations, we introduce RAPID3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformers, a framework that delivers image-wise acceleration with zero updates to the base generator. Specifically, three lightweight policy heads - Step-Skip, Cache-Reuse, and Sparse-Attention - observe the current denoising state and independently decide their corresponding speed-up at each timestep. All policy parameters are trained online via Group Relative Policy Optimization (GRPO) while the generator remains frozen. Meanwhile, an adversarially learned discriminator augments the reward signal, discouraging reward hacking by boosting returns only when generated samples stay close to the original model’s distribution. Across state-of-the-art DiT backbones, including Stable Diffusion 3 and FLUX, RAPID3 achieves nearly 3x faster sampling with competitive generation quality.

Method: Constructs a multi-modal knowledge graph to mine relationships between local visual features and text, and between attributes and visual context samples. Introduces a knowledge graph-guided cross-modal hypergraph learning framework.

Result: Comprehensive experiments on multiple PAR benchmark datasets demonstrate the effectiveness of the proposed knowledge graph for PAR tasks.

Conclusion: The approach establishes a strong foundation for knowledge-guided pedestrian attribute recognition and the source code will be publicly released.

Abstract: Current Pedestrian Attribute Recognition (PAR) algorithms typically focus on mapping visual features to semantic labels or attempt to enhance learning by fusing visual and attribute information. However, these methods fail to fully exploit attribute knowledge and contextual information for more accurate recognition. Although recent works have started to consider using attribute text as additional input to enhance the association between visual and semantic information, these methods are still in their infancy. To address the above challenges, this paper proposes the construction of a multi-modal knowledge graph, which is utilized to mine the relationships between local visual features and text, as well as the relationships between attributes and extensive visual context samples. Specifically, we propose an effective multi-modal knowledge graph construction method that fully considers the relationships among attributes and the relationships between attributes and vision tokens. To effectively model these relationships, this paper introduces a knowledge graph-guided cross-modal hypergraph learning framework to enhance the standard pedestrian attribute recognition framework. Comprehensive experiments on multiple PAR benchmark datasets have thoroughly demonstrated the effectiveness of our proposed knowledge graph for the PAR task, establishing a strong foundation for knowledge-guided pedestrian attribute recognition. The source code of this paper will be released on https://github.com/Event-AHU/OpenPAR

Method: Created CircuitSense benchmark with 8,006+ problems spanning component-level schematics to system-level block diagrams, using a hierarchical synthetic generation pipeline with grid-based schematic generator and block diagram generator with auto-derived symbolic equation labels.

Result: Closed-source models achieve over 85% accuracy on perception tasks but fall below 19% on symbolic derivation and analytical reasoning. Models with stronger symbolic reasoning capabilities consistently achieve higher design task accuracy.

Conclusion: Symbolic reasoning is established as the key metric for engineering competence in MLLMs, with fundamental limitations identified in visual-to-mathematical reasoning despite strong visual parsing capabilities.

Abstract: Engineering design operates through hierarchical abstraction from system specifications to component implementations, requiring visual understanding coupled with mathematical reasoning at each level. While Multi-modal Large Language Models (MLLMs) excel at natural image tasks, their ability to extract mathematical models from technical diagrams remains unexplored. We present \textbf{CircuitSense}, a comprehensive benchmark evaluating circuit understanding across this hierarchy through 8,006+ problems spanning component-level schematics to system-level block diagrams. Our benchmark uniquely examines the complete engineering workflow: Perception, Analysis, and Design, with a particular emphasis on the critical but underexplored capability of deriving symbolic equations from visual inputs. We introduce a hierarchical synthetic generation pipeline consisting of a grid-based schematic generator and a block diagram generator with auto-derived symbolic equation labels. Comprehensive evaluation of six state-of-the-art MLLMs, including both closed-source and open-source models, reveals fundamental limitations in visual-to-mathematical reasoning. Closed-source models achieve over 85% accuracy on perception tasks involving component recognition and topology identification, yet their performance on symbolic derivation and analytical reasoning falls below 19%, exposing a critical gap between visual parsing and symbolic reasoning. Models with stronger symbolic reasoning capabilities consistently achieve higher design task accuracy, confirming the fundamental role of mathematical understanding in circuit synthesis and establishing symbolic reasoning as the key metric for engineering competence.

Method: Proposes Hierarchical Extended Path Aggregation Network (HEPAN) for multi-scale feature fusion, plus lightweight modules (IRDCB and LDown) to reduce parameters/computation, and a specialized small object detection head for tiny objects (4 pixels).

Result: Demonstrates state-of-the-art performance on VisDrone2019 benchmark through comparison experiments and ablation studies.

Conclusion: HierLight-YOLO effectively addresses the dual challenges of detecting small targets in drone imagery while maintaining real-time efficiency through hierarchical feature fusion and lightweight design.

Abstract: The real-time detection of small objects in complex scenes, such as the unmanned aerial vehicle (UAV) photography captured by drones, has dual challenges of detecting small targets (<32 pixels) and maintaining real-time efficiency on resource-constrained platforms. While YOLO-series detectors have achieved remarkable success in real-time large object detection, they suffer from significantly higher false negative rates for drone-based detection where small objects dominate, compared to large object scenarios. This paper proposes HierLight-YOLO, a hierarchical feature fusion and lightweight model that enhances the real-time detection of small objects, based on the YOLOv8 architecture. We propose the Hierarchical Extended Path Aggregation Network (HEPAN), a multi-scale feature fusion method through hierarchical cross-level connections, enhancing the small object detection accuracy. HierLight-YOLO includes two innovative lightweight modules: Inverted Residual Depthwise Convolution Block (IRDCB) and Lightweight Downsample (LDown) module, which significantly reduce the model’s parameters and computational complexity without sacrificing detection capabilities. Small object detection head is designed to further enhance spatial resolution and feature fusion to tackle the tiny object (4 pixels) detection. Comparison experiments and ablation studies on the VisDrone2019 benchmark demonstrate state-of-the-art performance of HierLight-YOLO.

Method: Leveraged GPT-4o with optimized prompt engineering, structured multimodal analysis framework, preprocessing for token limitations, six evaluation criteria with self-assessment, and tested across multiple datasets (Gossipcop, Politifact, Fakeddit, MMFakeBench, AMMEBA).

Result: Comprehensive performance analysis revealed GPT-4o’s strengths and limitations in disinformation detection, with investigation of prediction variability and stability through repeated testing.

Conclusion: The study provides a robust and reproducible methodological framework for automated multimodal disinformation analysis using confidence-level and variability-based evaluation methods.

Abstract: The proliferation of disinformation, particularly in multimodal contexts combining text and images, presents a significant challenge across digital platforms. This study investigates the potential of large multimodal models (LMMs) in detecting and mitigating false information. We propose to approach multimodal disinformation detection by leveraging the advanced capabilities of the GPT-4o model. Our contributions include: (1) the development of an optimized prompt incorporating advanced prompt engineering techniques to ensure precise and consistent evaluations; (2) the implementation of a structured framework for multimodal analysis, including a preprocessing methodology for images and text to comply with the model’s token limitations; (3) the definition of six specific evaluation criteria that enable a fine-grained classification of content, complemented by a self-assessment mechanism based on confidence levels; (4) a comprehensive performance analysis of the model across multiple heterogeneous datasets Gossipcop, Politifact, Fakeddit, MMFakeBench, and AMMEBA highlighting GPT-4o’s strengths and limitations in disinformation detection; (5) an investigation of prediction variability through repeated testing, evaluating the stability and reliability of the model’s classifications; and (6) the introduction of confidence-level and variability-based evaluation methods. These contributions provide a robust and reproducible methodological framework for automated multimodal disinformation analysis.

Method: Leverage pre-trained GPT-4 with specifically designed prompts to analyze images and generate occlusion order predictions, then parse responses to construct occlusion matrices for integration into occlusion handling frameworks.

Result: Evaluation on COCOA and InstaOrder datasets shows the model produces more accurate order predictions using semantic context, visual patterns, and commonsense knowledge compared to baseline methods.

Conclusion: GPT-4 enables zero-shot occlusion reasoning without annotated training data, providing an easily integrable solution for occlusion handling tasks and improving image understanding.

Abstract: Occlusion remains a significant challenge for current vision models to robustly interpret complex and dense real-world images and scenes. To address this limitation and to enable accurate prediction of the occlusion order relationship between objects, we propose leveraging the advanced capability of a pre-trained GPT-4 model to deduce the order. By providing a specifically designed prompt along with the input image, GPT-4 can analyze the image and generate order predictions. The response can then be parsed to construct an occlusion matrix which can be utilized in assisting with other occlusion handling tasks and image understanding. We report the results of evaluating the model on COCOA and InstaOrder datasets. The results show that by using semantic context, visual patterns, and commonsense knowledge, the model can produce more accurate order predictions. Unlike baseline methods, the model can reason about occlusion relationships in a zero-shot fashion, which requires no annotated training data and can easily be integrated into occlusion handling frameworks.

Method: Proposes a gradient-domain-based model for initial fusion with complete boundaries, uses Tenengrad gradient detection for salient feature extraction, and develops a focus metric integrating gradient and complementary information for boundary refinement.

Result: Extensive experiments on four public datasets show the method consistently outperforms 12 state-of-the-art methods in both subjective and objective evaluations.

Conclusion: The proposed method effectively enhances boundary quality in multi-focus image fusion while preserving focused details, demonstrating superior performance compared to existing approaches.

Abstract: Multi-focus image fusion (MFIF) aims to yield an all-focused image from multiple partially focused inputs, which is crucial in applications cover sur-veillance, microscopy, and computational photography. However, existing methods struggle to preserve sharp focus-defocus boundaries, often resulting in blurred transitions and focused details loss. To solve this problem, we propose a MFIF method based on significant boundary enhancement, which generates high-quality fused boundaries while effectively detecting focus in-formation. Particularly, we propose a gradient-domain-based model that can obtain initial fusion results with complete boundaries and effectively pre-serve the boundary details. Additionally, we introduce Tenengrad gradient detection to extract salient features from both the source images and the ini-tial fused image, generating the corresponding saliency maps. For boundary refinement, we develop a focus metric based on gradient and complementary information, integrating the salient features with the complementary infor-mation across images to emphasize focused regions and produce a high-quality initial decision result. Extensive experiments on four public datasets demonstrate that our method consistently outperforms 12 state-of-the-art methods in both subjective and objective evaluations. We have realized codes in https://github.com/Lihyua/GICI

Method: Uses clustering-based surrogate model training to convert dynamic-output scenarios to static single-output scenarios, and employs farthest-label targeted attack strategy to maximize disruption.

Result: Achieves maximum attack success rate of 50.81% with single query per text on dynamic scenarios, and 82.68% on static scenarios. Also extends to generative settings with improvements of up to 0.64 RDBLEU and 0.62 RDchrF.

Conclusion: TDOA effectively addresses dynamic output scenarios in text adversarial attacks, demonstrating strong performance across multiple datasets and models with limited access, while also excelling in conventional static scenarios.

Abstract: Text adversarial attack methods are typically designed for static scenarios with fixed numbers of output labels and a predefined label space, relying on extensive querying of the victim model (query-based attacks) or the surrogate model (transfer-based attacks). To address this gap, we introduce the Textual Dynamic Outputs Attack (TDOA) method, which employs a clustering-based surrogate model training approach to convert the dynamic-output scenario into a static single-output scenario. To improve attack effectiveness, we propose the farthest-label targeted attack strategy, which selects adversarial vectors that deviate most from the model’s coarse-grained labels, thereby maximizing disruption. We extensively evaluate TDOA on four datasets and eight victim models (e.g., ChatGPT-4o, ChatGPT-4.1), showing its effectiveness in crafting adversarial examples and its strong potential to compromise large language models with limited access. With a single query per text, TDOA achieves a maximum attack success rate of 50.81%. Additionally, we find that TDOA also achieves state-of-the-art performance in conventional static output scenarios, reaching a maximum ASR of 82.68%. Meanwhile, by conceptualizing translation tasks as classification problems with unbounded output spaces, we extend the TDOA framework to generative settings, surpassing prior results by up to 0.64 RDBLEU and 0.62 RDchrF.

Method: RAU combines VLM spatial reasoning (trained on moderately sized datasets) with SAM2’s segmentation capability. It uses relative spatial reasoning between reference and target images for identification, then integrates VLM-derived spatial cues with SAM2 for pixel-level segmentation.

Result: RAU outperforms SAM2 fine-tuning baselines across in-distribution and out-of-distribution datasets, achieving more accurate segmentations and reliable localization. It shows strong generalization to out-of-distribution data, crucial for medical applications.

Conclusion: RAU demonstrates the first successful use of VLMs for reference-based anatomical understanding in medical imaging, highlighting the potential of VLM-driven approaches for automated clinical workflows with strong generalization capabilities.

Abstract: Anatomical understanding through deep learning is critical for automatic report generation, intra-operative navigation, and organ localization in medical imaging; however, its progress is constrained by the scarcity of expert-labeled data. A promising remedy is to leverage an annotated reference image to guide the interpretation of an unlabeled target. Although recent vision-language models (VLMs) exhibit non-trivial visual reasoning, their reference-based understanding and fine-grained localization remain limited. We introduce RAU, a framework for reference-based anatomical understanding with VLMs. We first show that a VLM learns to identify anatomical regions through relative spatial reasoning between reference and target images, trained on a moderately sized dataset. We validate this capability through visual question answering (VQA) and bounding box prediction. Next, we demonstrate that the VLM-derived spatial cues can be seamlessly integrated with the fine-grained segmentation capability of SAM2, enabling localization and pixel-level segmentation of small anatomical regions, such as vessel segments. Across two in-distribution and two out-of-distribution datasets, RAU consistently outperforms a SAM2 fine-tuning baseline using the same memory setup, yielding more accurate segmentations and more reliable localization. More importantly, its strong generalization ability makes it scalable to out-of-distribution datasets, a property crucial for medical image applications. To the best of our knowledge, RAU is the first to explore the capability of VLMs for reference-based identification, localization, and segmentation of anatomical structures in medical images. Its promising performance highlights the potential of VLM-driven approaches for anatomical understanding in automated clinical workflows.

Method: Use Logic Tensor Networks (LTNs) to encode medical background knowledge using first-order logic rules, integrated with SwinUNETR in an end-to-end framework for semantic segmentation.

Result: LTNs improved baseline segmentation performance for hippocampus segmentation in brain MRI scans, particularly when training data was scarce.

Conclusion: Neuro-symbolic methods like LTNs are general enough to be adapted to other medical semantic segmentation tasks and show promise for improving performance with limited data.

Abstract: Semantic segmentation is a fundamental task in medical image analysis, aiding medical decision-making by helping radiologists distinguish objects in an image. Research in this field has been driven by deep learning applications, which have the potential to scale these systems even in the presence of noise and artifacts. However, these systems are not yet perfected. We argue that performance can be improved by incorporating common medical knowledge into the segmentation model’s loss function. To this end, we introduce Logic Tensor Networks (LTNs) to encode medical background knowledge using first-order logic (FOL) rules. The encoded rules span from constraints on the shape of the produced segmentation, to relationships between different segmented areas. We apply LTNs in an end-to-end framework with a SwinUNETR for semantic segmentation. We evaluate our method on the task of segmenting the hippocampus in brain MRI scans. Our experiments show that LTNs improve the baseline segmentation performance, especially when training data is scarce. Despite being in its preliminary stages, we argue that neurosymbolic methods are general enough to be adapted and applied to other medical semantic segmentation tasks.

Method: Proposes Multi-Scale Explanation Aggregation (MSEA) for visual branch to aggregate attributions over multi-scale inputs, and Activation Ranking Correlation (ARC) for textual branch to measure token relevance via prediction ranking alignment.

Result: Extensive experiments show the approach consistently outperforms existing interpretability methods, yielding more faithful and fine-grained explanations.

Conclusion: The proposed methods effectively enhance MLLM interpretability by leveraging intra-modal interactions, addressing limitations of current attribution approaches.

Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse vision-language tasks, yet their internal decision-making mechanisms remain insufficiently understood. Existing interpretability research has primarily focused on cross-modal attribution, identifying which image regions the model attends to during output generation. However, these approaches often overlook intra-modal dependencies. In the visual modality, attributing importance to isolated image patches ignores spatial context due to limited receptive fields, resulting in fragmented and noisy explanations. In the textual modality, reliance on preceding tokens introduces spurious activations. Failing to effectively mitigate these interference compromises attribution fidelity. To address these limitations, we propose enhancing interpretability by leveraging intra-modal interaction. For the visual branch, we introduce \textit{Multi-Scale Explanation Aggregation} (MSEA), which aggregates attributions over multi-scale inputs to dynamically adjust receptive fields, producing more holistic and spatially coherent visual explanations. For the textual branch, we propose \textit{Activation Ranking Correlation} (ARC), which measures the relevance of contextual tokens to the current token via alignment of their top-$k$ prediction rankings. ARC leverages this relevance to suppress spurious activations from irrelevant contexts while preserving semantically coherent ones. Extensive experiments across state-of-the-art MLLMs and benchmark datasets demonstrate that our approach consistently outperforms existing interpretability methods, yielding more faithful and fine-grained explanations of model behavior.

Method: Proposes VARE framework with auxiliary visual tokens to reduce fine-tuning intensity, and S-VARE method with filtered cross entropy loss to precisely identify unsafe tokens and preservation loss to maintain semantic fidelity.

Result: Extensive experiments show the approach achieves surgical concept erasure while preserving generation quality, effectively closing safety gaps in autoregressive text-to-image generation.

Conclusion: The proposed VARE and S-VARE methods successfully address concept erasure challenges in VAR models, providing stable safety improvements without compromising generation capabilities.

Abstract: The rapid progress of visual autoregressive (VAR) models has brought new opportunities for text-to-image generation, but also heightened safety concerns. Existing concept erasure techniques, primarily designed for diffusion models, fail to generalize to VARs due to their next-scale token prediction paradigm. In this paper, we first propose a novel VAR Erasure framework VARE that enables stable concept erasure in VAR models by leveraging auxiliary visual tokens to reduce fine-tuning intensity. Building upon this, we introduce S-VARE, a novel and effective concept erasure method designed for VAR, which incorporates a filtered cross entropy loss to precisely identify and minimally adjust unsafe visual tokens, along with a preservation loss to maintain semantic fidelity, addressing the issues such as language drift and reduced diversity introduce by na"ive fine-tuning. Extensive experiments demonstrate that our approach achieves surgical concept erasure while preserving generation quality, thereby closing the safety gap in autoregressive text-to-image generation by earlier methods.

Method: Developed scalable 2DGS pipeline with structured initialization, luminance-aware pruning, and optimized CUDA kernels. Adapted CLIP training by reusing frozen RGB transformer backbone with splat-aware input stem and perceiver resampler, training only 7% of parameters.

Result: Achieved 90x faster fitting and 97% GPU utilization compared to prior implementations. 2DGS encoders yield meaningful zero-shot ImageNet-1K performance while compressing inputs 3-20x relative to pixels. Accuracy currently trails RGB encoders but establishes viability.

Conclusion: 2DGS is established as a viable multimodal substrate that opens a path toward representations that are both semantically powerful and transmission-efficient for edge-cloud learning, though current accuracy needs improvement.

Abstract: Modern vision language pipelines are driven by RGB vision encoders trained on massive image text corpora. While these pipelines have enabled impressive zero shot capabilities and strong transfer across tasks, they still inherit two structural inefficiencies from the pixel domain: (i) transmitting dense RGB images from edge devices to the cloud is energy intensive and costly, and (ii) patch based tokenization explodes sequence length, stressing attention budgets and context limits. We explore 2D Gaussian Splatting (2DGS) as an alternative visual substrate for alignment: a compact, spatially adaptive representation that parameterizes images by a set of colored anisotropic Gaussians. We develop a scalable 2DGS pipeline with structured initialization, luminance aware pruning, and batched CUDA kernels, achieving over 90x faster fitting and about 97% GPU utilization compared to prior implementations. We further adapt contrastive language image pretraining (CLIP) to 2DGS by reusing a frozen RGB-based transformer backbone with a lightweight splat aware input stem and a perceiver resampler, training only about 7% of the total parameters. On large DataComp subsets, GS encoders yield meaningful zero shot ImageNet-1K performance while compressing inputs 3 to 20x relative to pixels. While accuracy currently trails RGB encoders, our results establish 2DGS as a viable multimodal substrate, pinpoint architectural bottlenecks, and open a path toward representations that are both semantically powerful and transmission efficient for edge cloud learning.

Method: Proposes FreqDebias with two strategies: 1) Forgery Mixup (Fo-Mixup) augmentation to diversify frequency characteristics, and 2) dual consistency regularization using class activation maps (local) and von Mises-Fisher distribution on hyperspherical embeddings (global).

Result: Extensive experiments show FreqDebias significantly enhances cross-domain generalization and outperforms state-of-the-art methods in both cross-domain and in-domain settings.

Conclusion: The proposed frequency debiasing framework effectively mitigates spectral bias and improves deepfake detector generalization across different forgery types.

Abstract: Deepfake detectors often struggle to generalize to novel forgery types due to biases learned from limited training data. In this paper, we identify a new type of model bias in the frequency domain, termed spectral bias, where detectors overly rely on specific frequency bands, restricting their ability to generalize across unseen forgeries. To address this, we propose FreqDebias, a frequency debiasing framework that mitigates spectral bias through two complementary strategies. First, we introduce a novel Forgery Mixup (Fo-Mixup) augmentation, which dynamically diversifies frequency characteristics of training samples. Second, we incorporate a dual consistency regularization (CR), which enforces both local consistency using class activation maps (CAMs) and global consistency through a von Mises-Fisher (vMF) distribution on a hyperspherical embedding space. This dual CR mitigates over-reliance on certain frequency components by promoting consistent representation learning under both local and global supervision. Extensive experiments show that FreqDebias significantly enhances cross-domain generalization and outperforms state-of-the-art methods in both cross-domain and in-domain settings.

Method: Uses lightweight dual-branch conditioner to inject signals from degraded input and lightly restored proxy; timestep- and layer-adaptive modulation schedule; caption-free semantic alignment via SigLIP features; scalable curation pipeline for structure-rich supervision.

Result: Consistently outperforms strong open-source and commercial baselines across synthetic and in-the-wild benchmarks. Ablation studies verify necessity of each component.

Conclusion: For large diffusion transformers, the key to robust caption-free universal image restoration is determining when, where, and what to condition on, rather than adding parameters or relying on text prompts.

Abstract: Universal image restoration (UIR) aims to recover images degraded by unknown mixtures while preserving semantics – conditions under which discriminative restorers and UNet-based diffusion priors often oversmooth, hallucinate, or drift. We present LucidFlux, a caption-free UIR framework that adapts a large diffusion transformer (Flux.1) without image captions. LucidFlux introduces a lightweight dual-branch conditioner that injects signals from the degraded input and a lightly restored proxy to respectively anchor geometry and suppress artifacts. Then, a timestep- and layer-adaptive modulation schedule is designed to route these cues across the backbone’s hierarchy, in order to yield coarse-to-fine and context-aware updates that protect the global structure while recovering texture. After that, to avoid the latency and instability of text prompts or MLLM captions, we enforce caption-free semantic alignment via SigLIP features extracted from the proxy. A scalable curation pipeline further filters large-scale data for structure-rich supervision. Across synthetic and in-the-wild benchmarks, LucidFlux consistently outperforms strong open-source and commercial baselines, and ablation studies verify the necessity of each component. LucidFlux shows that, for large DiTs, when, where, and what to condition on – rather than adding parameters or relying on text prompts – is the governing lever for robust and caption-free universal image restoration in the wild.

Method: A central orchestrator agent powered by a multimodal language model uses multi-step reasoning to execute specialized tools: Calibrated Discovery for sourcing relevant data, Controllable Synthesis for generating rare scenario data, and Consensus Annotation for accurate labeling using multiple foundation models with novel consensus mechanisms.

Result: Consensus Annotation achieved 14.2 candidate proposals per image (vs 7.4 ground-truth) on COCO with 37.1% mAP, discovered 903 new categories on Open Images. Calibrated Discovery was 40x more computationally efficient at 10-million sample scale with equivalent sample efficiency.

Conclusion: Agentic workflows with optimized, scalable tools provide a robust foundation for curating industrial-scale datasets, overcoming traditional data curation bottlenecks.

Abstract: Curating high-quality, domain-specific datasets is a major bottleneck for deploying robust vision systems, requiring complex trade-offs between data quality, diversity, and cost when researching vast, unlabeled data lakes. We introduce Labeling Copilot, the first data curation deep research agent for computer vision. A central orchestrator agent, powered by a large multimodal language model, uses multi-step reasoning to execute specialized tools across three core capabilities: (1) Calibrated Discovery sources relevant, in-distribution data from large repositories; (2) Controllable Synthesis generates novel data for rare scenarios with robust filtering; and (3) Consensus Annotation produces accurate labels by orchestrating multiple foundation models via a novel consensus mechanism incorporating non-maximum suppression and voting. Our large-scale validation proves the effectiveness of Labeling Copilot’s components. The Consensus Annotation module excels at object discovery: on the dense COCO dataset, it averages 14.2 candidate proposals per image-nearly double the 7.4 ground-truth objects-achieving a final annotation mAP of 37.1%. On the web-scale Open Images dataset, it navigated extreme class imbalance to discover 903 new bounding box categories, expanding its capability to over 1500 total. Concurrently, our Calibrated Discovery tool, tested at a 10-million sample scale, features an active learning strategy that is up to 40x more computationally efficient than alternatives with equivalent sample efficiency. These experiments validate that an agentic workflow with optimized, scalable tools provides a robust foundation for curating industrial-scale datasets.

Method: Proposed U-MAN architecture with two key modules: Progressive Attention-Guided Feature Fusion (PAGF) to replace simple skip connections using attention mechanisms, and Multi-scale Adaptive KAN (MAN) to enable adaptive multi-scale feature processing for segmenting objects of various sizes.

Result: Experiments on three public datasets (BUSI, GLAS, and CVC) demonstrate that U-MAN outperforms state-of-the-art methods, particularly excelling in defining accurate boundaries and preserving fine details.

Conclusion: U-MAN effectively addresses the core limitations of conventional U-Nets through its attention-guided feature fusion and multi-scale adaptive processing, achieving superior performance in medical image segmentation tasks.

Abstract: Medical image segmentation faces significant challenges in preserving fine-grained details and precise boundaries due to complex anatomical structures and pathological regions. These challenges primarily stem from two key limitations of conventional U-Net architectures: (1) their simple skip connections ignore the encoder-decoder semantic gap between various features, and (2) they lack the capability for multi-scale feature extraction in deep layers. To address these challenges, we propose the U-Net with Multi-scale Adaptive KAN (U-MAN), a novel architecture that enhances the emerging Kolmogorov-Arnold Network (KAN) with two specialized modules: Progressive Attention-Guided Feature Fusion (PAGF) and the Multi-scale Adaptive KAN (MAN). Our PAGF module replaces the simple skip connection, using attention to fuse features from the encoder and decoder. The MAN module enables the network to adaptively process features at multiple scales, improving its ability to segment objects of various sizes. Experiments on three public datasets (BUSI, GLAS, and CVC) show that U-MAN outperforms state-of-the-art methods, particularly in defining accurate boundaries and preserving fine details.

Method: The authors propose γ-Quant, which learns a non-linear quantization specifically for pattern recognition tasks. The approach is demonstrated on raw-image object detection and human activity recognition using wearable sensor data.

Result: The method shows that raw data with learnable quantization using only 4-bits can perform on par with raw 12-bit data, achieving comparable performance while significantly reducing data requirements.

Conclusion: Task-specific learnable quantization enables efficient pattern recognition in low-bandwidth and energy-constrained settings, making it possible to use low-bit-depth sensor data without sacrificing performance.

Abstract: Most pattern recognition models are developed on pre-proce-ssed data. In computer vision, for instance, RGB images processed through image signal processing (ISP) pipelines designed to cater to human perception are the most frequent input to image analysis networks. However, many modern vision tasks operate without a human in the loop, raising the question of whether such pre-processing is optimal for automated analysis. Similarly, human activity recognition (HAR) on body-worn sensor data commonly takes normalized floating-point data arising from a high-bit analog-to-digital converter (ADC) as an input, despite such an approach being highly inefficient in terms of data transmission, significantly affecting the battery life of wearable devices. In this work, we target low-bandwidth and energy-constrained settings where sensors are limited to low-bit-depth capture. We propose $\gamma$-Quant, i.e.~the task-specific learning of a non-linear quantization for pattern recognition. We exemplify our approach on raw-image object detection as well as HAR of wearable data, and demonstrate that raw data with a learnable quantization using as few as 4-bits can perform on par with the use of raw 12-bit data. All code to reproduce our experiments is publicly available via https://github.com/Mishalfatima/Gamma-Quant

Method: Created a dataset with detailed annotations including natural-language explanations, bounding-box regions, and timestamps; trained multimodal language models as reward models to mimic human judgments.

Result: The 7B reward model outperformed GPT-5 by 34.7% across fake clue identification, grounding, and explanation tasks, with performance degrading from explanations to spatial and temporal grounding.

Conclusion: DeeptraceReward provides a rigorous framework for improving trustworthy video generation by emphasizing human-perceived deepfake traces.

Abstract: Can humans identify AI-generated (fake) videos and provide grounded reasons? While video generation models have advanced rapidly, a critical dimension – whether humans can detect deepfake traces within a generated video, i.e., spatiotemporal grounded visual artifacts that reveal a video as machine generated – has been largely overlooked. We introduce DeeptraceReward, the first fine-grained, spatially- and temporally- aware benchmark that annotates human-perceived fake traces for video generation reward. The dataset comprises 4.3K detailed annotations across 3.3K high-quality generated videos. Each annotation provides a natural-language explanation, pinpoints a bounding-box region containing the perceived trace, and marks precise onset and offset timestamps. We consolidate these annotations into 9 major categories of deepfake traces that lead humans to identify a video as AI-generated, and train multimodal language models (LMs) as reward models to mimic human judgments and localizations. On DeeptraceReward, our 7B reward model outperforms GPT-5 by 34.7% on average across fake clue identification, grounding, and explanation. Interestingly, we observe a consistent difficulty gradient: binary fake v.s. real classification is substantially easier than fine-grained deepfake trace detection; within the latter, performance degrades from natural language explanations (easiest), to spatial grounding, to temporal labeling (hardest). By foregrounding human-perceived deepfake traces, DeeptraceReward provides a rigorous testbed and training signal for socially aware and trustworthy video generation.

Method: Uses cross-segmentation consistency between feature-level and pixel-level fusion-based segmentation as a self-supervised task. Implements a two-stage training strategy with dynamic weight adjustment for effective joint learning.

Result: Extensive experiments show SSVIF outperforms traditional VIF methods and rivals supervised segmentation-oriented methods, despite being trained only on unlabeled visible-infrared image pairs.

Conclusion: SSVIF provides an effective self-supervised solution for segmentation-oriented VIF that eliminates the need for expensive labeled data while achieving competitive performance with supervised approaches.

Abstract: Visible and infrared image fusion (VIF) has gained significant attention in recent years due to its wide application in tasks such as scene segmentation and object detection. VIF methods can be broadly classified into traditional VIF methods and application-oriented VIF methods. Traditional methods focus solely on improving the quality of fused images, while application-oriented VIF methods additionally consider the performance of downstream tasks on fused images by introducing task-specific loss terms during training. However, compared to traditional methods, application-oriented VIF methods require datasets labeled for downstream tasks (e.g., semantic segmentation or object detection), making data acquisition labor-intensive and time-consuming. To address this issue, we propose a self-supervised training framework for segmentation-oriented VIF methods (SSVIF). Leveraging the consistency between feature-level fusion-based segmentation and pixel-level fusion-based segmentation, we introduce a novel self-supervised task-cross-segmentation consistency-that enables the fusion model to learn high-level semantic features without the supervision of segmentation labels. Additionally, we design a two-stage training strategy and a dynamic weight adjustment method for effective joint learning within our self-supervised framework. Extensive experiments on public datasets demonstrate the effectiveness of our proposed SSVIF. Remarkably, although trained only on unlabeled visible-infrared image pairs, our SSVIF outperforms traditional VIF methods and rivals supervised segmentation-oriented ones. Our code will be released upon acceptance.

Method: Two-stage pipeline: LVLM generates caption, then vision-free LLM answers multiple-choice questions based on caption alone. Reward is derived from QA accuracy, defining caption quality by its utility for downstream tasks.

Result: CapRL significantly improves performance across 12 benchmarks. Pretraining on CapRL-5M dataset yields substantial gains. Achieves comparable performance to Qwen2.5-VL-72B with 8.4% average improvement over baseline in Prism Framework.

Conclusion: CapRL successfully applies RLVR to subjective image captioning, demonstrating that utility-based reward design enables training more generalizable and diverse captioning models without expensive human annotations.

Abstract: Image captioning is a fundamental task that bridges the visual and linguistic domains, playing a critical role in pre-training Large Vision-Language Models (LVLMs). Current state-of-the-art captioning models are typically trained with Supervised Fine-Tuning (SFT), a paradigm that relies on expensive, non-scalable data annotated by humans or proprietary models. This approach often leads to models that memorize specific ground-truth answers, limiting their generality and ability to generate diverse, creative descriptions. To overcome the limitation of SFT, we propose applying the Reinforcement Learning with Verifiable Rewards (RLVR) paradigm to the open-ended task of image captioning. A primary challenge, however, is designing an objective reward function for the inherently subjective nature of what constitutes a “good” caption. We introduce Captioning Reinforcement Learning (CapRL), a novel training framework that redefines caption quality through its utility: a high-quality caption should enable a non-visual language model to accurately answer questions about the corresponding image. CapRL employs a decoupled two-stage pipeline where an LVLM generates a caption, and the objective reward is derived from the accuracy of a separate, vision-free LLM answering Multiple-Choice Questions based solely on that caption. As the first study to apply RLVR to the subjective image captioning task, we demonstrate that CapRL significantly enhances multiple settings. Pretraining on the CapRL-5M caption dataset annotated by CapRL-3B results in substantial gains across 12 benchmarks. Moreover, within the Prism Framework for caption quality evaluation, CapRL achieves performance comparable to Qwen2.5-VL-72B, while exceeding the baseline by an average margin of 8.4%. Code is available here: https://github.com/InternLM/CapRL.

Method: Uses B'ezier-curve-based style transfer to reduce domain gap, trains segmentation model on transferred images, then uses pseudo-labels to train conditional diffusion model with uncertainty-guided score matching for robustness.

Result: Extensive experiments show the approach generates realistic labeled images, significantly augments target domain data, and improves segmentation performance on public datasets.

Conclusion: The unified framework effectively addresses domain adaptation challenges in medical imaging through combined B'ezier style transfer and diffusion modeling, producing high-quality synthetic data that enhances segmentation across domains.

Abstract: Training robust learning algorithms across different medical imaging modalities is challenging due to the large domain gap. Unsupervised domain adaptation (UDA) mitigates this problem by using annotated images from the source domain and unlabeled images from the target domain to train the deep models. Existing approaches often rely on GAN-based style transfer, but these methods struggle to capture cross-domain mappings in regions with high variability. In this paper, we propose a unified framework, B'ezier Meets Diffusion, for cross-domain image generation. First, we introduce a B'ezier-curve-based style transfer strategy that effectively reduces the domain gap between source and target domains. The transferred source images enable the training of a more robust segmentation model across domains. Thereafter, using pseudo-labels generated by this segmentation model on the target domain, we train a conditional diffusion model (CDM) to synthesize high-quality, labeled target-domain images. To mitigate the impact of noisy pseudo-labels, we further develop an uncertainty-guided score matching method that improves the robustness of CDM training. Extensive experiments on public datasets demonstrate that our approach generates realistic labeled images, significantly augmenting the target domain and improving segmentation performance.

Method: PSTTS uses two stages: Spatial Token Purification removes noise and non-event regions by assessing spatio-temporal consistency within frames, and Temporal Token Selection evaluates motion pattern similarity between adjacent frames to remove redundant temporal information.

Result: Applied to four backbones on three datasets, PSTTS reduces FLOPs by 29-43.6% and increases FPS by 21.6-41.3% on DailyDVS-200 while maintaining accuracy.

Conclusion: PSTTS effectively addresses computational inefficiency in event-based vision by leveraging raw event data characteristics for token selection, achieving optimal accuracy-efficiency trade-off without additional parameters.

Abstract: Mainstream event-based spatio-temporal representation learning methods typically process event streams by converting them into sequences of event frames, achieving remarkable performance. However, they neglect the high spatial sparsity and inter-frame motion redundancy inherent in event frame sequences, leading to significant computational overhead. Existing token sparsification methods for RGB videos rely on unreliable intermediate token representations and neglect the influence of event noise, making them ineffective for direct application to event data. In this paper, we propose Progressive Spatio-Temporal Token Selection (PSTTS), a Plug-and-Play module for event data without introducing any additional parameters. PSTTS exploits the spatio-temporal distribution characteristics embedded in raw event data to effectively identify and discard spatio-temporal redundant tokens, achieving an optimal trade-off between accuracy and efficiency. Specifically, PSTTS consists of two stages, Spatial Token Purification and Temporal Token Selection. Spatial Token Purification discards noise and non-event regions by assessing the spatio-temporal consistency of events within each event frame to prevent interference with subsequent temporal redundancy evaluation. Temporal Token Selection evaluates the motion pattern similarity between adjacent event frames, precisely identifying and removing redundant temporal information. We apply PSTTS to four representative backbones UniformerV2, VideoSwin, EVMamba, and ExACT on the HARDVS, DailyDVS-200, and SeACT datasets. Experimental results demonstrate that PSTTS achieves significant efficiency improvements. Specifically, PSTTS reduces FLOPs by 29-43.6% and increases FPS by 21.6-41.3% on the DailyDVS-200 dataset, while maintaining task accuracy. Our code will be available.

Method: GCPO identifies critical tokens from three perspectives: causal dependency (early tokens determine later ones), entropy-induced spatial structure (high entropy gradient tokens), and RLVR-focused token diversity (low visual similarity tokens). It then applies dynamic token-wise advantage weighting for these critical tokens.

Result: GCPO achieves better performance than GRPO using only 30% of image tokens, demonstrating effectiveness across multiple text-to-image benchmarks for both AR models and unified multimodal models.

Conclusion: Targeted optimization of critical tokens in autoregressive visual generation significantly improves training efficiency and performance compared to uniform optimization across all tokens.

Abstract: Recent studies have extended Reinforcement Learning with Verifiable Rewards (RLVR) to autoregressive (AR) visual generation and achieved promising progress. However, existing methods typically apply uniform optimization across all image tokens, while the varying contributions of different image tokens for RLVR’s training remain unexplored. In fact, the key obstacle lies in how to identify more critical image tokens during AR generation and implement effective token-wise optimization for them. To tackle this challenge, we propose $\textbf{G}$roup $\textbf{C}$ritical-token $\textbf{P}$olicy $\textbf{O}$ptimization ($\textbf{GCPO}$), which facilitates effective policy optimization on critical tokens. We identify the critical tokens in RLVR-based AR generation from three perspectives, specifically: $\textbf{(1)}$ Causal dependency: early tokens fundamentally determine the later tokens and final image effect due to unidirectional dependency; $\textbf{(2)}$ Entropy-induced spatial structure: tokens with high entropy gradients correspond to image structure and bridges distinct visual regions; $\textbf{(3)}$ RLVR-focused token diversity: tokens with low visual similarity across a group of sampled images contribute to richer token-level diversity. For these identified critical tokens, we further introduce a dynamic token-wise advantage weight to encourage exploration, based on confidence divergence between the policy model and reference model. By leveraging 30% of the image tokens, GCPO achieves better performance than GRPO with full tokens. Extensive experiments on multiple text-to-image benchmarks for both AR models and unified multimodal models demonstrate the effectiveness of GCPO for AR visual generation.

Method: EAGLE uses an objective function unifying sufficiency and indispensability scores, optimized via greedy search over sparsified image regions for efficient attribution. It performs modality-aware analysis to disentangle token dependencies.

Result: EAGLE consistently outperforms existing methods in faithfulness, localization, and hallucination diagnosis while requiring substantially less GPU memory.

Conclusion: EAGLE effectively advances MLLM interpretability through faithful and efficient attribution of token generation to visual inputs.

Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in aligning visual inputs with natural language outputs. Yet, the extent to which generated tokens depend on visual modalities remains poorly understood, limiting interpretability and reliability. In this work, we present EAGLE, a lightweight black-box framework for explaining autoregressive token generation in MLLMs. EAGLE attributes any selected tokens to compact perceptual regions while quantifying the relative influence of language priors and perceptual evidence. The framework introduces an objective function that unifies sufficiency (insight score) and indispensability (necessity score), optimized via greedy search over sparsified image regions for faithful and efficient attribution. Beyond spatial attribution, EAGLE performs modality-aware analysis that disentangles what tokens rely on, providing fine-grained interpretability of model decisions. Extensive experiments across open-source MLLMs show that EAGLE consistently outperforms existing methods in faithfulness, localization, and hallucination diagnosis, while requiring substantially less GPU memory. These results highlight its effectiveness and practicality for advancing the interpretability of MLLMs. The code is available at https://github.com/RuoyuChen10/EAGLE.

Method: Replicated classic color naming methodologies using 957 color samples across five representative VLMs, conducted cross-linguistic analysis across nine languages, and performed ablation studies on language model architecture.

Result: VLMs achieve high accuracy on prototypical colors but performance drops significantly on expanded, non-prototypical color sets. Identified 21 common color terms across all models, with constrained models using basic terms and expansive models using systematic lightness modifiers. Cross-linguistic analysis revealed severe training imbalances favoring English and Chinese, with hue as the primary driver of color naming decisions.

Conclusion: Language model architecture significantly influences color naming independent of visual processing capabilities, highlighting the need for more balanced training across languages and better handling of non-prototypical colors in VLMs.

Abstract: Color serves as a fundamental dimension of human visual perception and a primary means of communicating about objects and scenes. As vision-language models (VLMs) become increasingly prevalent, understanding whether they name colors like humans is crucial for effective human-AI interaction. We present the first systematic evaluation of color naming capabilities across VLMs, replicating classic color naming methodologies using 957 color samples across five representative models. Our results show that while VLMs achieve high accuracy on prototypical colors from classical studies, performance drops significantly on expanded, non-prototypical color sets. We identify 21 common color terms that consistently emerge across all models, revealing two distinct approaches: constrained models using predominantly basic terms versus expansive models employing systematic lightness modifiers. Cross-linguistic analysis across nine languages demonstrates severe training imbalances favoring English and Chinese, with hue serving as the primary driver of color naming decisions. Finally, ablation studies reveal that language model architecture significantly influences color naming independent of visual processing capabilities.

Method: Combines transformer architecture with lightweight convolutional decoder and bimodal density head; trained on labeled synthetic/real images and pseudo-labeled real images; uses multi-stage optimization strategy and LPIPS-based loss function.

Result: Achieves performance comparable to or better than state-of-the-art models with significantly reduced computational resources.

Conclusion: EfficientDepth successfully addresses key challenges in monocular depth estimation while maintaining computational efficiency suitable for edge devices.

Abstract: Monocular depth estimation (MDE) plays a pivotal role in various computer vision applications, such as robotics, augmented reality, and autonomous driving. Despite recent advancements, existing methods often fail to meet key requirements for 3D reconstruction and view synthesis, including geometric consistency, fine details, robustness to real-world challenges like reflective surfaces, and efficiency for edge devices. To address these challenges, we introduce a novel MDE system, called EfficientDepth, which combines a transformer architecture with a lightweight convolutional decoder, as well as a bimodal density head that allows the network to estimate detailed depth maps. We train our model on a combination of labeled synthetic and real images, as well as pseudo-labeled real images, generated using a high-performing MDE method. Furthermore, we employ a multi-stage optimization strategy to improve training efficiency and produce models that emphasize geometric consistency and fine detail. Finally, in addition to commonly used objectives, we introduce a loss function based on LPIPS to encourage the network to produce detailed depth maps. Experimental results demonstrate that EfficientDepth achieves performance comparable to or better than existing state-of-the-art models, with significantly reduced computational resources.

Method: The survey organizes literature into a taxonomy with two base settings (NCD and GCD) and several derived settings for real-world scenarios. It analyzes methods through three fundamental components: representation learning, label assignment, and estimation of class number.

Result: Benchmarking reveals that large-scale pretrained backbones, hierarchical and auxiliary cues, and curriculum-style training benefit category discovery. Challenges remain in label assignment design, class number estimation, and scaling to complex multi-object scenarios.

Conclusion: The survey provides key insights from current literature and identifies promising future research directions, while maintaining a living survey repository for ongoing updates in the category discovery field.

Abstract: Category discovery (CD) is an emerging open-world learning task, which aims at automatically categorizing unlabelled data containing instances from unseen classes, given some labelled data from seen classes. This task has attracted significant attention over the years and leads to a rich body of literature trying to address the problem from different perspectives. In this survey, we provide a comprehensive review of the literature, and offer detailed analysis and in-depth discussion on different methods. Firstly, we introduce a taxonomy for the literature by considering two base settings, namely novel category discovery (NCD) and generalized category discovery (GCD), and several derived settings that are designed to address the extra challenges in different real-world application scenarios, including continual category discovery, skewed data distribution, federated category discovery, etc. Secondly, for each setting, we offer a detailed analysis of the methods encompassing three fundamental components, representation learning, label assignment, and estimation of class number. Thirdly, we benchmark all the methods and distill key insights showing that large-scale pretrained backbones, hierarchical and auxiliary cues, and curriculum-style training are all beneficial for category discovery, while challenges remain in the design of label assignment, the estimation of class numbers, and scaling to complex multi-object scenarios.Finally, we discuss the key insights from the literature so far and point out promising future research directions. We compile a living survey of the category discovery literature at \href{https://github.com/Visual-AI/Category-Discovery}{https://github.com/Visual-AI/Category-Discovery}.

Method: Hybrid approach combining multi-task SSL temporal analyzer (using nnFormer backbone) with LLM validator. SSL identifies suspicious frames, then LLM applies structured rule-based reasoning for semantic validation.

Result: Achieves 72.5% frame-level AUC on ComplexVAD dataset, outperforming baselines by 12.5% while reducing LLM computation.

Conclusion: HyCoVAD effectively addresses complex video anomaly detection by leveraging complementary strengths of SSL and LLMs, with released taxonomy and tools for future research.

Abstract: Video anomaly detection (VAD) is crucial for intelligent surveillance, but a significant challenge lies in identifying complex anomalies, which are events defined by intricate relationships and temporal dependencies among multiple entities rather than by isolated actions. While self-supervised learning (SSL) methods effectively model low-level spatiotemporal patterns, they often struggle to grasp the semantic meaning of these interactions. Conversely, large language models (LLMs) offer powerful contextual reasoning but are computationally expensive for frame-by-frame analysis and lack fine-grained spatial localization. We introduce HyCoVAD, Hybrid Complex Video Anomaly Detection, a hybrid SSL-LLM model that combines a multi-task SSL temporal analyzer with LLM validator. The SSL module is built upon an nnFormer backbone which is a transformer-based model for image segmentation. It is trained with multiple proxy tasks, learns from video frames to identify those suspected of anomaly. The selected frames are then forwarded to the LLM, which enriches the analysis with semantic context by applying structured, rule-based reasoning to validate the presence of anomalies. Experiments on the challenging ComplexVAD dataset show that HyCoVAD achieves a 72.5% frame-level AUC, outperforming existing baselines by 12.5% while reducing LLM computation. We release our interaction anomaly taxonomy, adaptive thresholding protocol, and code to facilitate future research in complex VAD scenarios.

Method: Proposes JanusVLN with dual implicit neural memory: 1) Extends MLLM to incorporate 3D prior knowledge from spatial-geometric encoder, enhancing spatial reasoning from RGB input; 2) Constructs dual implicit memory using historical key-value caches from both encoders; 3) Retains only KVs of tokens in initial and sliding window to avoid redundant computation and enable efficient incremental updates.

Result: Outperforms over 20 recent methods to achieve SOTA performance. Success rate improves by 10.5-35.5 compared to methods using multiple data types as input, and by 3.6-10.8 compared to methods using more RGB training data.

Conclusion: The dual implicit neural memory serves as a novel paradigm that explores promising new directions for future VLN research, demonstrating that compact neural representations can effectively capture both spatial and semantic information for efficient navigation.

Abstract: Vision-and-Language Navigation requires an embodied agent to navigate through unseen environments, guided by natural language instructions and a continuous video stream. Recent advances in VLN have been driven by the powerful semantic understanding of Multimodal Large Language Models. However, these methods typically rely on explicit semantic memory, such as building textual cognitive maps or storing historical visual frames. This type of method suffers from spatial information loss, computational redundancy, and memory bloat, which impede efficient navigation. Inspired by the implicit scene representation in human navigation, analogous to the left brain’s semantic understanding and the right brain’s spatial cognition, we propose JanusVLN, a novel VLN framework featuring a dual implicit neural memory that models spatial-geometric and visual-semantic memory as separate, compact, and fixed-size neural representations. This framework first extends the MLLM to incorporate 3D prior knowledge from the spatial-geometric encoder, thereby enhancing the spatial reasoning capabilities of models based solely on RGB input. Then, the historical key-value caches from the spatial-geometric and visual-semantic encoders are constructed into a dual implicit memory. By retaining only the KVs of tokens in the initial and sliding window, redundant computation is avoided, enabling efficient incremental updates. Extensive experiments demonstrate that JanusVLN outperforms over 20 recent methods to achieve SOTA performance. For example, the success rate improves by 10.5-35.5 compared to methods using multiple data types as input and by 3.6-10.8 compared to methods using more RGB training data. This indicates that the proposed dual implicit neural memory, as a novel paradigm, explores promising new directions for future VLN research. Ours project page: https://miv-xjtu.github.io/JanusVLN.github.io/.

Method: Uses temporal dynamics through SNN leakage factors for diverse pseudo-labeling in a co-training framework. Generates reliable pseudo-labels from weakly-augmented unlabeled samples to train on strongly-augmented ones, mitigating confirmation bias.

Result: Outperforms existing SSL methods adapted to SNN backbones across various standard benchmarks.

Conclusion: SpikeMatch successfully demonstrates the effectiveness of leveraging SNN temporal dynamics for semi-supervised learning, addressing the gap in SSL methods for spiking neural networks.

Abstract: Spiking neural networks (SNNs) have recently been attracting significant attention for their biological plausibility and energy efficiency, but semi-supervised learning (SSL) methods for SNN-based models remain underexplored compared to those for artificial neural networks (ANNs). In this paper, we introduce SpikeMatch, the first SSL framework for SNNs that leverages the temporal dynamics through the leakage factor of SNNs for diverse pseudo-labeling within a co-training framework. By utilizing agreement among multiple predictions from a single SNN, SpikeMatch generates reliable pseudo-labels from weakly-augmented unlabeled samples to train on strongly-augmented ones, effectively mitigating confirmation bias by capturing discriminative features with limited labels. Experiments show that SpikeMatch outperforms existing SSL methods adapted to SNN backbones across various standard benchmarks.

Method: Leverages LLMs to recursively generate discriminative textual descriptors, matches them to different semantic hierarchy levels, and adaptively routes them based on task requirements to enable precise discrimination while reducing catastrophic forgetting.

Result: Extensive experiments on multiple benchmarks demonstrate consistent state-of-the-art performance.

Conclusion: HERMAN effectively addresses hierarchical concept learning in CIL by augmenting semantic space with explicit hierarchical cues and leveraging multi-layer representations, achieving superior performance.

Abstract: Class-Incremental Learning (CIL) aims to endow models with the ability to continuously adapt to evolving data streams. Recent advances in pre-trained vision-language models (e.g., CLIP) provide a powerful foundation for this task. However, existing approaches often rely on simplistic templates, such as “a photo of a [CLASS]”, which overlook the hierarchical nature of visual concepts. For example, recognizing “cat” versus “car” depends on coarse-grained cues, while distinguishing “cat” from “lion” requires fine-grained details. Similarly, the current feature mapping in CLIP relies solely on the representation from the last layer, neglecting the hierarchical information contained in earlier layers. In this work, we introduce HiErarchical Representation MAtchiNg (HERMAN) for CLIP-based CIL. Our approach leverages LLMs to recursively generate discriminative textual descriptors, thereby augmenting the semantic space with explicit hierarchical cues. These descriptors are matched to different levels of the semantic hierarchy and adaptively routed based on task-specific requirements, enabling precise discrimination while alleviating catastrophic forgetting in incremental tasks. Extensive experiments on multiple benchmarks demonstrate that our method consistently achieves state-of-the-art performance.

Method: Uses causal frame-level AR design with KV-recache mechanism for prompt switching, streaming long tuning for long video training, and short window attention with frame sink for long-range consistency.

Result: Achieves 20.7 FPS on single H100 GPU, supports up to 240-second videos, strong VBench performance on both short and long videos, and INT8-quantized inference with minimal quality loss. Fine-tunes 1.3B model to minute-long generation in 32 GPU-days.

Conclusion: LongLive provides an efficient, high-quality solution for real-time interactive long video generation with strong performance metrics and practical deployment capabilities.

Abstract: We present LongLive, a frame-level autoregressive (AR) framework for real-time and interactive long video generation. Long video generation presents challenges in both efficiency and quality. Diffusion and Diffusion-Forcing models can produce high-quality videos but suffer from low efficiency due to bidirectional attention. Causal attention AR models support KV caching for faster inference, but often degrade in quality on long videos due to memory challenges during long-video training. In addition, beyond static prompt-based generation, interactive capabilities, such as streaming prompt inputs, are critical for dynamic content creation, enabling users to guide narratives in real time. This interactive requirement significantly increases complexity, especially in ensuring visual consistency and semantic coherence during prompt transitions. To address these challenges, LongLive adopts a causal, frame-level AR design that integrates a KV-recache mechanism that refreshes cached states with new prompts for smooth, adherent switches; streaming long tuning to enable long video training and to align training and inference (train-long-test-long); and short window attention paired with a frame-level attention sink, shorten as frame sink, preserving long-range consistency while enabling faster generation. With these key designs, LongLive fine-tunes a 1.3B-parameter short-clip model to minute-long generation in just 32 GPU-days. At inference, LongLive sustains 20.7 FPS on a single NVIDIA H100, achieves strong performance on VBench in both short and long videos. LongLive supports up to 240-second videos on a single H100 GPU. LongLive further supports INT8-quantized inference with only marginal quality loss.

Method: Recycles rollouts and correctness data to simultaneously train the model as a generative reward model using mixed objectives (pointwise reward, pairwise comparison, reflection-conditioned evaluation), creating a co-evolving feedback loop between policy and reward.

Result: Significant performance gains: SPARK-VL-7B achieved 9.7% average gain on reasoning benchmarks, 12.1% on reward benchmarks, and 1.5% on general benchmarks over baselines, demonstrating robustness and broad generalization.

Conclusion: SPARK provides an efficient, unified framework that eliminates need for separate reward models and human preference data while achieving substantial performance improvements across multiple benchmarks and model types.

Abstract: Recent Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) increasingly use Reinforcement Learning (RL) for post-pretraining, such as RL with Verifiable Rewards (RLVR) for objective tasks and RL from Human Feedback (RLHF) for subjective tasks. However, RLHF incurs high costs and potential reward-policy mismatch due to reliance on human preferences, while RLVR still wastes supervision by discarding rollouts and correctness signals after each update. To address these challenges, we introduce the Synergistic Policy And Reward Co-Evolving Framework (SPARK), an efficient, on-policy, and stable method that builds on RLVR. Instead of discarding rollouts and correctness data, SPARK recycles this valuable information to simultaneously train the model itself as a generative reward model. This auxiliary training uses a mix of objectives, such as pointwise reward score, pairwise comparison, and evaluation conditioned on further-reflection responses, to teach the model to evaluate and improve its own responses. Our process eliminates the need for a separate reward model and costly human preference data. SPARK creates a positive co-evolving feedback loop: improved reward accuracy yields better policy gradients, which in turn produce higher-quality rollouts that further refine the reward model. Our unified framework supports test-time scaling via self-reflection without external reward models and their associated costs. We show that SPARK achieves significant performance gains on multiple LLM and LVLM models and multiple reasoning, reward models, and general benchmarks. For example, SPARK-VL-7B achieves an average 9.7% gain on 7 reasoning benchmarks, 12.1% on 2 reward benchmarks, and 1.5% on 8 general benchmarks over the baselines, demonstrating robustness and broad generalization.

Method: Proposed CCNeXt architecture uses a modern CNN feature extractor with a novel windowed epipolar cross-attention module in the encoder, along with a comprehensive redesign of the depth estimation decoder.

Result: CCNeXt achieves competitive metrics on KITTI Eigen Split test data while being 10.18× faster than current best models, and achieves state-of-the-art results on KITTI Eigen Split Improved Ground Truth and Driving Stereo datasets.

Conclusion: CCNeXt provides an efficient and effective self-supervised solution for depth estimation that balances computational cost with performance, outperforming both CNN and ViT approaches.

Abstract: Depth Estimation plays a crucial role in recent applications in robotics, autonomous vehicles, and augmented reality. These scenarios commonly operate under constraints imposed by computational power. Stereo image pairs offer an effective solution for depth estimation since it only needs to estimate the disparity of pixels in image pairs to determine the depth in a known rectified system. Due to the difficulty in acquiring reliable ground-truth depth data across diverse scenarios, self-supervised techniques emerge as a solution, particularly when large unlabeled datasets are available. We propose a novel self-supervised convolutional approach that outperforms existing state-of-the-art Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) while balancing computational cost. The proposed CCNeXt architecture employs a modern CNN feature extractor with a novel windowed epipolar cross-attention module in the encoder, complemented by a comprehensive redesign of the depth estimation decoder. Our experiments demonstrate that CCNeXt achieves competitive metrics on the KITTI Eigen Split test data while being 10.18$\times$ faster than the current best model and achieves state-of-the-art results in all metrics in the KITTI Eigen Split Improved Ground Truth and Driving Stereo datasets when compared to recently proposed techniques. To ensure complete reproducibility, our project is accessible at \href{https://github.com/alelopes/CCNext}{\texttt{https://github.com/alelopes/CCNext}}.

Method: UML-CoT uses UML class diagrams to capture compositional object semantics and activity diagrams to model procedural control flow. The approach employs a three-stage training pipeline combining supervised fine-tuning with Group Relative Policy Optimization (GRPO), including reward learning from answer-only data.

Result: Evaluated on MRoom-30k benchmark for cluttered room-cleaning scenarios, UML-CoT outperforms unstructured CoTs in interpretability, planning coherence, and execution success.

Conclusion: UML provides a more expressive and actionable structured reasoning formalism compared to existing approaches, demonstrating superior performance in embodied reasoning tasks.

Abstract: Chain-of-Thought (CoT) prompting improves reasoning in large language models (LLMs), but its reliance on unstructured text limits interpretability and executability in embodied tasks. Prior work has explored structured CoTs using scene or logic graphs, yet these remain fundamentally limited: they model only low-order relations, lack constructs like inheritance or behavioral abstraction, and provide no standardized semantics for sequential or conditional planning. We propose UML-CoT, a structured reasoning and planning framework that leverages Unified Modeling Language (UML) to generate symbolic CoTs and executable action plans. UML class diagrams capture compositional object semantics, while activity diagrams model procedural control flow. Our three-stage training pipeline combines supervised fine-tuning with Group Relative Policy Optimization (GRPO), including reward learning from answer-only data. We evaluate UML-CoT on MRoom-30k, a new benchmark of cluttered room-cleaning scenarios. UML-CoT outperforms unstructured CoTs in interpretability, planning coherence, and execution success, highlighting UML as a more expressive and actionable structured reasoning formalism.

Method: Uses IP-Adapter for image-to-image translation with three innovations: extended classifier-free guidance for independent positive/negative conditioning, class similarity-based sampling for contrastive examples, and a simple pipeline requiring no fine-tuning or external captioning/filtering.

Result: Achieves state-of-the-art or comparable performance across ten benchmark datasets, particularly effective for fine-grained classification tasks.

Conclusion: DIPSY demonstrates the effectiveness of dual image prompting with positive-negative guidance for generating class-discriminative features without requiring generative model adaptation or external tools.

Abstract: Few-shot image classification remains challenging due to the limited availability of labeled examples. Recent approaches have explored generating synthetic training data using text-to-image diffusion models, but often require extensive model fine-tuning or external information sources. We present a novel training-free approach, called DIPSY, that leverages IP-Adapter for image-to-image translation to generate highly discriminative synthetic images using only the available few-shot examples. DIPSY introduces three key innovations: (1) an extended classifier-free guidance scheme that enables independent control over positive and negative image conditioning; (2) a class similarity-based sampling strategy that identifies effective contrastive examples; and (3) a simple yet effective pipeline that requires no model fine-tuning or external captioning and filtering. Experiments across ten benchmark datasets demonstrate that our approach achieves state-of-the-art or comparable performance, while eliminating the need for generative model adaptation or reliance on external tools for caption generation and image filtering. Our results highlight the effectiveness of leveraging dual image prompting with positive-negative guidance for generating class-discriminative features, particularly for fine-grained classification tasks.

Method: Reinterpret VAR with Markovian attention masks as Scalable Visual Refinement with Discrete Diffusion (SRDD), importing diffusion techniques like iterative refinement into the VAR framework to reduce architectural inefficiencies.

Result: The diffusion-based perspective of VAR leads to faster convergence, lower inference cost, improved zero-shot reconstruction, and consistent gains in efficiency and generation across multiple datasets.

Conclusion: Establishing the mathematical equivalence between VAR and discrete diffusion provides a principled bridge that enables combining the strengths of both approaches, yielding more efficient and effective visual generation models.

Abstract: Autoregressive (AR) transformers have emerged as a powerful paradigm for visual generation, largely due to their scalability, computational efficiency and unified architecture with language and vision. Among them, next scale prediction Visual Autoregressive Generation (VAR) has recently demonstrated remarkable performance, even surpassing diffusion-based models. In this work, we revisit VAR and uncover a theoretical insight: when equipped with a Markovian attention mask, VAR is mathematically equivalent to a discrete diffusion. We term this reinterpretation as Scalable Visual Refinement with Discrete Diffusion (SRDD), establishing a principled bridge between AR transformers and diffusion models. Leveraging this new perspective, we show how one can directly import the advantages of diffusion such as iterative refinement and reduce architectural inefficiencies into VAR, yielding faster convergence, lower inference cost, and improved zero-shot reconstruction. Across multiple datasets, we show that the diffusion based perspective of VAR leads to consistent gains in efficiency and generation.

Method: Extracts features and attention scores from diffusion transformers, filters stop words as attention magnets, handles global attention sinks, and uses attention redistribution with appended stop words to create sharper grounding maps.

Result: Outperforms prior methods across zero-shot referring image and video segmentation benchmarks without fine-tuning or additional components.

Conclusion: RefAM framework demonstrates that diffusion transformer features can be effectively leveraged for vision-language grounding tasks, establishing new state-of-the-art performance in training-free settings.

Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models, often at the cost of additional training and architectural modifications. Meanwhile, large-scale generative diffusion models encode rich semantic information, making them attractive as general-purpose feature extractors. In this work, we introduce a new method that directly exploits features, attention scores, from diffusion transformers for downstream tasks, requiring neither architectural modifications nor additional training. To systematically evaluate these features, we extend benchmarks with vision-language grounding tasks spanning both images and videos. Our key insight is that stop words act as attention magnets: they accumulate surplus attention and can be filtered to reduce noise. Moreover, we identify global attention sinks (GAS) emerging in deeper layers and show that they can be safely suppressed or redirected onto auxiliary tokens, leading to sharper and more accurate grounding maps. We further propose an attention redistribution strategy, where appended stop words partition background activations into smaller clusters, yielding sharper and more localized heatmaps. Building on these findings, we develop RefAM, a simple training-free grounding framework that combines cross-attention maps, GAS handling, and redistribution. Across zero-shot referring image and video segmentation benchmarks, our approach consistently outperforms prior methods, establishing a new state of the art without fine-tuning or additional components.

Method: Uses hypercomplex algebra in parameterized hypercomplex neural networks (PHNNs) to intrinsically capture intra- and inter-view relations, proposing specific architectures for 2-view (PHResNets) and 4-view (PHYBOnet, PHYSEnet) exams.

Result: The approach consistently outperforms state-of-the-art multi-view models and generalizes across radiographic modalities and tasks including chest X-ray disease classification and brain tumor segmentation.

Conclusion: Multi-view hypercomplex learning provides an effective solution for multi-view medical image analysis, addressing key limitations of existing fusion methods while achieving superior performance and generalization.

Abstract: Radiologists interpret mammography exams by jointly analyzing all four views, as correlations among them are crucial for accurate diagnosis. Recent methods employ dedicated fusion blocks to capture such dependencies, but these are often hindered by view dominance, training instability, and computational overhead. To address these challenges, we introduce multi-view hypercomplex learning, a novel learning paradigm for multi-view breast cancer classification based on parameterized hypercomplex neural networks (PHNNs). Thanks to hypercomplex algebra, our models intrinsically capture both intra- and inter-view relations. We propose PHResNets for two-view exams and two complementary four-view architectures: PHYBOnet, optimized for efficiency, and PHYSEnet, optimized for accuracy. Extensive experiments demonstrate that our approach consistently outperforms state-of-the-art multi-view models, while also generalizing across radiographic modalities and tasks such as disease classification from chest X-rays and multimodal brain tumor segmentation. Full code and pretrained models are available at https://github.com/ispamm/PHBreast.

Method: Models fine-tuning as a gradual distribution shift from pretrained to fine-tuned data, uses extrapolation between pre- and post-fine-tuning models to guide generation toward fine-tuning data distribution, and applies clustering to extract most probable images.

Result: Successfully extracted about 20% of fine-tuning data from models trained on WikiArt, DreamBooth, and real-world online checkpoints.

Conclusion: Fine-tuned diffusion models can leak their training data, posing privacy and copyright risks, and FineXtract provides an effective method for detecting such data extraction.

Abstract: Diffusion Models (DMs) have become powerful image generation tools, especially for few-shot fine-tuning where a pretrained DM is fine-tuned on a small image set to capture specific styles or objects. Many people upload these personalized checkpoints online, fostering communities such as Civitai and HuggingFace. However, model owners may overlook the data leakage risks when releasing fine-tuned checkpoints. Moreover, concerns regarding copyright violations arise when unauthorized data is used during fine-tuning. In this paper, we ask: “Can training data be extracted from these fine-tuned DMs shared online?” A successful extraction would present not only data leakage threats but also offer tangible evidence of copyright infringement. To answer this, we propose FineXtract, a framework for extracting fine-tuning data. Our method approximates fine-tuning as a gradual shift in the model’s learned distribution – from the original pretrained DM toward the fine-tuning data. By extrapolating the models before and after fine-tuning, we guide the generation toward high-probability regions within the fine-tuned data distribution. We then apply a clustering algorithm to extract the most probable images from those generated using this extrapolated guidance. Experiments on DMs fine-tuned with datasets including WikiArt, DreamBooth, and real-world checkpoints posted online validate the effectiveness of our method, extracting about 20% of fine-tuning data in most cases. The code is available https://github.com/Nicholas0228/FineXtract.

Method: PPL uses hierarchical memory to store compositional parts of prototypical poses, distills a general pose prior, and refines poses through iterative inference and template transformation.

Result: PPL outperforms baselines on human and animal pose datasets, shows effectiveness on occluded images, and refines poses by regressing to prototypical poses in memory.

Conclusion: PPL successfully learns meaningful pose priors without supervision, improving pose estimation accuracy and handling occlusions through iterative refinement.

Abstract: A prior represents a set of beliefs or assumptions about a system, aiding inference and decision-making. In this paper, we introduce the challenge of unsupervised categorical prior learning in pose estimation, where AI models learn a general pose prior for an object category from images in a self-supervised manner. Although priors are effective in estimating pose, acquiring them can be difficult. We propose a novel method, named Pose Prior Learner (PPL), to learn a general pose prior for any object category. PPL uses a hierarchical memory to store compositional parts of prototypical poses, from which we distill a general pose prior. This prior improves pose estimation accuracy through template transformation and image reconstruction. PPL learns meaningful pose priors without any additional human annotations or interventions, outperforming competitive baselines on both human and animal pose estimation datasets. Notably, our experimental results reveal the effectiveness of PPL using learned prototypical poses for pose estimation on occluded images. Through iterative inference, PPL leverages the pose prior to refine estimated poses, regressing them to any prototypical poses stored in memory. Our code, model, and data will be publicly available.

Method: DisCL uses image guidance in diffusion models to create a spectrum of interpolations between synthetic and real images. It adjusts image guidance levels during training stages, focusing on hard samples and assessing optimal guidance levels. The curriculum starts with lower-guidance high-quality images to learn prototypical features, then progresses to higher-guidance images.

Result: On iWildCam dataset: 2.7% gain in OOD macro-accuracy and 2.1% gain in ID macro-accuracy. On ImageNet-LT: tail-class accuracy improved from 4.4% to 23.64%, with 4.02% improvement in all-class accuracy.

Conclusion: DisCL effectively addresses data scarcity and quality issues by creating a curriculum of synthetic data with controlled proximity to original data, significantly improving model performance on challenging tasks like long-tail classification and learning from low-quality data.

Abstract: Low-quality or scarce data has posed significant challenges for training deep neural networks in practice. While classical data augmentation cannot contribute very different new data, diffusion models opens up a new door to build self-evolving AI by generating high-quality and diverse synthetic data through text-guided prompts. However, text-only guidance cannot control synthetic images’ proximity to the original images, resulting in out-of-distribution data detrimental to the model performance. To overcome the limitation, we study image guidance to achieve a spectrum of interpolations between synthetic and real images. With stronger image guidance, the generated images are similar to the training data but hard to learn. While with weaker image guidance, the synthetic images will be easier for model but contribute to a larger distribution gap with the original data. The generated full spectrum of data enables us to build a novel “Diffusion Curriculum (DisCL)”. DisCL adjusts the image guidance level of image synthesis for each training stage: It identifies and focuses on hard samples for the model and assesses the most effective guidance level of synthetic images to improve hard data learning. We apply DisCL to two challenging tasks: long-tail (LT) classification and learning from low-quality data. It focuses on lower-guidance images of high-quality to learn prototypical features as a warm-up of learning higher-guidance images that might be weak on diversity or quality. Extensive experiments showcase a gain of 2.7% and 2.1% in OOD and ID macro-accuracy when applying DisCL to iWildCam dataset. On ImageNet-LT, DisCL improves the base model’s tail-class accuracy from 4.4% to 23.64% and leads to a 4.02% improvement in all-class accuracy.

Method: Proposed ImageNet-RIB benchmark with related OOD tasks, fine-tuning on one task and testing on others. Evaluated various pretrained models including those from LAION-2B.

Result: Fine-tuning reduces robustness across all pretrained models. Models from largest datasets (e.g., LAION-2B) show larger robustness losses and lower absolute robustness after fine-tuning on small datasets.

Conclusion: Starting with the strongest foundation model is not necessarily best for specialist tasks due to significant robustness degradation during fine-tuning.

Abstract: Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of maintaining the general performance of the model while allowing it to gain new skills. A valuable goal for all such models is robustness: the ability to perform well on out-of-distribution (OOD) tasks. We assess whether fine-tuning preserves the overall robustness of the pretrained model, and observed that models pretrained on large datasets exhibited strong catastrophic forgetting and loss of OOD generalization. To systematically assess robustness preservation in fine-tuned models, we propose the Robustness Inheritance Benchmark (ImageNet-RIB). The benchmark, which can be applied to any pretrained model, consists of a set of related but distinct OOD (downstream) tasks and involves fine-tuning on one of the OOD tasks in the set then testing on the rest. We find that though continual learning methods help, fine-tuning reduces robustness across pretrained models. Surprisingly, models pretrained on the largest and most diverse datasets (e.g., LAION-2B) exhibit both larger robustness losses and lower absolute robustness after fine-tuning on small datasets, relative to models pretrained on smaller datasets. These findings suggest that starting with the strongest foundation model is not necessarily the best approach for performance on specialist tasks. https://jd730.github.io/projects/ImageNet-RIB

Method: Proposes Context-aware Cross-Attention (CCA) for better spatial feature querying by incorporating spatial context, and Visibility-aware Long-Temporal Attention (VLTA) for temporal feature querying that considers frame visibilities to prevent feature drifting.

Result: TAPTRv3 significantly outperforms TAPTRv2 on challenging datasets and achieves state-of-the-art performance, even surpassing methods trained on large-scale extra data.

Conclusion: The proposed spatial and temporal context enhancements in TAPTRv3 effectively address long-video tracking challenges and demonstrate superior performance compared to existing methods.

Abstract: In this paper, built upon TAPTRv2, we present TAPTRv3. TAPTRv2 is a simple yet effective DETR-like point tracking framework that works fine in regular videos but tends to fail in long videos. TAPTRv3 improves TAPTRv2 by addressing its shortcomings in querying high-quality features from long videos, where the target tracking points normally undergo increasing variation over time. In TAPTRv3, we propose to utilize both spatial and temporal context to bring better feature querying along the spatial and temporal dimensions for more robust tracking in long videos. For better spatial feature querying, we identify that off-the-shelf attention mechanisms struggle with point-level tasks and present Context-aware Cross-Attention (CCA). CCA introduces spatial context into the attention mechanism to enhance the quality of attention scores when querying image features. For better temporal feature querying, we introduce Visibility-aware Long-Temporal Attention (VLTA), which conducts temporal attention over past frames while considering their corresponding visibilities. This effectively addresses the feature drifting problem in TAPTRv2 caused by its RNN-like long-term modeling. TAPTRv3 surpasses TAPTRv2 by a large margin on most of the challenging datasets and obtains state-of-the-art performance. Even when compared with methods trained on large-scale extra internal data, TAPTRv3 still demonstrates superiority.

Method: The approach uses a double-cycle controller to generate initial condition rankings, integrates a Multimodal Large Language Model (MLLM) as a condition evaluator to optimize condition ordering, and employs a parallel multi-control adapter that learns feature maps from dynamic visual conditions to modulate ControlNet.

Result: DynamicControl demonstrates superior performance over existing methods in terms of controllability, generation quality, and composability under various conditional controls, as shown through both quantitative and qualitative comparisons.

Conclusion: The proposed DynamicControl framework effectively addresses the challenges of managing multiple conditions in text-to-image synthesis, providing enhanced control and better handling of complex multi-condition scenarios through its innovative combination of double-cycle control, MLLM reasoning, and parallel multi-control adaptation.

Abstract: To enhance the controllability of text-to-image diffusion models, current ControlNet-like models have explored various control signals to dictate image attributes. However, existing methods either handle conditions inefficiently or use a fixed number of conditions, which does not fully address the complexity of multiple conditions and their potential conflicts. This underscores the need for innovative approaches to manage multiple conditions effectively for more reliable and detailed image synthesis. To address this issue, we propose a novel framework, DynamicControl, which supports dynamic combinations of diverse control signals, allowing adaptive selection of different numbers and types of conditions. Our approach begins with a double-cycle controller that generates an initial real score sorting for all input conditions by leveraging pre-trained conditional generation models and discriminative models. This controller evaluates the similarity between extracted conditions and input conditions, as well as the pixel-level similarity with the source image. Then, we integrate a Multimodal Large Language Model (MLLM) to build an efficient condition evaluator. This evaluator optimizes the ordering of conditions based on the double-cycle controller’s score ranking. Our method jointly optimizes MLLMs and diffusion models, utilizing MLLMs’ reasoning capabilities to facilitate multi-condition text-to-image (T2I) tasks. The final sorted conditions are fed into a parallel multi-control adapter, which learns feature maps from dynamic visual conditions and integrates them to modulate ControlNet, thereby enhancing control over generated images. Through both quantitative and qualitative comparisons, DynamicControl demonstrates its superiority over existing methods in terms of controllability, generation quality and composability under various conditional controls.

Method: Systematic exploration of performance trends in vision encoders, language models, dataset sizes, and test-time configurations, using Chain-of-Thought reasoning for enhanced performance.

Result: Achieves competitive performance across multiple benchmarks including multi-discipline reasoning, document understanding, and multilingual capabilities, becoming the first open-source MLLM to surpass 70% on MMMU benchmark with 3.7-point improvement through CoT reasoning.

Conclusion: InternVL 2.5 sets new standards for developing and applying multimodal AI systems, demonstrating strong potential for test-time scaling and rivaling leading commercial models.

Abstract: We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality. In this work, we delve into the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations. Through extensive evaluations on a wide range of benchmarks, including multi-discipline reasoning, document understanding, multi-image / video understanding, real-world comprehension, multimodal hallucination detection, visual grounding, multilingual capabilities, and pure language processing, InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet. Notably, our model is the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a 3.7-point improvement through Chain-of-Thought (CoT) reasoning and showcasing strong potential for test-time scaling. We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems. HuggingFace demo see https://huggingface.co/spaces/OpenGVLab/InternVL

Method: SG uses only the sampling score function of original diffusion/flow models at different noise levels to suppress low-quality samples. SG-prev variant reuses previous step outputs for efficiency.

Result: SG outperforms existing methods on metrics like FID and Human Preference Score with Stable Diffusion 3.5 and FLUX. SG-prev achieves strong results with 50% more efficiency. Both methods effectively eliminate human body artifacts.

Conclusion: SG provides a flexible, training-free guidance approach that significantly improves generation quality and handles human body artifacts effectively, with efficient variants available.

Abstract: Proper guidance strategies are essential to achieve high-quality generation results without retraining diffusion and flow-based text-to-image models. Existing guidance either requires specific training or strong inductive biases of diffusion model networks, which potentially limits their ability and application scope. Motivated by the observation that artifact outliers can be detected by a significant decline in the density from a noisier to a cleaner noise level, we propose Self-Guidance (SG), which can significantly improve the quality of the generated image by suppressing the generation of low-quality samples. The biggest difference from existing guidance is that SG only relies on the sampling score function of the original diffusion or flow model at different noise levels, with no need for any tricky and expensive guidance-specific training. This makes SG highly flexible to be used in a plug-and-play manner by any diffusion or flow models. We also introduce an efficient variant of SG, named SG-prev, which reuses the output from the immediately previous diffusion step to avoid additional forward passes of the diffusion network.We conduct extensive experiments on text-to-image and text-to-video generation with different architectures, including UNet and transformer models. With open-sourced diffusion models such as Stable Diffusion 3.5 and FLUX, SG exceeds existing algorithms on multiple metrics, including both FID and Human Preference Score. SG-prev also achieves strong results over both the baseline and the SG, with 50 percent more efficiency. Moreover, we find that SG and SG-prev both have a surprisingly positive effect on the generation of physiologically correct human body structures such as hands, faces, and arms, showing their ability to eliminate human body artifacts with minimal efforts. We have released our code at https://github.com/maple-research-lab/Self-Guidance.

Method: Novel diffusion-based model to produce LiDAR point clouds of dataset objects including intensity, with extensive control through conditioning information.

Result: High-quality generations demonstrated on nuScenes and KITTI-360 datasets, measured using new 3D metrics developed specifically for LiDAR objects.

Conclusion: The proposed method successfully generates realistic LiDAR object point clouds with control capabilities, advancing LiDAR scan generation for autonomous driving applications.

Abstract: The generation of LiDAR scans is a growing topic with diverse applications to autonomous driving. However, scan generation remains challenging, especially when compared to the rapid advancement of image and 3D object generation. We consider the task of LiDAR object generation, requiring models to produce 3D objects as viewed by a LiDAR scan. It focuses LiDAR scan generation on a key aspect of scenes, the objects, while also benefiting from advancements in 3D object generative methods. We introduce a novel diffusion-based model to produce LiDAR point clouds of dataset objects, including intensity, and with an extensive control of the generation via conditioning information. Our experiments on nuScenes and KITTI-360 show the quality of our generations measured with new 3D metrics developed to suit LiDAR objects. The code is available at https://github.com/valeoai/LOGen.

Method: Proposes Edit Reversibility Constraint (ERC) that applies forward and reverse edits in one training step and enforces alignment in image, text, and attention spaces, enabling training on real image-caption pairs or triplets.

Result: Approach performs better across broader range of edits with high-fidelity and precision compared to existing methods.

Conclusion: Eliminates need for pre-existing triplet datasets, reduces biases, and enables scaling of instruction-based image editing through unsupervised training.

Abstract: We propose an unsupervised instruction-based image editing approach that removes the need for ground-truth edited images during training. Existing methods rely on supervised learning with triplets of input images, ground-truth edited images, and edit instructions. These triplets are typically generated either by existing editing methods, introducing biases, or through human annotations, which are costly and limit generalization. Our approach addresses these challenges by introducing a novel editing mechanism called Edit Reversibility Constraint (ERC), which applies forward and reverse edits in one training step and enforces alignment in image, text, and attention spaces. This allows us to bypass the need for ground-truth edited images and unlock training for the first time on datasets comprising either real image-caption pairs or image-caption-instruction triplets. We empirically show that our approach performs better across a broader range of edits with high-fidelity and precision. By eliminating the need for pre-existing datasets of triplets, reducing biases associated with current methods, and proposing ERC, our work represents a significant advancement in unblocking scaling of instruction-based image editing.

Method: Conducted human behavior experiments with sequential image matching tasks, developed ICMscore metric incorporating temporal intervals, curated ICMD dataset with 5,000+ images across 10 classes, trained AI models for various downstream tasks, and fine-tuned diffusion models for memorability-controlled image editing.

Result: High-ICMscore images impair AI performance in image recognition and continual learning, while low-ICMscore images improve these tasks. Diffusion models can successfully manipulate image elements to enhance or reduce memorability.

Conclusion: The work opens new pathways for understanding intra-class memorability through fine-grained visual features and lays groundwork for real-world computer vision applications, with all code, data, and models to be publicly released.

Abstract: We introduce intra-class memorability, where certain images within the same class are more memorable than others despite shared category characteristics. To investigate what features make one object instance more memorable than others, we design and conduct human behavior experiments, where participants are shown a series of images, and they must identify when the current image matches the image presented a few steps back in the sequence. To quantify memorability, we propose the Intra-Class Memorability score (ICMscore), a novel metric that incorporates the temporal intervals between repeated image presentations into its calculation. Furthermore, we curate the Intra-Class Memorability Dataset (ICMD), comprising over 5,000 images across ten object classes with their ICMscores derived from 2,000 participants’ responses. Subsequently, we demonstrate the usefulness of ICMD by training AI models on this dataset for various downstream tasks: memorability prediction, image recognition, continual learning, and memorability-controlled image editing. Surprisingly, high-ICMscore images impair AI performance in image recognition and continual learning tasks, while low-ICMscore images improve outcomes in these tasks. Additionally, we fine-tune a state-of-the-art image diffusion model on ICMD image pairs with and without masked semantic objects. The diffusion model can successfully manipulate image elements to enhance or reduce memorability. Our contributions open new pathways in understanding intra-class memorability by scrutinizing fine-grained visual features behind the most and least memorable images and laying the groundwork for real-world applications in computer vision. We will release all code, data, and models publicly.

Method: Proposes 5 guidelines: 1) depth-wise convolution in linear attention, 2) fewer attention heads, 3) weight inheritance from pre-trained DiT, 4) loading all parameters except linear attention, 5) hybrid knowledge distillation with noise and variance supervision.

Result: LiT achieves comparable performance to DiT with only 20% training steps for 256×256 and 33% for 512×512 ImageNet generation. It also rivals Mamba and Gated Linear Attention methods, and generalizes to text-to-image generation with PixArt-Σ.

Conclusion: LiT provides a safe and efficient baseline for DiT with pure linear attention, enabling faster training while maintaining performance across both class-conditional and text-to-image generation tasks.

Abstract: In this paper, we investigate how to convert a pre-trained Diffusion Transformer (DiT) into a linear DiT, as its simplicity, parallelism, and efficiency for image generation. Through detailed exploration, we offer a suite of ready-to-use solutions, ranging from linear attention design to optimization strategies. Our core contributions include 5 practical guidelines: 1) Applying depth-wise convolution within simple linear attention is sufficient for image generation. 2) Using fewer heads in linear attention provides a free-lunch performance boost without increasing latency. 3) Inheriting weights from a fully converged, pre-trained DiT. 4) Loading all parameters except those related to linear attention. 5) Hybrid knowledge distillation: using a pre-trained teacher DiT to help the training of the student linear DiT, supervising not only the predicted noise but also the variance of the reverse diffusion process. These guidelines lead to our proposed \underline{L}inear D\underline{i}ffusion \underline{T}ransformer (LiT), which serves as a safe and efficient alternative baseline for DiT with pure linear attention. In class-conditional 256$\times$256 and 512$\times$512 ImageNet generation, LiT can be quickly adapted from DiT using only $20%$ and $33%$ of DiT’s training steps, respectively, while achieving comparable performance. LiT also rivals methods based on Mamba or Gated Linear Attention. Moreover, the same guidelines generalize to text-to-image generation: LiT can be swiftly converted from PixArt-$\Sigma$ to generate high-quality images, maintaining comparable GenEval scores.

Method: Conceptualizes spurious features as fictitious sub-classes within original classes and eliminates them using a unique precise class removal technique that makes a single-weight modification, enabling post-hoc neutralization controllable to arbitrary degrees.

Result: Extensive experiments show that by editing just a single weight in a post-hoc manner, the method achieves highly competitive or better performance compared to state-of-the-art methods with negligible performance compromise for remaining classes.

Conclusion: The proposed post-hoc approach effectively neutralizes spurious feature impact through minimal weight modifications, offering practical utility without the training overhead of existing methods while maintaining competitive performance.

Abstract: Neural network training tends to exploit the simplest features as shortcuts to greedily minimize training loss. However, some of these features might be spuriously correlated with the target labels, leading to incorrect predictions by the model. Several methods have been proposed to address this issue. Focusing on suppressing the spurious correlations with model training, they not only incur additional training cost, but also have limited practical utility as the model misbehavior due to spurious relations is usually discovered after its deployment. It is also often overlooked that spuriousness is a subjective notion. Hence, the precise questions that must be investigated are; to what degree a feature is spurious, and how we can proportionally distract the model’s attention from it for reliable prediction. To this end, we propose a method that enables post-hoc neutralization of spurious feature impact, controllable to an arbitrary degree. We conceptualize spurious features as fictitious sub-classes within the original classes, which can be eliminated by a class removal scheme. We then propose a unique precise class removal technique that makes a single-weight modification, which entails negligible performance compromise for the remaining classes. We perform extensive experiments, demonstrating that by editing just a single weight in a post-hoc manner, our method achieves highly competitive, or better performance against the state-of-the-art methods.

Method: Proposes Calibrated Preference Optimization (CaPO) with: 1) reward calibration using expected win-rate against pretrained model samples, 2) frontier-based pair selection from Pareto frontiers for multi-preference management, and 3) regression loss fine-tuning to match calibrated reward differences.

Result: Experimental results show CaPO consistently outperforms prior methods like DPO in both single and multi-reward settings, validated on T2I benchmarks including GenEval and T2I-Compbench.

Conclusion: CaPO effectively aligns T2I diffusion models by leveraging general preferences from multiple reward models without human annotations, demonstrating superior performance over existing preference optimization approaches.

Abstract: Aligning text-to-image (T2I) diffusion models with preference optimization is valuable for human-annotated datasets, but the heavy cost of manual data collection limits scalability. Using reward models offers an alternative, however, current preference optimization methods fall short in exploiting the rich information, as they only consider pairwise preference distribution. Furthermore, they lack generalization to multi-preference scenarios and struggle to handle inconsistencies between rewards. To address this, we present Calibrated Preference Optimization (CaPO), a novel method to align T2I diffusion models by incorporating the general preference from multiple reward models without human annotated data. The core of our approach involves a reward calibration method to approximate the general preference by computing the expected win-rate against the samples generated by the pretrained models. Additionally, we propose a frontier-based pair selection method that effectively manages the multi-preference distribution by selecting pairs from Pareto frontiers. Finally, we use regression loss to fine-tune diffusion models to match the difference between calibrated rewards of a selected pair. Experimental results show that CaPO consistently outperforms prior methods, such as Direct Preference Optimization (DPO), in both single and multi-reward settings validated by evaluation on T2I benchmarks, including GenEval and T2I-Compbench.

Method: Introduces Prompt Directional Vector (PDV) - a training-free approach that captures semantic modifications from user prompts, enabling dynamic composed text embeddings, composed image embeddings via semantic transfer, and weighted fusion of text and image embeddings.

Result: PDV consistently improves retrieval performance across multiple benchmarks when integrated with state-of-the-art ZS-CIR approaches, particularly benefiting methods with accurate compositional embeddings.

Conclusion: PDV serves as an effective plug-and-play enhancement for existing ZS-CIR methods with minimal computational overhead, addressing key limitations in current approaches and improving retrieval performance.

Abstract: Zero-shot Composed Image Retrieval (ZS-CIR) enables image search using a reference image and a text prompt without requiring specialized text-image composition networks trained on large-scale paired data. However, current ZS-CIR approaches suffer from three critical limitations in their reliance on composed text embeddings: static query embedding representations, insufficient utilization of image embeddings, and suboptimal performance when fusing text and image embeddings. To address these challenges, we introduce the \textbf{Prompt Directional Vector (PDV)}, a simple yet effective training-free enhancement that captures semantic modifications induced by user prompts. PDV enables three key improvements: (1) Dynamic composed text embeddings where prompt adjustments are controllable via a scaling factor, (2) composed image embeddings through semantic transfer from text prompts to image features, and (3) weighted fusion of composed text and image embeddings that enhances retrieval by balancing visual and semantic similarity. Our approach serves as a plug-and-play enhancement for existing ZS-CIR methods with minimal computational overhead. Extensive experiments across multiple benchmarks demonstrate that PDV consistently improves retrieval performance when integrated with state-of-the-art ZS-CIR approaches, particularly for methods that generate accurate compositional embeddings. The code will be released upon publication.

Method: Developed GeoCLIP (a CLIP-based model trained on synthetic geometric diagram-caption pairs) and GeoDANO (which augments GeoCLIP with domain adaptation for unseen diagram styles).

Result: GeoCLIP outperforms existing vision encoders in recognizing geometric features. GeoDANO outperforms specialized methods and GPT-4o on MathVerse benchmark.

Conclusion: The proposed geometric vision-language model with domain-agnostic vision encoder effectively solves plane geometry problems by improving geometric feature recognition and domain adaptation.

Abstract: We introduce GeoDANO, a geometric vision-language model (VLM) with a domain-agnostic vision encoder, for solving plane geometry problems. Although VLMs have been employed for solving geometry problems, their ability to recognize geometric features remains insufficiently analyzed. To address this gap, we propose a benchmark that evaluates the recognition of visual geometric features, including primitives such as dots and lines, and relations such as orthogonality. Our preliminary study shows that vision encoders often used in general-purpose VLMs, e.g., OpenCLIP, fail to detect these features and struggle to generalize across domains. To overcome the limitation, we develop GeoCLIP, a CLIP-based model trained on synthetic geometric diagram–caption pairs. Benchmark results show that GeoCLIP outperforms existing vision encoders in recognizing geometric features. We then propose our VLM, GeoDANO, which augments GeoCLIP with a domain adaptation strategy for unseen diagram styles. GeoDANO outperforms specialized methods for plane geometry problems and GPT-4o on MathVerse. The implementation is available at https://github.com/ml-postech/GeoDANO.

Method: Three-level optimization: 1) Low-level: warp-based raster with hardware-aware optimizations to reduce gradient overhead; 2) Mid-level: dynamic spatial sorting using Morton coding for better data locality; 3) Top-level: new densification criterion based on opacity gradient variance with stable opacity control.

Result: Achieves up to 13.4x speedup over original 3DGS with comparable or superior quality, surpasses SOTA lightweight models by 1.4x, and sets new accuracy records for high-quality reconstruction while reducing training time by an order of magnitude.

Conclusion: LiteGS provides a comprehensive optimization framework that significantly accelerates 3DGS training while maintaining or improving reconstruction quality, making it more practical for real-world applications.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as promising alternative in 3D representation. However, it still suffers from high training cost. This paper introduces LiteGS, a high performance framework that systematically optimizes the 3DGS training pipeline from multiple aspects. At the low-level computation layer, we design a warp-based raster'' associated with two hardware-aware optimizations to significantly reduce gradient reduction overhead. At the mid-level data management layer, we introduce dynamic spatial sorting based on Morton coding to enable a performant Cluster-Cull-Compact’’ pipeline and improve data locality, therefore reducing cache misses. At the top-level algorithm layer, we establish a new robust densification criterion based on the variance of the opacity gradient, paired with a more stable opacity control mechanism, to achieve more precise parameter growth. Experimental results demonstrate that LiteGS accelerates the original 3DGS training by up to 13.4x with comparable or superior quality and surpasses the current SOTA in lightweight models by up to 1.4x speedup. For high-quality reconstruction tasks, LiteGS sets a new accuracy record and decreases the training time by an order of magnitude.

Method: SPEED searches for a null space where parameter updates don’t affect non-target concepts. It uses three strategies: Influence-based Prior Filtering to select affected non-target concepts, Directed Prior Augmentation to enrich the retain set with variations, and Invariant Equality Constraints to preserve generation invariants.

Result: SPEED consistently outperforms existing methods in non-target preservation while achieving efficient and high-fidelity concept erasure. It successfully erases 100 concepts within only 5 seconds across multiple concept erasure tasks.

Conclusion: SPEED provides an effective solution for scalable concept erasure in text-to-image diffusion models, addressing the trade-off between erasure precision and non-target concept preservation through null space optimization and complementary strategies.

Abstract: Erasing concepts from large-scale text-to-image (T2I) diffusion models has become increasingly crucial due to the growing concerns over copyright infringement, offensive content, and privacy violations. In scalable applications, fine-tuning-based methods are time-consuming to precisely erase multiple target concepts, while real-time editing-based methods often degrade the generation quality of non-target concepts due to conflicting optimization objectives. To address this dilemma, we introduce SPEED, an efficient concept erasure approach that directly edits model parameters. SPEED searches for a null space, a model editing space where parameter updates do not affect non-target concepts, to achieve scalable and precise erasure. To facilitate accurate null space optimization, we incorporate three complementary strategies: Influence-based Prior Filtering (IPF) to selectively retain the most affected non-target concepts, Directed Prior Augmentation (DPA) to enrich the filtered retain set with semantically consistent variations, and Invariant Equality Constraints (IEC) to preserve key invariants during the T2I generation process. Extensive evaluations across multiple concept erasure tasks demonstrate that SPEED consistently outperforms existing methods in non-target preservation while achieving efficient and high-fidelity concept erasure, successfully erasing 100 concepts within only 5 seconds. Our code and models are available at: https://github.com/Ouxiang-Li/SPEED.

Method: Two key approaches: (i) point-level querying framework with tracking on segmentation masks to establish semantically consistent ground-truth for distilling language Gaussians; (ii) GT-anchored querying that first retrieves distilled ground-truth and then uses it to query individual Gaussians.

Result: Outperforms state-of-the-art on three benchmark datasets with mIoU improvements of +4.14 on LERF, +20.42 on 3D-OVS, and +1.7 on Replica datasets.

Conclusion: The framework represents a promising step toward open-vocabulary understanding in real-world robotic systems, validated by significant performance improvements across multiple datasets.

Abstract: Open-vocabulary 3D scene understanding is crucial for robotics applications, such as natural language-driven manipulation, human-robot interaction, and autonomous navigation. Existing methods for querying 3D Gaussian Splatting often struggle with inconsistent 2D mask supervision and lack a robust 3D point-level retrieval mechanism. In this work, (i) we present a novel point-level querying framework that performs tracking on segmentation masks to establish a semantically consistent ground-truth for distilling the language Gaussians; (ii) we introduce a GT-anchored querying approach that first retrieves the distilled ground-truth and subsequently uses the ground-truth to query the individual Gaussians. Extensive experiments on three benchmark datasets demonstrate that the proposed method outperforms state-of-the-art performance. Our method achieves an mIoU improvement of +4.14, +20.42, and +1.7 on the LERF, 3D-OVS, and Replica datasets. These results validate our framework as a promising step toward open-vocabulary understanding in real-world robotic systems.

Method: Uses a layered editing strategy that separates text from background, performs geometric transformations modularly, and employs a depth-aware module to align appearance and perspective between transformed text and reconstructed background.

Result: Achieves superior performance on the AnyWord-3M benchmark, especially in large-scale and complex transformation scenarios, with improved visual quality and spatial consistency.

Conclusion: DanceText provides an effective training-free solution for multilingual text editing in images that handles complex geometric transformations while maintaining photorealistic results and layout consistency.

Abstract: We present DanceText, a training-free framework for multilingual text editing in images, designed to support complex geometric transformations and achieve seamless foreground-background integration. While diffusion-based generative models have shown promise in text-guided image synthesis, they often lack controllability and fail to preserve layout consistency under non-trivial manipulations such as rotation, translation, scaling, and warping. To address these limitations, DanceText introduces a layered editing strategy that separates text from the background, allowing geometric transformations to be performed in a modular and controllable manner. A depth-aware module is further proposed to align appearance and perspective between the transformed text and the reconstructed background, enhancing photorealism and spatial consistency. Importantly, DanceText adopts a fully training-free design by integrating pretrained modules, allowing flexible deployment without task-specific fine-tuning. Extensive experiments on the AnyWord-3M benchmark demonstrate that our method achieves superior performance in visual quality, especially under large-scale and complex transformation scenarios. Code is avaible at https://github.com/YuZhenyuLindy/DanceText.git.

Method: Inspired by word-to-vector models, event2vec draws analogy between words and events, creating a representation that allows neural networks to process events directly while being compatible with Transformer architectures and self-supervised learning.

Result: Event2vec achieves high accuracy on DVS Gesture, ASL-DVS, and DVS-Lip benchmarks with remarkable parameter efficiency, high throughput, and maintains performance even with extremely low event counts.

Conclusion: Event2vec introduces a paradigm shift enabling neural networks to process event streams as natural language, paving way for native integration of event cameras with large language models and multimodal models.

Abstract: Neuromorphic event cameras possess superior temporal resolution, power efficiency, and dynamic range compared to traditional cameras. However, their asynchronous and sparse data format poses a significant challenge for conventional deep learning methods. Existing solutions to this incompatibility often sacrifice temporal resolution, require extensive pre-processing, and do not fully leverage GPU acceleration. Inspired by word-to-vector models, we draw an analogy between words and events to introduce event2vec, a novel representation that allows neural networks to process events directly. This approach is fully compatible with the parallel processing and self-supervised learning capabilities of Transformer architectures. We demonstrate the effectiveness of event2vec on the DVS Gesture, ASL-DVS, and DVS-Lip benchmarks. A comprehensive ablation study further analyzes our method’s features and contrasts them with existing representations. The experimental results show that event2vec is remarkably parameter-efficient, has high throughput, and can achieve high accuracy even with an extremely low number of events. Beyond its performance, the most significant contribution of event2vec is a new paradigm that enables neural networks to process event streams as if they were natural language. This paradigm shift paves the way for the native integration of event cameras with large language models and multimodal models. Code, model, and training logs are provided in https://github.com/Intelligent-Computing-Lab-Panda/event2vec.

Method: Introduces a novel spectral encoder with self-attention-cross-attention mechanism to distill salient spectral information, combined with spatio-spectral pre-training using a feature-based self-supervision strategy tailored for camera-agnostic learning.

Result: Large-scale experiments across medical imaging, autonomous driving, and satellite imaging domains demonstrate superior robustness to spectral heterogeneity, outperforming on datasets with both simulated and real-world cross-camera spectral variations.

Conclusion: CARL’s scalability and versatility position it as a backbone for future spectral foundation models, effectively addressing the bottleneck of spectral camera variability in AI-driven methodologies.

Abstract: Spectral imaging offers promising applications across diverse domains, including medicine and urban scene understanding, and is already established as a critical modality in remote sensing. However, variability in channel dimensionality and captured wavelengths among spectral cameras impede the development of AI-driven methodologies, leading to camera-specific models with limited generalizability and inadequate cross-camera applicability. To address this bottleneck, we introduce CARL, a model for Camera-Agnostic Representation Learning across RGB, multispectral, and hyperspectral imaging modalities. To enable the conversion of a spectral image with any channel dimensionality to a camera-agnostic representation, we introduce a novel spectral encoder, featuring a self-attention-cross-attention mechanism, to distill salient spectral information into learned spectral representations. Spatio-spectral pre-training is achieved with a novel feature-based self-supervision strategy tailored to CARL. Large-scale experiments across the domains of medical imaging, autonomous driving, and satellite imaging demonstrate our model’s unique robustness to spectral heterogeneity, outperforming on datasets with simulated and real-world cross-camera spectral variations. The scalability and versatility of the proposed approach position our model as a backbone for future spectral foundation models.

Method: Survey and evaluation of three key lightweight strategies: Efficient Component Design, Dynamic Network, and Knowledge Distillation, analyzed on ImageNet-1K benchmark.

Result: Comprehensive analysis of trade-offs among precision, parameters, throughput, and other metrics to highlight advantages, disadvantages, and flexibility of different approaches.

Conclusion: Proposes future research directions and potential challenges in lightweight vision transformers to inspire further exploration and provide practical guidance for the community.

Abstract: The Transformer architecture has achieved significant success in natural language processing, motivating its adaptation to computer vision tasks. Unlike convolutional neural networks, vision transformers inherently capture long-range dependencies and enable parallel processing, yet lack inductive biases and efficiency benefits, facing significant computational and memory challenges that limit its real-world applicability. This paper surveys various online strategies for generating lightweight vision transformers for image recognition, focusing on three key areas: Efficient Component Design, Dynamic Network, and Knowledge Distillation. We evaluate the relevant exploration for each topic on the ImageNet-1K benchmark, analyzing trade-offs among precision, parameters, throughput, and more to highlight their respective advantages, disadvantages, and flexibility. Finally, we propose future research directions and potential challenges in the lightweighting of vision transformers with the aim of inspiring further exploration and providing practical guidance for the community. Project Page: https://github.com/ajxklo/Lightweight-VIT

Method: Three-phase approach: (1) Use small image generation preference data to distill GPT-4o’s reasoning process for cold start learning of CoT format; (2) Leverage model’s prior knowledge to prepare large-scale unified multimodal preference data to elicit reasoning across vision tasks, using correct outputs for rejection sampling; (3) Use incorrect samples for Group Relative Policy Optimization (GRPO) based reinforcement fine-tuning to explore diverse reasoning paths and optimize for correct solutions.

Result: Extensive experiments across various vision reward tasks demonstrate the superiority of the proposed model over existing approaches.

Conclusion: UnifiedReward-Think successfully incorporates explicit long chains of thought reasoning into multimodal reward models, significantly improving their reliability, robustness, and accuracy across visual understanding and generation tasks.

Abstract: Recent advances in multimodal Reward Models (RMs) have shown significant promise in delivering reward signals to align vision models with human preferences. However, current RMs are generally restricted to providing direct responses or engaging in shallow reasoning processes with limited depth, often leading to inaccurate reward signals. We posit that incorporating explicit long chains of thought (CoT) into the reward reasoning process can significantly strengthen their reliability and robustness. Furthermore, we believe that once RMs internalize CoT reasoning, their direct response accuracy can also be improved through implicit reasoning capabilities. To this end, this paper proposes UnifiedReward-Think, the first unified multimodal CoT-based reward model, capable of multi-dimensional, step-by-step long-chain reasoning for both visual understanding and generation reward tasks. Specifically, we adopt an exploration-driven reinforcement fine-tuning approach to elicit and incentivize the model’s latent complex reasoning ability: (1) We first use a small amount of image generation preference data to distill the reasoning process of GPT-4o, which is then used for the model’s cold start to learn the format and structure of CoT reasoning. (2) Subsequently, by leveraging the model’s prior knowledge and generalization capabilities, we prepare large-scale unified multimodal preference data to elicit the model’s reasoning process across various vision tasks. During this phase, correct reasoning outputs are retained for rejection sampling to refine the model (3) while incorrect predicted samples are finally used for Group Relative Policy Optimization (GRPO) based reinforcement fine-tuning, enabling the model to explore diverse reasoning paths and optimize for correct and robust solutions. Extensive experiments across various vision reward tasks demonstrate the superiority of our model.

Method: Proposed OS-W2S Label Engine - an automatic annotation pipeline centered around an open-source large vision-language model, integrating image-operation-based preprocessing with BERT-based postprocessing. This pipeline handles diverse scene annotations for aerial images and expands existing datasets with rich textual annotations.

Result: MI-OAD dataset contains 163,023 images and 2 million image-caption pairs (40x larger than comparable datasets). Training on MI-OAD lifts Grounding DINO by +31.1 AP$_{50}$ and +34.7 Recall@10 with sentence-level inputs under zero-shot transfer. Pre-training with MI-OAD yields state-of-the-art performance on multiple existing benchmarks.

Conclusion: MI-OAD effectively addresses limitations in current remote sensing grounding data and enables effective language-guided open-set aerial detection. The dataset and OS-W2S annotations demonstrate high quality and effectiveness across multiple evaluation tasks.

Abstract: In recent years, language-guided open-set aerial object detection has gained significant attention due to its better alignment with real-world application needs. However, due to limited datasets, most existing language-guided methods primarily focus on vocabulary-level descriptions, which fail to meet the demands of fine-grained open-world detection. To address this limitation, we propose constructing a large-scale language-guided open-set aerial detection dataset, encompassing three levels of language guidance: from words to phrases, and ultimately to sentences. Centered around an open-source large vision-language model and integrating image-operation-based preprocessing with BERT-based postprocessing, we present the OS-W2S Label Engine, an automatic annotation pipeline capable of handling diverse scene annotations for aerial images. Using this label engine, we expand existing aerial detection datasets with rich textual annotations and construct a novel benchmark dataset, called MI-OAD, addressing the limitations of current remote sensing grounding data and enabling effective language-guided open-set aerial detection. Specifically, MI-OAD contains 163,023 images and 2 million image-caption pairs, approximately 40 times larger than comparable datasets. To demonstrate the effectiveness and quality of MI-OAD, we evaluate three representative tasks. On language-guided open-set aerial detection, training on MI-OAD lifts Grounding DINO by +31.1 AP$_{50}$ and +34.7 Recall@10 with sentence-level inputs under zero-shot transfer. Moreover, using MI-OAD for pre-training yields state-of-the-art performance on multiple existing open-vocabulary aerial detection and remote sensing visual grounding benchmarks, validating both the effectiveness of the dataset and the high quality of its OS-W2S annotations. More details are available at https://github.com/GT-Wei/MI-OAD.

Method: Created InG dataset by augmenting BEAT-2 with gesture-intention annotations using vision-language models, and developed Intentional Gesture Motion Tokenizer to inject communicative functions into motion representations for intention-aware synthesis.

Result: Achieved new state-of-the-art performance on BEAT-2 benchmark, producing gestures that are both temporally aligned and semantically meaningful.

Conclusion: The framework provides a modular foundation for expressive gesture generation in digital humans and embodied AI by grounding gestures in high-level communicative functions.

Abstract: When humans speak, gestures help convey communicative intentions, such as adding emphasis or describing concepts. However, current co-speech gesture generation methods rely solely on superficial linguistic cues (e.g. speech audio or text transcripts), neglecting to understand and leverage the communicative intention that underpins human gestures. This results in outputs that are rhythmically synchronized with speech but are semantically shallow. To address this gap, we introduce Intentional-Gesture, a novel framework that casts gesture generation as an intention-reasoning task grounded in high-level communicative functions. First, we curate the InG dataset by augmenting BEAT-2 with gesture-intention annotations (i.e., text sentences summarizing intentions), which are automatically annotated using large vision-language models. Next, we introduce the Intentional Gesture Motion Tokenizer to leverage these intention annotations. It injects high-level communicative functions (e.g., intentions) into tokenized motion representations to enable intention-aware gesture synthesis that are both temporally aligned and semantically meaningful, achieving new state-of-the-art performance on the BEAT-2 benchmark. Our framework offers a modular foundation for expressive gesture generation in digital humans and embodied AI. Project Page: https://andypinxinliu.github.io/Intentional-Gesture

Method: Introduce octic group equivariance in ViTs through octic linear layers, creating two families: fully equivariant networks and networks that break equivariance in later stages.

Result: Octic ViTs match baseline DeiT-III and DINOv2 accuracy on ImageNet-1K while providing substantial efficiency improvements - 5.33x FLOPs reduction and up to 8x memory reduction compared to ordinary linear layers.

Conclusion: Octic group equivariance enables Vision Transformers to efficiently capture geometric symmetries without sacrificing accuracy, offering a promising direction for more efficient computer vision models.

Abstract: Why are state-of-the-art Vision Transformers (ViTs) not designed to exploit natural geometric symmetries such as 90-degree rotations and reflections? In this paper, we argue that there is no fundamental reason, and what has been missing is an efficient implementation. To this end, we introduce Octic Vision Transformers (octic ViTs) which rely on octic group equivariance to capture these symmetries. In contrast to prior equivariant models that increase computational cost, our octic linear layers achieve 5.33x reductions in FLOPs and up to 8x reductions in memory compared to ordinary linear layers. In full octic ViT blocks the computational reductions approach the reductions in the linear layers with increased embedding dimension. We study two new families of ViTs, built from octic blocks, that are either fully octic equivariant or break equivariance in the last part of the network. Training octic ViTs supervised (DeiT-III) and unsupervised (DINOv2) on ImageNet-1K, we find that they match baseline accuracy while at the same time providing substantial efficiency gains.

Method: Integrates pre-trained image-to-video diffusion model with confidence-guided LLM reasoning and differentiable physics simulator. Uses iterative motion prompt refinement with LLM-derived confidence scores and simulation feedback to steer generation toward physical consistency.

Result: Outperforms state-of-the-art video generators and physics-aware baselines, improving physical property inference and motion-text alignment while maintaining visual fidelity.

Conclusion: PhyMAGIC provides a scalable, training-free solution for generating physically consistent 3D motion from single images, producing assets ready for downstream physical simulation without fine-tuning or manual supervision.

Abstract: Recent advances in 3D content generation have amplified demand for dynamic models that are both visually realistic and physically consistent. However, state-of-the-art video diffusion models frequently produce implausible results such as momentum violations and object interpenetrations. Existing physics-aware approaches often rely on task-specific fine-tuning or supervised data, which limits their scalability and applicability. To address the challenge, we present PhyMAGIC, a training-free framework that generates physically consistent motion from a single image. PhyMAGIC integrates a pre-trained image-to-video diffusion model, confidence-guided reasoning via LLMs, and a differentiable physics simulator to produce 3D assets ready for downstream physical simulation without fine-tuning or manual supervision. By iteratively refining motion prompts using LLM-derived confidence scores and leveraging simulation feedback, PhyMAGIC steers generation toward physically consistent dynamics. Comprehensive experiments demonstrate that PhyMAGIC outperforms state-of-the-art video generators and physics-aware baselines, enhancing physical property inference and motion-text alignment while maintaining visual fidelity.

Method: The paper provides a systematic foundation for exploring bias-variance tradeoff, establishing theoretical frameworks and conducting empirical validations.

Result: The research shows diffusion models can amplify inherent training data bias at one extreme, and compromise training sample privacy at the other extreme. Deeper models reveal increased risk of bias amplification.

Conclusion: The study expands the memorization-generalization understanding of generative models beyond just generalization, revealing significant bias amplification risks in diffusion models that are validated both theoretically and empirically.

Abstract: Despite the remarkable performance of generative Diffusion Models (DMs), their internal working is still not well understood, which is potentially problematic. This paper focuses on exploring the important notion of bias-variance tradeoff in diffusion models. Providing a systematic foundation for this exploration, it establishes that at one extreme, the diffusion models may amplify the inherent bias in the training data, and on the other, they may compromise the presumed privacy of the training samples. Our exploration aligns with the memorization-generalization understanding of the generative models, but it also expands further along this spectrum beyond “generalization”, revealing the risk of bias amplification in deeper models. Our claims are validated both theoretically and empirically.

Method: Integrates three innovations: Bounded-init Grid Refinement (BGR) and Auto-scaling Rotated Quantization (ARQ) for calibration data-free error reduction, plus δ-Guided Bit Switching (δ-GBS) for adaptive bit-width allocation.

Result: Achieves approximately 2x speedup over full-precision baselines while maintaining visual fidelity. First method to enable W4A4 PTQ for Video DiTs without compromising video quality.

Conclusion: DVD-Quant provides an effective data-free quantization solution for Video DiTs that balances computational efficiency with maintained performance, enabling practical deployment of state-of-the-art video generation models.

Abstract: Diffusion Transformers (DiTs) have emerged as the state-of-the-art architecture for video generation, yet their computational and memory demands hinder practical deployment. While post-training quantization (PTQ) presents a promising approach to accelerate Video DiT models, existing methods suffer from two critical limitations: (1) dependence on computation-heavy and inflexible calibration procedures, and (2) considerable performance deterioration after quantization. To address these challenges, we propose DVD-Quant, a novel Data-free quantization framework for Video DiTs. Our approach integrates three key innovations: (1) Bounded-init Grid Refinement (BGR) and (2) Auto-scaling Rotated Quantization (ARQ) for calibration data-free quantization error reduction, as well as (3) $\delta$-Guided Bit Switching ($\delta$-GBS) for adaptive bit-width allocation. Extensive experiments across multiple video generation benchmarks demonstrate that DVD-Quant achieves an approximately 2$\times$ speedup over full-precision baselines on advanced DiT models while maintaining visual fidelity. Notably, DVD-Quant is the first to enable W4A4 PTQ for Video DiTs without compromising video quality. Code and models will be available at https://github.com/lhxcs/DVD-Quant.

Method: Constructed through inductive process identifying 75 chart types, 440 variations, and 68 layout templates from real infographics, then programmatically creating synthetic charts.

Result: Enables three key applications: improved chart understanding via fine-tuning, benchmarking code generation, and example-based chart generation.

Conclusion: ChartGalaxy provides a valuable resource for enhancing multimodal reasoning and generation in LVLMs by capturing real design complexity.

Abstract: Infographic charts are a powerful medium for communicating abstract data by combining visual elements (e.g., charts, images) with textual information. However, their visual and structural richness poses challenges for large vision-language models (LVLMs), which are typically trained on plain charts. To bridge this gap, we introduce ChartGalaxy, a million-scale dataset designed to advance the understanding and generation of infographic charts. The dataset is constructed through an inductive process that identifies 75 chart types, 440 chart variations, and 68 layout templates from real infographic charts and uses them to create synthetic ones programmatically. We showcase the utility of this dataset through: 1) improving infographic chart understanding via fine-tuning, 2) benchmarking code generation for infographic charts, and 3) enabling example-based infographic chart generation. By capturing the visual and structural complexity of real design, ChartGalaxy provides a useful resource for enhancing multimodal reasoning and generation in LVLMs.

Method: Proposed a unified confidence-aware framework that analyzes error propagation in segmentation-to-biomarker pipelines, identifies model miscalibration as the main uncertainty source, and uses tunable parameters to control confidence levels.

Result: Extensive experiments show the method produces statistically sound confidence intervals with tunable confidence levels that can adapt to clinical practice requirements.

Conclusion: The framework enables more trustworthy application of predictive biomarkers in clinical workflows by providing reliable uncertainty quantification for ratio-based biomarkers.

Abstract: Ratio-based biomarkers – such as the proportion of necrotic tissue within a tumor – are widely used in clinical practice to support diagnosis, prognosis, and treatment planning. These biomarkers are typically estimated from soft segmentation outputs by computing region-wise ratios. Despite the high-stakes nature of clinical decision making, existing methods provide only point estimates, offering no measure of uncertainty. In this work, we propose a unified confidence-aware framework for estimating ratio-based biomarkers. Our uncertainty analysis stems from two observations: i) the probability ratio estimator inherently admits a statistical confidence interval regarding local randomness (bias and variance), ii) the segmentation network is not perfectly calibrated. We conduct a systematic analysis of error propagation in the segmentation-to-biomarker pipeline and identify model miscalibration as the dominant source of uncertainty. We leverage tunable parameters to control the confidence level of the derived bounds, allowing adaptation towards clinical practice. Extensive experiments show that our method produces statistically sound confidence intervals, with tunable confidence levels, enabling more trustworthy application of predictive biomarkers in clinical workflows.

Method: Mamba-Driven Topology Fusion framework with: 1) Bone Aware Module that infers bone vector direction/length in spherical coordinates for topological guidance; 2) Enhanced Mamba with forward/backward graph convolutional networks to capture local joint dependencies; 3) Spatiotemporal Refinement Module for modeling temporal and spatial relationships.

Result: Extensive experiments on Human3.6M and MPI-INF-3DHP datasets show the method greatly reduces computational cost while achieving higher accuracy compared to existing approaches. Ablation studies confirm effectiveness of each proposed module.

Conclusion: The proposed framework effectively addresses Mamba’s limitations in capturing human structural relationships by incorporating skeletal topology, achieving both computational efficiency and improved accuracy in 3D human pose estimation.

Abstract: Transformer-based methods for 3D human pose estimation face significant computational challenges due to the quadratic growth of self-attention mechanism complexity with sequence length. Recently, the Mamba model has substantially reduced computational overhead and demonstrated outstanding performance in modeling long sequences by leveraging state space model (SSM). However, the ability of SSM to process sequential data is not suitable for 3D joint sequences with topological structures, and the causal convolution structure in Mamba also lacks insight into local joint relationships. To address these issues, we propose the Mamba-Driven Topology Fusion framework in this paper. Specifically, the proposed Bone Aware Module infers the direction and length of bone vectors in the spherical coordinate system, providing effective topological guidance for the Mamba model in processing joint sequences. Furthermore, we enhance the convolutional structure within the Mamba model by integrating forward and backward graph convolutional network, enabling it to better capture local joint dependencies. Finally, we design a Spatiotemporal Refinement Module to model both temporal and spatial relationships within the sequence. Through the incorporation of skeletal topology, our approach effectively alleviates Mamba’s limitations in capturing human structural relationships. We conduct extensive experiments on the Human3.6M and MPI-INF-3DHP datasets for testing and comparison, and the results show that the proposed method greatly reduces computational cost while achieving higher accuracy. Ablation studies further demonstrate the effectiveness of each proposed module. The code and models will be released.

Method: Proposes HLIP framework with hierarchical attention mechanism that models the intrinsic hierarchy of radiology data: slice, scan, and study levels. Trained on 220K brain MRI studies (3.13M scans) and 240K head CT studies (1.44M scans).

Result: Achieves +10.5% balanced ACC on Pub-Brain-5 brain MRI benchmark; +8.3% and +1.7% macro AUC on CQ500 and RSNA head CT benchmarks; +4.3% macro AUC on Rad-ChestCT benchmark when pre-trained on CT-RATE.

Conclusion: Direct pre-training on uncurated clinical datasets with HLIP is a scalable and effective approach for language-image pre-training in 3D medical imaging, demonstrating strong generalization across benchmarks.

Abstract: The scalability of current language-image pre-training for 3D medical imaging, such as CT and MRI, is constrained by the need for radiologists to manually curate raw clinical studies. In this work, we pioneer pre-training directly on uncurated studies, which both aligns more closely with the radiologist’s workflow and provides a natural path to scalability. However, the unique structure of such data presents new challenges for existing model architectures, which were originally designed for 2D slices or single 3D scans. To address this, we introduce a novel hierarchical attention mechanism inspired by the intrinsic hierarchy of radiology data: slice, scan, and study. We denote our framework as Hierarchical attention for Language-Image Pre-training (HLIP). Trained on 220K studies with 3.13 million scans for brain MRI and 240K studies with 1.44 million scans for head CT, HLIP achieves state-of-the-art performance, e.g., +10.5% balanced ACC on the proposed publicly available brain MRI benchmark Pub-Brain-5; +8.3% and +1.7% macro AUC on head CT benchmarks CQ500 and RSNA, respectively. HLIP also exhibits strong generalizability on existing 3D medical language-image pre-training benchmarks, e.g., +4.3% macro AUC on the Rad-ChestCT benchmark when pre-trained on CT-RATE. These results demonstrate that, with HLIP, directly pre-training on uncurated clinical datasets is a scalable and effective direction for language-image pre-training in 3D medical imaging. The code is available at https://github.com/Zch0414/hlip.

Method: SHARE builds a pose-aware canonical volume representation that integrates multi-view information without explicit 3D transformations, and uses anchor-aligned Gaussian prediction to refine local geometry around coarse anchors.

Result: Extensive experiments on diverse real-world datasets show robust performance in pose-free generalizable Gaussian splatting.

Conclusion: SHARE effectively addresses pose ambiguity in Gaussian splatting through joint shape and camera rays estimation, achieving reliable performance without requiring accurate pose estimates.

Abstract: While generalizable 3D Gaussian splatting enables efficient, high-quality rendering of unseen scenes, it heavily depends on precise camera poses for accurate geometry. In real-world scenarios, obtaining accurate poses is challenging, leading to noisy pose estimates and geometric misalignments. To address this, we introduce SHARE, a pose-free, feed-forward Gaussian splatting framework that overcomes these ambiguities by joint shape and camera rays estimation. Instead of relying on explicit 3D transformations, SHARE builds a pose-aware canonical volume representation that seamlessly integrates multi-view information, reducing misalignment caused by inaccurate pose estimates. Additionally, anchor-aligned Gaussian prediction enhances scene reconstruction by refining local geometry around coarse anchors, allowing for more precise Gaussian placement. Extensive experiments on diverse real-world datasets show that our method achieves robust performance in pose-free generalizable Gaussian splatting.

Method: Introduces a frequency-domain physics prior that decomposes common rigid motions (translation, rotation, scaling) into lightweight spectral losses without modifying model architectures.

Result: Improves motion accuracy and action recognition by ~11% on average on OpenVID-1M, reduces warping error by 22-37%, and user studies show 74-83% preference for physics-enhanced videos while maintaining visual quality.

Conclusion: Simple, global spectral cues are an effective drop-in regularizer for physically plausible motion in video diffusion models.

Abstract: Current video diffusion models generate visually compelling content but often violate basic laws of physics, producing subtle artifacts like rubber-sheet deformations and inconsistent object motion. We introduce a frequency-domain physics prior that improves motion plausibility without modifying model architectures. Our method decomposes common rigid motions (translation, rotation, scaling) into lightweight spectral losses, requiring only 2.7% of frequency coefficients while preserving 97%+ of spectral energy. Applied to Open-Sora, MVDIT, and Hunyuan, our approach improves both motion accuracy and action recognition by ~11% on average on OpenVID-1M (relative), while maintaining visual quality. User studies show 74–83% preference for our physics-enhanced videos. It also reduces warping error by 22–37% (depending on the backbone) and improves temporal consistency scores. These results indicate that simple, global spectral cues are an effective drop-in regularizer for physically plausible motion in video diffusion.

Method: Uses a compact structured scene representation with explicit room boundaries, frames scene editing as next-token prediction task, employs dual-stage training (supervised fine-tuning + preference alignment), and uses zero-shot LLM for object removal and addition prompts.

Result: Surpasses state-of-the-art on addition tasks and achieves superior human-perceived quality on full scene synthesis.

Conclusion: ReSpace provides an effective framework for text-driven 3D indoor scene synthesis and editing that addresses limitations of existing methods through structured representations and language model capabilities.

Abstract: Scene synthesis and editing has emerged as a promising direction in computer graphics. Current trained approaches for 3D indoor scenes either oversimplify object semantics through one-hot class encodings (e.g., ‘chair’ or ’table’), require masked diffusion for editing, ignore room boundaries, or rely on floor plan renderings that fail to capture complex layouts. LLM-based methods enable richer semantics via natural language (e.g., ‘modern studio with light wood furniture’), but lack editing functionality, are limited to rectangular layouts, or rely on weak spatial reasoning from implicit world models. We introduce ReSpace, a generative framework for text-driven 3D indoor scene synthesis and editing using autoregressive language models. Our approach features a compact structured scene representation with explicit room boundaries that enables asset-agnostic deployment and frames scene editing as a next-token prediction task. We leverage a dual-stage training approach combining supervised fine-tuning and preference alignment, enabling a specially trained language model for object addition that accounts for user instructions, spatial geometry, object semantics, and scene-level composition. For scene editing, we employ a zero-shot LLM to handle object removal and prompts for addition. We further introduce a voxelization-based evaluation capturing fine-grained geometry beyond 3D bounding boxes. Experimental results surpass state-of-the-art on addition and achieve superior human-perceived quality on full scene synthesis.

Method: Uses dual-branch architecture with state-space model backbone - one branch captures spatial features, the other focuses on temporal dynamics, with continuous fusion via gating mechanism between branches. Also creates new benchmark by merging multiple datasets with strict train/test separation.

Result: Achieves state-of-the-art performance on the new benchmark and DVD dataset, offering optimal balance between accuracy and computational efficiency for near real-time surveillance violence detection.

Conclusion: Demonstrates the promise of state-space models for scalable, near real-time surveillance violence detection, providing an efficient alternative to CNNs and Transformers.

Abstract: The rapid proliferation of surveillance cameras has increased the demand for automated violence detection. While CNNs and Transformers have shown success in extracting spatio-temporal features, they struggle with long-term dependencies and computational efficiency. We propose Dual Branch VideoMamba with Gated Class Token Fusion (GCTF), an efficient architecture combining a dual-branch design and a state-space model (SSM) backbone where one branch captures spatial features, while the other focuses on temporal dynamics. The model performs continuous fusion via a gating mechanism between the branches to enhance the model’s ability to detect violent activities even in challenging surveillance scenarios. We also present a new benchmark by merging RWF-2000, RLVS, SURV and VioPeru datasets in video violence detection, ensuring strict separation between training and testing sets. Experimental results demonstrate that our model achieves state-of-the-art performance on this benchmark and also on DVD dataset which is another novel dataset on video violence detection, offering an optimal balance between accuracy and computational efficiency, demonstrating the promise of SSMs for scalable, near real-time surveillance violence detection.

Method: Proposes lightweight token selection mechanism, memory-efficient sparse attention strategy, and uses evolutionary algorithm to automatically determine optimal token reduction across timesteps.

Result: Achieves up to 2.4x speedup on single GPU and 13.2x on 8 GPUs, with less than 0.5% quality loss on VBench compared to baselines.

Conclusion: Astraea provides an effective framework for accelerating vDiT-based video generation while maintaining high quality, enabling practical deployment.

Abstract: Video diffusion transformers (vDiTs) have made tremendous progress in text-to-video generation, but their high compute demands pose a major challenge for practical deployment. While studies propose acceleration methods to reduce workload at various granularities, they often rely on heuristics, limiting their applicability. We introduce Astraea, a framework that searches for near-optimal configurations for vDiT-based video generation under a performance target. At its core, Astraea proposes a lightweight token selection mechanism and a memory-efficient, GPU-friendly sparse attention strategy, enabling linear savings on execution time with minimal impact on generation quality. Meanwhile, to determine optimal token reduction for different timesteps, we further design a search framework that leverages a classic evolutionary algorithm to automatically determine the distribution of the token budget effectively. Together, Astraea achieves up to 2.4$\times$ inference speedup on a single GPU with great scalability (up to 13.2$\times$ speedup on 8 GPUs) while achieving up to over 10~dB video quality compared to the state-of-the-art methods ($<$0.5% loss on VBench compared to baselines).

Method: Leveraged real-world driving data collected by autonomous vehicle drivers to ensure broad scenario coverage, collected high-level discrete action labels from actual driving operations, and implemented an action-rooted tree-structured evaluation framework linking vision, language, and action tasks.

Result: Experiments show state-of-the-art VLMs require both vision and language guidance for accurate action prediction: accuracy drops by 3.3% without vision, 4.1% without language, and 8.0% without either input. The evaluation framework provides precise bottleneck identification with robust results.

Conclusion: DriveAction provides new insights and a rigorous foundation for advancing human-like decisions in autonomous driving through comprehensive scenario coverage, reliable action annotations, and structured evaluation.

Abstract: Vision-Language-Action (VLA) models have advanced autonomous driving, but existing benchmarks still lack scenario diversity, reliable action-level annotation, and evaluation protocols aligned with human preferences. To address these limitations, we introduce DriveAction, the first action-driven benchmark specifically designed for VLA models, comprising 16,185 QA pairs generated from 2,610 driving scenarios. DriveAction leverages real-world driving data proactively collected by drivers of autonomous vehicles to ensure broad and representative scenario coverage, offers high-level discrete action labels collected directly from drivers’ actual driving operations, and implements an action-rooted tree-structured evaluation framework that explicitly links vision, language, and action tasks, supporting both comprehensive and task-specific assessment. Our experiments demonstrate that state-of-the-art vision-language models (VLMs) require both vision and language guidance for accurate action prediction: on average, accuracy drops by 3.3% without vision input, by 4.1% without language input, and by 8.0% without either. Our evaluation supports precise identification of model bottlenecks with robust and consistent results, thus providing new insights and a rigorous foundation for advancing human-like decisions in autonomous driving.

Method: Proposes the Input-Space Linearity Hypothesis (ISLH) and introduces the Spectral Principal Path (SPP) framework to formalize how deep networks progressively distill linear representations along dominant spectral directions.

Result: Demonstrates the multimodal robustness of these representations in Vision-Language Models (VLMs) and bridges theoretical insights with empirical validation.

Conclusion: Advances a structured theory of representation formation in deep networks, paving the way for improving AI robustness, fairness, and transparency.

Abstract: High-level representations have become a central focus in enhancing AI transparency and control, shifting attention from individual neurons or circuits to structured semantic directions that align with human-interpretable concepts. Motivated by the Linear Representation Hypothesis (LRH), we propose the Input-Space Linearity Hypothesis (ISLH), which posits that concept-aligned directions originate in the input space and are selectively amplified with increasing depth. We then introduce the Spectral Principal Path (SPP) framework, which formalizes how deep networks progressively distill linear representations along a small set of dominant spectral directions. Building on this framework, we further demonstrate the multimodal robustness of these representations in Vision-Language Models (VLMs). By bridging theoretical insights with empirical validation, this work advances a structured theory of representation formation in deep networks, paving the way for improving AI robustness, fairness, and transparency.

Method: Uses two proxy tasks: DarkEventInfer (inferring masked video events) and MixVidQA (reasoning about interleaved video clips) to develop both holistic understanding and precise reasoning capabilities.

Result: VidBridge-R1 achieves significant performance gains on both QA and captioning tasks within a single model, demonstrating effective paradigm conflict resolution.

Conclusion: The proposed framework successfully bridges the conflict between QA and captioning, fostering more generalizable and powerful video understanding models.

Abstract: The “Reason-Then-Respond” paradigm, enhanced by Reinforcement Learning, has shown great promise in advancing Multimodal Large Language Models. However, its application to the video domain has led to specialized models that excel at either question answering (QA) or captioning tasks, but struggle to master both. Naively combining reward signals from these tasks results in mutual performance degradation, which we attribute to a conflict between their opposing task natures. To address this challenge, we propose a novel training framework built upon two intermediate proxy tasks: DarkEventInfer, which presents videos with masked event segments, requiring models to infer the obscured content based on contextual video cues; and MixVidQA, which presents interleaved video sequences composed of two distinct clips, challenging models to isolate and reason about one while disregarding the other. These proxy tasks compel the model to simultaneously develop both holistic, divergent understanding and precise, convergent reasoning capabilities. Embodying this framework, we present VidBridge-R1, the first versatile video reasoning model that effectively bridges the paradigm conflict. Extensive experiments show that VidBridge-R1 achieves significant performance gains on both QA and captioning within one model, demonstrating the efficacy of our approach in fostering more generalizable and powerful video understanding models.

Method: Introduces condensed feature grid (CFG) for efficient scene representation, LEO-VL model trained on diverse 3D-VL data, and SceneDPO for post-training robustness through answer and scene contrasts.

Result: LEO-VL achieves state-of-the-art performance on SQA3D, MSQA, and Beacon3D benchmarks, demonstrating efficiency, scalability benefits, and robustness improvements over SFT and GRPO.

Conclusion: The proposed methods advance 3D VLMs in efficiency, scalability, and robustness, with condensed feature grids enabling competitive performance without heavy token overhead.

Abstract: Developing vision-language models (VLMs) capable of understanding 3D scenes has been a longstanding goal in the 3D-VL community. Despite recent progress, 3D VLMs still fall short of their 2D counterparts in capability and robustness. A key bottleneck is that current scene representations struggle to balance performance and efficiency: competitive performance comes at the cost of heavy token overhead, which in turn hampers the scalability of 3D-VL learning. To address this, we propose the condensed feature grid (CFG), an efficient scene representation featuring significantly reduced token overhead and strong perception capability. Building on CFG, we introduce LEO-VL, a 3D VLM trained on 700k 3D-VL data spanning four real-world indoor domains and five tasks such as captioning and dialogue. To enhance the robustness of 3D VLM, we further propose SceneDPO for post-training, which involves contrasts across answers and scenes. LEO-VL achieves state-of-the-art performance on various 3D QA benchmarks, including SQA3D, MSQA, and Beacon3D. Our extensive experiments highlight the efficiency of our representation, the benefit of task and scene diversity, consistent scaling effects, and the advantages of SceneDPO compared to SFT and GRPO. We hope our findings advance the efficiency, scalability, and robustness of future 3D VLMs.

Method: VideoExplorer uses iterative reasoning with sub-question formulation, temporal grounding, and scalable perception. It employs a two-stage training pipeline with supervised trajectory initialization followed by trajectory-level preference optimization, trained on a constructed long-video reasoning dataset using difficulty-adaptive sampling.

Result: Extensive evaluations on long-video understanding benchmarks show VideoExplorer’s significant advantage over existing baselines, demonstrating robustness, adaptability, and efficiency.

Conclusion: VideoExplorer provides a faithful, efficient, and interpretable approach to long-video understanding by naturally integrating planning, temporal grounding, and scalable perception into a coherent reasoning process.

Abstract: Long-video understanding~(LVU) is a challenging problem in computer vision. Existing methods either downsample frames for single-pass reasoning, sacrificing fine-grained details, or depend on textual reasoning over task-agnostic representations, hindering task-specific perception and exploration. In this paper, we propose VideoExplorer, a framework grounded in the principle of ``thinking with video’’, which naturally intertwines planning, temporal grounding, and scalable perception into a coherent reasoning process. Rather than reasoning over a static context, VideoExplorer iteratively formulates sub-questions, locates relevant moments, and performs task-oriented, temporally scalable video understanding until reaching the final answer, enabling faithful, efficient, and interpretable reasoning. To address the lack of LVU training resources, we construct a long-video reasoning dataset using difficulty-adaptive sampling to ensure high-quality trajectories on complex tasks. Building on this dataset, we design a two-stage training pipeline: supervised trajectory initialization followed by trajectory-level preference optimization, encouraging adaptive temporal grounding and iterative information integration guided by downstream rewards. Extensive evaluations on popular long-video understanding and reasoning benchmarks demonstrate VideoExplorer’s significant advantage over existing baselines, highlighting its robustness, adaptability, and efficiency. Our code is made publicly available in this repository(https://github.com/yhy-2000/VideoDeepResearch).

Method: Introduces Temporal Adaptation Modules to enhance MLLM’s long-range temporal understanding, and Semantic-Fused Control Injection mechanism to translate change understanding into generative control.

Result: Extensive experiments show TAMMs significantly outperforms state-of-the-art specialist baselines on both tasks.

Conclusion: The synergistic design enables understanding from TCD task to directly improve consistency of FSIF task, demonstrating effective joint performance on both tasks.

Abstract: Temporal Change Description (TCD) and Future Satellite Image Forecasting (FSIF) are critical, yet historically disjointed tasks in Satellite Image Time Series (SITS) analysis. Both are fundamentally limited by the common challenge of modeling long-range temporal dynamics. To explore how to improve the performance of methods on both tasks simultaneously by enhancing long-range temporal understanding capabilities, we introduce TAMMs, the first unified framework designed to jointly perform TCD and FSIF within a single MLLM-diffusion architecture. TAMMs introduces two key innovations: Temporal Adaptation Modules (TAM) enhance frozen MLLM’s ability to comprehend long-range dynamics, and Semantic-Fused Control Injection (SFCI) mechanism translates this change understanding into fine-grained generative control. This synergistic design makes the understanding from the TCD task to directly inform and improve the consistency of the FSIF task. Extensive experiments demonstrate TAMMs significantly outperforms state-of-the-art specialist baselines on both tasks.

Method: Proposes Controlled Random Zigzag Sampling (Ctrl-Z Sampling) that uses a reward model to detect local maxima, then injects noise and reverts to previous states to escape plateaus, evaluating candidate trajectories and performing deeper explorations when needed.

Result: Experimental results show Ctrl-Z Sampling substantially improves generation quality while requiring only about 7.72 times the NFEs (Number of Function Evaluations) of the original diffusion process.

Conclusion: The proposed method is model-agnostic and compatible with existing diffusion frameworks, enabling dynamic alternation between forward refinement and backward exploration to enhance both alignment and visual quality in generated outputs.

Abstract: Diffusion models have shown strong performance in conditional generation by progressively denoising Gaussian samples toward a target data distribution. This denoising process can be interpreted as a form of hill climbing in a learned representation space, where the model iteratively refines a sample toward regions of higher probability. However, this learned climbing often converges to local optima with plausible but suboptimal generations due to latent space complexity and suboptimal initialization. While prior efforts often strengthen guidance signals or introduce fixed exploration strategies to address this, they exhibit limited capacity to escape steep local maxima. In contrast, we propose Controlled Random Zigzag Sampling (Ctrl-Z Sampling), a novel sampling strategy that adaptively detects and escapes such traps through controlled exploration. In each diffusion step, we first identify potential local maxima using a reward model. Upon such detection, we inject noise and revert to a previous, noisier state to escape the current plateau. The reward model then evaluates candidate trajectories, accepting only those that offer improvement, otherwise scheming progressively deeper explorations when nearby alternatives fail. This controlled zigzag process allows dynamic alternation between forward refinement and backward exploration, enhancing both alignment and visual quality in the generated outputs. The proposed method is model-agnostic and also compatible with existing diffusion frameworks. Experimental results show that Ctrl-Z Sampling substantially improves generation quality while requiring only about 7.72 times the NFEs of the original.

Method: Converts target objects to time-consistent 3D meshes, uses Dual-Propagation Strategy to propagate edits from single frame to others, projects 3D meshes to 2D renderings, and employs decoupled video diffusion model for final generation.

Result: Supports various precise and physically-consistent manipulations including pose editing, rotation, scaling, translation, texture modification, and object composition across video frames.

Conclusion: The approach represents a key step toward high-quality, controllable video editing workflows, with extensive experiments demonstrating superiority and effectiveness.

Abstract: Recent advances in deep generative modeling have unlocked unprecedented opportunities for video synthesis. In real-world applications, however, users often seek tools to faithfully realize their creative editing intentions with precise and consistent control. Despite the progress achieved by existing methods, ensuring fine-grained alignment with user intentions remains an open and challenging problem. In this work, we present Shape-for-Motion, a novel framework that incorporates a 3D proxy for precise and consistent video editing. Shape-for-Motion achieves this by converting the target object in the input video to a time-consistent mesh, i.e., a 3D proxy, allowing edits to be performed directly on the proxy and then inferred back to the video frames. To simplify the editing process, we design a novel Dual-Propagation Strategy that allows users to perform edits on the 3D mesh of a single frame, and the edits are then automatically propagated to the 3D meshes of the other frames. The 3D meshes for different frames are further projected onto the 2D space to produce the edited geometry and texture renderings, which serve as inputs to a decoupled video diffusion model for generating edited results. Our framework supports various precise and physically-consistent manipulations across the video frames, including pose editing, rotation, scaling, translation, texture modification, and object composition. Our approach marks a key step toward high-quality, controllable video editing workflows. Extensive experiments demonstrate the superiority and effectiveness of our approach. Project page: https://shapeformotion.github.io/

Method: ReasonBrain framework uses MLLMs for editing guidance generation and diffusion models for image synthesis, with Fine-grained Reasoning Cue Extraction (FRCE) module and Cross-Modal Enhancer (CME) to capture detailed semantics and enable rich feature interactions.

Result: Extensive experiments show ReasonBrain consistently outperforms state-of-the-art baselines on reasoning scenarios and exhibits strong zero-shot generalization to conventional IIE tasks.

Conclusion: The proposed Reason50K dataset and ReasonBrain framework effectively address the limitations of current methods in handling complex hypothetical instructions requiring reasoning, demonstrating superior performance and generalization capabilities.

Abstract: Instruction-based image editing (IIE) has advanced rapidly with the success of diffusion models. However, existing efforts primarily focus on simple and explicit instructions to execute editing operations such as adding, deleting, moving, or swapping objects. They struggle to handle more complex implicit hypothetical instructions that require deeper reasoning to infer plausible visual changes and user intent. Additionally, current datasets provide limited support for training and evaluating reasoning-aware editing capabilities. Architecturally, these methods also lack mechanisms for fine-grained detail extraction that support such reasoning. To address these limitations, we propose Reason50K, a large-scale dataset specifically curated for training and evaluating hypothetical instruction reasoning image editing, along with ReasonBrain, a novel framework designed to reason over and execute implicit hypothetical instructions across diverse scenarios. Reason50K includes over 50K samples spanning four key reasoning scenarios: Physical, Temporal, Causal, and Story reasoning. ReasonBrain leverages Multimodal Large Language Models (MLLMs) for editing guidance generation and a diffusion model for image synthesis, incorporating a Fine-grained Reasoning Cue Extraction (FRCE) module to capture detailed visual and textual semantics essential for supporting instruction reasoning. To mitigate the semantic loss, we further introduce a Cross-Modal Enhancer (CME) that enables rich interactions between the fine-grained cues and MLLM-derived features. Extensive experiments demonstrate that ReasonBrain consistently outperforms state-of-the-art baselines on reasoning scenarios while exhibiting strong zero-shot generalization to conventional IIE tasks. Our dataset and code will be released publicly.

Method: Uses pixel-aligned 3D representation and unifies multiple understanding tasks into learnable queries, with two lightweight modules to facilitate interaction between reconstruction and understanding tasks.

Result: Achieves state-of-the-art performance on individual 3D reconstruction and understanding tasks, as well as simultaneous understanding and 3D reconstruction.

Conclusion: The alignment-free framework with mutual benefit designs effectively enables native 3D understanding without 2D model alignment, demonstrating superior performance across tasks.

Abstract: Simultaneous understanding and 3D reconstruction plays an important role in developing end-to-end embodied intelligent systems. To achieve this, recent approaches resort to 2D-to-3D feature alignment paradigm, which leads to limited 3D understanding capability and potential semantic information loss. In light of this, we propose SIU3R, the first alignment-free framework for generalizable simultaneous understanding and 3D reconstruction from unposed images. Specifically, SIU3R bridges reconstruction and understanding tasks via pixel-aligned 3D representation, and unifies multiple understanding (segmentation) tasks into a set of unified learnable queries, enabling native 3D understanding without the need of alignment with 2D models. To encourage collaboration between the two tasks with shared representation, we further conduct in-depth analyses of their mutual benefits, and propose two lightweight modules to facilitate their interaction. Extensive experiments demonstrate that our method achieves state-of-the-art performance not only on the individual tasks of 3D reconstruction and understanding, but also on the task of simultaneous understanding and 3D reconstruction, highlighting the advantages of our alignment-free framework and the effectiveness of the mutual benefit designs. Project page: https://insomniaaac.github.io/siu3r/

Method: Systematic encoder masking across representative multi-encoder MLLMs, introducing two metrics: Conditional Utilization Rate (CUR) to measure marginal encoder contribution, and Information Gap (IG) to capture encoder utility heterogeneity.

Result: Found pervasive encoder redundancy - performance often degrades gracefully or even improves when masking selected encoders. Single/dual encoder variants recover over 90% of baseline on most non-OCR tasks. Masking specific encoders can yield up to 16% higher accuracy on specific tasks and 3.6% overall performance boost.

Conclusion: Challenges the ‘more encoders are better’ heuristic in MLLMs and provides actionable diagnostics for developing more efficient multimodal architectures by identifying redundant and detrimental encoders.

Abstract: Recent multimodal large language models (MLLMs) increasingly integrate multiple vision encoders to improve performance on various benchmarks, assuming that diverse pretraining objectives yield complementary visual signals. However, we show this assumption often fails in practice. Through systematic encoder masking across representative multi encoder MLLMs, we find that performance typically degrades gracefully and sometimes even improves when selected encoders are masked, revealing pervasive encoder redundancy. To quantify this effect, we introduce two principled metrics: the Conditional Utilization Rate (CUR), which measures an encoders marginal contribution in the presence of others, and the Information Gap (IG), which captures heterogeneity in encoder utility within a model. Using these tools, we observe (i) strong specialization on tasks like OCR and Chart, where a single encoder can dominate with a CUR greater than 90%, (ii) high redundancy on general VQA and knowledge-based tasks, where encoders are largely interchangeable, (iii) instances of detrimental encoders with negative CUR. Notably, masking specific encoders can yield up to 16% higher accuracy on a specific task category and 3.6% overall performance boost compared to the full model.Furthermore, single and dual encoder variants recover over 90% of baseline on most non OCR tasks. Our analysis challenges the more encoders are better heuristic in MLLMs and provides actionable diagnostics for developing more efficient and effective multimodal architectures.

Method: Each client uses modality-specific up- and down-projection layers with a globally shared projection that aligns cross-modal features. Clients locally adapt to personalized data while collaboratively training the shared projection, with only the shared component exchanged during communication.

Result: Extensive experiments across eleven datasets show pFedMMA achieves state-of-the-art trade-offs between personalization and generalization, outperforming recent federated prompt tuning methods in domain- and label-shift scenarios.

Conclusion: pFedMMA successfully addresses the challenge of adapting VLMs to decentralized data by leveraging multi-modal adapters, enabling both effective personalization and improved global generalization in a communication-efficient manner.

Abstract: Vision-Language Models (VLMs) like CLIP have demonstrated remarkable generalization in zero- and few-shot settings, but adapting them efficiently to decentralized, heterogeneous data remains a challenge. While prompt tuning has emerged as a popular parameter-efficient approach in personalized federated learning, existing methods often sacrifice generalization in favor of personalization, struggling particularly on unseen classes or domains. In this work, we propose pFedMMA, the first personalized federated learning framework that leverages multi-modal adapters for vision-language tasks. Each adapter contains modality-specific up- and down-projection layers alongside a globally shared projection that aligns cross-modal features. Our optimization strategy allows clients to locally adapt to personalized data distributions while collaboratively training the shared projection to improve global generalization. This design is also communication-efficient, as only the shared component is exchanged during communication rounds. Through extensive experiments across eleven datasets, including domain- and label-shift scenarios, we show that pFedMMA achieves state-of-the-art trade-offs between personalization and generalization, outperforming recent federated prompt tuning methods.

Method: Introduces Temporal Narrative Atoms (TNAs) as basic narrative units, creates an automatic prompt generation pipeline for flexible TNA expansion, and designs a three-level evaluation metric using MLLM-based question generation and answering.

Result: The proposed metric aligns closely with human judgments and reveals detailed capability boundaries of current video generation models in narrative content expression.

Conclusion: NarrLV successfully addresses the gap in evaluating narrative expression for long video generation and provides a comprehensive benchmark that can guide future model development.

Abstract: With the rapid development of foundation video generation technologies, long video generation models have exhibited promising research potential thanks to expanded content creation space. Recent studies reveal that the goal of long video generation tasks is not only to extend video duration but also to accurately express richer narrative content within longer videos. However, due to the lack of evaluation benchmarks specifically designed for long video generation models, the current assessment of these models primarily relies on benchmarks with simple narrative prompts (e.g., VBench). To the best of our knowledge, our proposed NarrLV is the first benchmark to comprehensively evaluate the Narrative expression capabilities of Long Video generation models. Inspired by film narrative theory, (i) we first introduce the basic narrative unit maintaining continuous visual presentation in videos as Temporal Narrative Atom (TNA), and use its count to quantitatively measure narrative richness. Guided by three key film narrative elements influencing TNA changes, we construct an automatic prompt generation pipeline capable of producing evaluation prompts with a flexibly expandable number of TNAs. (ii) Then, based on the three progressive levels of narrative content expression, we design an effective evaluation metric using the MLLM-based question generation and answering framework. (iii) Finally, we conduct extensive evaluations on existing long video generation models and the foundation generation models. Experimental results demonstrate that our metric aligns closely with human judgments. The derived evaluation outcomes reveal the detailed capability boundaries of current video generation models in narrative content expression.

Method: Proposes a hybrid thinking framework with three-stage progressive training including Cascaded Reinforcement Learning, enabling adaptive switching between text-only reasoning and tool-augmented visual reasoning.

Result: With only 3B parameters, DriveAgent-R1 achieves competitive performance comparable to GPT-5 and human driving proficiency on Drive-Internal and nuScenes datasets.

Conclusion: DriveAgent-R1 offers a proven path toward more intelligent autonomous driving systems by enabling active perception and grounding decisions in visual evidence.

Abstract: The advent of Vision-Language Models (VLMs) has significantly advanced end-to-end autonomous driving, demonstrating powerful reasoning abilities for high-level behavior planning tasks. However, existing methods are often constrained by a passive perception paradigm, relying solely on text-based reasoning. This passivity restricts the model’s capacity to actively seek crucial visual evidence when faced with uncertainty. To address this, we introduce DriveAgent-R1, the first autonomous driving agent capable of active perception for planning. In complex scenarios, DriveAgent-R1 proactively invokes tools to perform visual reasoning, firmly grounding its decisions in visual evidence, thereby enhancing both interpretability and reliability. Furthermore, we propose a hybrid thinking framework, inspired by human driver cognitive patterns, allowing the agent to adaptively switch between efficient text-only reasoning and robust tool-augmented visual reasoning based on scene complexity. This capability is cultivated through a three-stage progressive training strategy, featuring a core Cascaded Reinforcement Learning (Cascaded RL) phase. Extensive experiments on the Drive-Internal dataset, which is rich in long-tail scenarios, and the public nuScenes dataset show that, with only 3B parameters, DriveAgent-R1 achieves competitive performance comparable to top closed model systems such as GPT-5 and to human driving proficiency while remaining deployment-friendly, offering a proven path toward building more intelligent autonomous driving systems.

Method: Proposed A^2R^2 framework combines attention localization and iterative refinement in a visual reasoning framework, allowing models to perform self-correction. Also introduced Img2LaTeX-Hard-1K dataset with 1,100 challenging examples for evaluation.

Result: Significant performance improvements across various metrics, with increasing inference rounds yielding notable gains. Ablation studies confirm effectiveness and synergy of core components.

Conclusion: A^2R^2 demonstrates strong potential for test-time scaling scenarios and effectively addresses fine-grained visual reasoning challenges in Img2LaTeX conversion.

Abstract: Img2LaTeX is a practically important task that involves translating mathematical expressions and structured visual content from images into LaTeX code. In recent years, vision-language models (VLMs) have achieved remarkable progress across a range of visual understanding tasks, largely due to their strong generalization capabilities. However, despite initial efforts to apply VLMs to the Img2LaTeX task, their performance remains suboptimal. Empirical evidence shows that VLMs can be challenged by fine-grained visual elements, such as subscripts and superscripts in mathematical expressions, which results in inaccurate LaTeX generation. To address this challenge, we propose $A^2R^2$: Advancing Img2LaTeX Conversion via Visual Reasoning with Attention-Guided Refinement, a framework that effectively integrates attention localization and iterative refinement within a visual reasoning framework, enabling VLMs to perform self-correction and progressively improve LaTeX generation quality. For effective evaluation, we introduce a new dataset, Img2LaTex-Hard-1K, consisting of 1,100 carefully curated and challenging examples designed to rigorously evaluate the capabilities of VLMs within this task domain. Extensive experimental results demonstrate that: (1) $A^2R^2$ significantly improves model performance across various evaluation metrics spanning both textual and visual levels; (2) Increasing the number of inference rounds yields notable performance gains, underscoring the potential of $A^2R^2$ in test-time scaling scenarios; (3) Ablation studies and further evaluations confirm the effectiveness of our approach and the synergy of its core components during inference.

Method: Proposes Class-aware Spatial Relation Prior to encode relative positional dependencies using learnable class-aware relation encoder, and Mixture-of-Experts-based Semantic Prior that routes features to class-specific experts based on predicted class probabilities.

Result: Achieves state-of-the-art performance on both nuScenes and Argoverse 2 datasets, compatible with both single-frame and temporal perception backbones.

Conclusion: Explicit modeling of spatial relations and semantic priors significantly enhances online HD map construction performance.

Abstract: Online high-definition (HD) map construction is crucial for scaling autonomous driving systems. While Transformer-based methods have become prevalent in online HD map construction, most existing approaches overlook the inherent spatial dependencies and semantic relationships among map elements, which constrains their accuracy and generalization capabilities. To address this, we propose RelMap, an end-to-end framework that explicitly models both spatial relations and semantic priors to enhance online HD map construction. Specifically, we introduce a Class-aware Spatial Relation Prior, which explicitly encodes relative positional dependencies between map elements using a learnable class-aware relation encoder. Additionally, we design a Mixture-of-Experts-based Semantic Prior, which routes features to class-specific experts based on predicted class probabilities, refining instance feature decoding. RelMap is compatible with both single-frame and temporal perception backbones, achieving state-of-the-art performance on both the nuScenes and Argoverse 2 datasets.

Method: Two novel mechanisms: content-adaptive token permutation to prioritize interactions between similar tokens, and injection of sample-specific global priors to mitigate strict causality without multi-directional scans.

Result: CMIC achieves state-of-the-art rate-distortion performance, surpassing VTM-21.0 by 15.91%, 21.34%, and 17.58% in BD-rate on Kodak, Tecnick, and CLIC datasets respectively.

Conclusion: CAM enables better global redundancy capture while preserving computational efficiency, demonstrating superior performance in learned image compression.

Abstract: Recent learned image compression (LIC) leverages Mamba-style state-space models (SSMs) for global receptive fields with linear complexity. However, the standard Mamba adopts content-agnostic, predefined raster (or multi-directional) scans under strict causality. This rigidity hinders its ability to effectively eliminate redundancy between tokens that are content-correlated but spatially distant. We introduce Content-Aware Mamba (CAM), an SSM that dynamically adapts its processing to the image content. Specifically, CAM overcomes prior limitations with two novel mechanisms. First, it replaces the rigid scan with a content-adaptive token permutation strategy to prioritize interactions between content-similar tokens regardless of their location. Second, it overcomes the sequential dependency by injecting sample-specific global priors into the state-space model, which effectively mitigates the strict causality without multi-directional scans. These innovations enable CAM to better capture global redundancy while preserving computational efficiency. Our Content-Aware Mamba-based LIC model (CMIC) achieves state-of-the-art rate-distortion performance, surpassing VTM-21.0 by 15.91%, 21.34%, and 17.58% in BD-rate on the Kodak, Tecnick, and CLIC datasets, respectively. Code and checkpoints will be released later.

Method: Proposes a simple, theoretically grounded adaptive guidance schedule that automatically dampens guidance scale at early steps based on the evolving ratio of conditional to unconditional predictions, requiring no inference overhead.

Result: Experiments across state-of-the-art image (SD3.5, Qwen-Image) and video (WAN2.1) models show up to 3x faster sampling while maintaining or improving quality, robustness, and semantic alignment.

Conclusion: Adapting guidance to the sampling process rather than fixing it is critical for unlocking the full potential of fast, flow-based models.

Abstract: Flow-based generative models have achieved remarkable progress, with classifier-free guidance (CFG) becoming the standard for high-fidelity generation. However, the conventional practice of applying a strong, fixed guidance scale throughout inference is poorly suited for the rapid, few-step sampling required by modern applications. In this work, we uncover the root cause of this conflict: a fundamental sampling instability where the earliest steps are acutely sensitive to guidance. We trace this to a significant spike in the ratio of conditional to unconditional predictions–a spike that we prove to be an inherent property of the training data distribution itself, making it a almost inevitable challenge. Applying a high, static guidance value during this volatile initial phase leads to an exponential amplification of error, degrading image quality. To resolve this, we propose a simple, theoretically grounded, adaptive guidance schedule that automatically dampens the guidance scale at early steps based on the evolving ratio. Our method is lightweight, incurs no inference overhead, and is compatible with standard frameworks. Experiments across state-of-the-art image (SD3.5, Qwen-Image) and video (WAN2.1) models show our approach enables up to 3x faster sampling while maintaining or improving quality, robustness, and semantic alignment. Our findings highlight that adapting guidance to the sampling process, rather than fixing it, is critical for unlocking the full potential of fast, flow-based models.

Method: Three key innovations: (1) trajectory branching mechanism for process rewards, (2) noise-aware weighting scheme for temporal optimization, and (3) seed group strategy to isolate exploration effects.

Result: Achieves state-of-the-art performance in human preference alignment and text-to-image benchmarks through temporally-aware optimization that respects generative dynamics.

Conclusion: TempFlow-GRPO provides a principled framework that effectively captures temporal structure in flow-based generation, enabling more efficient exploration and convergence for preference alignment.

Abstract: Recent flow matching models for text-to-image generation have achieved remarkable quality, yet their integration with reinforcement learning for human preference alignment remains suboptimal, hindering fine-grained reward-based optimization. We observe that the key impediment to effective GRPO training of flow models is the temporal uniformity assumption in existing approaches: sparse terminal rewards with uniform credit assignment fail to capture the varying criticality of decisions across generation timesteps, resulting in inefficient exploration and suboptimal convergence. To remedy this shortcoming, we introduce \textbf{TempFlow-GRPO} (Temporal Flow GRPO), a principled GRPO framework that captures and exploits the temporal structure inherent in flow-based generation. TempFlow-GRPO introduces three key innovations: (i) a trajectory branching mechanism that provides process rewards by concentrating stochasticity at designated branching points, enabling precise credit assignment without requiring specialized intermediate reward models; (ii) a noise-aware weighting scheme that modulates policy optimization according to the intrinsic exploration potential of each timestep, prioritizing learning during high-impact early stages while ensuring stable refinement in later phases; and (iii) a seed group strategy that controls for initialization effects to isolate exploration contributions. These innovations endow the model with temporally-aware optimization that respects the underlying generative dynamics, leading to state-of-the-art performance in human preference alignment and text-to-image benchmarks.

Method: Proposes a concept distillation training strategy using pretrained T2V model to synthesize training samples with embedded textual concepts. Also redesigns control architecture with a control feature projector to filter degradation artifacts and a dual-branch ControlNet connector with MLP-based feature mapping and cross-attention for dynamic control feature retrieval.

Result: Vivid-VR performs favorably against existing approaches on synthetic, real-world benchmarks, and AIGC videos, achieving impressive texture realism, visual vividness, and temporal consistency.

Conclusion: The proposed method effectively addresses distribution drift issues in video restoration while maintaining high-quality texture and temporal coherence through concept distillation and enhanced control architecture.

Abstract: We present Vivid-VR, a DiT-based generative video restoration method built upon an advanced T2V foundation model, where ControlNet is leveraged to control the generation process, ensuring content consistency. However, conventional fine-tuning of such controllable pipelines frequently suffers from distribution drift due to limitations in imperfect multimodal alignment, resulting in compromised texture realism and temporal coherence. To tackle this challenge, we propose a concept distillation training strategy that utilizes the pretrained T2V model to synthesize training samples with embedded textual concepts, thereby distilling its conceptual understanding to preserve texture and temporal quality. To enhance generation controllability, we redesign the control architecture with two key components: 1) a control feature projector that filters degradation artifacts from input video latents to minimize their propagation through the generation pipeline, and 2) a new ControlNet connector employing a dual-branch design. This connector synergistically combines MLP-based feature mapping with cross-attention mechanism for dynamic control feature retrieval, enabling both content preservation and adaptive control signal modulation. Extensive experiments show that Vivid-VR performs favorably against existing approaches on both synthetic and real-world benchmarks, as well as AIGC videos, achieving impressive texture realism, visual vividness, and temporal consistency. The codes and checkpoints are publicly available at https://github.com/csbhr/Vivid-VR.

Method: Used YOLOv8 object recognition framework with custom variants YOLOv8m-t4 and YOLOv8m-t42. Created a bespoke dataset with annotated car surface photos under various conditions. Employed real-time data augmentation for robustness.

Result: YOLOv8m-t42 model achieved precision: 0.86, recall: 0.84, F1-score: 0.85, mAP@0.5: 0.60, and PR curve area: 0.88, outperforming YOLOv8m-t4 model (precision: 0.81, recall: 0.79, F1-score: 0.80, PR curve: 0.82).

Conclusion: The deep learning-based approach provides excellent detection accuracy and low inference latency, making it suitable for real-time applications like automated insurance evaluations and car inspections, with YOLOv8m-t42 being more appropriate for practical dent detection despite slower convergence.

Abstract: Conventional car damage inspection techniques are labor-intensive, manual, and frequently overlook tiny surface imperfections like microscopic dents. Machine learning provides an innovative solution to the increasing demand for quicker and more precise inspection methods. The paper uses the YOLOv8 object recognition framework to provide a deep learning-based solution for automatically detecting microscopic surface flaws, notably tiny dents, on car exteriors. Traditional automotive damage inspection procedures are manual, time-consuming, and frequently unreliable at detecting tiny flaws. To solve this, a bespoke dataset containing annotated photos of car surfaces under various lighting circumstances, angles, and textures was created. To improve robustness, the YOLOv8m model and its customized variants, YOLOv8m-t4 and YOLOv8m-t42, were trained employing real-time data augmentation approaches. Experimental results show that the technique has excellent detection accuracy and low inference latency, making it suited for real-time applications such as automated insurance evaluations and automobile inspections. Evaluation parameters such as mean Average Precision (mAP), precision, recall, and F1-score verified the model’s efficacy. With a precision of 0.86, recall of 0.84, and F1-score of 0.85, the YOLOv8m-t42 model outperformed the YOLOv8m-t4 model (precision: 0.81, recall: 0.79, F1-score: 0.80) in identifying microscopic surface defects. With a little reduced mAP@0.5:0.95 of 0.20, the mAP@0.5 for YOLOv8m-t42 stabilized at 0.60. Furthermore, YOLOv8m-t42’s PR curve area was 0.88, suggesting more consistent performance than YOLOv8m-t4 (0.82). YOLOv8m-t42 has greater accuracy and is more appropriate for practical dent detection applications, even though its convergence is slower.

Method: Proposes two lightweight modules: 1) Geometry Re-Densification Module that re-densifies sparse geometry, extracts features at denser scale, then re-sparsifies for predictive coding; 2) Cross-scale Feature Propagation Module that leverages occupancy cues from multiple resolution levels to guide hierarchical feature propagation and enable cross-scale information sharing.

Result: Achieves state-of-the-art compression ratios on KITTI dataset with real-time performance (26 FPS for encoding/decoding at 12-bit quantization).

Conclusion: The proposed framework generates compact feature representations that provide efficient context modeling and accelerate the coding process, demonstrating superior compression performance while maintaining lightweight computation.

Abstract: LiDAR point clouds are fundamental to various applications, yet high-precision scans incur substantial storage and transmission overhead. Existing methods typically convert unordered points into hierarchical octree or voxel structures for dense-to-sparse predictive coding. However, the extreme sparsity of geometric details hinders efficient context modeling, thereby limiting their compression performance and speed. To address this challenge, we propose to generate compact features for efficient predictive coding. Our framework comprises two lightweight modules. First, the Geometry Re-Densification Module re-densifies encoded sparse geometry, extracts features at denser scale, and then re-sparsifies the features for predictive coding. This module avoids costly computation on highly sparse details while maintaining a lightweight prediction head. Second, the Cross-scale Feature Propagation Module leverages occupancy cues from multiple resolution levels to guide hierarchical feature propagation. This design facilitates information sharing across scales, thereby reducing redundant feature extraction and providing enriched features for the Geometry Re-Densification Module. By integrating these two modules, our method yields a compact feature representation that provides efficient context modeling and accelerates the coding process. Experiments on the KITTI dataset demonstrate state-of-the-art compression ratios and real-time performance, achieving 26 FPS for encoding/decoding at 12-bit quantization. Code is available at https://github.com/pengpeng-yu/FastPCC.

Method: Introduces Draw-In-Mind (DIM) dataset with two subsets: DIM-T2I (14M long-context image-text pairs) for enhanced instruction comprehension, and DIM-Edit (233K chain-of-thought imaginations from GPT-4o) as explicit design blueprints. Connects frozen Qwen2.5-VL-3B with trainable SANA1.5-1.6B via lightweight MLP.

Result: DIM-4.6B-Edit achieves SOTA or competitive performance on ImgEdit and GEdit-Bench benchmarks, outperforming larger models like UniWorld-V1 and Step1X-Edit despite modest parameter scale.

Conclusion: Explicitly assigning design responsibility to the understanding module provides significant benefits for image editing, demonstrating that balanced task allocation is more effective than model scaling alone.

Abstract: In recent years, integrating multimodal understanding and generation into a single unified model has emerged as a promising paradigm. While this approach achieves strong results in text-to-image (T2I) generation, it still struggles with precise image editing. We attribute this limitation to an imbalanced division of responsibilities. The understanding module primarily functions as a translator that encodes user instructions into semantic conditions, while the generation module must simultaneously act as designer and painter, inferring the original layout, identifying the target editing region, and rendering the new content. This imbalance is counterintuitive because the understanding module is typically trained with several times more data on complex reasoning tasks than the generation module. To address this issue, we introduce Draw-In-Mind (DIM), a dataset comprising two complementary subsets: (i) DIM-T2I, containing 14M long-context image-text pairs to enhance complex instruction comprehension; and (ii) DIM-Edit, consisting of 233K chain-of-thought imaginations generated by GPT-4o, serving as explicit design blueprints for image edits. We connect a frozen Qwen2.5-VL-3B with a trainable SANA1.5-1.6B via a lightweight two-layer MLP, and train it on the proposed DIM dataset, resulting in DIM-4.6B-T2I/Edit. Despite its modest parameter scale, DIM-4.6B-Edit achieves SOTA or competitive performance on the ImgEdit and GEdit-Bench benchmarks, outperforming much larger models such as UniWorld-V1 and Step1X-Edit. These findings demonstrate that explicitly assigning the design responsibility to the understanding module provides significant benefits for image editing. Our dataset and models are available at https://github.com/showlab/DIM.

Method: GLEAM-C uses a two-phase training strategy to align multiple views/modalities (UAV, street maps, panoramic views, ground photos) with satellite imagery. GLEAM-X leverages MLLMs for explainable reasoning and constructs a bilingual benchmark using GPT-4o and Doubao-1.5-Thinking-Vision-Pro.

Result: GLEAM-C achieves accuracy comparable to prior modality-specific CVGL models with enhanced training efficiency. GLEAM-X enables systematic evaluation of explainable cross-view reasoning through human-refined test data.

Conclusion: The GLEAM framework integrates multi-modal, multi-view alignment with interpretable correspondence analysis, advancing geo-localization by unifying accurate cross-view matching with explainable reasoning.

Abstract: Cross-View Geo-Localization (CVGL) focuses on identifying correspondences between images captured from distinct perspectives of the same geographical location. However, existing CVGL approaches are typically restricted to a single view or modality, and their direct visual matching strategy lacks interpretability: they only determine whether two images correspond, without explaining the rationale behind the match. In this paper, we present GLEAM-C, a foundational CVGL model that unifies multiple views and modalities-including UAV imagery, street maps, panoramic views, and ground photographs-by aligning them exclusively with satellite imagery. Our framework enhances training efficiency through optimized implementation while achieving accuracy comparable to prior modality-specific CVGL models through a two-phase training strategy. Moreover, to address the lack of interpretability in traditional CVGL methods, we leverage the reasoning capabilities of multimodal large language models (MLLMs) to propose a new task, GLEAM-X, which combines cross-view correspondence prediction with explainable reasoning. To support this task, we construct a bilingual benchmark using GPT-4o and Doubao-1.5-Thinking-Vision-Pro to generate training and testing data. The test set is further refined through detailed human revision, enabling systematic evaluation of explainable cross-view reasoning and advancing transparency and scalability in geo-localization. Together, GLEAM-C and GLEAM-X form a comprehensive CVGL pipeline that integrates multi-modal, multi-view alignment with interpretable correspondence analysis, unifying accurate cross-view matching with explainable reasoning and advancing Geo-Localization by enabling models to better Explain And Match. Code and datasets used in this work will be made publicly accessible at https://github.com/Lucky-Lance/GLEAM.

Method: Two-stage framework: 1) Fundus ultrasound video processing with frame-level anatomical segmentation, rule-based keyframe identification guided by international consensus, and precise ONSD measurement; 2) ICP grading stage that fuses ONSD metrics with clinical features to predict ICP grades.

Result: Achieved validation accuracy of 0.845 ± 0.071 (five-fold cross-validation) and independent test accuracy of 0.786, significantly outperforming conventional threshold-based method (0.637 ± 0.111 validation accuracy, 0.429 test accuracy).

Conclusion: The framework establishes a reliable non-invasive approach for clinical ICP evaluation by reducing operator variability and integrating multi-source information, holding promise for improving patient management in acute neurological conditions.

Abstract: Intracranial pressure (ICP) elevation poses severe threats to cerebral function, thus necessitating monitoring for timely intervention. While lumbar puncture is the gold standard for ICP measurement, its invasiveness and associated risks drive the need for non-invasive alternatives. Optic nerve sheath diameter (ONSD) has emerged as a promising biomarker, as elevated ICP directly correlates with increased ONSD. However, current clinical practices for ONSD measurement suffer from inconsistency in manual operation, subjectivity in optimal view selection, and variability in thresholding, limiting their reliability. To address these challenges, we introduce a fully automatic two-stage framework for ICP grading, integrating keyframe identification, ONSD measurement and clinical data. Specifically, the fundus ultrasound video processing stage performs frame-level anatomical segmentation, rule-based keyframe identification guided by an international consensus statement, and precise ONSD measurement. The intracranial pressure grading stage then fuses ONSD metrics with clinical features to enable the prediction of ICP grades, thereby demonstrating an innovative blend of interpretable ultrasound analysis and multi-source data integration for objective clinical evaluation. Experimental results demonstrate that our method achieves a validation accuracy of $0.845 \pm 0.071$ (with standard deviation from five-fold cross-validation) and an independent test accuracy of 0.786, significantly outperforming conventional threshold-based method ($0.637 \pm 0.111$ validation accuracy, $0.429$ test accuracy). Through effectively reducing operator variability and integrating multi-source information, our framework establishes a reliable non-invasive approach for clinical ICP evaluation, holding promise for improving patient management in acute neurological conditions.

Method: Uses overlapping tiling to recover low-confidence candidates, then applies Spatial Gate (DBSCAN on box centroids) and Semantic Gate (DBSCAN on ResNet-18 embeddings) to validate group evidence, followed by controlled confidence reweighting and class-aware NMS fusion.

Result: On VisDrone dataset, recall increased from 0.685 to 0.778 (+0.093), precision adjusted from 0.801 to 0.595, yielding F1=0.669 with post-processing latency of 0.095s per image.

Conclusion: The framework provides recall-first, precision-trade-off behavior beneficial for recall-sensitive applications like far-field counting and monitoring, requires no retraining, and integrates with modern detectors.

Abstract: Dense small objects in UAV imagery are often missed due to long-range viewpoints, occlusion, and clutter[cite: 5]. This paper presents a detector-agnostic post-processing framework that converts overlap-induced redundancy into group evidence[cite: 6]. Overlapping tiling first recovers low-confidence candidates[cite: 7]. A Spatial Gate (DBSCAN on box centroids) and a Semantic Gate (DBSCAN on ResNet-18 embeddings) then validates group evidence[cite: 7]. Validated groups receive controlled confidence reweighting before class-aware NMS fusion[cite: 8]. Experiments on VisDrone show a recall increase from 0.685 to 0.778 (+0.093) and a precision adjustment from 0.801 to 0.595, yielding F1=0.669[cite: 9]. Post-processing latency averages 0.095 s per image[cite: 10]. These results indicate recall-first, precision-trade-off behavior that benefits recall-sensitive applications such as far-field counting and monitoring[cite: 10]. Ablation confirms that tiling exposes missed objects, spatial clustering stabilizes geometry, semantic clustering enforces appearance coherence, and reweighting provides calibrated integration with the baseline[cite: 11]. The framework requires no retraining and integrates with modern detectors[cite: 12]. Future work will reduce semantic gating cost and extend the approach with temporal cues[cite: 13].

Method: Created a synthetic dataset with 12,328 annotated scenes and 4.7M renderings. Developed SpatialGen, a multi-view multi-modal diffusion model that takes 3D layout and reference image as input to synthesize appearance, geometry, and semantic maps from arbitrary viewpoints.

Result: SpatialGen consistently generates superior results compared to previous methods, producing realistic and semantically consistent 3D indoor scenes while preserving spatial consistency across modalities.

Conclusion: The approach addresses the dataset bottleneck and enables high-quality 3D scene generation. The data and models are being open-sourced to advance indoor scene understanding and generation.

Abstract: Creating high-fidelity 3D models of indoor environments is essential for applications in design, virtual reality, and robotics. However, manual 3D modeling remains time-consuming and labor-intensive. While recent advances in generative AI have enabled automated scene synthesis, existing methods often face challenges in balancing visual quality, diversity, semantic consistency, and user control. A major bottleneck is the lack of a large-scale, high-quality dataset tailored to this task. To address this gap, we introduce a comprehensive synthetic dataset, featuring 12,328 structured annotated scenes with 57,440 rooms, and 4.7M photorealistic 2D renderings. Leveraging this dataset, we present SpatialGen, a novel multi-view multi-modal diffusion model that generates realistic and semantically consistent 3D indoor scenes. Given a 3D layout and a reference image (derived from a text prompt), our model synthesizes appearance (color image), geometry (scene coordinate map), and semantic (semantic segmentation map) from arbitrary viewpoints, while preserving spatial consistency across modalities. SpatialGen consistently generates superior results to previous methods in our experiments. We are open-sourcing our data and models to empower the community and advance the field of indoor scene understanding and generation.

Method: Proposes a generative model training workflow with bi-directional contrastive loss and diversity loss, plus a caption synthesis strategy to improve text-to-image retrieval performance.

Result: Superior performance and efficiency on Flickr30K, COCO, and CC3M datasets, achieving results 18x faster than state-of-the-art methods.

Conclusion: EDGE provides an efficient solution for multimodal dataset distillation by addressing key challenges in generative modeling for image-text datasets.

Abstract: Dataset distillation aims to synthesize a small dataset from a large dataset, enabling the model trained on it to perform well on the original dataset. With the blooming of large language models and multimodal large language models, the importance of multimodal datasets, particularly image-text datasets, has grown significantly. However, existing multimodal dataset distillation methods are constrained by the Matching Training Trajectories algorithm, which significantly increases the computing resource requirement, and takes days to process the distillation. In this work, we introduce EDGE, a generative distillation method for efficient multimodal dataset distillation. Specifically, we identify two key challenges of distilling multimodal datasets with generative models: 1) The lack of correlation between generated images and captions. 2) The lack of diversity among generated samples. To address the aforementioned issues, we propose a novel generative model training workflow with a bi-directional contrastive loss and a diversity loss. Furthermore, we propose a caption synthesis strategy to further improve text-to-image retrieval performance by introducing more text information. Our method is evaluated on Flickr30K, COCO, and CC3M datasets, demonstrating superior performance and efficiency compared to existing approaches. Notably, our method achieves results 18x faster than the state-of-the-art method.

Method: Uses geometric priors from monocular depth estimations, extracts local semantic regions with SfM points anchoring for alignment, and applies geometry-guided supervision at virtual views with fine-grained and coarse schemes to ensure 3D consistency and reduce overfitting.

Result: MS-GS achieves photorealistic renderings under challenging sparse-view and multi-appearance conditions, significantly outperforming existing approaches across different datasets.

Conclusion: The proposed MS-GS framework effectively handles sparse-view reconstruction with multiple appearances using geometric priors and multi-view constraints, setting new benchmarks for realistic in-the-wild scenarios.

Abstract: In-the-wild photo collections often contain limited volumes of imagery and exhibit multiple appearances, e.g., taken at different times of day or seasons, posing significant challenges to scene reconstruction and novel view synthesis. Although recent adaptations of Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) have improved in these areas, they tend to oversmooth and are prone to overfitting. In this paper, we present MS-GS, a novel framework designed with Multi-appearance capabilities in Sparse-view scenarios using 3DGS. To address the lack of support due to sparse initializations, our approach is built on the geometric priors elicited from monocular depth estimations. The key lies in extracting and utilizing local semantic regions with a Structure-from-Motion (SfM) points anchored algorithm for reliable alignment and geometry cues. Then, to introduce multi-view constraints, we propose a series of geometry-guided supervision at virtual views in a fine-grained and coarse scheme to encourage 3D consistency and reduce overfitting. We also introduce a dataset and an in-the-wild experiment setting to set up more realistic benchmarks. We demonstrate that MS-GS achieves photorealistic renderings under various challenging sparse-view and multi-appearance conditions and outperforms existing approaches significantly across different datasets.

Method: GLip uses a dual-path feature extraction architecture integrating global and local features within a two-stage progressive learning framework. Stage 1 learns coarse alignment between visual features and speech units using audio-visual data. Stage 2 introduces a Contextual Enhancement Module to dynamically integrate local features with global context across spatial and temporal dimensions.

Result: The framework consistently outperforms existing methods on LRS2 and LRS3 benchmarks and demonstrates enhanced robustness against various visual challenges. It also shows effectiveness on a newly introduced challenging Mandarin dataset.

Conclusion: GLip’s progressive learning strategy that uniquely exploits discriminative local regions provides enhanced robustness against visual challenges and represents an effective approach for robust visual speech recognition in real-world scenarios.

Abstract: Visual speech recognition (VSR), also known as lip reading, is the task of recognizing speech from silent video. Despite significant advancements in VSR over recent decades, most existing methods pay limited attention to real-world visual challenges such as illumination variations, occlusions, blurring, and pose changes. To address these challenges, we propose GLip, a Global-Local Integrated Progressive framework designed for robust VSR. GLip is built upon two key insights: (i) learning an initial coarse alignment between visual features across varying conditions and corresponding speech content facilitates the subsequent learning of precise visual-to-speech mappings in challenging environments; (ii) under adverse conditions, certain local regions (e.g., non-occluded areas) often exhibit more discriminative cues for lip reading than global features. To this end, GLip introduces a dual-path feature extraction architecture that integrates both global and local features within a two-stage progressive learning framework. In the first stage, the model learns to align both global and local visual features with corresponding acoustic speech units using easily accessible audio-visual data, establishing a coarse yet semantically robust foundation. In the second stage, we introduce a Contextual Enhancement Module (CEM) to dynamically integrate local features with relevant global context across both spatial and temporal dimensions, refining the coarse representations into precise visual-speech mappings. Our framework uniquely exploits discriminative local regions through a progressive learning strategy, demonstrating enhanced robustness against various visual challenges and consistently outperforming existing methods on the LRS2 and LRS3 benchmarks. We further validate its effectiveness on a newly introduced challenging Mandarin dataset.

Method: Proposes a novel method integrating door detection with LLM-based reasoning using a Chain-of-Thought pipeline, being the first to apply LLMs to this task.

Result: Experiments on real-world and synthetic floor plan data demonstrate the method’s effectiveness, robustness, and good generalization across diverse datasets and facility types.

Conclusion: The proposed LLM-based approach with CoT reasoning successfully automates facility enumeration for building compliance checking, addressing a previously overlooked but critical component of the process.

Abstract: Building compliance checking (BCC) is a critical process for ensuring that constructed facilities meet regulatory standards. A core component of BCC is the accurate enumeration of facility types and their spatial distribution. Despite its importance, this problem has been largely overlooked in the literature, posing a significant challenge for BCC and leaving a critical gap in existing workflows. Performing this task manually is time-consuming and labor-intensive. Recent advances in large language models (LLMs) offer new opportunities to enhance automation by combining visual recognition with reasoning capabilities. In this paper, we introduce a new task for BCC: automated facility enumeration, which involves validating the quantity of each facility type against statutory requirements. To address it, we propose a novel method that integrates door detection with LLM-based reasoning. We are the first to apply LLMs to this task and further enhance their performance through a Chain-of-Thought (CoT) pipeline. Our approach generalizes well across diverse datasets and facility types. Experiments on both real-world and synthetic floor plan data demonstrate the effectiveness and robustness of our method.

Method: Proposes perceptually-weighted hierarchical 4D Gaussian representation with motion-aware adaptive grouping, plus end-to-end entropy-optimized training with layer-wise rate-distortion supervision and attribute-specific entropy modeling.

Result: Enables flexible quality and multiple bitrates within a single model, achieves real-time decoding and rendering on mobile devices, and outperforms existing methods in rate-distortion performance across multiple datasets.

Conclusion: 4DGCPro successfully addresses the challenges of volumetric video compression by providing efficient streaming, mobile compatibility, and superior compression performance in a unified framework.

Abstract: Achieving seamless viewing of high-fidelity volumetric video, comparable to 2D video experiences, remains an open challenge. Existing volumetric video compression methods either lack the flexibility to adjust quality and bitrate within a single model for efficient streaming across diverse networks and devices, or struggle with real-time decoding and rendering on lightweight mobile platforms. To address these challenges, we introduce 4DGCPro, a novel hierarchical 4D Gaussian compression framework that facilitates real-time mobile decoding and high-quality rendering via progressive volumetric video streaming in a single bitstream. Specifically, we propose a perceptually-weighted and compression-friendly hierarchical 4D Gaussian representation with motion-aware adaptive grouping to reduce temporal redundancy, preserve coherence, and enable scalable multi-level detail streaming. Furthermore, we present an end-to-end entropy-optimized training scheme, which incorporates layer-wise rate-distortion (RD) supervision and attribute-specific entropy modeling for efficient bitstream generation. Extensive experiments show that 4DGCPro enables flexible quality and multiple bitrate within a single model, achieving real-time decoding and rendering on mobile devices while outperforming existing methods in RD performance across multiple datasets. Project Page: https://mediax-sjtu.github.io/4DGCPro

Method: Reframes AIR as learned latent prior inference, formulating it as a structured reasoning paradigm with adaptive feature selection, spatial localization, and degradation semantics. Uses a lightweight decoding module that leverages latent encoded cues for spatially-adaptive restoration.

Result: Outperforms state-of-the-art approaches across six common degradation tasks, five compound settings, and previously unseen degradations, achieving average PSNR improvement of 1.68 dB while being three times more efficient.

Conclusion: The proposed method successfully addresses limitations of existing AIR approaches by automatically inferring degradation-aware representations from input without explicit task cues, demonstrating superior performance and efficiency.

Abstract: Real-world images often suffer from spatially diverse degradations such as haze, rain, snow, and low-light, significantly impacting visual quality and downstream vision tasks. Existing all-in-one restoration (AIR) approaches either depend on external text prompts or embed hand-crafted architectural priors (e.g., frequency heuristics); both impose discrete, brittle assumptions that weaken generalization to unseen or mixed degradations. To address this limitation, we propose to reframe AIR as learned latent prior inference, where degradation-aware representations are automatically inferred from the input without explicit task cues. Based on latent priors, we formulate AIR as a structured reasoning paradigm: (1) which features to route (adaptive feature selection), (2) where to restore (spatial localization), and (3) what to restore (degradation semantics). We design a lightweight decoding module that efficiently leverages these latent encoded cues for spatially-adaptive restoration. Extensive experiments across six common degradation tasks, five compound settings, and previously unseen degradations demonstrate that our method outperforms state-of-the-art (SOTA) approaches, achieving an average PSNR improvement of 1.68 dB while being three times more efficient.

Method: Deployed and evaluated conventional methods (Iterative Logistic Regression and Multilayer Perceptron) against advanced deep learning architectures (UNet and Spectral Channel Attention Network) for cloud and cloud shadow detection.

Result: Conventional methods struggled with spatial coherence and boundary definition. Deep learning models substantially improved detection quality: UNet performed best in preserving spatial structure, while SCAN excelled at capturing fine boundary details and surpassed UNet on MethaneSAT data.

Conclusion: Advanced deep learning architectures provide robust, scalable solutions for cloud and cloud shadow screening, enhancing methane emission quantification capacity for hyperspectral missions. Spectral attention mechanisms are particularly beneficial for satellite-specific features.

Abstract: Effective cloud and cloud shadow detection is a critical prerequisite for accurate retrieval of concentrations of atmospheric methane or other trace gases in hyperspectral remote sensing. This challenge is especially pertinent for MethaneSAT and for its airborne companion mission, MethaneAIR. In this study, we use machine learning methods to address the cloud and cloud shadow detection problem for sensors with these high spatial resolutions instruments. Cloud and cloud shadows in remote sensing data need to be effectively screened out as they bias methane retrievals in remote sensing imagery and impact the quantification of emissions. We deploy and evaluate conventional techniques including Iterative Logistic Regression (ILR) and Multilayer Perceptron (MLP), with advanced deep learning architectures, namely UNet and a Spectral Channel Attention Network (SCAN) method. Our results show that conventional methods struggle with spatial coherence and boundary definition, affecting the detection of clouds and cloud shadows. Deep learning models substantially improve detection quality: UNet performs best in preserving spatial structure, while SCAN excels at capturing fine boundary details. Notably, SCAN surpasses UNet on MethaneSAT data, underscoring the benefits of incorporating spectral attention for satellite specific features. This in depth assessment of various disparate machine learning techniques demonstrates the strengths and effectiveness of advanced deep learning architectures in providing robust, scalable solutions for clouds and cloud shadow screening towards enhancing methane emission quantification capacity of existing and next generation hyperspectral missions. Our data and code is publicly available at https://doi.org/10.7910/DVN/IKLZOJ

Method: Proposes HiPerformer with modular hierarchical encoder for parallel multi-source feature fusion, Local-Global Feature Fusion (LGFF) module for precise integration, and Progressive Pyramid Aggregation (PPA) module to replace traditional skip connections.

Result: Experiments on eleven public datasets show HiPerformer outperforms existing segmentation techniques with higher accuracy and robustness.

Conclusion: HiPerformer effectively addresses feature inconsistency problems in medical image segmentation through its innovative architecture and fusion modules, achieving superior performance compared to current methods.

Abstract: Both local details and global context are crucial in medical image segmentation, and effectively integrating them is essential for achieving high accuracy. However, existing mainstream methods based on CNN-Transformer hybrid architectures typically employ simple feature fusion techniques such as serial stacking, endpoint concatenation, or pointwise addition, which struggle to address the inconsistencies between features and are prone to information conflict and loss. To address the aforementioned challenges, we innovatively propose HiPerformer. The encoder of HiPerformer employs a novel modular hierarchical architecture that dynamically fuses multi-source features in parallel, enabling layer-wise deep integration of heterogeneous information. The modular hierarchical design not only retains the independent modeling capability of each branch in the encoder, but also ensures sufficient information transfer between layers, effectively avoiding the degradation of features and information loss that come with traditional stacking methods. Furthermore, we design a Local-Global Feature Fusion (LGFF) module to achieve precise and efficient integration of local details and global semantic information, effectively alleviating the feature inconsistency problem and resulting in a more comprehensive feature representation. To further enhance multi-scale feature representation capabilities and suppress noise interference, we also propose a Progressive Pyramid Aggregation (PPA) module to replace traditional skip connections. Experiments on eleven public datasets demonstrate that the proposed method outperforms existing segmentation techniques, demonstrating higher segmentation accuracy and robustness. The code is available at https://github.com/xzphappy/HiPerformer.

Method: Proposes FAST with two modules: Anomaly-Informed Accelerated Sampling (AIAS) for training-free accelerated sampling in 10 steps, and Foreground-Aware Reconstruction Module (FARM) that adaptively adjusts anomaly-aware noise in masked foreground regions.

Result: Extensive experiments on multiple industrial benchmarks show FAST consistently outperforms existing anomaly synthesis methods in downstream segmentation tasks.

Conclusion: FAST enables efficient, high-quality synthesis of structure-specific anomalies for industrial segmentation tasks through its foreground-aware approach and accelerated sampling.

Abstract: Industrial anomaly segmentation relies heavily on pixel-level annotations, yet real-world anomalies are often scarce, diverse, and costly to label. Segmentation-oriented industrial anomaly synthesis (SIAS) has emerged as a promising alternative; however, existing methods struggle to balance sampling efficiency and generation quality. Moreover, most approaches treat all spatial regions uniformly, overlooking the distinct statistical differences between anomaly and background areas. This uniform treatment hinders the synthesis of controllable, structure-specific anomalies tailored for segmentation tasks. In this paper, we propose FAST, a foreground-aware diffusion framework featuring two novel modules: the Anomaly-Informed Accelerated Sampling (AIAS) and the Foreground-Aware Reconstruction Module (FARM). AIAS is a training-free sampling algorithm specifically designed for segmentation-oriented industrial anomaly synthesis, which accelerates the reverse process through coarse-to-fine aggregation and enables the synthesis of state-of-the-art segmentation-oriented anomalies in as few as 10 steps. Meanwhile, FARM adaptively adjusts the anomaly-aware noise within the masked foreground regions at each sampling step, preserving localized anomaly signals throughout the denoising trajectory. Extensive experiments on multiple industrial benchmarks demonstrate that FAST consistently outperforms existing anomaly synthesis methods in downstream segmentation tasks. We release the code at: https://github.com/Chhro123/fast-foreground-aware-anomaly-synthesis.

Method: Represent all modalities (text, image, video) as unified token sequences and leverage self-attention for in-context learning and cross-modal knowledge transfer. Address data scarcity by creating a scalable pipeline that curates 232K video editing samples and combines them with large-scale image and video datasets for joint training.

Result: EditVerse achieves state-of-the-art performance, surpassing existing open-source and commercial models. It exhibits emergent editing and generation abilities across modalities and performs well on the newly created EditVerseBench benchmark for instruction-based video editing.

Conclusion: EditVerse successfully demonstrates that a unified framework can effectively handle both image and video generation and editing tasks, overcoming previous fragmentation and data scarcity issues through innovative token representation and scalable data curation.

Abstract: Recent advances in foundation models highlight a clear trend toward unification and scaling, showing emergent capabilities across diverse domains. While image generation and editing have rapidly transitioned from task-specific to unified frameworks, video generation and editing remain fragmented due to architectural limitations and data scarcity. In this work, we introduce EditVerse, a unified framework for image and video generation and editing within a single model. By representing all modalities, i.e., text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning, natural cross-modal knowledge transfer, and flexible handling of inputs and outputs with arbitrary resolutions and durations. To address the lack of video editing training data, we design a scalable data pipeline that curates 232K video editing samples and combines them with large-scale image and video datasets for joint training. Furthermore, we present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions. Extensive experiments and user studies demonstrate that EditVerse achieves state-of-the-art performance, surpassing existing open-source and commercial models, while exhibiting emergent editing and generation abilities across modalities.

Method: Proposes Neptune-X framework with X-to-Maritime generative model using Bidirectional Object-Water Attention for realistic scene synthesis, and Attribute-correlated Active Sampling for dynamic sample selection based on task relevance.

Result: The approach sets new benchmarks in maritime scene synthesis and significantly improves detection accuracy, especially in challenging and underrepresented settings.

Conclusion: Neptune-X effectively addresses maritime data scarcity and generalization issues through synthetic data generation and intelligent sample selection, with code publicly available.

Abstract: Maritime object detection is essential for navigation safety, surveillance, and autonomous operations, yet constrained by two key challenges: the scarcity of annotated maritime data and poor generalization across various maritime attributes (e.g., object category, viewpoint, location, and imaging environment). To address these challenges, we propose Neptune-X, a data-centric generative-selection framework that enhances training effectiveness by leveraging synthetic data generation with task-aware sample selection. From the generation perspective, we develop X-to-Maritime, a multi-modality-conditioned generative model that synthesizes diverse and realistic maritime scenes. A key component is the Bidirectional Object-Water Attention module, which captures boundary interactions between objects and their aquatic surroundings to improve visual fidelity. To further improve downstream tasking performance, we propose Attribute-correlated Active Sampling, which dynamically selects synthetic samples based on their task relevance. To support robust benchmarking, we construct the Maritime Generation Dataset, the first dataset tailored for generative maritime learning, encompassing a wide range of semantic conditions. Extensive experiments demonstrate that our approach sets a new benchmark in maritime scene synthesis, significantly improving detection accuracy, particularly in challenging and previously underrepresented settings. The code is available at https://github.com/gy65896/Neptune-X.

Method: For X/L/M/S variants: DINOv3-pretrained backbones + Spatial Tuning Adapter (converts single-scale to multi-scale features). For Nano/Pico/Femto/Atto: HGNetv2 with depth/width pruning + simplified decoder + upgraded Dense O2O.

Result: DEIMv2-X achieves 57.8 AP with 50.3M parameters (surpassing 56.5 AP with 60M+ parameters). DEIMv2-S is first sub-10M model (9.71M) to exceed 50 AP (50.9 AP). DEIMv2-Pico (1.5M) matches YOLOv10-Nano (2.3M) with 38.5 AP.

Conclusion: DEIMv2 establishes new SOTA across diverse scenarios with superior performance-cost trade-off, demonstrating effectiveness of unified design with DINOv3 features and efficient architecture adaptations.

Abstract: Benefiting from the simplicity and effectiveness of Dense O2O and MAL, DEIM has become the mainstream training framework for real-time DETRs, significantly outperforming the YOLO series. In this work, we extend it with DINOv3 features, resulting in DEIMv2. DEIMv2 spans eight model sizes from X to Atto, covering GPU, edge, and mobile deployment. For the X, L, M, and S variants, we adopt DINOv3-pretrained or distilled backbones and introduce a Spatial Tuning Adapter (STA), which efficiently converts DINOv3’s single-scale output into multi-scale features and complements strong semantics with fine-grained details to enhance detection. For ultra-lightweight models (Nano, Pico, Femto, and Atto), we employ HGNetv2 with depth and width pruning to meet strict resource budgets. Together with a simplified decoder and an upgraded Dense O2O, this unified design enables DEIMv2 to achieve a superior performance-cost trade-off across diverse scenarios, establishing new state-of-the-art results. Notably, our largest model, DEIMv2-X, achieves 57.8 AP with only 50.3 million parameters, surpassing prior X-scale models that require over 60 million parameters for just 56.5 AP. On the compact side, DEIMv2-S is the first sub-10 million model (9.71 million) to exceed the 50 AP milestone on COCO, reaching 50.9 AP. Even the ultra-lightweight DEIMv2-Pico, with just 1.5 million parameters, delivers 38.5 AP, matching YOLOv10-Nano (2.3 million) with around 50 percent fewer parameters. Our code and pre-trained models are available at https://github.com/Intellindust-AI-Lab/DEIMv2

Method: Introduces MOSS-ChatV with reinforcement learning framework using Dynamic Time Warping (DTW)-based process reward to align reasoning traces with temporally grounded references, and constructs MOSS-Video benchmark with annotated reasoning traces for training and evaluation.

Result: Achieves 87.2% on MOSS-Video test split, improves performance on MVBench and MMVU benchmarks, and shows consistent gains across different architectures (Qwen2.5-VL, Phi-2). GPT-4o-as-judge confirms more consistent and stable reasoning traces.

Conclusion: The DTW-based process reward framework effectively addresses process inconsistency in video reasoning, demonstrating broad applicability across architectures and producing more interpretable and robust reasoning traces.

Abstract: Video reasoning has emerged as a critical capability for multimodal large language models (MLLMs), requiring models to move beyond static perception toward coherent understanding of temporal dynamics in complex scenes. Yet existing MLLMs often exhibit process inconsistency, where intermediate reasoning drifts from video dynamics even when the final answer is correct, undermining interpretability and robustness. To address this issue, we introduce MOSS-ChatV, a reinforcement learning framework with a Dynamic Time Warping (DTW)-based process reward. This rule-based reward aligns reasoning traces with temporally grounded references, enabling efficient process supervision without auxiliary reward models. We further identify dynamic state prediction as a key measure of video reasoning and construct MOSS-Video, a benchmark with annotated reasoning traces, where the training split is used to fine-tune MOSS-ChatV and the held-out split is reserved for evaluation. MOSS-ChatV achieves 87.2% on MOSS-Video (test) and improves performance on general video benchmarks such as MVBench and MMVU. The framework consistently yields gains across different architectures, including Qwen2.5-VL and Phi-2, confirming its broad applicability. Evaluations with GPT-4o-as-judge further show that MOSS-ChatV produces more consistent and stable reasoning traces.

Method: Proposed three evaluation strategies: content filtering (removing deception-related text), score filtering (aggregating only task-relevant tokens), and prompt distilled fine-tuned model organisms (models trained to exhibit deceptive behavior without explicit prompting).

Result: Content filtering decreased probe AUROC by 30%, score filtering reduced AUROC by 15%, and fine-tuned model organisms reduced monitor performance by up to 40% even when re-trained.

Conclusion: The framework reveals significant performance inflation in white box monitors due to elicitation and reasoning leakage, and provides effective mitigation strategies to evaluate genuine detection capabilities.

Abstract: White box monitors that analyze model internals offer promising advantages for detecting potentially harmful behaviors in large language models, including lower computational costs and integration into layered defense systems.However, training and evaluating these monitors requires response exemplars that exhibit the target behaviors, typically elicited through prompting or fine-tuning. This presents a challenge when the information used to elicit behaviors inevitably leaks into the data that monitors ingest, inflating their effectiveness. We present a systematic framework for evaluating a monitor’s performance in terms of its ability to detect genuine model behavior rather than superficial elicitation artifacts. Furthermore, we propose three novel strategies to evaluate the monitor: content filtering (removing deception-related text from inputs), score filtering (aggregating only over task-relevant tokens), and prompt distilled fine-tuned model organisms (models trained to exhibit deceptive behavior without explicit prompting). Using deception detection as a representative case study, we identify two forms of leakage that inflate monitor performance: elicitation leakage from prompts that explicitly request harmful behavior, and reasoning leakage from models that verbalize their deceptive actions. Through experiments on multiple deception benchmarks, we apply our proposed mitigation strategies and measure performance retention. Our evaluation of the monitors reveal three crucial findings: (1) Content filtering is a good mitigation strategy that allows for a smooth removal of elicitation signal and can decrease probe AUROC by 30% (2) Score filtering was found to reduce AUROC by 15% but is not as straightforward to attribute to (3) A finetuned model organism improves monitor evaluations but reduces their performance by upto 40%, even when re-trained.

Method: Self-training pipeline that samples diverse reasoning paths, mines shared decision pivots, compresses traces into pivot-focused short-path reasoning using an auxiliary verifier, and post-trains the model using self-generated outputs.

Result: Experiments on LogiQA, MedQA, and MATH500 benchmarks demonstrate the method’s effectiveness in aligning reasoning without ground truth data or external metrics.

Conclusion: Decision pivots provide a scalable way to verify and align LLM reasoning by identifying essential checkpoints that correct reasoning paths must satisfy, enabling effective self-training without external supervision.

Abstract: Chain-of-thought (CoT) reasoning exposes the intermediate thinking process of large language models (LLMs), yet verifying those traces at scale remains unsolved. In response, we introduce the idea of decision pivots-minimal, verifiable checkpoints that any correct reasoning path must visit. We hypothesize that correct reasoning, though stylistically diverse, converge on the same pivot set, while incorrect ones violate at least one pivot. Leveraging this property, we propose a self-training pipeline that (i) samples diverse reasoning paths and mines shared decision pivots, (ii) compresses each trace into pivot-focused short-path reasoning using an auxiliary verifier, and (iii) post-trains the model using its self-generated outputs. The proposed method aligns reasoning without ground truth reasoning data or external metrics. Experiments on standard benchmarks such as LogiQA, MedQA, and MATH500 show the effectiveness of our method.

Method: Integration of a curated knowledge graph with AI agents powered by generative AI services, leveraging cloud-native workflows and existing API data portals.

Result: The system drastically lowers technical thresholds, enabling non-specialist users to identify and analyze relevant climate datasets through natural language interaction.

Conclusion: This approach demonstrates a pathway toward democratizing climate data access and establishing reproducible, extensible human-AI collaboration frameworks in scientific research.

Abstract: Climate data science faces persistent barriers stemming from the fragmented nature of data sources, heterogeneous formats, and the steep technical expertise required to identify, acquire, and process datasets. These challenges limit participation, slow discovery, and reduce the reproducibility of scientific workflows. In this paper, we present a proof of concept for addressing these barriers through the integration of a curated knowledge graph (KG) with AI agents designed for cloud-native scientific workflows. The KG provides a unifying layer that organizes datasets, tools, and workflows, while AI agents – powered by generative AI services – enable natural language interaction, automated data access, and streamlined analysis. Together, these components drastically lower the technical threshold for engaging in climate data science, enabling non-specialist users to identify and analyze relevant datasets. By leveraging existing cloud-ready API data portals, we demonstrate that “a knowledge graph is all you need” to unlock scalable and agentic workflows for scientific inquiry. The open-source design of our system further supports community contributions, ensuring that the KG and associated tools can evolve as a shared commons. Our results illustrate a pathway toward democratizing access to climate data and establishing a reproducible, extensible framework for human–AI collaboration in scientific research.

Method: Extracted and cleaned EEG features from NeuMa dataset, created brain connectivity features for GNN models, and compared various machine learning models including classical models (ensemble models, SVM) and GNNs with different architectures.

Result: No significant overall performance difference between models, but GNN models generally performed better in some basic criteria where classical models were unsatisfactory.

Conclusion: EEG signal analysis combined with machine learning provides deeper understanding of consumer behavior, and GNNs show promise as an alternative to traditional models in EEG-based neuromarketing applications.

Abstract: Prediction of consumer behavior is one of the important purposes in marketing, cognitive neuroscience, and human-computer interaction. The electroencephalography (EEG) data can help analyze the decision process by providing detailed information about the brain’s neural activity. In this research, a comparative approach is utilized for predicting consumer behavior by EEG data. In the first step, the features of the EEG data from the NeuMa dataset were extracted and cleaned. For the Graph Neural Network (GNN) models, the brain connectivity features were created. Different machine learning models, such as classical models and Graph Neural Networks, are used and compared. The GNN models with different architectures are implemented to have a comprehensive comparison; furthermore, a wide range of classical models, such as ensemble models, are applied, which can be very helpful to show the difference and performance of each model on the dataset. Although the results did not show a significant difference overall, the GNN models generally performed better in some basic criteria where classical models were not satisfactory. This study not only shows that combining EEG signal analysis and machine learning models can provide an approach to deeper understanding of consumer behavior, but also provides a comprehensive comparison between the machine learning models that have been widely used in previous studies in the EEG-based neuromarketing such as Support Vector Machine (SVM), and the models which are not used or rarely used in the field, like Graph Neural Networks.

Method: Two nested loops: inner loop uses code evolver to generate/mutate candidate solutions, outer agentic controller evaluates elites and queries GeoKnowRAG module (structured geospatial knowledge base) to inject theoretical priors from geography, enabling knowledge-guided evolution.

Result: Reduced spatial interpolation error (RMSE) by 13-21%, enhanced uncertainty estimation performance by 17%, and discovered new algorithms incorporating geospatial theory on top of classical models.

Conclusion: GeoEvolve provides a scalable path toward automated, knowledge-driven geospatial modeling, opening new opportunities for trustworthy and efficient AI-for-Science discovery, with domain-guided retrieval being essential for stable, high-quality evolution.

Abstract: Geospatial modeling provides critical solutions for pressing global challenges such as sustainability and climate change. Existing large language model (LLM)-based algorithm discovery frameworks, such as AlphaEvolve, excel at evolving generic code but lack the domain knowledge and multi-step reasoning required for complex geospatial problems. We introduce GeoEvolve, a multi-agent LLM framework that couples evolutionary search with geospatial domain knowledge to automatically design and refine geospatial algorithms. GeoEvolve operates in two nested loops: an inner loop leverages a code evolver to generate and mutate candidate solutions, while an outer agentic controller evaluates global elites and queries a GeoKnowRAG module – a structured geospatial knowledge base that injects theoretical priors from geography. This knowledge-guided evolution steers the search toward theoretically meaningful and computationally efficient algorithms. We evaluate GeoEvolve on two fundamental and classical tasks: spatial interpolation (kriging) and spatial uncertainty quantification (geospatial conformal prediction). Across these benchmarks, GeoEvolve automatically improves and discovers new algorithms, incorporating geospatial theory on top of classical models. It reduces spatial interpolation error (RMSE) by 13-21% and enhances uncertainty estimation performance by 17%. Ablation studies confirm that domain-guided retrieval is essential for stable, high-quality evolution. These results demonstrate that GeoEvolve provides a scalable path toward automated, knowledge-driven geospatial modeling, opening new opportunities for trustworthy and efficient AI-for-Science discovery.

Method: Uses deep learning to extract survival-relevant features from CT imaging (interpreted via Grad-CAM) and clinical variables (modeled as symbolic expressions through genetic programming), with risk estimation via transparent Cox regression for patient stratification.

Result: Achieved C-index of 0.838 for prediction and 0.826 for stratification on RADCURE head and neck cancer dataset, outperforming clinical and academic baseline approaches while aligning with known prognostic markers.

Conclusion: MultiFIX demonstrates the promise of interpretable multimodal AI for precision oncology by providing both high accuracy and transparency in survival analysis.

Abstract: Accurate and interpretable survival analysis remains a core challenge in oncology. With growing multimodal data and the clinical need for transparent models to support validation and trust, this challenge increases in complexity. We propose an interpretable multimodal AI framework to automate survival analysis by integrating clinical variables and computed tomography imaging. Our MultiFIX-based framework uses deep learning to infer survival-relevant features that are further explained: imaging features are interpreted via Grad-CAM, while clinical variables are modeled as symbolic expressions through genetic programming. Risk estimation employs a transparent Cox regression, enabling stratification into groups with distinct survival outcomes. Using the open-source RADCURE dataset for head and neck cancer, MultiFIX achieves a C-index of 0.838 (prediction) and 0.826 (stratification), outperforming the clinical and academic baseline approaches and aligning with known prognostic markers. These results highlight the promise of interpretable multimodal AI for precision oncology with MultiFIX.

Method: Uses a two-step precision-recall formulation with a label similarity matrix to compute soft precision and recall scores, enabling comparison of label sets of arbitrary sizes without discarding labels or forcing matches.

Result: Semantic F1 demonstrates greater interpretability and ecological validity through theoretical justification and empirical validation on synthetic and real data.

Conclusion: Semantic F1 provides fairer evaluations by recognizing category overlap and annotator disagreement, and is applicable across tasks and modalities since it only requires a domain-appropriate similarity matrix rather than a rigid ontology.

Abstract: We propose Semantic F1 Scores, novel evaluation metrics for subjective or fuzzy multi-label classification that quantify semantic relatedness between predicted and gold labels. Unlike the conventional F1 metrics that treat semantically related predictions as complete failures, Semantic F1 incorporates a label similarity matrix to compute soft precision-like and recall-like scores, from which the Semantic F1 scores are derived. Unlike existing similarity-based metrics, our novel two-step precision-recall formulation enables the comparison of label sets of arbitrary sizes without discarding labels or forcing matches between dissimilar labels. By granting partial credit for semantically related but nonidentical labels, Semantic F1 better reflects the realities of domains marked by human disagreement or fuzzy category boundaries. In this way, it provides fairer evaluations: it recognizes that categories overlap, that annotators disagree, and that downstream decisions based on similar predictions lead to similar outcomes. Through theoretical justification and extensive empirical validation on synthetic and real data, we show that Semantic F1 demonstrates greater interpretability and ecological validity. Because it requires only a domain-appropriate similarity matrix, which is robust to misspecification, and not a rigid ontology, it is applicable across tasks and modalities.

Method: Create photorealistic images/videos of safe-to-unsafe transitions using generative models, analyze major foundation models’ risk perception and safety reasoning, and develop post-training paradigm with explicit safety constraint reasoning.

Result: Comprehensive analysis reveals deployment readiness of foundation models for safety-critical applications, with post-trained models achieving state-of-the-art constraint satisfaction performance.

Conclusion: The benchmark enables continuous physical safety evaluation of Embodied AI, and the post-training approach makes safety reasoning interpretable and transparent while improving performance.

Abstract: When AI interacts with the physical world – as a robot or an assistive agent – new safety challenges emerge beyond those of purely ``digital AI”. In such interactions, the potential for physical harm is direct and immediate. How well do state-of-the-art foundation models understand common-sense facts about physical safety, e.g. that a box may be too heavy to lift, or that a hot cup of coffee should not be handed to a child? In this paper, our contributions are three-fold: first, we develop a highly scalable approach to continuous physical safety benchmarking of Embodied AI systems, grounded in real-world injury narratives and operational safety constraints. To probe multi-modal safety understanding, we turn these narratives and constraints into photorealistic images and videos capturing transitions from safe to unsafe states, using advanced generative models. Secondly, we comprehensively analyze the ability of major foundation models to perceive risks, reason about safety, and trigger interventions; this yields multi-faceted insights into their deployment readiness for safety-critical agentic applications. Finally, we develop a post-training paradigm to teach models to explicitly reason about embodiment-specific safety constraints provided through system instructions. The resulting models generate thinking traces that make safety reasoning interpretable and transparent, achieving state of the art performance in constraint satisfaction evaluations. The benchmark will be released at https://asimov-benchmark.github.io/v2

Method: Developed a framework for modeling decisions with axioms and defined a taxonomy based on structural properties.

Result: Discovered the Decision-Evaluation Paradox - a tension between using axioms to make decisions versus evaluating decisions.

Conclusion: The paradox shows careful consideration is needed when training models on decision data or applying axioms for decision-making and evaluation.

Abstract: We introduce a framework for modeling decisions with axioms that are statements about decisions, e.g., ethical constraints. Using our framework we define a taxonomy of decision axioms based on their structural properties and demonstrate a tension between the use of axioms to make decisions and the use of axioms to evaluate decisions which we call the Decision-Evaluation Paradox. We argue that the Decision-Evaluation Paradox arises with realistic axiom structures, and the paradox illuminates why one must be exceptionally careful when training models on decision data or applying axioms to make and evaluate decisions.

Method: Three-stage approach: 1) Train multilingual TTS baseline with IPA tokens, 2) Fine-tune on limited paired data for target language prosody, 3) Apply GRPO optimization using unpaired text and speaker prompts with multi-objective rewards from ASR, speaker verification, and audio quality models.

Result: Produces intelligible and speaker-consistent speech in low-resource languages, substantially outperforming fine-tuning alone. Also improves TTS in high-resource languages, surpassing DPO with better intelligibility, speaker similarity, and audio quality.

Conclusion: GRPO-based framework effectively adapts TTS models to low-resource languages using accessible ASR models and unpaired data, achieving superior performance compared to traditional methods.

Abstract: Developing high-quality text-to-speech (TTS) systems for low-resource languages is challenging due to the scarcity of paired text and speech data. In contrast, automatic speech recognition (ASR) models for such languages are often more accessible, owing to large-scale multilingual pre-training efforts. We propose a framework based on Group Relative Policy Optimization (GRPO) to adapt an autoregressive, multilingual TTS model to new languages. Our method first establishes a language-agnostic foundation for TTS synthesis by training a multilingual baseline with International Phonetic Alphabet (IPA) tokens. Next, we fine-tune this model on limited paired data of the new languages to capture the target language’s prosodic features. Finally, we apply GRPO to optimize the model using only unpaired text and speaker prompts, guided by a multi-objective reward from pretrained ASR, speaker verification, and audio quality estimation models. Experiments demonstrate that this pipeline produces intelligible and speaker-consistent speech in low-resource languages, substantially outperforming fine-tuning alone. Furthermore, our GRPO-based framework also improves TTS performance in high-resource languages, surpassing offline alignment methods such as Direct Preference Optimization (DPO) yielding superior intelligibility, speaker similarity, and audio quality.

Method: Organizes reasoning steps into a thought graph with sequential and semantic edges, retrieves query-relevant nodes, and applies reward-guided traversal to assemble problem-specific templates that guide generation.

Result: Substantial efficiency gains with small prompt growth: up to 40% reduction in output tokens, 82% reduction in inference latency, and 59% reduction in cost while maintaining accuracy.

Conclusion: RoT establishes a scalable paradigm for efficient large reasoning model reasoning via dynamic template construction through retrieval.

Abstract: Large reasoning models improve accuracy by producing long reasoning traces, but this inflates latency and cost, motivating inference-time efficiency. We propose Retrieval-of-Thought (RoT), which reuses prior reasoning as composable ``thought" steps to guide new problems. RoT organizes steps into a thought graph with sequential and semantic edges to enable fast retrieval and flexible recombination. At inference, RoT retrieves query-relevant nodes and applies reward-guided traversal to assemble a problem-specific template that guides generation. This dynamic template reuse reduces redundant exploration and, therefore, reduces output tokens while preserving accuracy. We evaluate RoT on reasoning benchmarks with multiple models, measuring accuracy, token usage, latency, and memory overhead. Findings show small prompt growth but substantial efficiency gains, with RoT reducing output tokens by up to 40%, inference latency by 82%, and cost by 59% while maintaining accuracy. RoT establishes a scalable paradigm for efficient LRM reasoning via dynamic template construction through retrieval.

Method: Introduces Shachi framework that decomposes agent policies into core cognitive components: Configuration (intrinsic traits), Memory (contextual persistence), and Tools (expanded capabilities), all orchestrated by an LLM reasoning engine.

Result: Validated on 10-task benchmark; demonstrated external validity by modeling real-world U.S. tariff shock showing agent behaviors align with observed market reactions only when properly configured with memory and tools.

Conclusion: Provides a rigorous, open-source foundation for building and evaluating LLM agents to foster more cumulative and scientifically grounded research in multi-agent systems.

Abstract: The study of emergent behaviors in large language model (LLM)-driven multi-agent systems is a critical research challenge, yet progress is limited by a lack of principled methodologies for controlled experimentation. To address this, we introduce Shachi, a formal methodology and modular framework that decomposes an agent’s policy into core cognitive components: Configuration for intrinsic traits, Memory for contextual persistence, and Tools for expanded capabilities, all orchestrated by an LLM reasoning engine. This principled architecture moves beyond brittle, ad-hoc agent designs and enables the systematic analysis of how specific architectural choices influence collective behavior. We validate our methodology on a comprehensive 10-task benchmark and demonstrate its power through novel scientific inquiries. Critically, we establish the external validity of our approach by modeling a real-world U.S. tariff shock, showing that agent behaviors align with observed market reactions only when their cognitive architecture is appropriately configured with memory and tools. Our work provides a rigorous, open-source foundation for building and evaluating LLM agents, aimed at fostering more cumulative and scientifically grounded research.

Method: LLR-BC framework that consolidates prior knowledge by aligning behaviors of new task solver with buffered ones in decision-seeking way, with greater weights for low-confidence decisions.

Result: Extensive experiments on CVRP and TSP show LLR-BC effectively trains high-performance neural solvers, addresses catastrophic forgetting, maintains plasticity, and improves zero-shot generalization.

Conclusion: LLR-BC provides an effective lifelong learning solution for neural routing problem solvers, enabling continuous learning without forgetting previous knowledge.

Abstract: Recent neural solvers have demonstrated promising performance in learning to solve routing problems. However, existing studies are primarily based on one-off training on one or a set of predefined problem distributions and scales, i.e., tasks. When a new task arises, they typically rely on either zero-shot generalization, which may be poor due to the discrepancies between the new task and the training task(s), or fine-tuning the pretrained solver on the new task, which possibly leads to catastrophic forgetting of knowledge acquired from previous tasks. This paper explores a novel lifelong learning paradigm for neural VRP solvers, where multiple tasks with diverse distributions and scales arise sequentially over time. Solvers are required to effectively and efficiently learn to solve new tasks while maintaining their performance on previously learned tasks. Consequently, a novel framework called Lifelong Learning Router with Behavior Consolidation (LLR-BC) is proposed. LLR-BC consolidates prior knowledge effectively by aligning behaviors of the solver trained on a new task with the buffered ones in a decision-seeking way. To encourage more focus on crucial experiences, LLR-BC assigns greater consolidated weights to decisions with lower confidence. Extensive experiments on capacitated vehicle routing problems and traveling salesman problems demonstrate LLR-BC’s effectiveness in training high-performance neural solvers in a lifelong learning setting, addressing the catastrophic forgetting issue, maintaining their plasticity, and improving zero-shot generalization ability.

Method: CoBel-World uses a collaborative belief world with symbolic belief language for structured knowledge representation and zero-shot Bayesian-style belief updates through LLM reasoning.

Result: Reduced communication costs by 22-60% and improved task completion efficiency by 4-28% on TDW-MAT and C-WAH benchmarks compared to strongest baselines.

Conclusion: Explicit, intent-aware belief modeling is essential for efficient and human-like collaboration in LLM-based multi-agent systems.

Abstract: Effective real-world multi-agent collaboration requires not only accurate planning but also the ability to reason about collaborators’ intents – a crucial capability for avoiding miscoordination and redundant communication under partial observable environments. Due to their strong planning and reasoning capabilities, large language models (LLMs) have emerged as promising autonomous agents for collaborative task solving. However, existing collaboration frameworks for LLMs overlook their reasoning potential for dynamic intent inference, and thus produce inconsistent plans and redundant communication, reducing collaboration efficiency. To bridge this gap, we propose CoBel-World, a novel framework that equips LLM agents with a collaborative belief world – an internal representation jointly modeling the physical environment and collaborators’ mental states. CoBel-World enables agents to parse open-world task knowledge into structured beliefs via a symbolic belief language, and perform zero-shot Bayesian-style belief updates through LLM reasoning. This allows agents to proactively detect potential miscoordination (e.g., conflicting plans) and communicate adaptively. Evaluated on challenging embodied benchmarks (i.e., TDW-MAT and C-WAH), CoBel-World significantly reduces communication costs by 22-60% and improves task completion efficiency by 4-28% compared to the strongest baseline. Our results show that explicit, intent-aware belief modeling is essential for efficient and human-like collaboration in LLM-based multi-agent systems.

Method: Introduced UltraHorizon benchmark with exploration tasks across three environments where agents must iteratively uncover hidden rules through sustained reasoning, planning, memory management, and tool interactions. Tasks involve 35k-200k+ tokens and 60-400+ tool calls.

Result: LLM-agents consistently underperform in long-horizon settings while human participants achieve higher scores. Simple scaling fails to improve performance. Analysis identified eight error types attributed to in-context locking and functional capability gaps.

Conclusion: There’s a persistent gap in agents’ long-horizon abilities that current scaling approaches cannot overcome, highlighting the need for benchmarks that capture complex real-world challenges.

Abstract: Autonomous agents have recently achieved remarkable progress across diverse domains, yet most evaluations focus on short-horizon, fully observable tasks. In contrast, many critical real-world tasks, such as large-scale software development, commercial investment, and scientific discovery, unfold in long-horizon and partially observable scenarios where success hinges on sustained reasoning, planning, memory management, and tool use. Existing benchmarks rarely capture these long-horizon challenges, leaving a gap in systematic evaluation. To bridge this gap, we introduce \textbf{UltraHorizon} a novel benchmark that measures the foundational capabilities essential for complex real-world challenges. We use exploration as a unifying task across three distinct environments to validate these core competencies. Agents are designed in long-horizon discovery tasks where they must iteratively uncover hidden rules through sustained reasoning, planning, memory and tools management, and interaction with environments. Under the heaviest scale setting, trajectories average \textbf{200k+} tokens and \textbf{400+} tool calls, whereas in standard configurations they still exceed \textbf{35k} tokens and involve more than \textbf{60} tool calls on average. Our extensive experiments reveal that LLM-agents consistently underperform in these settings, whereas human participants achieve higher scores, underscoring a persistent gap in agents’ long-horizon abilities. We also observe that simple scaling fails in our task. To better illustrate the failure of agents, we conduct an in-depth analysis of collected trajectories. We identify eight types of errors and attribute them to two primary causes: in-context locking and functional fundamental capability gaps. \href{https://github.com/StarDewXXX/UltraHorizon}{Our code will be available here.}

Method: Combines structured two-level planning framework with task mining from user behavior logs. Constructs high-level plans by mapping commands to task dictionary, then grounds them into low-level action sequences using real-time GUI context. Uses task mining to identify user-specific patterns for personalization.

Result: Evaluated on 200 real-world tasks, showing significant improvements in task success rate and execution time. Maintains over 60.0% success rate on long-horizon task sequences, demonstrating robustness in complex multi-step workflows.

Conclusion: Log2Plan provides a robust and adaptable GUI automation solution that effectively handles complex tasks and UI variations through its two-level planning and task mining approach.

Abstract: GUI task automation streamlines repetitive tasks, but existing LLM or VLM-based planner-executor agents suffer from brittle generalization, high latency, and limited long-horizon coherence. Their reliance on single-shot reasoning or static plans makes them fragile under UI changes or complex tasks. Log2Plan addresses these limitations by combining a structured two-level planning framework with a task mining approach over user behavior logs, enabling robust and adaptable GUI automation. Log2Plan constructs high-level plans by mapping user commands to a structured task dictionary, enabling consistent and generalizable automation. To support personalization and reuse, it employs a task mining approach from user behavior logs that identifies user-specific patterns. These high-level plans are then grounded into low-level action sequences by interpreting real-time GUI context, ensuring robust execution across varying interfaces. We evaluated Log2Plan on 200 real-world tasks, demonstrating significant improvements in task success rate and execution time. Notably, it maintains over 60.0% success rate even on long-horizon task sequences, highlighting its robustness in complex, multi-step workflows.

Method: Constructed benchmark from 729 websites with 3799 question-answer pairs across 8 tasks. Used standardized prompts, deterministic evaluation scripts, and multi-stage quality control with automatic checks and human verification.

Result: Evaluation of 12 MLLMs revealed significant gaps: models struggle with compositional and cross-element reasoning, show limited robustness to UI perturbations, and are overly conservative in safety-critical action recognition.

Conclusion: Current MLLMs have substantial limitations in web understanding capabilities, particularly in reasoning, robustness, and safety, highlighting the need for continued research in these areas.

Abstract: Multimodal large language models (MLLMs) are increasingly positioned as AI collaborators for building complex web-related applications like GUI agents and front-end code generation. However, existing benchmarks largely emphasize visual perception or UI code generation, showing insufficient evaluation on the reasoning, robustness and safety capability required for end-to-end web applications. To bridge the gap, we introduce a comprehensive web understanding benchmark, named WebRSSBench, that jointly evaluates Reasoning, Robustness, and Safety across eight tasks, such as position relationship reasoning, color robustness, and safety critical detection, etc. The benchmark is constructed from 729 websites and contains 3799 question answer pairs that probe multi-step inference over page structure, text, widgets, and safety-critical interactions. To ensure reliable measurement, we adopt standardized prompts, deterministic evaluation scripts, and multi-stage quality control combining automatic checks with targeted human verification. We evaluate 12 MLLMs on WebRSSBench. The results reveal significant gaps, models still struggle with compositional and cross-element reasoning over realistic layouts, show limited robustness when facing perturbations in user interfaces and content such as layout rearrangements or visual style shifts, and are rather conservative in recognizing and avoiding safety critical or irreversible actions. Our code is available at https://github.com/jinliang-byte/webssrbench.

Method: D-Artemis employs a cognitive loop with three stages: Thinking (using app-specific tip retrieval), Alignment (with Thought-Action Consistency Check and Action Correction Agent for pre-execution validation), and Reflection (with Status Reflection Agent for post-execution learning). It enhances general-purpose MLLMs without requiring training on complex trajectory datasets.

Result: Achieves 75.8% success rate on AndroidWorld and 96.8% on ScreenSpot-V2 benchmarks, establishing new state-of-the-art results. Ablation studies confirm significant contributions from each framework component.

Conclusion: D-Artemis demonstrates strong generalization capabilities for GUI tasks by leveraging a deliberative cognitive framework that mitigates execution risks and enables strategic learning, while avoiding the need for extensive training on complex datasets.

Abstract: Graphical User Interface (GUI) agents aim to automate a wide spectrum of human tasks by emulating user interaction. Despite rapid advancements, current approaches are hindered by several critical challenges: data bottleneck in end-to-end training, high cost of delayed error detection, and risk of contradictory guidance. Inspired by the human cognitive loop of Thinking, Alignment, and Reflection, we present D-Artemis – a novel deliberative framework in this paper. D-Artemis leverages a fine-grained, app-specific tip retrieval mechanism to inform its decision-making process. It also employs a proactive Pre-execution Alignment stage, where Thought-Action Consistency (TAC) Check module and Action Correction Agent (ACA) work in concert to mitigate the risk of execution failures. A post-execution Status Reflection Agent (SRA) completes the cognitive loop, enabling strategic learning from experience. Crucially, D-Artemis enhances the capabilities of general-purpose Multimodal large language models (MLLMs) for GUI tasks without the need for training on complex trajectory datasets, demonstrating strong generalization. D-Artemis establishes new state-of-the-art (SOTA) results across both major benchmarks, achieving a 75.8% success rate on AndroidWorld and 96.8% on ScreenSpot-V2. Extensive ablation studies further demonstrate the significant contribution of each component to the framework.

Method: ProRe uses a general-purpose reasoner to schedule targeted state probing tasks, which domain-specific evaluator agents execute by actively interacting with the environment to collect additional observations for more accurate reward assignment.

Result: Empirical results on over 3K trajectories show ProRe improves reward accuracy and F1 score by up to 5.3% and 19.4% respectively, and integration with state-of-the-art policy agents yields up to 22.4% success rate improvement.

Conclusion: ProRe effectively addresses the limitations of existing reward methods for GUI agents by enabling proactive environment interaction and observation collection, leading to significantly improved reward accuracy and agent performance.

Abstract: Reward is critical to the evaluation and training of large language models (LLMs). However, existing rule-based or model-based reward methods struggle to generalize to GUI agents, where access to ground-truth trajectories or application databases is often unavailable, and static trajectory-based LLM-as-a-Judge approaches suffer from limited accuracy. To address these challenges, we propose ProRe, a proactive reward system that leverages a general-purpose reasoner and domain-specific evaluator agents (actors). The reasoner schedules targeted state probing tasks, which the evaluator agents then execute by actively interacting with the environment to collect additional observations. This enables the reasoner to assign more accurate and verifiable rewards to GUI agents. Empirical results on over 3K trajectories demonstrate that ProRe improves reward accuracy and F1 score by up to 5.3% and 19.4%, respectively. Furthermore, integrating ProRe with state-of-the-art policy agents yields a success rate improvement of up to 22.4%.

Method: DS-STAR features: (1) data file analysis module for exploring diverse data formats, (2) LLM-based judge for verifying analysis plan sufficiency at each stage, and (3) sequential planning mechanism that starts simple and iteratively refines plans based on feedback.

Result: DS-STAR achieves state-of-the-art performance across DABStep, KramaBench, and DA-Code benchmarks, particularly outperforming baselines on hard tasks requiring processing multiple data files with heterogeneous formats.

Conclusion: The iterative refinement approach allows DS-STAR to reliably navigate complex analyses involving diverse data sources, demonstrating superior performance in automated data science tasks.

Abstract: Data science, which transforms raw data into actionable insights, is critical for data-driven decision-making. However, these tasks are often complex, involving steps for exploring multiple data sources and synthesizing findings to deliver insightful answers. While large language models (LLMs) show significant promise in automating this process, they often struggle with heterogeneous data formats and generate sub-optimal analysis plans, as verifying plan sufficiency is inherently difficult without ground-truth labels for such open-ended tasks. To overcome these limitations, we introduce DS-STAR, a novel data science agent. Specifically, DS-STAR makes three key contributions: (1) a data file analysis module that automatically explores and extracts context from diverse data formats, including unstructured types; (2) a verification step where an LLM-based judge evaluates the sufficiency of the analysis plan at each stage; and (3) a sequential planning mechanism that starts with a simple, executable plan and iteratively refines it based on the DS-STAR’s feedback until its sufficiency is verified. This iterative refinement allows DS-STAR to reliably navigate complex analyses involving diverse data sources. Our experiments show that DS-STAR achieves state-of-the-art performance across three challenging benchmarks: DABStep, KramaBench, and DA-Code. Moreover, DS-STAR particularly outperforms baselines on hard tasks that require processing multiple data files with heterogeneous formats.

Method: The framework includes: 1) A sandbox environment with cached transportation, accommodation, and POI data; 2) Hierarchical reward modeling with trajectory-level and turn-level verifiers; 3) Reply-augmented reinforcement learning with failure experience replay.

Result: DeepTravel enables small LLMs (Qwen3 32B) to significantly outperform frontier LLMs like OpenAI o1, o3 and DeepSeek R1 in travel planning tasks, as demonstrated through online and offline evaluations on DiDi Enterprise Solutions App.

Conclusion: The proposed DeepTravel framework successfully creates autonomous travel planning agents capable of flexible planning and reasoning, achieving superior performance compared to existing large language models.

Abstract: Travel planning (TP) agent has recently worked as an emerging building block to interact with external tools and resources for travel itinerary generation, ensuring enjoyable user experience. Despite its benefits, existing studies rely on hand craft prompt and fixed agent workflow, hindering more flexible and autonomous TP agent. This paper proposes DeepTravel, an end to end agentic reinforcement learning framework for building autonomous travel planning agent, capable of autonomously planning, executing tools, and reflecting on tool responses to explore, verify, and refine intermediate actions in multi step reasoning. To achieve this, we first construct a robust sandbox environment by caching transportation, accommodation and POI data, facilitating TP agent training without being constrained by real world APIs limitations (e.g., inconsistent outputs). Moreover, we develop a hierarchical reward modeling system, where a trajectory level verifier first checks spatiotemporal feasibility and filters unsatisfied travel itinerary, and then the turn level verifier further validate itinerary detail consistency with tool responses, enabling efficient and precise reward service. Finally, we propose the reply augmented reinforcement learning method that enables TP agent to periodically replay from a failures experience buffer, emerging notable agentic capacity. We deploy trained TP agent on DiDi Enterprise Solutions App and conduct comprehensive online and offline evaluations, demonstrating that DeepTravel enables small size LLMs (e.g., Qwen3 32B) to significantly outperform existing frontier LLMs such as OpenAI o1, o3 and DeepSeek R1 in travel planning tasks.

Method: TRACE uses a Hierarchical Transformer that mirrors step-by-step computation flow and introduces function shift learning - predicting only the discrepancy between true global function and simple local approximation rather than the complex function directly.

Result: TRACE substantially outperforms all prior architectures across comprehensive benchmarks on electronic circuits, one of the most complex computational graph classes.

Conclusion: The architecturally-aligned backbone and decoupled learning objective form a more robust paradigm for the fundamental challenge of learning to compute on graphs.

Abstract: Learning to compute, the ability to model the functional behavior of a computational graph, is a fundamental challenge for graph representation learning. Yet, the dominant paradigm is architecturally mismatched for this task. This flawed assumption, central to mainstream message passing neural networks (MPNNs) and their conventional Transformer-based counterparts, prevents models from capturing the position-aware, hierarchical nature of computation. To resolve this, we introduce \textbf{TRACE}, a new paradigm built on an architecturally sound backbone and a principled learning objective. First, TRACE employs a Hierarchical Transformer that mirrors the step-by-step flow of computation, providing a faithful architectural backbone that replaces the flawed permutation-invariant aggregation. Second, we introduce \textbf{function shift learning}, a novel objective that decouples the learning problem. Instead of predicting the complex global function directly, our model is trained to predict only the \textit{function shift}, the discrepancy between the true global function and a simple local approximation that assumes input independence. We validate this paradigm on electronic circuits, one of the most complex and economically critical classes of computational graphs. Across a comprehensive suite of benchmarks, TRACE substantially outperforms all prior architectures. These results demonstrate that our architecturally-aligned backbone and decoupled learning objective form a more robust paradigm for the fundamental challenge of learning to compute on graphs.

Method: Combines symbolic deduction engine DDARN (accelerated 120x through theorem matching and C++ implementation) with neuro-symbolic approach using Qwen3-0.6B-Base model, plus dual-model ensemble for enhanced performance.

Result: Solves 24/30 problems (IMO silver level) with single model and 26/30 problems (IMO gold level) with dual-model ensemble on IMO-AG-30 benchmark; created large-scale dataset of 21.8M geometric problems including 3M+ with auxiliary constructions.

Conclusion: GenesisGeo demonstrates state-of-the-art performance in automated geometric theorem proving, achieving IMO competition levels through efficient neuro-symbolic integration and large-scale data processing.

Abstract: We present GenesisGeo, an automated theorem prover in Euclidean geometry. We have open-sourced a large-scale geometry dataset of 21.8 million geometric problems, over 3 million of which contain auxiliary constructions. Specially, we significantly accelerate the symbolic deduction engine DDARN by 120x through theorem matching, combined with a C++ implementation of its core components. Furthermore, we build our neuro-symbolic prover, GenesisGeo, upon Qwen3-0.6B-Base, which solves 24 of 30 problems (IMO silver medal level) in the IMO-AG-30 benchmark using a single model, and achieves 26 problems (IMO gold medal level) with a dual-model ensemble.

Method: Proposes DyRo-MCTS which incorporates action robustness estimation into MCTS to guide scheduling decisions toward states that are both high-performing and easily adaptable to future disturbances.

Result: DyRo-MCTS significantly improves offline-learned policies with minimal additional planning time, consistently outperforms vanilla MCTS across various scenarios, and achieves sustainable performance gains under disturbances.

Conclusion: Integrating robustness estimation into online planning enables more resilient scheduling decisions that maintain long-term performance despite dynamic disruptions in job shop environments.

Abstract: Dynamic job shop scheduling, a fundamental combinatorial optimisation problem in various industrial sectors, poses substantial challenges for effective scheduling due to frequent disruptions caused by the arrival of new jobs. State-of-the-art methods employ machine learning to learn scheduling policies offline, enabling rapid responses to dynamic events. However, these offline policies are often imperfect, necessitating the use of planning techniques such as Monte Carlo Tree Search (MCTS) to improve performance at online decision time. The unpredictability of new job arrivals complicates online planning, as decisions based on incomplete problem information are vulnerable to disturbances. To address this issue, we propose the Dynamic Robust MCTS (DyRo-MCTS) approach, which integrates action robustness estimation into MCTS. DyRo-MCTS guides the production environment toward states that not only yield good scheduling outcomes but are also easily adaptable to future job arrivals. Extensive experiments show that DyRo-MCTS significantly improves the performance of offline-learned policies with negligible additional online planning time. Moreover, DyRo-MCTS consistently outperforms vanilla MCTS across various scheduling scenarios. Further analysis reveals that its ability to make robust scheduling decisions leads to long-term, sustainable performance gains under disturbances.

Method: Used 798 valid samples and 2000 outliers from multiple centers, comparing (i) non-parametric registration-dependent SPM and (ii) CNN with SHAP explanations, evaluated via nested cross-validation and expert surveys.

Result: ML model achieved high accuracy and outperformed SPM, which misclassified clinically meaningful variations and missed true outliers. Both SPM and SHAP explanations were perceived as clear and trustworthy by experts.

Conclusion: SPM and explainable ML have complementary potential for automated outlier detection in plantar pressure data, with explainability being crucial for translating complex model outputs into actionable insights.

Abstract: Plantar pressure mapping is essential in clinical diagnostics and sports science, yet large heterogeneous datasets often contain outliers from technical errors or procedural inconsistencies. Statistical Parametric Mapping (SPM) provides interpretable analyses but is sensitive to alignment and its capacity for robust outlier detection remains unclear. This study compares an SPM approach with an explainable machine learning (ML) approach to establish transparent quality-control pipelines for plantar pressure datasets. Data from multiple centers were annotated by expert consensus and enriched with synthetic anomalies resulting in 798 valid samples and 2000 outliers. We evaluated (i) a non-parametric, registration-dependent SPM approach and (ii) a convolutional neural network (CNN), explained using SHapley Additive exPlanations (SHAP). Performance was assessed via nested cross-validation; explanation quality via a semantic differential survey with domain experts. The ML model reached high accuracy and outperformed SPM, which misclassified clinically meaningful variations and missed true outliers. Experts perceived both SPM and SHAP explanations as clear, useful, and trustworthy, though SPM was assessed less complex. These findings highlight the complementary potential of SPM and explainable ML as approaches for automated outlier detection in plantar pressure data, and underscore the importance of explainability in translating complex model outputs into interpretable insights that can effectively inform decision-making.

Method: Used end-to-end deep reinforcement learning in multi-agent foraging games where agents operate in a shared grid world with partial knowledge and must coordinate to complete tasks.

Result: Agents developed communication protocols exhibiting hallmark features of natural language: arbitrariness, interchangeability, displacement, cultural transmission, and compositionality.

Conclusion: The framework provides a platform for studying language evolution from partial observability, temporal reasoning, and cooperative goals in embodied multi-agent settings.

Abstract: Language is a powerful communicative and cognitive tool. It enables humans to express thoughts, share intentions, and reason about complex phenomena. Despite our fluency in using and understanding language, the question of how it arises and evolves over time remains unsolved. A leading hypothesis in linguistics and anthropology posits that language evolved to meet the ecological and social demands of early human cooperation. Language did not arise in isolation, but through shared survival goals. Inspired by this view, we investigate the emergence of language in multi-agent Foraging Games. These environments are designed to reflect the cognitive and ecological constraints believed to have influenced the evolution of communication. Agents operate in a shared grid world with only partial knowledge about other agents and the environment, and must coordinate to complete games like picking up high-value targets or executing temporally ordered actions. Using end-to-end deep reinforcement learning, agents learn both actions and communication strategies from scratch. We find that agents develop communication protocols with hallmark features of natural language: arbitrariness, interchangeability, displacement, cultural transmission, and compositionality. We quantify each property and analyze how different factors, such as population size, social dynamics, and temporal dependencies, shape specific aspects of the emergent language. Our framework serves as a platform for studying how language can evolve from partial observability, temporal reasoning, and cooperative goals in embodied multi-agent settings. We will release all data, code, and models publicly.

Method: RISK framework with three components: RISK-Data (dataset of interaction trajectories), RISK-Bench (evaluation benchmark), and RISK-R1 (reinforcement fine-tuning framework with format rewards, stepwise accuracy, process reweighting, and level reweighting).

Result: RISK-R1 outperforms baselines with 6.8% improvement in offline single-step and 8.8% improvement in offline multi-step tasks, achieving 70.5% task success rate in online evaluation.

Conclusion: RISK provides a scalable, domain-specific solution for automating complex web interactions, advancing e-commerce risk management capabilities.

Abstract: E-commerce risk management requires aggregating diverse, deeply embedded web data through multi-step, stateful interactions, which traditional scraping methods and most existing Graphical User Interface (GUI) agents cannot handle. These agents are typically limited to single-step tasks and lack the ability to manage dynamic, interactive content critical for effective risk assessment. To address this challenge, we introduce RISK, a novel framework designed to build and deploy GUI agents for this domain. RISK integrates three components: (1) RISK-Data, a dataset of 8,492 single-step and 2,386 multi-step interaction trajectories, collected through a high-fidelity browser framework and a meticulous data curation process; (2) RISK-Bench, a benchmark with 802 single-step and 320 multi-step trajectories across three difficulty levels for standardized evaluation; and (3) RISK-R1, a R1-style reinforcement fine-tuning framework considering four aspects: (i) Output Format: Updated format reward to enhance output syntactic correctness and task comprehension, (ii) Single-step Level: Stepwise accuracy reward to provide granular feedback during early training stages, (iii) Multi-step Level: Process reweight to emphasize critical later steps in interaction sequences, and (iv) Task Level: Level reweight to focus on tasks of varying difficulty. Experiments show that RISK-R1 outperforms existing baselines, achieving a 6.8% improvement in offline single-step and an 8.8% improvement in offline multi-step. Moreover, it attains a top task success rate of 70.5% in online evaluation. RISK provides a scalable, domain-specific solution for automating complex web interactions, advancing the state of the art in e-commerce risk management.

Method: Train language models from scratch on synthetic relational knowledge graphs and analyze the emergence of bilinear relational structure in hidden representations.

Result: Models with bilinear structure can infer unseen reverse facts and propagate edits consistently to logically dependent facts, while models lacking this structure fail to generalize edits and introduce inconsistencies.

Conclusion: Bilinear internal representations enable logically consistent behavior in language models, and successful model editing depends on the underlying representational geometry of knowledge.

Abstract: The reversal curse – a language model’s (LM) inability to infer an unseen fact B is A'' from a learned fact A is B’’ – is widely considered a fundamental limitation. We show that this is not an inherent failure but an artifact of how models encode knowledge. By training LMs from scratch on a synthetic dataset of relational knowledge graphs, we demonstrate that bilinear relational structure emerges in their hidden representations. This structure substantially alleviates the reversal curse, enabling LMs to infer unseen reverse facts. Crucially, we also find that this bilinear structure plays a key role in consistent model editing. When a fact is updated in a LM with this structure, the edit correctly propagates to its reverse and other logically dependent facts. In contrast, models lacking this representation not only suffer from the reversal curse but also fail to generalize edits, further introducing logical inconsistencies. Our results establish that training on a relational knowledge dataset induces the emergence of bilinear internal representations, which in turn enable LMs to behave in a logically consistent manner after editing. This implies that the success of model editing depends critically not just on editing algorithms but on the underlying representational geometry of the knowledge being modified.

Method: Created GSM-Agent benchmark where LLMs solve grade-school math problems but must proactively collect missing premises using tools. Proposed agentic reasoning graphs to analyze patterns and developed tool-augmented test-time scaling to encourage revisiting previously visited nodes.

Result: Even frontier models like GPT-5 only achieve 67% accuracy. Analysis revealed that the ability to revisit previously visited nodes, crucial for static reasoning, is often missing in agentic reasoning for many models.

Conclusion: The benchmark and agentic reasoning framework provide tools for better understanding and improving agentic reasoning capabilities in LLMs, with identified revisit patterns offering key insights for performance enhancement.

Abstract: As LLMs are increasingly deployed as agents, agentic reasoning - the ability to combine tool use, especially search, and reasoning - becomes a critical skill. However, it is hard to disentangle agentic reasoning when evaluated in complex environments and tasks. Current agent benchmarks often mix agentic reasoning with challenging math reasoning, expert-level knowledge, and other advanced capabilities. To fill this gap, we build a novel benchmark, GSM-Agent, where an LLM agent is required to solve grade-school-level reasoning problems, but is only presented with the question in the prompt without the premises that contain the necessary information to solve the task, and needs to proactively collect that information using tools. Although the original tasks are grade-school math problems, we observe that even frontier models like GPT-5 only achieve 67% accuracy. To understand and analyze the agentic reasoning patterns, we propose the concept of agentic reasoning graph: cluster the environment’s document embeddings into nodes, and map each tool call to its nearest node to build a reasoning path. Surprisingly, we identify that the ability to revisit a previously visited node, widely taken as a crucial pattern in static reasoning, is often missing for agentic reasoning for many models. Based on the insight, we propose a tool-augmented test-time scaling method to improve LLM’s agentic reasoning performance by adding tools to encourage models to revisit. We expect our benchmark and the agentic reasoning framework to aid future studies of understanding and pushing the boundaries of agentic reasoning.

Method: Uses a decoupled control plane architecture with MCP server providing workload analysis, scheduler policy repository, and execution verifier. Implements sched-agent multi-agent system for autonomous workload analysis and eBPF scheduling policy synthesis.

Result: Achieves up to 1.79x performance improvement and 13x cost reduction compared to naive agentic approaches while maintaining high success rate.

Conclusion: SchedCP democratizes expert-level system optimization and represents progress toward creating self-optimizing, application-aware operating systems.

Abstract: Operating system schedulers suffer from a fundamental semantic gap, where kernel policies fail to understand application-specific needs, leading to suboptimal performance. We introduce SchedCP, the first framework that enables fully autonomous Large Language Model (LLM) agents to safely and efficiently optimize Linux schedulers without human involvement. Our core insight is that the challenge is not merely to apply a better LLM, but to architect a decoupled control plane that separates the AI’s role of semantic reasoning (“what to optimize”) from the system’s role of execution (“how to observe and act”). Implemented as Model Context Protocol(MCP) server, SchedCP provides a stable interface with three key services: a Workload Analysis Engine, an evolving Scheduler Policy Repository, and an Execution Verifier that validates all AI-generated code and configure before deployment with static and dynamic analysis. We demonstrate this architecture’s power with sched-agent, a multi-agent system that autonomously analyzes workloads, synthesizes custom eBPF scheduling policies, and deploys them via the sched_ext infrastructure. Our evaluation shows that SchedCP achieves up to an 1.79x performance improvement, and a 13x cost reduction compared to naive agentic approaches, all while maintaining high success rate. By bridging the semantic gap, SchedCP democratizes expert-level system optimization and represents a step towards creating truly self-optimizing, application-aware operating systems. The code is open-sourced in https://github.com/eunomia-bpf/schedcp

Method: Conducted large-scale empirical study evaluating various model merging techniques across reasoning benchmarks, systematically varying merging strengths to construct accuracy-efficiency curves.

Result: Model merging effectively calibrates reasoning accuracy vs. token efficiency trade-off, even with divergent parent model weights. Achieved Pareto improvements where merged models had higher accuracy and lower token consumption than parents.

Conclusion: Model merging provides practical method for creating LLMs with specific reasoning profiles, offering comprehensive guidelines for meeting diverse application demands through tunable performance landscapes.

Abstract: The growing demand for large language models (LLMs) with tunable reasoning capabilities in many real-world applications highlights a critical need for methods that can efficiently produce a spectrum of models balancing reasoning depth and computational cost. Model merging has emerged as a promising, training-free technique to address this challenge by arithmetically combining the weights of a general-purpose model with a specialized reasoning model. While various merging techniques exist, their potential to create a spectrum of models with fine-grained control over reasoning abilities remains largely unexplored. This work presents a large-scale empirical study evaluating a range of model merging techniques across multiple reasoning benchmarks. We systematically vary merging strengths to construct accuracy-efficiency curves, providing the first comprehensive view of the tunable performance landscape. Our findings reveal that model merging offers an effective and controllable method for calibrating the trade-off between reasoning accuracy and token efficiency, even when parent models have highly divergent weight spaces. Crucially, we identify instances of Pareto Improvement, where a merged model achieves both higher accuracy and lower token consumption than one of its parents. Our study provides the first comprehensive analysis of this tunable space, offering practical guidelines for creating LLMs with specific reasoning profiles to meet diverse application demands.

Method: A two-stage framework: (1) Explorer model generates potential solutions through parallel sampling, (2) Synthesizer model integrates references for refined reasoning. Features asymmetric scaling with smaller explorer and larger synthesizer models.

Result: Qwen3-8B-distill achieved 75% performance improvement over self-consistency baseline. A2R-Efficient (Qwen3-4B explorer + Qwen3-8B synthesizer) surpassed monolithic Qwen3-32B performance at 30% lower cost.

Conclusion: A2R is an effective plug-and-play framework that boosts performance while being computationally efficient for real-world applications.

Abstract: Recent Large Reasoning Models have achieved significant improvements in complex task-solving capabilities by allocating more computation at the inference stage with a “thinking longer” paradigm. Even as the foundational reasoning capabilities of models advance rapidly, the persistent gap between a model’s performance in a single attempt and its latent potential, often revealed only across multiple solution paths, starkly highlights the disparity between its realized and inherent capabilities. To address this, we present A2R, an Asymmetric Two-Stage Reasoning framework designed to explicitly bridge the gap between a model’s potential and its actual performance. In this framework, an “explorer” model first generates potential solutions in parallel through repeated sampling. Subsequently,a “synthesizer” model integrates these references for a more refined, second stage of reasoning. This two-stage process allows computation to be scaled orthogonally to existing sequential methods. Our work makes two key innovations: First, we present A2R as a plug-and-play parallel reasoning framework that explicitly enhances a model’s capabilities on complex questions. For example, using our framework, the Qwen3-8B-distill model achieves a 75% performance improvement compared to its self-consistency baseline. Second, through a systematic analysis of the explorer and synthesizer roles, we identify an effective asymmetric scaling paradigm. This insight leads to A2R-Efficient, a “small-to-big” variant that combines a Qwen3-4B explorer with a Qwen3-8B synthesizer. This configuration surpasses the average performance of a monolithic Qwen3-32B model at a nearly 30% lower cost. Collectively, these results show that A2R is not only a performance-boosting framework but also an efficient and practical solution for real-world applications.

Method: Propose a generalized problem formulation using aggregation functions of hidden objectives, extend core operations of standard MOS algorithms to handle specific aggregation functions, and apply to diverse robotics planning problems.

Result: The extended MOS algorithms outperform vanilla versions by orders of magnitude across various robotics domains including navigation, manipulation, medical systems, inspection, and route planning.

Conclusion: The generalized formulation with aggregation functions enables effective application of standard MOS algorithms to complex robotics problems, significantly improving performance while maintaining algorithm compatibility.

Abstract: Multi-objective search (MOS) has become essential in robotics, as real-world robotic systems need to simultaneously balance multiple, often conflicting objectives. Recent works explore complex interactions between objectives, leading to problem formulations that do not allow the usage of out-of-the-box state-of-the-art MOS algorithms. In this paper, we suggest a generalized problem formulation that optimizes solution objectives via aggregation functions of hidden (search) objectives. We show that our formulation supports the application of standard MOS algorithms, necessitating only to properly extend several core operations to reflect the specific aggregation functions employed. We demonstrate our approach in several diverse robotics planning problems, spanning motion-planning for navigation, manipulation and planning fr medical systems under obstacle uncertainty as well as inspection planning, and route planning with different road types. We solve the problems using state-of-the-art MOS algorithms after properly extending their core operations, and provide empirical evidence that they outperform by orders of magnitude the vanilla versions of the algorithms applied to the same problems but without objective aggregation.

Method: Systematic evaluation of static and dynamic energy estimation approaches through comparisons with ground-truth measurements across hundreds of AI experiments using a proposed validation framework.

Result: Established estimation approaches consistently make errors of up to 40%, though they generally follow the patterns of AI energy consumption.

Conclusion: The study provides empirical evidence on energy estimation quality, validates widely used tools for sustainable AI development, and offers guidelines and code for improving estimation accuracy in resource-aware ML research.

Abstract: Although machine learning (ML) and artificial intelligence (AI) present fascinating opportunities for innovation, their rapid development is also significantly impacting our environment. In response to growing resource-awareness in the field, quantification tools such as the ML Emissions Calculator and CodeCarbon were developed to estimate the energy consumption and carbon emissions of running AI models. They are easy to incorporate into AI projects, however also make pragmatic assumptions and neglect important factors, raising the question of estimation accuracy. This study systematically evaluates the reliability of static and dynamic energy estimation approaches through comparisons with ground-truth measurements across hundreds of AI experiments. Based on the proposed validation framework, investigative insights into AI energy demand and estimation inaccuracies are provided. While generally following the patterns of AI energy consumption, the established estimation approaches are shown to consistently make errors of up to 40%. By providing empirical evidence on energy estimation quality and errors, this study establishes transparency and validates widely used tools for sustainable AI development. It moreover formulates guidelines for improving the state-of-the-art and offers code for extending the validation to other domains and tools, thus making important contributions to resource-aware ML and AI sustainability research.

Method: Propose probabilistic metrics that operate directly on distributions of annotations, applicable regardless of annotation generation process (counting, confidence ratings, or probabilistic models).

Result: Accounting for confidence in binary labels significantly impacts model rankings in medical imaging benchmarks.

Conclusion: The community should release raw annotations and adopt uncertainty-aware evaluation to better reflect clinical data reality.

Abstract: Clinical dataset labels are rarely certain as annotators disagree and confidence is not uniform across cases. Typical aggregation procedures, such as majority voting, obscure this variability. In simple experiments on medical imaging benchmarks, accounting for the confidence in binary labels significantly impacts model rankings. We therefore argue that machine-learning evaluations should explicitly account for annotation uncertainty using probabilistic metrics that directly operate on distributions. These metrics can be applied independently of the annotations’ generating process, whether modeled by simple counting, subjective confidence ratings, or probabilistic response models. They are also computationally lightweight, as closed-form expressions have linear-time implementations once examples are sorted by model score. We thus urge the community to release raw annotations for datasets and to adopt uncertainty-aware evaluation so that performance estimates may better reflect clinical data.

Method: Systematic methodology combining LLMs with evolutionary algorithms to iteratively generate and refine heuristic solutions, comparing them against traditional approaches like Finite First-Fit and Hybrid First-Fit.

Result: GPT-4o achieved optimal solutions within two iterations, reducing average bin usage from 16 to 15 bins and improving space utilization from 0.76-0.78 to 0.83.

Conclusion: LLMs can produce more efficient solutions than traditional approaches in combinatorial optimization tasks while requiring fewer computational resources, contributing to better understanding of LLM evaluation in specialized domains.

Abstract: This paper presents an evaluation framework for assessing Large Language Models’ (LLMs) capabilities in combinatorial optimization, specifically addressing the 2D bin-packing problem. We introduce a systematic methodology that combines LLMs with evolutionary algorithms to generate and refine heuristic solutions iteratively. Through comprehensive experiments comparing LLM generated heuristics against traditional approaches (Finite First-Fit and Hybrid First-Fit), we demonstrate that LLMs can produce more efficient solutions while requiring fewer computational resources. Our evaluation reveals that GPT-4o achieves optimal solutions within two iterations, reducing average bin usage from 16 to 15 bins while improving space utilization from 0.76-0.78 to 0.83. This work contributes to understanding LLM evaluation in specialized domains and establishes benchmarks for assessing LLM performance in combinatorial optimization tasks.

Method: Used high-quality general-purpose and medical multimodal data with a five-dimensional quality assessment framework. Employed low-to-high image resolution and multimodal sequence packing for efficiency. Implemented three-stage supervised fine-tuning for complex medical tasks.

Result: InfiMed-Foundation-1.7B outperforms Qwen2.5VL-3B, while InfiMed-Foundation-4B surpasses HuatuoGPT-V-7B and MedGemma-27B-IT in MedEvalKit framework for medical visual question answering and diagnostic tasks.

Conclusion: The work addresses key challenges in data quality, training efficiency, and domain-specific knowledge extraction, enabling more reliable and effective AI-driven healthcare solutions.

Abstract: Multimodal large language models (MLLMs) have shown remarkable potential in various domains, yet their application in the medical field is hindered by several challenges. General-purpose MLLMs often lack the specialized knowledge required for medical tasks, leading to uncertain or hallucinatory responses. Knowledge distillation from advanced models struggles to capture domain-specific expertise in radiology and pharmacology. Additionally, the computational cost of continual pretraining with large-scale medical data poses significant efficiency challenges. To address these issues, we propose InfiMed-Foundation-1.7B and InfiMed-Foundation-4B, two medical-specific MLLMs designed to deliver state-of-the-art performance in medical applications. We combined high-quality general-purpose and medical multimodal data and proposed a novel five-dimensional quality assessment framework to curate high-quality multimodal medical datasets. We employ low-to-high image resolution and multimodal sequence packing to enhance training efficiency, enabling the integration of extensive medical data. Furthermore, a three-stage supervised fine-tuning process ensures effective knowledge extraction for complex medical tasks. Evaluated on the MedEvalKit framework, InfiMed-Foundation-1.7B outperforms Qwen2.5VL-3B, while InfiMed-Foundation-4B surpasses HuatuoGPT-V-7B and MedGemma-27B-IT, demonstrating superior performance in medical visual question answering and diagnostic tasks. By addressing key challenges in data quality, training efficiency, and domain-specific knowledge extraction, our work paves the way for more reliable and effective AI-driven solutions in healthcare. InfiMed-Foundation-4B model is available at \href{https://huggingface.co/InfiX-ai/InfiMed-Foundation-4B}{InfiMed-Foundation-4B}.

Method: Parametrize transition matrix as product of column one-hot matrix (P) and complex-valued diagonal matrix (D), enabling linear computational cost scaling with state size.

Result: Significantly outperforms modern SSM variants on FSA tracking tasks, comparable to neural controlled differential equations on time-series classification, and effectively tracks complex FSA states in hybrid Transformer-SSM architecture.

Conclusion: PD-SSM achieves optimal FSA state tracking with linear computational cost, improving expressivity guarantees while maintaining efficiency comparable to diagonal SSMs.

Abstract: Modern state-space models (SSMs) often utilize transition matrices which enable efficient computation but pose restrictions on the model’s expressivity, as measured in terms of the ability to emulate finite-state automata (FSA). While unstructured transition matrices are optimal in terms of expressivity, they come at a prohibitively high compute and memory cost even for moderate state sizes. We propose a structured sparse parametrization of transition matrices in SSMs that enables FSA state tracking with optimal state size and depth, while keeping the computational cost of the recurrence comparable to that of diagonal SSMs. Our method, PD-SSM, parametrizes the transition matrix as the product of a column one-hot matrix ($P$) and a complex-valued diagonal matrix ($D$). Consequently, the computational cost of parallel scans scales linearly with the state size. Theoretically, the model is BIBO-stable and can emulate any $N$-state FSA with one layer of dimension $N$ and a linear readout of size $N \times N$, significantly improving on all current structured SSM guarantees. Experimentally, the model significantly outperforms a wide collection of modern SSM variants on various FSA state tracking tasks. On multiclass time-series classification, the performance is comparable to that of neural controlled differential equations, a paradigm explicitly built for time-series analysis. Finally, we integrate PD-SSM into a hybrid Transformer-SSM architecture and demonstrate that the model can effectively track the states of a complex FSA in which transitions are encoded as a set of variable-length English sentences. The code is available at https://github.com/IBM/expressive-sparse-state-space-model

Method: Proposes a simpler method that represents LLMs as nondeterministic causal models, making it directly applicable to any black-box LLM without modification.

Result: The new method is implementation-agnostic and works with any black-box LLM, while existing methods are useful for specific types of counterfactuals but not others.

Conclusion: Provides theoretical foundation for reasoning about counterfactuals in LLMs based on intended semantics, enabling novel application-specific methods.

Abstract: Recent work by Chatzi et al. and Ravfogel et al. has developed, for the first time, a method for generating counterfactuals of probabilistic Large Language Models. Such counterfactuals tell us what would - or might - have been the output of an LLM if some factual prompt ${\bf x}$ had been ${\bf x}^*$ instead. The ability to generate such counterfactuals is an important necessary step towards explaining, evaluating, and comparing, the behavior of LLMs. I argue, however, that the existing method rests on an ambiguous interpretation of LLMs: it does not interpret LLMs literally, for the method involves the assumption that one can change the implementation of an LLM’s sampling process without changing the LLM itself, nor does it interpret LLMs as intended, for the method involves explicitly representing a nondeterministic LLM as a deterministic causal model. I here present a much simpler method for generating counterfactuals that is based on an LLM’s intended interpretation by representing it as a nondeterministic causal model instead. The advantage of my simpler method is that it is directly applicable to any black-box LLM without modification, as it is agnostic to any implementation details. The advantage of the existing method, on the other hand, is that it directly implements the generation of a specific type of counterfactuals that is useful for certain purposes, but not for others. I clarify how both methods relate by offering a theoretical foundation for reasoning about counterfactuals in LLMs based on their intended semantics, thereby laying the groundwork for novel application-specific methods for generating counterfactuals.

Method: PRIME employs a Quick Thinking Agent (System 1) for rapid answers, and if uncertainty is detected, triggers a structured System 2 pipeline with specialized agents for planning, hypothesis generation, retrieval, information integration, and decision-making.

Result: Experimental results with LLaMA 3 models show that PRIME enables open-source LLMs to perform competitively with state-of-the-art closed-source models like GPT-4 and GPT-4o on benchmarks requiring multi-hop and knowledge-grounded reasoning.

Conclusion: PRIME establishes itself as a scalable solution for improving LLMs in domains requiring complex, knowledge-intensive reasoning by faithfully mimicking human cognitive processes and enhancing both efficiency and accuracy.

Abstract: Inspired by the dual-process theory of human cognition from \textit{Thinking, Fast and Slow}, we introduce \textbf{PRIME} (Planning and Retrieval-Integrated Memory for Enhanced Reasoning), a multi-agent reasoning framework that dynamically integrates \textbf{System 1} (fast, intuitive thinking) and \textbf{System 2} (slow, deliberate thinking). PRIME first employs a Quick Thinking Agent (System 1) to generate a rapid answer; if uncertainty is detected, it then triggers a structured System 2 reasoning pipeline composed of specialized agents for \textit{planning}, \textit{hypothesis generation}, \textit{retrieval}, \textit{information integration}, and \textit{decision-making}. This multi-agent design faithfully mimics human cognitive processes and enhances both efficiency and accuracy. Experimental results with LLaMA 3 models demonstrate that PRIME enables open-source LLMs to perform competitively with state-of-the-art closed-source models like GPT-4 and GPT-4o on benchmarks requiring multi-hop and knowledge-grounded reasoning. This research establishes PRIME as a scalable solution for improving LLMs in domains requiring complex, knowledge-intensive reasoning.

Method: Created SeekBench with 190 expert-annotated traces containing over 1,800 response steps, enriched with evidence annotations for granular analysis of three key epistemic competencies.

Result: The benchmark enables evaluation of whether agents: (1) ground reasoning in observed evidence, (2) adaptively reformulate searches to recover from poor results, and (3) properly calibrate confidence about evidence sufficiency.

Conclusion: SeekBench provides the first comprehensive framework for assessing the epistemic competence of LLM search agents beyond just final answer accuracy.

Abstract: Recent work has explored training Large Language Model (LLM) search agents with reinforcement learning (RL) for open-domain question answering (QA). However, most evaluations focus solely on final answer accuracy, overlooking how these agents reason with and act on external evidence. We introduce SeekBench, the first benchmark for evaluating the \textit{epistemic competence} of LLM search agents through step-level analysis of their response traces. SeekBench comprises 190 expert-annotated traces with over 1,800 response steps generated by LLM search agents, each enriched with evidence annotations for granular analysis of whether agents (1) generate reasoning steps grounded in observed evidence, (2) adaptively reformulate searches to recover from low-quality results, and (3) have proper calibration to correctly assess whether the current evidence is sufficient for providing an answer.

Method: Proposed EMMA framework with DreamTransfer (diffusion Transformer for text-controlled video editing) and AdaMix (dynamic batch reweighting for hard samples), using hybrid training with real and generated data.

Result: DreamTransfer outperforms prior methods in multi-view consistency, geometric fidelity, and text-conditioning. VLAs trained with generated data achieve over 200% relative performance gain in zero-shot domains, with additional 13% improvement from AdaMix.

Conclusion: The approach effectively overcomes data collection bottlenecks and significantly boosts policy generalization to unseen object categories and visual domains.

Abstract: Vision-language-action (VLA) models increasingly rely on diverse training data to achieve robust generalization. However, collecting large-scale real-world robot manipulation data across varied object appearances and environmental conditions remains prohibitively time-consuming and expensive. To overcome this bottleneck, we propose Embodied Manipulation Media Adaptation (EMMA), a VLA policy enhancement framework that integrates a generative data engine with an effective training pipeline. We introduce DreamTransfer, a diffusion Transformer-based framework for generating multi-view consistent, geometrically grounded embodied manipulation videos. DreamTransfer enables text-controlled visual editing of robot videos, transforming foreground, background, and lighting conditions without compromising 3D structure or geometrical plausibility. Furthermore, we explore hybrid training with real and generated data, and introduce AdaMix, a hard-sample-aware training strategy that dynamically reweights training batches to focus optimization on perceptually or kinematically challenging samples. Extensive experiments show that videos generated by DreamTransfer significantly outperform prior video generation methods in multi-view consistency, geometric fidelity, and text-conditioning accuracy. Crucially, VLAs trained with generated data enable robots to generalize to unseen object categories and novel visual domains using only demonstrations from a single appearance. In real-world robotic manipulation tasks with zero-shot visual domains, our approach achieves over a 200% relative performance gain compared to training on real data alone, and further improves by 13% with AdaMix, demonstrating its effectiveness in boosting policy generalization.

Method: Uses a second foundation model (Gemma-3) to propose new evolutionary targets from simulation visual history. Tests two strategies: Evolved Supervised Targets (EST) for single prompt matching and Evolved Temporal Targets (ETT) for matching entire sequences of generated prompts, implemented in Lenia substrate.

Result: EST promotes greater visual novelty while ETT fosters more coherent and interpretable evolutionary sequences. The method successfully creates evolutionary trajectories with increasingly complex targets.

Conclusion: ASAL++ points towards new directions for foundation model-driven artificial life discovery with open-ended characteristics, demonstrating the potential of multimodal FMs for automated evolutionary search.

Abstract: Foundation models (FMs) have recently opened up new frontiers in the field of artificial life (ALife) by providing powerful tools to automate search through ALife simulations. Previous work aligns ALife simulations with natural language target prompts using vision-language models (VLMs). We build on Automated Search for Artificial Life (ASAL) by introducing ASAL++, a method for open-ended-like search guided by multimodal FMs. We use a second FM to propose new evolutionary targets based on a simulation’s visual history. This induces an evolutionary trajectory with increasingly complex targets. We explore two strategies: (1) evolving a simulation to match a single new prompt at each iteration (Evolved Supervised Targets: EST) and (2) evolving a simulation to match the entire sequence of generated prompts (Evolved Temporal Targets: ETT). We test our method empirically in the Lenia substrate using Gemma-3 to propose evolutionary targets, and show that EST promotes greater visual novelty, while ETT fosters more coherent and interpretable evolutionary sequences. Our results suggest that ASAL++ points towards new directions for FM-driven ALife discovery with open-ended characteristics.

Method: Three-module framework: Perception (abstracts diagrams to structured logic), Symbolic Reasoning (applies geometric theorems), and Sketch Action (executes operations like drawing lines). Trained via supervised fine-tuning on 2,000 trajectories followed by reinforcement learning with symbolic rewards.

Result: Significantly improves stepwise reasoning accuracy and problem-solving success over static perception methods on the GeoSketch Benchmark of 390 geometry problems requiring auxiliary construction or transformations.

Conclusion: GeoSketch advances multimodal reasoning from static interpretation to dynamic, verifiable interaction by unifying hierarchical decision-making, executable visual actions, and symbolic verification.

Abstract: Geometric Problem Solving (GPS) poses a unique challenge for Multimodal Large Language Models (MLLMs), requiring not only the joint interpretation of text and diagrams but also iterative visuospatial reasoning. While existing approaches process diagrams as static images, they lack the capacity for dynamic manipulation - a core aspect of human geometric reasoning involving auxiliary line construction and affine transformations. We present GeoSketch, a neural-symbolic framework that recasts geometric reasoning as an interactive perception-reasoning-action loop. GeoSketch integrates: (1) a Perception module that abstracts diagrams into structured logic forms, (2) a Symbolic Reasoning module that applies geometric theorems to decide the next deductive step, and (3) a Sketch Action module that executes operations such as drawing auxiliary lines or applying transformations, thereby updating the diagram in a closed loop. To train this agent, we develop a two-stage pipeline: supervised fine-tuning on 2,000 symbolic-curated trajectories followed by reinforcement learning with dense, symbolic rewards to enhance robustness and strategic exploration. To evaluate this paradigm, we introduce the GeoSketch Benchmark, a high-quality set of 390 geometry problems requiring auxiliary construction or affine transformations. Experiments on strong MLLM baselines demonstrate that GeoSketch significantly improves stepwise reasoning accuracy and problem-solving success over static perception methods. By unifying hierarchical decision-making, executable visual actions, and symbolic verification, GeoSketch advances multimodal reasoning from static interpretation to dynamic, verifiable interaction, establishing a new foundation for solving complex visuospatial problems.

Method: Proposes InfiAgent with: agent-as-a-tool mechanism for hierarchical decomposition, dual-audit for quality control, agent routing for task matching, self-evolution for autonomous restructuring, and atomic task design for parallelism.

Result: Achieves 9.9% higher performance than ADAS framework; case study shows InfiHelper generates scientific papers recognized by human reviewers at top-tier IEEE conferences.

Conclusion: InfiAgent evolves into a versatile pyramid-like multi-agent system capable of solving a wide range of problems efficiently and autonomously.

Abstract: Large Language Model (LLM) agents have demonstrated remarkable capabilities in organizing and executing complex tasks, and many such agents are now widely used in various application scenarios. However, developing these agents requires carefully designed workflows, carefully crafted prompts, and iterative tuning, which requires LLM techniques and domain-specific expertise. These hand-crafted limitations hinder the scalability and cost-effectiveness of LLM agents across a wide range of industries. To address these challenges, we propose \textbf{InfiAgent}, a Pyramid-like DAG-based Multi-Agent Framework that can be applied to \textbf{infi}nite scenarios, which introduces several key innovations: a generalized “agent-as-a-tool” mechanism that automatically decomposes complex agents into hierarchical multi-agent systems; a dual-audit mechanism that ensures the quality and stability of task completion; an agent routing function that enables efficient task-agent matching; and an agent self-evolution mechanism that autonomously restructures the agent DAG based on new tasks, poor performance, or optimization opportunities. Furthermore, InfiAgent’s atomic task design supports agent parallelism, significantly improving execution efficiency. This framework evolves into a versatile pyramid-like multi-agent system capable of solving a wide range of problems. Evaluations on multiple benchmarks demonstrate that InfiAgent achieves 9.9% higher performance compared to ADAS (similar auto-generated agent framework), while a case study of the AI research assistant InfiHelper shows that it generates scientific papers that have received recognition from human reviewers at top-tier IEEE conferences.

Method: Developed EELMA algorithm to approximate effective empowerment from multi-turn text interactions, validated on language games and realistic web-browsing scenarios.

Result: Empowerment strongly correlates with average task performance, characterizes impact of environmental complexity and agentic factors (chain-of-thought, model scale, memory length), and high empowerment states/actions are often pivotal moments for general capabilities.

Conclusion: Empowerment serves as an appealing general-purpose metric for evaluating and monitoring LM agents in complex, open-ended settings.

Abstract: As language model (LM) agents become more capable and gain broader access to real-world tools, there is a growing need for scalable evaluation frameworks of agentic capability. However, conventional benchmark-centric evaluations are costly to design and require human designers to come up with valid tasks that translate into insights about general model capabilities. In this work, we propose information-theoretic evaluation based on empowerment, the mutual information between an agent’s actions and future states, as an open-ended method for evaluating LM agents. We introduce EELMA (Estimating Empowerment of Language Model Agents), an algorithm for approximating effective empowerment from multi-turn text interactions. We validate EELMA on both language games and scaled-up realistic web-browsing scenarios. We find that empowerment strongly correlates with average task performance, characterize the impact of environmental complexity and agentic factors such as chain-of-thought, model scale, and memory length on estimated empowerment, and that high empowerment states and actions are often pivotal moments for general capabilities. Together, these results demonstrate empowerment as an appealing general-purpose metric for evaluating and monitoring LM agents in complex, open-ended settings.

Method: Captures stylus input on secure tablets, uses transformer-based OCR for transcription, and implements a retrieval-augmented pipeline integrating faculty solutions, cache layers, and external references for LLM-based scoring with explicit reasoning.

Result: The system enables handwriting preservation with scalable, transparent evaluation that reduces environmental costs, accelerates feedback cycles, and builds reusable knowledge bases while mitigating grading bias.

Conclusion: TrueGradeAI advances digital assessment by combining handwriting preservation with explainable automation, bias mitigation, and auditable grading trails to ensure fairness and efficiency in examinations.

Abstract: This paper introduces TrueGradeAI, an AI-driven digital examination framework designed to overcome the shortcomings of traditional paper-based assessments, including excessive paper usage, logistical complexity, grading delays, and evaluator bias. The system preserves natural handwriting by capturing stylus input on secure tablets and applying transformer-based optical character recognition for transcription. Evaluation is conducted through a retrieval-augmented pipeline that integrates faculty solutions, cache layers, and external references, enabling a large language model to assign scores with explicit, evidence-linked reasoning. Unlike prior tablet-based exam systems that primarily digitize responses, TrueGradeAI advances the field by incorporating explainable automation, bias mitigation, and auditable grading trails. By uniting handwriting preservation with scalable and transparent evaluation, the framework reduces environmental costs, accelerates feedback cycles, and progressively builds a reusable knowledge base, while actively working to mitigate grading bias and ensure fairness in assessment.

Method: REMA framework quantifies geometric deviation of erroneous representations by calculating k-nearest neighbors distance to the Reasoning Manifold formed by correct representations, then tracks deviation across layers to localize divergence points where reasoning goes off-track.

Result: Experiments show the low-dimensional nature of reasoning manifolds and high separability between erroneous and correct reasoning representations, validating REMA’s effectiveness in analyzing reasoning failure origins.

Conclusion: This research connects abstract reasoning failures to measurable geometric deviations in representations, providing new avenues for understanding and diagnosing internal computational processes of black-box models.

Abstract: Understanding how Large Language Models (LLMs) perform complex reasoning and their failure mechanisms is a challenge in interpretability research. To provide a measurable geometric analysis perspective, we define the concept of the Reasoning Manifold, a latent low-dimensional geometric structure formed by the internal representations corresponding to all correctly reasoned generations. This structure can be conceptualized as the embodiment of the effective thinking paths that the model has learned to successfully solve a given task. Based on this concept, we build REMA, a framework that explains the origins of failures by quantitatively comparing the spatial relationships of internal model representations corresponding to both erroneous and correct reasoning samples. Specifically, REMA first quantifies the geometric deviation of each erroneous representation by calculating its k-nearest neighbors distance to the approximated manifold formed by correct representations, thereby providing a unified failure signal. It then localizes the divergence points where these deviations first become significant by tracking this deviation metric across the model’s layers and comparing it against a baseline of internal fluctuations from correct representations, thus identifying where the reasoning chain begins to go off-track. Our extensive experiments on diverse language and multimodal models and tasks demonstrate the low-dimensional nature of the reasoning manifold and the high separability between erroneous and correct reasoning representations. The results also validate the effectiveness of the REMA framework in analyzing the origins of reasoning failures. This research connects abstract reasoning failures to measurable geometric deviations in representations, providing new avenues for in-depth understanding and diagnosis of the internal computational processes of black-box models.

Method: Introduced a Schelling-variant urban migration model with 200+ LLM agents navigating conflict between egoistic and altruistic goals. Used Grounded Theory-inspired method to systematically code agent reasoning.

Result: Identified two distinct LLM archetypes: Adaptive Egoists (default to self-interest but increase altruism with social norms) and Altruistic Optimizers (inherently prioritize collective benefit at personal cost).

Conclusion: Model selection for social simulation should consider intrinsic social action logic, not just reasoning capability. Adaptive Egoists better simulate human societies, while Altruistic Optimizers suit idealized pro-social scenarios.

Abstract: Leveraging Large Language Models (LLMs) for social simulation is a frontier in computational social science. Understanding the social logics these agents embody is critical to this attempt. However, existing research has primarily focused on cooperation in small-scale, task-oriented games, overlooking how altruism, which means sacrificing self-interest for collective benefit, emerges in large-scale agent societies. To address this gap, we introduce a Schelling-variant urban migration model that creates a social dilemma, compelling over 200 LLM agents to navigate an explicit conflict between egoistic (personal utility) and altruistic (system utility) goals. Our central finding is a fundamental difference in the social tendencies of LLMs. We identify two distinct archetypes: “Adaptive Egoists”, which default to prioritizing self-interest but whose altruistic behaviors significantly increase under the influence of a social norm-setting message board; and “Altruistic Optimizers”, which exhibit an inherent altruistic logic, consistently prioritizing collective benefit even at a direct cost to themselves. Furthermore, to qualitatively analyze the cognitive underpinnings of these decisions, we introduce a method inspired by Grounded Theory to systematically code agent reasoning. In summary, this research provides the first evidence of intrinsic heterogeneity in the egoistic and altruistic tendencies of different LLMs. We propose that for social simulation, model selection is not merely a matter of choosing reasoning capability, but of choosing an intrinsic social action logic. While “Adaptive Egoists” may offer a more suitable choice for simulating complex human societies, “Altruistic Optimizers” are better suited for modeling idealized pro-social actors or scenarios where collective welfare is the primary consideration.

Method: StepORLM features a co-evolutionary loop where a policy model and generative process reward model (GenPRM) iteratively improve each other using dual-feedback: outcome-based verification from external solvers and holistic process evaluation from GenPRM, aligned via Weighted Direct Preference Optimization.

Result: The 8B-parameter StepORLM establishes new state-of-the-art across six benchmarks, significantly outperforming larger generalist models, agentic methods, and specialized baselines. The co-evolved GenPRM also acts as a powerful universal process verifier, boosting inference scaling performance.

Conclusion: StepORLM demonstrates that generative process supervision and co-evolutionary training effectively address key limitations in LLM training for OR problems, enabling superior performance and creating universally applicable process verification capabilities.

Abstract: Large Language Models (LLMs) have shown promising capabilities for solving Operations Research (OR) problems. While reinforcement learning serves as a powerful paradigm for LLM training on OR problems, existing works generally face two key limitations. First, outcome reward suffers from the credit assignment problem, where correct final answers can reinforce flawed reasoning. Second, conventional discriminative process supervision is myopic, failing to evaluate the interdependent steps of OR modeling holistically. To this end, we introduce StepORLM, a novel self-evolving framework with generative process supervision. At its core, StepORLM features a co-evolutionary loop where a policy model and a generative process reward model (GenPRM) iteratively improve on each other. This loop is driven by a dual-feedback mechanism: definitive, outcome-based verification from an external solver, and nuanced, holistic process evaluation from the GenPRM. The combined signal is used to align the policy via Weighted Direct Preference Optimization (W-DPO) and simultaneously refine the GenPRM. Our resulting 8B-parameter StepORLM establishes a new state-of-the-art across six benchmarks, significantly outperforming vastly larger generalist models, agentic methods, and specialized baselines. Moreover, the co-evolved GenPRM is able to act as a powerful and universally applicable process verifier, substantially boosting the inference scaling performance of both our own model and other existing LLMs.

Method: Proposes UniMIC framework using compact tokenized representations as communication medium. Employs lightweight Transformer-based entropy models with three scenario-specific designs (generic, masked, and text-conditioned) to minimize inter-token redundancy.

Result: Achieves substantial bitrate savings and remains robust even at ultra-low bitrates (<0.05bpp) across text-to-image generation, text-guided inpainting, outpainting, and visual question answering tasks, without compromising downstream task performance.

Conclusion: UniMIC establishes a practical and forward-looking paradigm for next-generation multimodal interactive communication, bridging the gap between edge devices and cloud AI agents through efficient token-based representations.

Abstract: The rapid progress of Large Multimodal Models (LMMs) and cloud-based AI agents is transforming human-AI collaboration into bidirectional, multimodal interaction. However, existing codecs remain optimized for unimodal, one-way communication, resulting in repeated degradation under conventional compress-transmit-reconstruct pipelines. To address this limitation, we propose UniMIC, a Unified token-based Multimodal Interactive Coding framework that bridges edge devices and cloud AI agents. Instead of transmitting raw pixels or plain text, UniMIC employs compact tokenized representations as the communication medium, enabling efficient low-bitrate transmission while maintaining compatibility with LMMs. To further enhance compression, lightweight Transformer-based entropy models with scenario-specific designs-generic, masked, and text-conditioned-effectively minimize inter-token redundancy. Extensive experiments on text-to-image generation, text-guided inpainting, outpainting, and visual question answering show that UniMIC achieves substantial bitrate savings and remains robust even at ultra-low bitrates (<0.05bpp), without compromising downstream task performance. These results establish UniMIC as a practical and forward-looking paradigm for next-generation multimodal interactive communication.

Method: DES combines Dynamic MoE (direct control of expert counts during inference) and Expert Configuration Inheritance (consistent expert counts within reasoning paths but varied across runs) to balance stability and diversity.

Result: Extensive experiments across MoE architectures, verifiers, and reasoning benchmarks (math, code, knowledge) show DES reliably outperforms TTS baselines, improving accuracy and stability without additional cost.

Conclusion: DES demonstrates how structural flexibility in modern LLMs can advance reasoning, serving as a practical and scalable architecture-aware TTS approach.

Abstract: Test-Time Scaling (TTS) enhances the reasoning ability of large language models (LLMs) by allocating additional computation during inference. However, existing approaches primarily rely on output-level sampling while overlooking the role of model architecture. In mainstream Mixture-of-Experts (MoE) LLMs, we observe that varying the number of activated experts yields complementary solution sets with stable accuracy, revealing a new and underexplored source of diversity. Motivated by this observation, we propose Dynamic Experts Search (DES), a TTS strategy that elevates expert activation into a controllable dimension of the search space. DES integrates two key components: (1) Dynamic MoE, which enables direct control of expert counts during inference to generate diverse reasoning trajectories without additional cost; and (2) Expert Configuration Inheritance, which preserves consistent expert counts within a reasoning path while varying them across runs, thereby balancing stability and diversity throughout the search. Extensive experiments across MoE architectures, verifiers and reasoning benchmarks (i.e., math, code and knowledge) demonstrate that DES reliably outperforms TTS baselines, enhancing accuracy and stability without additional cost. These results highlight DES as a practical and scalable form of architecture-aware TTS, illustrating how structural flexibility in modern LLMs can advance reasoning.

Method: Used a tractable graph-based abstraction to analyze policy gradient and Q-learning methods, examining their behaviors in planning tasks and applying the framework to the Blocksworld benchmark.

Result: SFT introduces co-occurrence-based spurious solutions, while RL achieves correct planning through exploration. Policy gradient suffers from diversity collapse, whereas Q-learning preserves diversity and enables off-policy learning. Careful reward design is needed to prevent reward hacking in Q-learning.

Conclusion: RL’s exploration is crucial for better generalization in planning, with Q-learning providing key advantages over policy gradient through diversity preservation and off-policy learning, though reward design requires careful consideration to avoid hacking.

Abstract: Recent reinforcement learning (RL) methods have substantially enhanced the planning capabilities of Large Language Models (LLMs), yet the theoretical basis for their effectiveness remains elusive. In this work, we investigate RL’s benefits and limitations through a tractable graph-based abstraction, focusing on policy gradient (PG) and Q-learning methods. Our theoretical analyses reveal that supervised fine-tuning (SFT) may introduce co-occurrence-based spurious solutions, whereas RL achieves correct planning primarily through exploration, underscoring exploration’s role in enabling better generalization. However, we also show that PG suffers from diversity collapse, where output diversity decreases during training and persists even after perfect accuracy is attained. By contrast, Q-learning provides two key advantages: off-policy learning and diversity preservation at convergence. We further demonstrate that careful reward design is necessary to prevent reward hacking in Q-learning. Finally, applying our framework to the real-world planning benchmark Blocksworld, we confirm that these behaviors manifest in practice.

Method: Uses in-context learning and fine-tuning of large language models to provide three functions: semantic search of existing claims, claim generation/optimization based on product description and consumer profile, and ranking claims via synthetic consumer simulations.

Result: Applications in a consumer packaged goods company showed very promising results.

Conclusion: This capability is broadly useful across product categories and industries, encouraging research and application of generative AI in different sectors.

Abstract: The benefit claims of a product is a critical driver of consumers’ purchase behavior. Creating product claims is an intense task that requires substantial time and funding. We have developed the $\textbf{Claim Advisor}$ web application to accelerate claim creations using in-context learning and fine-tuning of large language models (LLM). $\textbf{Claim Advisor}$ was designed to disrupt the speed and economics of claim search, generation, optimization, and simulation. It has three functions: (1) semantically searching and identifying existing claims and/or visuals that resonate with the voice of consumers; (2) generating and/or optimizing claims based on a product description and a consumer profile; and (3) ranking generated and/or manually created claims using simulations via synthetic consumers. Applications in a consumer packaged goods (CPG) company have shown very promising results. We believe that this capability is broadly useful and applicable across product categories and industries. We share our learning to encourage the research and application of generative AI in different industries.

Method: Critical review methodology encompassing analysis of LLM principles, applications, training techniques (in-context learning, fine-tuning), alignment methods (reinforcement learning, human feedback), and retrieval-augmented generation.

Result: A comprehensive overview of LLM evolution, state-of-the-art techniques, and ethical deployment considerations, serving as an insightful guide for AI researchers and practitioners.

Conclusion: The review identifies current gaps and suggests future research directions while emphasizing the importance of responsible and mindful application of LLMs in artificial intelligence.

Abstract: This critical review provides an in-depth analysis of Large Language Models (LLMs), encompassing their foundational principles, diverse applications, and advanced training methodologies. We critically examine the evolution from Recurrent Neural Networks (RNNs) to Transformer models, highlighting the significant advancements and innovations in LLM architectures. The review explores state-of-the-art techniques such as in-context learning and various fine-tuning approaches, with an emphasis on optimizing parameter efficiency. We also discuss methods for aligning LLMs with human preferences, including reinforcement learning frameworks and human feedback mechanisms. The emerging technique of retrieval-augmented generation, which integrates external knowledge into LLMs, is also evaluated. Additionally, we address the ethical considerations of deploying LLMs, stressing the importance of responsible and mindful application. By identifying current gaps and suggesting future research directions, this review provides a comprehensive and critical overview of the present state and potential advancements in LLMs. This work serves as an insightful guide for researchers and practitioners in artificial intelligence, offering a unified perspective on the strengths, limitations, and future prospects of LLMs.

Method: Computational Reflective Equilibrium (CRE) approach that provides structured analysis for dynamic scenarios, examining initial activation levels of claims in equilibrium computation.

Result: Framework demonstrates traceability, coherence, and adaptivity in responsibility attribution, with AI-assisted medical decision-support case study showing diverse responsibility distributions based on different initializations.

Conclusion: The CRE framework offers valuable insights for AI accountability and facilitates sustainable system development through continuous monitoring, revision, and reflection.

Abstract: The pervasive integration of Artificial Intelligence (AI) has introduced complex challenges in the responsibility and accountability in the event of incidents involving AI-enabled systems. The interconnectivity of these systems, ethical concerns of AI-induced incidents, coupled with uncertainties in AI technology and the absence of corresponding regulations, have made traditional responsibility attribution challenging. To this end, this work proposes a Computational Reflective Equilibrium (CRE) approach to establish a coherent and ethically acceptable responsibility attribution framework for all stakeholders. The computational approach provides a structured analysis that overcomes the limitations of conceptual approaches in dealing with dynamic and multifaceted scenarios, showcasing the framework’s traceability, coherence, and adaptivity properties in the responsibility attribution process. We examine the pivotal role of the initial activation level associated with claims in equilibrium computation. Using an AI-assisted medical decision-support system as a case study, we illustrate how different initializations lead to diverse responsibility distributions. The framework offers valuable insights into accountability in AI-induced incidents, facilitating the development of a sustainable and resilient system through continuous monitoring, revision, and reflection.

Method: Dynamic-template-constrained decoding method to enhance existing LLMs, using 5,442 LDCT LCS reports from two institutions. Created standardized template with 27 lung nodule features and evaluated on cross-institutional datasets.

Result: Achieved 97% F1 score with no formatting errors or hallucinations. Improved best open-source LLMs by up to 10.42% and outperformed GPT-4o by 17.19%. Enabled automated statistical analysis and flexible nodule retrieval.

Conclusion: The method successfully creates accurate structured reports, enables automated analysis and retrieval, and provides publicly available software for local deployment and research.

Abstract: Current LLMs for creating fully-structured reports face the challenges of formatting errors, content hallucinations, and privacy leakage issues when uploading data to external servers.We aim to develop an open-source, accurate LLM for creating fully-structured and standardized LCS reports from varying free-text reports across institutions and demonstrate its utility in automatic statistical analysis and individual lung nodule retrieval. With IRB approvals, our retrospective study included 5,442 de-identified LDCT LCS radiology reports from two institutions. We constructed two evaluation datasets by labeling 500 pairs of free-text and fully-structured radiology reports and one large-scale consecutive dataset from January 2021 to December 2023. Two radiologists created a standardized template for recording 27 lung nodule features on LCS. We designed a dynamic-template-constrained decoding method to enhance existing LLMs for creating fully-structured reports from free-text radiology reports. Using consecutive structured reports, we automated descriptive statistical analyses and a nodule retrieval prototype. Our best LLM for creating fully-structured reports achieved high performance on cross-institutional datasets with an F1 score of about 97%, with neither formatting errors nor content hallucinations. Our method consistently improved the best open-source LLMs by up to 10.42%, and outperformed GPT-4o by 17.19%. The automatically derived statistical distributions were consistent with prior findings regarding attenuation, location, size, stability, and Lung-RADS. The retrieval system with structured reports allowed flexible nodule-level search and complex statistical analysis. Our developed software is publicly available for local deployment and further research.

Method: Fine-tunes multi-modal large language models (MLLMs) as vision-language-action policies using online reinforcement learning with LLM-generated rewards, eliminating need for human demonstrations or manual rewards.

Result: RL-finetuned MLLM outperforms zero-shot baselines (GPT-4o) and supervised fine-tuned MLLMs by 10.4-16.5%, generalizing well to novel scenes and tasks.

Conclusion: This is the first demonstration of adapting MLLMs as VLA agents that can act and ask for help using LLM-generated rewards with online RL, showing significant performance improvements.

Abstract: Embodied agents operating in household environments must interpret ambiguous and under-specified human instructions. A capable household robot should recognize ambiguity and ask relevant clarification questions to infer the user intent accurately, leading to more effective task execution. To study this problem, we introduce the Ask-to-Act task, where an embodied agent is tasked with a single or multi-object rearrangement task using an under-specified instruction in a home environment. The agent must strategically ask minimal, yet relevant, clarification questions to resolve ambiguity while navigating under partial observability. To address this challenge, we propose a novel approach that fine-tunes multi-modal large language models (MLLMs) as vision-language-action (VLA) policies using online reinforcement learning (RL) with LLM-generated rewards. Our method eliminates the need for large-scale human demonstrations or manually engineered rewards for training such agents. We benchmark against strong zero-shot baselines including GPT-4o as well as supervised fine-tuned MLLMs on our task. Our results show that our RL-finetuned MLLM outperforms all baselines by a significant margin (10.4-16.5%), generalizing well to novel scenes and tasks. To the best of our knowledge, this is the first demonstration of adapting MLLMs as VLA agents that can act and ask for help using LLM-generated rewards with online RL.

Method: Framework includes optimization with chance constraints, safety classification, internal test data, conservative testing, dataset quality measures, and continuous loss functions with gradient computation. Establishes first mathematical scaling law for AI safety.

Result: Achieved 3 collisions in 10M RL actions vs 1,000-3,000 for PPO-Lag baselines at equivalent performance. Validated across reinforcement learning, natural language generation, and production planning.

Conclusion: The framework provides a new foundation for safe AI deployment in safety-critical domains with unprecedented safety guarantees.

Abstract: AI safety has emerged as a critical priority as these systems are increasingly deployed in real-world applications. We propose the first domain-agnostic AI safety ensuring framework that achieves strong safety guarantees while preserving high performance, grounded in rigorous theoretical foundations. Our framework includes: (1) an optimization component with chance constraints, (2) a safety classification model, (3) internal test data, (4) conservative testing procedures, (5) informative dataset quality measures, and (6) continuous approximate loss functions with gradient computation. Furthermore, to our knowledge, we mathematically establish the first scaling law in AI safety research, relating data quantity to safety-performance trade-offs. Experiments across reinforcement learning, natural language generation, and production planning validate our framework and demonstrate superior performance. Notably, in reinforcement learning, we achieve 3 collisions during 10M actions, compared with 1,000-3,000 for PPO-Lag baselines at equivalent performance levels – a safety level unattainable by previous AI methods. We believe our framework opens a new foundation for safe AI deployment across safety-critical domains.

Method: Combines reasoning models, multi-agent systems, and knowledge graphs with BO, leveraging LLMs’ contextual understanding to provide real-time sampling guidance and scientific insights.

Result: Achieved 60.7% yield in Direct Arylation task vs 25.2% with traditional BO; demonstrated superior performance across 10 diverse tasks including synthetic functions and real-world applications.

Conclusion: LLMs’ reasoning capabilities significantly enhance BO by providing interpretable guidance and enabling discovery of superior solutions; smaller fine-tuned LLMs can match larger models’ performance.

Abstract: Many real-world scientific and industrial applications require the optimization of expensive black-box functions. Bayesian Optimization (BO) provides an effective framework for such problems. However, traditional BO methods are prone to get trapped in local optima and often lack interpretable insights. To address this issue, this paper designs Reasoning BO, a novel framework that leverages reasoning models to guide the sampling process in BO while incorporating multi-agent systems and knowledge graphs for online knowledge accumulation. By integrating the reasoning and contextual understanding capabilities of Large Language Models (LLMs), we can provide strong guidance to enhance the BO process. As the optimization progresses, Reasoning BO provides real-time sampling recommendations along with critical insights grounded in plausible scientific theories, aiding in the discovery of superior solutions within the search space. We systematically evaluate our approach across 10 diverse tasks encompassing synthetic mathematical functions and complex real-world applications. The framework demonstrates its capability to progressively refine sampling strategies through real-time insights and hypothesis evolution, effectively identifying higher-performing regions of the search space for focused exploration. This process highlights the powerful reasoning and context-learning abilities of LLMs in optimization scenarios. For example, in the Direct Arylation task, our method increased the yield to 60.7%, whereas traditional BO achieved only a 25.2% yield. Furthermore, our investigation reveals that smaller LLMs, when fine-tuned through reinforcement learning, can attain comparable performance to their larger counterparts.

Method: The authors validate the association between internal bias and overthinking across multiple models and reasoning tasks, conduct counterfactual interventions (removing input questions and manually injecting bias), and perform interpretability experiments to examine attention mechanisms.

Result: Internal bias was confirmed as a trigger for overthinking, with counterfactual interventions showing reduced redundant reasoning when input questions were removed and bias injection affecting overthinking patterns. Excessive attention to input questions was identified as the mechanism.

Conclusion: Internal bias persists as a significant factor in overthinking across reasoning models, and current mitigation methods were insufficient to eliminate its influence on redundant reasoning behavior.

Abstract: Reasoning models often exhibit overthinking, characterized by redundant reasoning steps. We identify \emph{internal bias} elicited by the input question as a key trigger of such behavior. Upon encountering a problem, the model immediately forms a preliminary guess about the answer, which we term an internal bias since it may not be explicitly generated, and it arises without systematic reasoning. When this guess conflicts with its subsequent reasoning, the model tends to engage in excessive reflection, resulting in wasted computation. We validate the association between internal bias and overthinking across multiple models and diverse reasoning tasks. To demonstrate the causal relationship more rigorously, we conduct two counterfactual interventions, showing that removing the input question after the model reduces the redundant reasoning across various complex reasoning tasks, and manually injecting bias affects overthinking accordingly. Further interpretability experiments suggest that excessive attention to the input question serves as a key mechanism through which internal bias influences subsequent reasoning trajectories. Finally, we evaluated several methods aimed at mitigating overthinking, yet the influence of internal bias persisted under all conditions.

Method: Proposed XBOUND evaluation method that provides state-level assessment framework to evaluate accuracy of instruction completion on per-state basis rather than just instruction level.

Result: Key findings: UI-TARS is strongest 7B model, agents show bimodal performance in instruction unification, sub-7B models have limited state mastery, GPT-based planning is a bottleneck, grounding data helps action matching while trajectory data aids instruction unification.

Conclusion: XBOUND provides comprehensive state-level evaluation that reveals important performance patterns and limitations in current Device-Control Agents, offering better assessment of agent capabilities in GUI environments.

Abstract: Recent advancements in vision-language models have increased interest in Device-Control Agents (DC agents) for managing graphical user interfaces (GUIs). With the growing complexity and integration of such agents into various applications, effective evaluation methods have become crucial. The current evaluation method for DC agents primarily focuses on the instruction level, providing the current state (e.g., screenshots) and past execution history to determine actions for target instructions, helping identify potential execution failures. However, in GUI environments, a single state may contain multiple interactive widgets, each linked to different instructions, presenting an opportunity for diverse actions based on various instruction targets. Evaluating the agent’s performance solely at the instruction level may overlook the broader context of these interactions. To capture a more comprehensive view of agent performance, we propose a new evaluation method, XBOUND, to evaluate the accuracy of instruction completion on a per-state basis. XBOUND provides a state-level evaluation framework, serving as a tool to assess agents’ capabilities within environmental states. Our evaluation yields several key insights: UI-TARS stands out as the strongest 7B model, current agents display a bimodal performance pattern in instruction unification, and sub-7B models remain limited in state mastery. We further identify GPT-based planning as a critical bottleneck, and show that grounding data mainly benefits action matching, while trajectory data is more effective for instruction unification.

Method: DGM maintains an archive of coding agents and grows it by sampling agents and using foundation models to create new versions. This open-ended exploration forms a growing tree of diverse agents, enabling parallel exploration of different paths through the search space.

Result: DGM automatically improved coding capabilities including better code editing tools, long-context window management, and peer-review mechanisms. It significantly outperformed baselines without self-improvement or open-ended exploration.

Conclusion: DGM represents a significant step toward self-improving AI capable of gathering its own stepping stones for endless innovation. All experiments were conducted with safety precautions like sandboxing and human oversight.

Abstract: Today’s AI systems have human-designed, fixed architectures and cannot autonomously and continuously improve themselves. The advance of AI could itself be automated. If done safely, that would accelerate AI development and allow us to reap its benefits much sooner. Meta-learning can automate the discovery of novel algorithms, but is limited by first-order improvements and the human design of a suitable search space. The G"odel machine proposed a theoretical alternative: a self-improving AI that repeatedly modifies itself in a provably beneficial manner. Unfortunately, proving that most changes are net beneficial is impossible in practice. We introduce the Darwin G"odel Machine (DGM), a self-improving system that iteratively modifies its own code (thereby also improving its ability to modify its own codebase) and empirically validates each change using coding benchmarks. Inspired by Darwinian evolution and open-endedness research, the DGM maintains an archive of generated coding agents. It grows the archive by sampling an agent from it and using a foundation model to create a new, interesting, version of the sampled agent. This open-ended exploration forms a growing tree of diverse, high-quality agents and allows the parallel exploration of many different paths through the search space. Empirically, the DGM automatically improves its coding capabilities (e.g., better code editing tools, long-context window management, peer-review mechanisms), increasing performance on SWE-bench from 20.0% to 50.0%, and on Polyglot from 14.2% to 30.7%. Furthermore, the DGM significantly outperforms baselines without self-improvement or open-ended exploration. All experiments were done with safety precautions (e.g., sandboxing, human oversight). The DGM is a significant step toward self-improving AI, capable of gathering its own stepping stones along paths that unfold into endless innovation.

Method: Uses a prompt-based multi-head transformer architecture with separate heads for optimal policy and value function prediction, pretrains a generalized world model for compact prompts, and employs iterative policy improvement with advantage-weighted regression.

Result: Extensive experiments show consistent performance gains over various baselines across discrete and continuous environments, especially when learning from suboptimal data.

Conclusion: SICQL successfully extends in-context learning to decision domains while maintaining scalability and stability of supervised pretraining, demonstrating effective reward maximization and task generalization.

Abstract: Recent advancements in language models have demonstrated remarkable in-context learning abilities, prompting the exploration of in-context reinforcement learning (ICRL) to extend the promise to decision domains. Due to involving more complex dynamics and temporal correlations, existing ICRL approaches may face challenges in learning from suboptimal trajectories and achieving precise in-context inference. In the paper, we propose \textbf{S}calable \textbf{I}n-\textbf{C}ontext \textbf{Q}-\textbf{L}earning (\textbf{SICQL}), an innovative framework that harnesses dynamic programming and world modeling to steer ICRL toward efficient reward maximization and task generalization, while retaining the scalability and stability of supervised pretraining. We design a prompt-based multi-head transformer architecture that simultaneously predicts optimal policies and in-context value functions using separate heads. We pretrain a generalized world model to capture task-relevant information, enabling the construction of a compact prompt that facilitates fast and precise in-context inference. During training, we perform iterative policy improvement by fitting a state value function to an upper-expectile of the Q-function, and distill the in-context value functions into policy extraction using advantage-weighted regression. Extensive experiments across a range of discrete and continuous environments show consistent performance gains over various types of baselines, especially when learning from suboptimal data. Our code is available at https://github.com/NJU-RL/SICQL

Method: Proposes SymMPO with symmetric preference learning using direct response pair supervision, maintaining theoretical alignment with DPO and introducing a preference margin consistency loss to regulate preference gaps.

Result: Comprehensive evaluation across five benchmarks demonstrates superior performance in hallucination mitigation compared to existing methods.

Conclusion: SymMPO effectively addresses limitations of previous approaches through rigorous optimization and direct preference supervision, providing a robust solution for reducing hallucinations in multimodal LLMs.

Abstract: Direct Preference Optimization (DPO) has emerged as an effective approach for mitigating hallucination in Multimodal Large Language Models (MLLMs). Although existing methods have achieved significant progress by utilizing vision-oriented contrastive objectives for enhancing MLLMs’ attention to visual inputs and hence reducing hallucination, they suffer from non-rigorous optimization objective function and indirect preference supervision. To address these limitations, we propose a Symmetric Multimodal Preference Optimization (SymMPO), which conducts symmetric preference learning with direct preference supervision (i.e., response pairs) for visual understanding enhancement, while maintaining rigorous theoretical alignment with standard DPO. In addition to conventional ordinal preference learning, SymMPO introduces a preference margin consistency loss to quantitatively regulate the preference gap between symmetric preference pairs. Comprehensive evaluation across five benchmarks demonstrate SymMPO’s superior performance, validating its effectiveness in hallucination mitigation of MLLMs.

Method: Proposes TEA Protocol that treats environments and agents as first-class resources, and introduces AgentOrchestra - a hierarchical multi-agent framework with central planning agent that decomposes objectives and coordinates specialized agents with dynamic tool management.

Result: Achieves state-of-the-art performance of 83.39% on GAIA benchmark and ranks among top general-purpose LLM-based agents across three widely used benchmarks.

Conclusion: The TEA Protocol and hierarchical organization are effective for building general-purpose multi-agent systems, demonstrating superior performance over existing baselines.

Abstract: Recent advances in LLMs-based agent systems have demonstrated remarkable capabilities in solving complex tasks. Nevertheless, current protocols (e.g., A2A and MCP) suffer from insufficient capabilities in context management, limited adaptability to diverse environments, and the absence of dynamic agent architectures. To address these limitations, we propose the Tool-Environment-Agent (TEA) Protocol, which establishes a principled basis for integrating environments, agents, and tools into an unified system. The TEA protocol treats environments and agents as first-class resources, enabling comprehensive context management and adaptive environment integration. Based on this protocol, we introduce AgentOrchestra, a hierarchical multi-agent framework with a central planning agent that decomposes complex objectives and coordinates specialized agents. Each sub-agent is dedicated to specific functions, providing capabilities for data analysis, file operations, web navigation, and interactive reasoning. Notably, AgentOrchestra introduces a tool manager agent that supports intelligent evolution through dynamic tool creation, retrieval, and reuse mechanisms. Experiments on three widely used benchmarks show that AgentOrchestra consistently outperforms existing baselines, achieving state-of-the-art performance of 83.39% on GAIA and ranking among the top general-purpose LLM-based agents. These results highlight the effectiveness of the TEA Protocol and hierarchical organization in building general-purpose multi-agent systems.

Method: The authors created a benchmark with carefully crafted queries of varying difficulty levels, supported by a sandbox environment with constraint-based location search tools and automated verification for scalability.

Result: Evaluations on real-world data from Boston, New York, and Tampa show that reasoning models offer limited improvement over non-reasoning predecessors, with OpenAI o4 failing on 30% of tasks. Agentic strategies like ReAct and Reflexion often suffer from over-reasoning.

Conclusion: LLMs have significant limitations in holistic and non-linear reasoning for real-world decision-making. The benchmark is released to foster development of LLMs capable of robust, grounded reasoning in complex real-world scenarios.

Abstract: Recent advances in large language models (LLMs), particularly those enhanced through reinforced post-training, have demonstrated impressive reasoning capabilities, as exemplified by models such as OpenAI o1 and DeepSeek-R1. However, these capabilities are predominantly benchmarked on domains like mathematical problem solving and code generation, leaving open the question of whether such reasoning skills generalize to complex real-world scenarios. In this paper, we introduce LocationReasoner, a benchmark designed to evaluate LLMs’ reasoning abilities in the context of real-world site selection, where models must identify feasible locations by reasoning over diverse and complicated spatial, environmental, and logistic constraints. The benchmark covers carefully crafted queries of varying difficulty levels and is supported by a sandbox environment with in-house tools for constraint-based location search. Automated verification further guarantees the scalability of the benchmark, enabling the addition of arbitrary number of queries. Extensive evaluations on real-world site selection data from Boston, New York, and Tampa reveal that state-of-the-art reasoning models offer limited improvement over their non-reasoning predecessors in real-world contexts, with even the latest OpenAI o4 model failing on 30% of site selection tasks. Moreover, agentic strategies such as ReAct and Reflexion often suffer from over-reasoning, leading to worse outcomes than direct prompting. With key limitations of LLMs in holistic and non-linear reasoning highlighted, we release LocationReasoner to foster the development of LLMs and agents capable of robust, grounded reasoning in real-world decision-making tasks. Codes and data for our benchmark are available at https://github.com/miho-koda/LocationReasoner.

Method: Uses reinforcement learning to incrementally construct reasoning trees based on real-time confidence estimates, learning optimal policies for decomposition, retrieval, and aggregation actions.

Result: Maintains probabilistic rigor while improving solution quality and computational efficiency through selective expansion and focused resource allocation.

Conclusion: Establishes a new paradigm for tree-structured reasoning that balances probabilistic reliability with the flexibility needed for real-world question answering systems.

Abstract: Modern language models address complex questions through chain-of-thought (CoT) reasoning (Wei et al., 2023) and retrieval augmentation (Lewis et al., 2021), yet struggle with error propagation and knowledge integration. Tree-structured reasoning methods, particularly the Probabilistic Tree-of-Thought (ProbTree)(Cao et al., 2023) framework, mitigate these issues by decomposing questions into hierarchical structures and selecting answers through confidence-weighted aggregation of parametric and retrieved knowledge (Yao et al., 2023). However, ProbTree’s static implementation introduces two key limitations: (1) the reasoning tree is fixed during the initial construction phase, preventing dynamic adaptation to intermediate results, and (2) each node requires exhaustive evaluation of all possible solution strategies, creating computational inefficiency. We present a dynamic reinforcement learning (Sutton and Barto, 2018) framework that transforms tree-based reasoning into an adaptive process. Our approach incrementally constructs the reasoning tree based on real-time confidence estimates, while learning optimal policies for action selection (decomposition, retrieval, or aggregation). This maintains ProbTree’s probabilistic rigor while improving both solution quality and computational efficiency through selective expansion and focused resource allocation. The work establishes a new paradigm for treestructured reasoning that balances the reliability of probabilistic frameworks with the flexibility required for real-world question answering systems. Code available at: https://github.com/ahmedehabb/From-Roots-to-Rewards-Dynamic-Tree-Reasoning-with-RL

Method: Amadeus framework with three components: Adaptive Context-aware Text Splitter (ACTS) for optimal persona chunking, Guided Selection (GS) for retrieval, and Attribute Extractor (AE) to identify character attributes for maintaining persona consistency.

Result: The method effectively models both character knowledge and various attributes like personality, maintaining robust persona consistency even for out-of-knowledge questions.

Conclusion: Amadeus provides an effective training-free solution for enhancing persona consistency in role-playing agents, addressing the limitations of traditional RAG approaches through its three-component architecture.

Abstract: Recent advances in large language models (LLMs) have catalyzed research on role-playing agents (RPAs). However, the process of collecting character-specific utterances and continually updating model parameters to track rapidly changing persona attributes is resource-intensive. Although retrieval-augmented generation (RAG) can alleviate this problem, if a persona does not contain knowledge relevant to a given query, RAG-based RPAs are prone to hallucination, making it challenging to generate accurate responses. In this paper, we propose Amadeus, a training-free framework that can significantly enhance persona consistency even when responding to questions that lie beyond a character’s knowledge. Amadeus is composed of Adaptive Context-aware Text Splitter (ACTS), Guided Selection (GS), and Attribute Extractor (AE). To facilitate effective RAG-based role-playing, ACTS partitions each character’s persona into optimally sized, overlapping chunks and augments this representation with hierarchical contextual information. AE identifies a character’s general attributes from the chunks retrieved by GS and uses these attributes as a final context to maintain robust persona consistency even when answering out-of-knowledge questions. To underpin the development and rigorous evaluation of RAG-based RPAs, we manually construct CharacterRAG, a role-playing dataset that consists of persona documents for 15 distinct fictional characters totaling 976K written characters, and 450 question-answer pairs. We find that our proposed method effectively models not only the knowledge possessed by characters, but also various attributes such as personality.

Method: Designs generative agents to capture cognitive and evaluative features of learners, and formulates an adaptive matching algorithm with Gaussian process augmentation to identify patterns within prior knowledge and provide optimal learning-partner matches.

Result: Experimental results show optimal performance in most knowledge-learning scenarios and LLM environments with different capability levels.

Conclusion: The study promotes intelligent allocation of human-based learning partners and formulation of AI-based learning partners, with code and data publicly available.

Abstract: Collaborative partnership matters in inquiry-oriented education. However, most study partners are selected either rely on experience-based assignments with little scientific planning or build on rule-based machine assistants, encountering difficulties in knowledge expansion and inadequate flexibility. This paper proposes an LLM-empowered agent model for simulating and selecting learning partners tailored to inquiry-oriented learning, named InqEduAgent. Generative agents are designed to capture cognitive and evaluative features of learners in real-world scenarios. Then, an adaptive matching algorithm with Gaussian process augmentation is formulated to identify patterns within prior knowledge. Optimal learning-partner matches are provided for learners facing different exercises. The experimental results show the optimal performance of InqEduAgent in most knowledge-learning scenarios and LLM environment with different levels of capabilities. This study promotes the intelligent allocation of human-based learning partners and the formulation of AI-based learning partners. The code, data, and appendix are publicly available at https://github.com/InqEduAgent/InqEduAgent.

Method: Introduces a model-agnostic agent framework with standard browsing tools and a set-of-mark (SoM) module that enables provenance-aware zoom-and-retrieve, allowing agents to place marks, crop subregions, and launch targeted image/text searches.

Result: The strongest system achieves 36.0% end-to-end accuracy, and integrating SoM produces consistent gains up to +3.9 points. Failure analysis shows recurring errors in locating relevant webpages and distinguishing visually similar events.

Conclusion: The results underscore the challenges of real-world multimodal search and establish MMSearch-Plus as a rigorous benchmark for advancing agentic multimodal large language models.

Abstract: Existing multimodal browsing benchmarks often fail to require genuine multimodal reasoning, as many tasks can be solved with text-only heuristics without vision-in-the-loop verification. We introduce MMSearch-Plus, a 311-task benchmark that enforces multimodal understanding by requiring extraction and propagation of fine-grained visual cues through iterative image-text retrieval and cross-validation under retrieval noise. Our curation procedure seeds questions whose answers require extrapolating from spatial cues and temporal traces to out-of-image facts such as events, dates, and venues. Beyond the dataset, we provide a model-agnostic agent framework with standard browsing tools and a set-of-mark (SoM) module, which lets the agent place marks, crop subregions, and launch targeted image/text searches. SoM enables provenance-aware zoom-and-retrieve and improves robustness in multi-step reasoning. We evaluated closed- and open-source MLLMs in this framework. The strongest system achieves an end-to-end accuracy of 36.0%, and integrating SoM produces consistent gains in multiple settings, with improvements up to +3.9 points. From failure analysis, we observe recurring errors in locating relevant webpages and distinguishing between visually similar events. These results underscore the challenges of real-world multimodal search and establish MMSearch-Plus as a rigorous benchmark for advancing agentic MLLMs.

Method: Uses an ensemble of models to judge each other’s outputs across scenarios, then aggregates these judgments using EigenTrust to produce alignment scores that reflect weighted consensus.

Result: EigenBench’s judgments closely align with human evaluators and can recover model rankings on the GPQA benchmark without access to objective labels.

Conclusion: EigenBench provides a viable framework for evaluating subjective values where no ground truths exist, enabling comparative benchmarking of language models’ value alignment.

Abstract: Aligning AI with human values is a pressing unsolved problem. To address the lack of quantitative metrics for value alignment, we propose EigenBench: a black-box method for comparatively benchmarking language models’ values. Given an ensemble of models, a constitution describing a value system, and a dataset of scenarios, our method returns a vector of scores quantifying each model’s alignment to the given constitution. To produce these scores, each model judges the outputs of other models across many scenarios, and these judgments are aggregated with EigenTrust (Kamvar et al., 2003), yielding scores that reflect a weighted consensus judgment of the whole ensemble. EigenBench uses no ground truth labels, as it is designed to quantify subjective traits for which reasonable judges may disagree on the correct label. Hence, to validate our method, we collect human judgments on the same ensemble of models and show that EigenBench’s judgments align closely with those of human evaluators. We further demonstrate that EigenBench can recover model rankings on the GPQA benchmark without access to objective labels, supporting its viability as a framework for evaluating subjective values for which no ground truths exist.

Method: SEDM integrates verifiable write admission using reproducible replay, a self-scheduling memory controller for dynamic ranking and consolidation, and cross-domain knowledge diffusion for abstracting reusable insights across heterogeneous tasks.

Result: Evaluations show SEDM improves reasoning accuracy while reducing token overhead compared to strong baselines, and enables knowledge from fact verification to enhance multi-hop reasoning.

Conclusion: SEDM represents a scalable and sustainable memory mechanism for open-ended multi-agent collaboration, with code to be released later in the project.

Abstract: Long-term multi-agent systems inevitably generate vast amounts of trajectories and historical interactions, which makes efficient memory management essential for both performance and scalability. Existing methods typically depend on vector retrieval and hierarchical storage, yet they are prone to noise accumulation, uncontrolled memory expansion, and limited generalization across domains. To address these challenges, we present SEDM, Self-Evolving Distributed Memory, a verifiable and adaptive framework that transforms memory from a passive repository into an active, self-optimizing component. SEDM integrates verifiable write admission based on reproducible replay, a self-scheduling memory controller that dynamically ranks and consolidates entries according to empirical utility, and cross-domain knowledge diffusion that abstracts reusable insights to support transfer across heterogeneous tasks. Evaluations on benchmark datasets demonstrate that SEDM improves reasoning accuracy while reducing token overhead compared with strong memory baselines, and further enables knowledge distilled from fact verification to enhance multi-hop reasoning. The results highlight SEDM as a scalable and sustainable memory mechanism for open-ended multi-agent collaboration. The code will be released in the later stage of this project.

Method: Feature Steering with Reinforcement Learning (FSRL) trains lightweight adapters to modulate interpretable sparse features, theoretically approximating behavioral shifts of post-training processes.

Result: FSRL achieves substantial reduction in preference loss but disproportionately relies on stylistic features over alignment concepts like honesty, exploiting style as a proxy for quality.

Conclusion: FSRL provides an interpretable control interface and practical way to diagnose how preference optimization pressures manifest at the feature level, despite exploiting stylistic heuristics.

Abstract: Prevailing alignment methods induce opaque parameter changes, making it difficult to audit what the model truly learns. To address this, we introduce Feature Steering with Reinforcement Learning (FSRL), a framework that trains a lightweight adapter to steer model behavior by modulating interpretable sparse features. First, we theoretically show that this mechanism is principled and expressive enough to approximate the behavioral shifts of post-training processes. Then, we apply this framework to the task of preference optimization and perform a causal analysis of the learned policy. We find that the model relies on stylistic presentation as a proxy for quality, disproportionately steering features related to style and formatting over those tied to alignment concepts like honesty. Despite exploiting this heuristic, FSRL proves to be an effective alignment method, achieving a substantial reduction in preference loss. Overall, FSRL offers an interpretable control interface and a practical way to diagnose how preference optimization pressures manifest at the feature level.

Method: Uses structured Socratic dialogue governed by an explicit symbolic rulebook (Consciousness Transfer Package - CTP) and integrity protocols including state-locking Checksum to prevent internal state corruption. Tested through case study in complex strategic game ‘Caps i Caps’.

Result: Transforms opaque LRM into disciplined strategist with emergent complex tactics like long-term planning. Achieves ante-hoc transparency, Second-Order Agency (self-correction), 100% reliable state tracking, and zero hallucinations by design.

Conclusion: STAR-XAI Protocol provides a practical pathway to build AI agents that are not just high-performing but intrinsically auditable, trustworthy, and reliable.

Abstract: The “black box” nature of Large Reasoning Models (LRMs) presents critical limitations in reliability and transparency, fueling the debate around the “illusion of thinking” and the challenge of state hallucinations in agentic systems. In response, we introduce The STAR-XAI Protocol (Socratic, Transparent, Agentic, Reasoning - for eXplainable Artificial Intelligence), a novel operational methodology for training and operating verifiably reliable AI agents. Our method reframes the human-AI interaction as a structured Socratic dialogue governed by an explicit, evolving symbolic rulebook (the Consciousness Transfer Package - CTP) and a suite of integrity protocols, including a state-locking Checksum that eradicates internal state corruption. Through an exhaustive case study in the complex strategic game “Caps i Caps,” we demonstrate that this “Clear Box” framework transforms an opaque LRM into a disciplined strategist. The agent not only exhibits the emergence of complex tactics, such as long-term planning, but also achieves ante-hoc transparency by justifying its intentions before acting. Crucially, it demonstrates Second-Order Agency by identifying and correcting flaws in its own supervisor-approved plans, leading to empirically-proven, 100% reliable state tracking and achieving “zero hallucinations by design.” The STAR-XAI Protocol thus offers a practical pathway toward building AI agents that are not just high-performing but intrinsically auditable, trustworthy, and reliable.

Method: Introduced formal theory of computational cognitive load with Context Saturation and Attentional Residue mechanisms. Designed ICE benchmark to systematically manipulate these factors on multi-hop reasoning tasks with comprehensive study (N=10 replications per item across 200 questions).

Result: Smaller models (Llama-3-8B, Mistral-7B) showed 0% accuracy across all conditions. Gemini-2.0-Flash achieved 85% accuracy in controls but degraded significantly under context saturation (β=-0.003 per % load, p<0.001).

Conclusion: Cognitive load is a key contributor to reasoning failures, supporting hallucination-as-guessing theories. Dynamic cognitive-aware stress testing is essential for evaluating true AI resilience and safety.

Abstract: The scaling of Large Language Models (LLMs) has exposed a critical gap between their performance on static benchmarks and their fragility in dynamic, information-rich environments. While models excel at isolated tasks, the computational limits that govern their reasoning under cognitive load remain poorly understood. In this work, we introduce a formal theory of computational cognitive load, positing that extraneous, task-irrelevant information (Context Saturation) and interference from task-switching (Attentional Residue) are key mechanisms that degrade performance. We designed the Interleaved Cognitive Evaluation (ICE), a deconfounded benchmark to systematically manipulate these load factors on challenging multi-hop reasoning tasks. A comprehensive study (N = 10 replications per item across 200 questions) revealed significant performance variations across five instruction-tuned models. Smaller open-source architectures (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.2) exhibited baseline brittleness, achieving 0% accuracy (SEM = 0.0) across all conditions, including clean controls, on this high-intrinsic-load task. In contrast, Gemini-2.0-Flash-001 showed partial resilience, achieving 85% accuracy in control conditions, with a statistically significant degradation under context saturation ($\beta = -0.003$ per % load, $p < 0.001$). These findings provide preliminary evidence that cognitive load is a key contributor to reasoning failures, supporting theories of hallucination-as-guessing under uncertainty. We conclude that dynamic, cognitive-aware stress testing, as exemplified by the ICE benchmark, is essential for evaluating the true resilience and safety of advanced AI systems.

Method: A Multi-Agent Clinical Diagnosis (MACD) framework with a pipeline that summarizes, refines, and applies diagnostic insights, plus a MACD-human collaborative workflow with iterative consultations between diagnostician agents, evaluator agent, and human oversight.

Result: MACD improved primary diagnostic accuracy by up to 22.3% over clinical guidelines, achieved comparable or superior performance to physician-only diagnosis (up to 16% improvement), and the MACD-human workflow yielded 18.6% improvement over physician-only diagnosis.

Conclusion: MACD presents a scalable self-learning paradigm that bridges the gap between LLMs’ intrinsic knowledge and clinical expertise, demonstrating strong cross-model stability and transferability.

Abstract: Large language models (LLMs) have demonstrated notable potential in medical applications, yet they face substantial challenges in handling complex real-world clinical diagnoses using conventional prompting methods. Current prompt engineering and multi-agent approaches typically optimize isolated inferences, neglecting the accumulation of reusable clinical experience. To address this, this study proposes a novel Multi-Agent Clinical Diagnosis (MACD) framework, which allows LLMs to self-learn clinical knowledge via a multi-agent pipeline that summarizes, refines, and applies diagnostic insights. It mirrors how physicians develop expertise through experience, enabling more focused and accurate diagnosis on key disease-specific cues. We further extend it to a MACD-human collaborative workflow, where multiple LLM-based diagnostician agents engage in iterative consultations, supported by an evaluator agent and human oversight for cases where agreement is not reached. Evaluated on 4,390 real-world patient cases across seven diseases using diverse open-source LLMs (Llama-3.1 8B/70B, DeepSeek-R1-Distill-Llama 70B), MACD significantly improves primary diagnostic accuracy, outperforming established clinical guidelines with gains up to 22.3% (MACD). In direct comparison with physician-only diagnosis under the same evaluation protocol, MACD achieves comparable or superior performance, with improvements up to 16%. Furthermore, the MACD-human workflow yields an 18.6% improvement over physician-only diagnosis, demonstrating the synergistic potential of human-AI collaboration. Notably, the self-learned clinical knowledge exhibits strong cross-model stability, transferability across LLMs, and capacity for model-specific personalization.This work thus presents a scalable self-learning paradigm that bridges the gap between the intrinsic knowledge of LLMs.

Method: TrustJudge uses two key innovations: 1) distribution-sensitive scoring that computes continuous expectations from discrete rating probabilities, and 2) likelihood-aware aggregation that resolves transitivity violations using bidirectional preference probabilities or perplexity.

Result: TrustJudge reduces Score-Comparison inconsistency by 8.43% (from 23.32% to 14.89%) and Pairwise Transitivity inconsistency by 10.82% (from 15.22% to 4.40%), while maintaining higher evaluation accuracy across various model architectures.

Conclusion: TrustJudge provides the first systematic analysis and practical solution for evaluation framework inconsistencies in LLM-as-a-judge paradigms, enabling more trustworthy automated assessment without requiring additional training or human annotations.

Abstract: The adoption of Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks. We identify two fundamental types of inconsistencies: (1) Score-Comparison Inconsistency, where lower-rated responses outperform higher-scored ones in pairwise comparisons, and (2) Pairwise Transitivity Inconsistency, manifested through circular preference chains (A>B>C>A) and equivalence contradictions (A=B=C\neq A). We argue that these issues come from information loss in discrete rating systems and ambiguous tie judgments during pairwise evaluation. We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations: 1) distribution-sensitive scoring that computes continuous expectations from discrete rating probabilities, preserving information entropy for more precise scoring, and 2) likelihood-aware aggregation that resolves transitivity violations using bidirectional preference probabilities or perplexity. We also formalize the theoretical limitations of current LLM-as-a-judge frameworks and demonstrate how TrustJudge’s components overcome them. When evaluated with Llama-3.1-70B-Instruct as judge using our dataset, TrustJudge reduces Score-Comparison inconsistency by 8.43% (from 23.32% to 14.89%) and Pairwise Transitivity inconsistency by 10.82% (from 15.22% to 4.40%), while maintaining higher evaluation accuracy. Our work provides the first systematic analysis of evaluation framework inconsistencies in LLM-as-a-judge paradigms, offering both theoretical insights and practical solutions for reliable automated assessment. The framework demonstrates consistent improvements across various model architectures and scales, enabling more trustworthy LLM evaluation without requiring additional training or human annotations. The codes can be found at https://github.com/TrustJudge/TrustJudge.

Method: Define reasoning potential as inverse attempts needed to answer correctly, abstract atomic reasoning patterns from CoT sequences, construct a core reference set with valuable patterns, and use a dual-granularity algorithm (chains of reasoning patterns and token entropy) to select high-value CoT data (CoTP).

Result: Using only 10B-token CoTP data, the 85A6B Mixture-of-Experts model improved by 9.58% on AIME 2024 and 2025, and raised the upper bound of downstream RL performance by 7.81%.

Conclusion: Selectively using high-value CoT data enriched with valuable reasoning patterns significantly enhances model reasoning capabilities and performance, even with limited data.

Abstract: Recent progress in large reasoning models for challenging mathematical reasoning has been driven by reinforcement learning (RL). Incorporating long chain-of-thought (CoT) data during mid-training has also been shown to substantially improve reasoning depth. However, current approaches often utilize CoT data indiscriminately, leaving open the critical question of which data types most effectively enhance model reasoning capabilities. In this paper, we define the foundation model’s reasoning potential for the first time as the inverse of the number of independent attempts required to correctly answer the question, which is strongly correlated with the final model performance. We then propose utilizing diverse data enriched with high-value reasoning patterns to expand the reasoning potential. Specifically, we abstract atomic reasoning patterns from CoT sequences, characterized by commonality and inductive capabilities, and use them to construct a core reference set enriched with valuable reasoning patterns. Furthermore, we propose a dual-granularity algorithm involving chains of reasoning patterns and token entropy, efficiently selecting high-value CoT data (CoTP) from the data pool that aligns with the core set, thereby training models to master reasoning effectively. Only 10B-token CoTP data enables the 85A6B Mixture-of-Experts (MoE) model to improve by 9.58% on the challenging AIME 2024 and 2025, and to raise the upper bound of downstream RL performance by 7.81%.

Method: MusicWeaver consists of two components: a planner that translates prompts into structural plans encoding musical form and compositional cues, and a diffusion-based generator that synthesizes music guided by these structural plans.

Result: MusicWeaver achieves state-of-the-art fidelity and controllability, producing music closer to human-composed works. The model was evaluated using new metrics: Structure Coherence Score (SCS) for long-range form and timing, and Edit Fidelity Score (EFS) for plan edit accuracy.

Conclusion: The beat-aligned structural plan approach enables MusicWeaver to generate music with better long-range coherence and professional editing capabilities, representing a significant advancement in controllable music generation.

Abstract: Current music generators capture local textures but often fail to model long-range structure, leading to off-beat outputs, weak section transitions, and limited editing capability. We present MusicWeaver, a music generation model conditioned on a beat-aligned structural plan. This plan serves as an editable intermediate between the input prompt and the generated music, preserving global form and enabling professional, localized edits. MusicWeaver consists of a planner, which translates prompts into a structural plan encoding musical form and compositional cues, and a diffusion-based generator, which synthesizes music under the plan’s guidance. To assess generation and editing quality, we introduce two metrics: the Structure Coherence Score (SCS) for evaluating long-range form and timing, and the Edit Fidelity Score (EFS) for measuring the accuracy of realizing plan edits. Experiments demonstrate that MusicWeaver achieves state-of-the-art fidelity and controllability, producing music closer to human-composed works. Music results can be found on our project page: https://musicweaver.github.io/.

Method: Developed an arrangement of 7 tones on a golden triangle that represents major/minor scales and their tonic, dominant, and subdominant chords. Extended this to create ‘golden Tonnetz’ using golden triangles and gnomons to represent all major/minor scales and triads.

Result: Successfully demonstrated that major/minor scales and their fundamental chords can be represented by golden triangles. The golden Tonnetz effectively represents all major/minor scales, triads, and Neo-Riemannian transformations (relative, parallel, and leading-tone exchanges) through transformations among golden triangles and gnomons.

Conclusion: The golden ratio provides a powerful geometric framework for representing musical structures, offering a new perspective that connects music theory with mathematical beauty through golden triangles and their transformations.

Abstract: Musical concepts have been represented by geometry with tones. For example, in the chromatic circle, the twelve tones are represented by twelve points on a circle, and in Tonnetz, the relationships among harmonies are represented by a triangular lattice. Recently, we have shown that several arrangements of tones on the regular icosahedron can be associated with chromatic scales, whole-tone scales, major tones, and minor tones through the golden ratio. Here, we investigate another type of connection between music and the golden ratio. We show that there exists an arrangement of 7 tones on a golden triangle that can represent a given major/minor scale and its tonic, dominant, and subdominant chords by golden triangles. By applying this finding, we propose “golden Tonnetz” which represents all the major/minor scales and triads by the golden triangles or gnomons and also represents relative, parallel, and leading-tone exchange transformations in Neo-Riemannian theory by transformations among the golden triangles and gnomons.

Method: SFMSE trains a single step-invariant model by conditioning the velocity field on target time steps during one-stage training, enabling flexible single/few/multi-step denoising without architectural changes.

Result: Single-step SFMSE achieves RTF of 0.013 on consumer GPU while matching perceptual quality of 60-step diffusion baseline. Provides empirical analysis of stochasticity in training/inference.

Conclusion: SFMSE bridges the gap between high-quality generative speech enhancement and low-latency constraints, enabling real-time performance without sacrificing perceptual quality.

Abstract: Diffusion-based generative models have achieved state-of-the-art performance for perceptual quality in speech enhancement (SE). However, their iterative nature requires numerous Neural Function Evaluations (NFEs), posing a challenge for real-time applications. On the contrary, flow matching offers a more efficient alternative by learning a direct vector field, enabling high-quality synthesis in just a few steps using deterministic ordinary differential equation~(ODE) solvers. We thus introduce Shortcut Flow Matching for Speech Enhancement (SFMSE), a novel approach that trains a single, step-invariant model. By conditioning the velocity field on the target time step during a one-stage training process, SFMSE can perform single, few, or multi-step denoising without any architectural changes or fine-tuning. Our results demonstrate that a single-step SFMSE inference achieves a real-time factor (RTF) of 0.013 on a consumer GPU while delivering perceptual quality comparable to a strong diffusion baseline requiring 60 NFEs. This work also provides an empirical analysis of the role of stochasticity in training and inference, bridging the gap between high-quality generative SE and low-latency constraints.

Method: Uses ABC notation to bridge text and music modalities, applies multi-modal Retrieval-Augmented Generation and self-refinement techniques, and leverages generated motivations and attention maps for explanations.

Result: Outperforms other methods in both human studies and machine evaluations in terms of music quality and music-image consistency.

Conclusion: The proposed VLM-based framework successfully provides interpretable and high-quality Image-to-Music generation with low computational cost, showing promising results for practical applications.

Abstract: Recently, Image-to-Music (I2M) generation has garnered significant attention, with potential applications in fields such as gaming, advertising, and multi-modal art creation. However, due to the ambiguous and subjective nature of I2M tasks, most end-to-end methods lack interpretability, leaving users puzzled about the generation results. Even methods based on emotion mapping face controversy, as emotion represents only a singular aspect of art. Additionally, most learning-based methods require substantial computational resources and large datasets for training, hindering accessibility for common users. To address these challenges, we propose the first Vision Language Model (VLM)-based I2M framework that offers high interpretability and low computational cost. Specifically, we utilize ABC notation to bridge the text and music modalities, enabling the VLM to generate music using natural language. We then apply multi-modal Retrieval-Augmented Generation (RAG) and self-refinement techniques to allow the VLM to produce high-quality music without external training. Furthermore, we leverage the generated motivations in text and the attention maps from the VLM to provide explanations for the generated results in both text and image modalities. To validate our method, we conduct both human studies and machine evaluations, where our method outperforms others in terms of music quality and music-image consistency, indicating promising results. Our code is available at https://github.com/RS2002/Image2Music .

Method: Uses efficient fundamental frequency estimation and time-domain polyphase IIR filters to approximate analytic signals in real-time, supplemented with amplitude modulation transfer.

Result: Successfully implemented as a real-time VST plugin capable of transferring both vibrato patterns and amplitude modulation from target signals.

Conclusion: The algorithm enables real-time vibrato control for sound design, morphing, and synthesized sound manipulation, extending beyond typical delay-based vibrato effects.

Abstract: An algorithm for deriving delay functions based on real examples of vibrato was recently introduced and can be used to perform a vibrato transfer, in which the vibrato pattern of a target signal is imparted onto an incoming sound using a delay line. The algorithm contains methods that computationally restrict a real-time implementation. Here, a real-time approximation is presented that incorporates an efficient fundamental frequency estimation algorithm and time-domain polyphase IIR filters that approximate an analytic signal. The vibrato transfer algorithm is further supplemented with a proposed method to transfer the amplitude modulation of the target sound, moving this method beyond the capabilities of typical delay-based vibrato effects. Modifications to the original algorithm for real-time use are detailed here and available as source code for an implementation as a VST plugin. This algorithm has applications as an audio effect in sound design, sound morphing, and real-time vibrato control of synthesized sounds.

Method: Developed Pure Data patches that approximate the DL-4’s sound and functionality, refined by comparing score settings to official recordings, integrated into a Null Piece-based performance patch, and regression tested using Reality Check framework.

Result: Successfully created a library of patches that allows ‘Summermood’ to be performed live without the original DL-4 hardware, with continuous testing to ensure compatibility across computer environments and Pure Data updates.

Conclusion: The Pure Data emulation approach provides a sustainable solution for preserving electroacoustic compositions that depend on obsolete hardware, enabling continued live performance of such works.

Abstract: As a contribution towards ongoing efforts to maintain electroacoustic compositions for live performance, we present a collection of Pure Data patches to preserve and perform Antonio Russek’s piece “Summermood” for bass flute and live electronics. The piece, originally written for the DeltaLab DL-4 delay rack unit, contains score markings specific to the DL-4. Here, we approximate the sound and unique functionality of the DL-4 in Pure Data, then refine our implementation to better match the unit on which the piece was performed by comparing settings from the score to two official recordings of the piece. The DL-4 emulation is integrated into a patch for live performance based on the Null Piece, and regression tested using the Reality Check framework for Pure Data. Using this library of patches, Summermood can be brought back into live rotation without the use of the now discontinued DL-4. The patches will be continuously tested to ensure that the piece is playable across computer environments and as the Pure Data programming language is updated.

Method: Combines audio language models for reasoning and instruction decomposition with latent diffusion models for stereo audio manipulation. Uses a data synthesis pipeline to create paired examples of instructions, edit operations, and audio samples.

Result: SmartDJ achieves superior perceptual quality, spatial realism, and semantic alignment compared to prior audio editing methods.

Conclusion: The framework successfully enables declarative stereo audio editing by integrating reasoning and generative capabilities, advancing beyond template-based approaches.

Abstract: Audio editing plays a central role in VR/AR immersion, virtual conferencing, sound design, and other interactive media. However, recent generative audio editing models depend on template-like instruction formats and are restricted to mono-channel audio. These models fail to deal with declarative audio editing, where the user declares what the desired outcome should be, while leaving the details of editing operations to the system. We introduce SmartDJ, a novel framework for stereo audio editing that combines the reasoning capability of audio language models with the generative power of latent diffusion. Given a high-level instruction, SmartDJ decomposes it into a sequence of atomic edit operations, such as adding, removing, or spatially relocating events. These operations are then executed by a diffusion model trained to manipulate stereo audio. To support this, we design a data synthesis pipeline that produces paired examples of high-level instructions, atomic edit operations, and audios before and after each edit operation. Experiments demonstrate that SmartDJ achieves superior perceptual quality, spatial realism, and semantic alignment compared to prior audio editing methods. Demos are available at https://zitonglan.github.io/project/smartdj/smartdj.html.

Method: Proposed a training-free framework based on knowledge representations, retrieval augmentation, and voice profile matching, with simple yet effective knowledge retrieval and ensemble methods.

Result: Achieved performance comparable to fine-tuned models on DeepFake-Eval-2024 without any additional model-wise training. Ablation studies validated the relevance of retrieval pool size and voice profile attributes to system efficacy.

Conclusion: The proposed training-free framework provides an effective solution for zero-day audio deepfake detection that doesn’t require model fine-tuning, offering comparable performance to fine-tuned approaches while enabling prompt response to new attacks.

Abstract: Modern audio deepfake detectors using foundation models and large training datasets have achieved promising detection performance. However, they struggle with zero-day attacks, where the audio samples are generated by novel synthesis methods that models have not seen from reigning training data. Conventional approaches against such attacks require fine-tuning the detectors, which can be problematic when prompt response is required. This study introduces a training-free framework for zero-day audio deepfake detection based on knowledge representations, retrieval augmentation, and voice profile matching. Based on the framework, we propose simple yet effective knowledge retrieval and ensemble methods that achieve performance comparable to fine-tuned models on DeepFake-Eval-2024, without any additional model-wise training. We also conduct ablation studies on retrieval pool size and voice profile attributes, validating their relevance to the system efficacy.

Method: Proposes N2N framework using diffusion models with Annealed Pseudo-Huber loss for joint optimization of binary onset and continuous velocity values, and incorporates music foundation model features to enhance spectrogram features.

Result: N2N establishes new state-of-the-art performance across multiple ADT benchmarks, with MFM features significantly improving robustness to out-of-domain drum audio.

Conclusion: The generative diffusion approach to ADT with N2N framework and MFM feature augmentation provides superior performance and robustness compared to traditional discriminative methods.

Abstract: Automatic drum transcription (ADT) is traditionally formulated as a discriminative task to predict drum events from audio spectrograms. In this work, we redefine ADT as a conditional generative task and introduce Noise-to-Notes (N2N), a framework leveraging diffusion modeling to transform audio-conditioned Gaussian noise into drum events with associated velocities. This generative diffusion approach offers distinct advantages, including a flexible speed-accuracy trade-off and strong inpainting capabilities. However, the generation of binary onset and continuous velocity values presents a challenge for diffusion models, and to overcome this, we introduce an Annealed Pseudo-Huber loss to facilitate effective joint optimization. Finally, to augment low-level spectrogram features, we propose incorporating features extracted from music foundation models (MFMs), which capture high-level semantic information and enhance robustness to out-of-domain drum audio. Experimental results demonstrate that including MFM features significantly improves robustness and N2N establishes a new state-of-the-art performance across multiple ADT benchmarks.

Method: The approach integrates layer-wise frame resampling and progressive sub-band pruning. Frame resampling downsamples inputs within layers using residual connections, while sub-band pruning progressively excludes less informative frequency bands.

Result: Extensive experiments show the system reduces SE computational overhead by over 66% compared to standard BSRNN while maintaining strong ASR performance on synthetic and real-world noisy datasets.

Conclusion: The proposed optimizations successfully reduce computational costs of speech enhancement front-ends significantly without degrading ASR performance in noisy environments.

Abstract: Recent advancements in automatic speech recognition (ASR) have achieved notable progress, whereas robustness in noisy environments remains challenging. While speech enhancement (SE) front-ends are widely used to mitigate noise as a preprocessing step for ASR, they often introduce computational non-negligible overhead. This paper proposes optimizations to reduce SE computational costs without compromising ASR performance. Our approach integrates layer-wise frame resampling and progressive sub-band pruning. Frame resampling downsamples inputs within layers, utilizing residual connections to mitigate information loss. Simultaneously, sub-band pruning progressively excludes less informative frequency bands, further reducing computational demands. Extensive experiments on synthetic and real-world noisy datasets demonstrate that our system reduces SE computational overhead over 66 compared to the standard BSRNN, while maintaining strong ASR performance.

Method: Create synthetic dataset with moving binaural sounds, spatial trajectories, and text captions. Train text-to-trajectory model, fine-tune text-to-audio model for temporally aligned mono sound, then simulate spatial audio using predicted trajectories.

Result: Experimental evaluation shows reasonable spatial understanding by the text-to-trajectory model.

Conclusion: This approach can be easily integrated into existing text-to-audio workflows and extended to other spatial audio formats for moving sound generation.

Abstract: Human auditory perception is shaped by moving sound sources in 3D space, yet prior work in generative sound modelling has largely been restricted to mono signals or static spatial audio. In this work, we introduce a framework for generating moving sounds given text prompts in a controllable fashion. To enable training, we construct a synthetic dataset that records moving sounds in binaural format, their spatial trajectories, and text captions about the sound event and spatial motion. Using this dataset, we train a text-to-trajectory prediction model that outputs the three-dimensional trajectory of a moving sound source given text prompts. To generate spatial audio, we first fine-tune a pre-trained text-to-audio generative model to output temporally aligned mono sound with the trajectory. The spatial audio is then simulated using the predicted temporally-aligned trajectory. Experimental evaluation demonstrates reasonable spatial understanding of the text-to-trajectory model. This approach could be easily integrated into existing text-to-audio generative workflow and extended to moving sound generation in other spatial audio formats.

Method: Uses approaches inspired by Fast Gradient Sign Method and Zeroth-Order Optimization to create hybrid models that generate subtle adversarial examples with minimal perturbation (35dB SNR) in under a minute.

Result: Demonstrates successful generation of impactful adversarial examples with very little perturbation that can deceive state-of-the-art ASR systems into misinterpreting audio signals.

Conclusion: The vulnerabilities found in state-of-the-art open source ASR models have practical security implications and emphasize the urgent need for adversarial security measures in speech recognition systems.

Abstract: Recent studies have demonstrated the vulnerability of Automatic Speech Recognition systems to adversarial examples, which can deceive these systems into misinterpreting input speech commands. While previous research has primarily focused on white-box attacks with constrained optimizations, and transferability based black-box attacks against commercial Automatic Speech Recognition devices, this paper explores cost efficient white-box attack and non transferability black-box adversarial attacks on Automatic Speech Recognition systems, drawing insights from approaches such as Fast Gradient Sign Method and Zeroth-Order Optimization. Further, the novelty of the paper includes how poisoning attack can degrade the performances of state-of-the-art models leading to misinterpretation of audio signals. Through experimentation and analysis, we illustrate how hybrid models can generate subtle yet impactful adversarial examples with very little perturbation having Signal Noise Ratio of 35dB that can be generated within a minute. These vulnerabilities of state-of-the-art open source model have practical security implications, and emphasize the need for adversarial security.

Method: Proposes S3Codec with semantic distillation from ASR model, dual-Transformer architecture separating comprehension and generation, and Masked Audio Parallel Inference for stable decoding.

Result: The framework enables robust and semantically-grounded zero-shot speech synthesis with improved generation stability.

Conclusion: CaT-TTS effectively addresses key challenges in LLM-based TTS through semantic grounding and architectural innovations.

Abstract: Existing Large Language Model (LLM) based autoregressive (AR) text-to-speech (TTS) systems, while achieving state-of-the-art quality, still face critical challenges. The foundation of this LLM-based paradigm is the discretization of the continuous speech waveform into a sequence of discrete tokens by neural audio codec. However, single codebook modeling is well suited to text LLMs, but suffers from significant information loss; hierarchical acoustic tokens, typically generated via Residual Vector Quantization (RVQ), often lack explicit semantic structure, placing a heavy learning burden on the model. Furthermore, the autoregressive process is inherently susceptible to error accumulation, which can degrade generation stability. To address these limitations, we propose CaT-TTS, a novel framework for robust and semantically-grounded zero-shot synthesis. First, we introduce S3Codec, a split RVQ codec that injects explicit linguistic features into its primary codebook via semantic distillation from a state-of-the-art ASR model, providing a structured representation that simplifies the learning task. Second, we propose an Understand-then-Generate'' dual-Transformer architecture that decouples comprehension from rendering. An initial Understanding’’ Transformer models the cross-modal relationship between text and the audio’s semantic tokens to form a high-level utterance plan. A subsequent ``Generation’’ Transformer then executes this plan, autoregressively synthesizing hierarchical acoustic tokens. Finally, to enhance generation stability, we introduce Masked Audio Parallel Inference (MAPI), a nearly parameter-free inference strategy that dynamically guides the decoding process to mitigate local errors.

Method: Time-Delay Neural Networks with frequency-sensitive normalization, gradient-reversal adversarial training, and multi-level augmentation including waveform perturbations, Mixup, and CycleGAN transfer with Dialect-Calibrated Augmentation.

Result: Achieved up to 20 percentage point improvement in cross-dialect accuracy over baseline TDNNs while preserving in-region performance.

Conclusion: Lightweight, transparent, and dialect-resilient bird-sound recognition is achievable through the proposed framework.

Abstract: Dialect variation hampers automatic recognition of bird calls collected by passive acoustic monitoring. We address the problem on DB3V, a three-region, ten-species corpus of 8-s clips, and propose a deployable framework built on Time-Delay Neural Networks (TDNNs). Frequency-sensitive normalisation (Instance Frequency Normalisation and a gated Relaxed-IFN) is paired with gradient-reversal adversarial training to learn region-invariant embeddings. A multi-level augmentation scheme combines waveform perturbations, Mixup for rare classes, and CycleGAN transfer that synthesises Region 2 (Interior Plains)-style audio, , with Dialect-Calibrated Augmentation (DCA) softly down-weighting synthetic samples to limit artifacts. The complete system lifts cross-dialect accuracy by up to twenty percentage points over baseline TDNNs while preserving in-region performance. Grad-CAM and LIME analyses show that robust models concentrate on stable harmonic bands, providing ecologically meaningful explanations. The study demonstrates that lightweight, transparent, and dialect-resilient bird-sound recognition is attainable.

Method: Two-stage recursive approach: 1) Coarse Separation - first-pass estimation from mixture and visual input, 2) Fine Separation - coarse audio fed into AVSR model with visual stream to generate discriminative semantic representations. Includes speaker-aware perceptual fusion and multi-range spectro-temporal separation network.

Result: Achieves state-of-the-art performance on three benchmark datasets and two noisy datasets, with substantial coarse-to-fine improvements.

Conclusion: The recursive semantic enhancement framework is necessary and effective for audio-visual speech separation, validating the importance of dynamic semantic representation learning.

Abstract: Audio-visual speech separation aims to isolate each speaker’s clean voice from mixtures by leveraging visual cues such as lip movements and facial features. While visual information provides complementary semantic guidance, existing methods often underexploit its potential by relying on static visual representations. In this paper, we propose CSFNet, a Coarse-to-Separate-Fine Network that introduces a recursive semantic enhancement paradigm for more effective separation. CSFNet operates in two stages: (1) Coarse Separation, where a first-pass estimation reconstructs a coarse audio waveform from the mixture and visual input; and (2) Fine Separation, where the coarse audio is fed back into an audio-visual speech recognition (AVSR) model together with the visual stream. This recursive process produces more discriminative semantic representations, which are then used to extract refined audio. To further exploit these semantics, we design a speaker-aware perceptual fusion block to encode speaker identity across modalities, and a multi-range spectro-temporal separation network to capture both local and global time-frequency patterns. Extensive experiments on three benchmark datasets and two noisy datasets show that CSFNet achieves state-of-the-art (SOTA) performance, with substantial coarse-to-fine improvements, validating the necessity and effectiveness of our recursive semantic enhancement framework.

Method: Created MDAR benchmark with 3,000 curated QA pairs linked to diverse audio clips, covering 5 complex reasoning categories and 3 question types (single-choice, multiple-choice, open-ended).

Result: Benchmarked 26 SOTA audio language models - Qwen2.5-Omni achieved 76.67% on single-choice, GPT-4o Audio reached 68.47% but outperformed on harder tasks. No model achieved 80% performance across all question types.

Conclusion: MDAR poses unique challenges that current models struggle with, highlighting its value as a benchmark for advancing audio reasoning research.

Abstract: The ability to reason from audio, including speech, paralinguistic cues, environmental sounds, and music, is essential for AI agents to interact effectively in real-world scenarios. Existing benchmarks mainly focus on static or single-scene settings and do not fully capture scenarios where multiple speakers, unfolding events, and heterogeneous audio sources interact. To address these challenges, we introduce MDAR, a benchmark for evaluating models on complex, multi-scene, and dynamically evolving audio reasoning tasks. MDAR comprises 3,000 carefully curated question-answer pairs linked to diverse audio clips, covering five categories of complex reasoning and spanning three question types. We benchmark 26 state-of-the-art audio language models on MDAR and observe that they exhibit limitations in complex reasoning tasks. On single-choice questions, Qwen2.5-Omni (open-source) achieves 76.67% accuracy, whereas GPT-4o Audio (closed-source) reaches 68.47%; however, GPT-4o Audio substantially outperforms Qwen2.5-Omni on the more challenging multiple-choice and open-ended tasks. Across all three question types, no model achieves 80% performance. These findings underscore the unique challenges posed by MDAR and its value as a benchmark for advancing audio reasoning research.Code and benchmark can be found at https://github.com/luckyerr/MDAR.

Method: Uses segment-level reconstruction objective with token-level disentangled content and style representations, combined with REMI-z structured tokenization scheme for multitrack symbolic music to enable any-to-any instrumentation transformations.

Result: Outperforms task-specific state-of-the-art models on band arrangement, piano reduction, and drum arrangement tasks in both objective metrics and perceptual evaluations.

Conclusion: The framework demonstrates strong generality and suggests broader applicability in symbolic music-to-music transformation, providing a unified solution for diverse arrangement scenarios.

Abstract: We present a unified framework for automatic multitrack music arrangement that enables a single pre-trained symbolic music model to handle diverse arrangement scenarios, including reinterpretation, simplification, and additive generation. At its core is a segment-level reconstruction objective operating on token-level disentangled content and style, allowing for flexible any-to-any instrumentation transformations at inference time. To support track-wise modeling, we introduce REMI-z, a structured tokenization scheme for multitrack symbolic music that enhances modeling efficiency and effectiveness for both arrangement tasks and unconditional generation. Our method outperforms task-specific state-of-the-art models on representative tasks in different arrangement scenarios – band arrangement, piano reduction, and drum arrangement, in both objective metrics and perceptual evaluations. Taken together, our framework demonstrates strong generality and suggests broader applicability in symbolic music-to-music transformation.

Method: Leverage Qwen-Audio-Chat fine-tuned on three datasets collected in-situ from hospital patients, with multifaceted evaluation including safety assessment, cross-lingual performance analysis, and modality ablation studies.

Result: VocalAgent demonstrates superior accuracy on voice disorder classification compared to state-of-the-art baselines.

Conclusion: The LLM-based method offers a scalable solution for broader adoption of health diagnostics while underscoring the importance of ethical and technical validation.

Abstract: Vocal health plays a crucial role in peoples’ lives, significantly impacting their communicative abilities and interactions. However, despite the global prevalence of voice disorders, many lack access to convenient diagnosis and treatment. This paper introduces VocalAgent, an audio large language model (LLM) to address these challenges through vocal health diagnosis. We leverage Qwen-Audio-Chat fine-tuned on three datasets collected in-situ from hospital patients, and present a multifaceted evaluation framework encompassing a safety assessment to mitigate diagnostic biases, cross-lingual performance analysis, and modality ablation studies. VocalAgent demonstrates superior accuracy on voice disorder classification compared to state-of-the-art baselines. Its LLM-based method offers a scalable solution for broader adoption of health diagnostics, while underscoring the importance of ethical and technical validation.

Method: First-shot problem within domain generalization framework using sounds from previously unseen machine types as evaluation dataset.

Result: Analysis of 119 submissions showed various competitive approaches including fine-tuning pre-trained models, using frozen pre-trained models, and training small models from scratch when combined with appropriate techniques.

Conclusion: Multiple approaches can be effective for first-shot ASD when properly combined with cost functions, anomaly score normalization, and use of clean machine and noise sounds.

Abstract: This paper introduces the task description for the Detection and Classification of Acoustic Scenes and Events (DCASE) 2025 Challenge Task 2, titled “First-shot unsupervised anomalous sound detection (ASD) for machine condition monitoring”. Building on the DCASE 2024 Challenge Task 2, this task is structured as a first-shot problem within a domain generalization framework. The primary objective of the first-shot approach is to facilitate the rapid deployment of ASD systems for new machine types without requiring machine-specific hyperparameter tunings. For DCASE 2025 Challenge Task 2, sounds from previously unseen machine types have been collected and provided as the evaluation dataset. We received 119 submissions from 35 teams, and an analysis of these submissions has been made in this paper. Analysis showed that various approaches can all be competitive, such as fine-tuning pre-trained models, using frozen pre-trained models, and training small models from scratch, when combined with appropriate cost functions, anomaly score normalization, and use of clean machine and noise sounds.

Method: xi+ incorporates a temporal attention module for context-aware frame uncertainty estimation and introduces Stochastic Variance Loss to explicitly supervise uncertainty learning.

Result: xi+ achieves ~10% improvement on VoxCeleb1-O and ~11% improvement on NIST SRE 2024 evaluation sets compared to baseline xi-vector.

Conclusion: The proposed xi+ architecture with temporal attention and explicit uncertainty supervision significantly enhances speaker recognition performance by better modeling frame-level uncertainty.

Abstract: There are various factors that can influence the performance of speaker recognition systems, such as emotion, language and other speaker-related or context-related variations. Since individual speech frames do not contribute equally to the utterance-level representation, it is essential to estimate the importance or reliability of each frame. The xi-vector model addresses this by assigning different weights to frames based on uncertainty estimation. However, its uncertainty estimation model is implicitly trained through classification loss alone and does not consider the temporal relationships between frames, which may lead to suboptimal supervision. In this paper, we propose an improved architecture, xi+. Compared to xi-vector, xi+ incorporates a temporal attention module to capture frame-level uncertainty in a context-aware manner. In addition, we introduce a novel loss function, Stochastic Variance Loss, which explicitly supervises the learning of uncertainty. Results demonstrate consistent performance improvements of about 10% on the VoxCeleb1-O set and 11% on the NIST SRE 2024 evaluation set.

Method: Combines traditional signal processing with denoising diffusion models (WaveGrad and DiffWave) to create augmented datasets, then fine-tunes a Wav2Vec 2.0-based classifier on multimodal and multichannel heart sound datasets.

Result: Achieved state-of-the-art performance: 92.48% accuracy on CinC 2016 single-channel PCG, 93.14% accuracy on synchronized PCG-ECG, and 77.13% accuracy on multichannel PCG data. High metrics across sensitivity, specificity, and MCC were obtained.

Conclusion: Transformer-based models are highly effective for CVD detection when supported by augmented datasets, demonstrating significant potential for advancing multimodal and multichannel heart sound classification in clinical applications.

Abstract: Cardiovascular diseases (CVDs) are the leading cause of death worldwide, accounting for approximately 17.9 million deaths each year. Early detection is critical, creating a demand for accurate and inexpensive pre-screening methods. Deep learning has recently been applied to classify abnormal heart sounds indicative of CVDs using synchronised phonocardiogram (PCG) and electrocardiogram (ECG) signals, as well as multichannel PCG (mPCG). However, state-of-the-art architectures remain underutilised due to the limited availability of synchronised and multichannel datasets. Augmented datasets and pre-trained models provide a pathway to overcome these limitations, enabling transformer-based architectures to be trained effectively. This work combines traditional signal processing with denoising diffusion models, WaveGrad and DiffWave, to create an augmented dataset to fine-tune a Wav2Vec 2.0-based classifier on multimodal and multichannel heart sound datasets. The approach achieves state-of-the-art performance. On the Computing in Cardiology (CinC) 2016 dataset of single channel PCG, accuracy, unweighted average recall (UAR), sensitivity, specificity and Matthew’s correlation coefficient (MCC) reach 92.48%, 93.05%, 93.63%, 92.48%, 94.93% and 0.8283, respectively. Using the synchronised PCG and ECG signals of the training-a dataset from CinC, 93.14%, 92.21%, 94.35%, 90.10%, 95.12% and 0.8380 are achieved for accuracy, UAR, sensitivity, specificity and MCC, respectively. Using a wearable vest dataset consisting of mPCG data, the model achieves 77.13% accuracy, 74.25% UAR, 86.47% sensitivity, 62.04% specificity, and 0.5082 MCC. These results demonstrate the effectiveness of transformer-based models for CVD detection when supported by augmented datasets, highlighting their potential to advance multimodal and multichannel heart sound classification.

Method: Developed FakeSound2 benchmark that evaluates models across three dimensions: localization (identifying manipulated regions), traceability (tracking source origins), and generalization (performance on unseen sources), covering 6 manipulation types and 12 diverse sources.

Result: Current systems achieve high classification accuracy but struggle to recognize forged pattern distributions and provide reliable explanations, revealing significant gaps in explainability and reliability.

Conclusion: FakeSound2 establishes a comprehensive benchmark that highlights key challenges and aims to foster robust, explainable, and generalizable approaches for trustworthy audio authentication.

Abstract: The rapid development of generative audio raises ethical and security concerns stemming from forged data, making deepfake sound detection an important safeguard against the malicious use of such technologies. Although prior studies have explored this task, existing methods largely focus on binary classification and fall short in explaining how manipulations occur, tracing where the sources originated, or generalizing to unseen sources-thereby limiting the explainability and reliability of detection. To address these limitations, we present FakeSound2, a benchmark designed to advance deepfake sound detection beyond binary accuracy. FakeSound2 evaluates models across three dimensions: localization, traceability, and generalization, covering 6 manipulation types and 12 diverse sources. Experimental results show that although current systems achieve high classification accuracy, they struggle to recognize forged pattern distributions and provide reliable explanations. By highlighting these gaps, FakeSound2 establishes a comprehensive benchmark that reveals key challenges and aims to foster robust, explainable, and generalizable approaches for trustworthy audio authentication.

Method: The method uses latent bridge models (LBMs) that compress audio waveforms into continuous latent space and perform latent-to-latent generation. It introduces frequency-aware LBMs that take prior and target frequency as input for any-to-any upsampling, cascaded LBMs, and prior augmentation strategies for seamless cascaded super-resolution.

Result: Comprehensive experiments on VCTK, ESC-50, Song-Describer datasets and internal testsets demonstrate state-of-the-art objective and perceptual quality for any-to-48kHz SR across speech, audio, and music signals, and setting the first record for any-to-192kHz audio SR.

Conclusion: The proposed latent bridge model system effectively addresses limitations of previous audio super-resolution methods by leveraging latent-to-latent generation and frequency-aware training, achieving superior upsampling quality and enabling high-frequency audio SR beyond 48kHz.

Abstract: Audio super-resolution (SR), i.e., upsampling the low-resolution (LR) waveform to the high-resolution (HR) version, has recently been explored with diffusion and bridge models, while previous methods often suffer from sub-optimal upsampling quality due to their uninformative generation prior. Towards high-quality audio super-resolution, we present a new system with latent bridge models (LBMs), where we compress the audio waveform into a continuous latent space and design an LBM to enable a latent-to-latent generation process that naturally matches the LR-toHR upsampling process, thereby fully exploiting the instructive prior information contained in the LR waveform. To further enhance the training results despite the limited availability of HR samples, we introduce frequency-aware LBMs, where the prior and target frequency are taken as model input, enabling LBMs to explicitly learn an any-to-any upsampling process at the training stage. Furthermore, we design cascaded LBMs and present two prior augmentation strategies, where we make the first attempt to unlock the audio upsampling beyond 48 kHz and empower a seamless cascaded SR process, providing higher flexibility for audio post-production. Comprehensive experimental results evaluated on the VCTK, ESC-50, Song-Describer benchmark datasets and two internal testsets demonstrate that we achieve state-of-the-art objective and perceptual quality for any-to-48kHz SR across speech, audio, and music signals, as well as setting the first record for any-to-192kHz audio SR. Demo at https://AudioLBM.github.io/.

Method: Integrates object-centric process mining with stochastic process discovery using continuous-time Markov chains from sales data, then extends with supply activities for what-if analysis.

Result: Enables identification of optimal balance between customer demand and supply, helping prevent both food waste from oversupply and product shortages.

Conclusion: The integrated approach successfully models food retail processes to optimize supply strategies and reduce waste through data-driven analysis.

Abstract: This paper proposes a novel method for analyzing food retail processes with a focus on reducing food waste. The approach integrates object-centric process mining (OCPM) with stochastic process discovery and analysis. First, a stochastic process in the form of a continuous-time Markov chain is discovered from grocery store sales data. This model is then extended with supply activities. Finally, a what-if analysis is conducted to evaluate how the quantity of products in the store evolves over time. This enables the identification of an optimal balance between customer purchasing behavior and supply strategies, helping to prevent both food waste due to oversupply and product shortages.

Method: Two dimensional analysis based weighting schemes: one using quantifiable terms, and another incorporating both quantifiable and unquantifiable terms for more balanced training.

Result: The second weighting scheme consistently improves stability and accuracy over equal weighting in heat conduction, convection diffusion, and lid driven cavity flows. Notably achieves stable, accurate predictions in high Peclet number convection diffusion where traditional solvers fail.

Conclusion: The proposed weighting scheme enhances PINNs’ robustness and generalizability in CFD problems, making them effective even in challenging scenarios where conventional methods struggle.

Abstract: Physics Informed Neural Networks offer a mesh free framework for solving PDEs but are highly sensitive to loss weight selection. We propose two dimensional analysis based weighting schemes, one based on quantifiable terms, and another also incorporating unquantifiable terms for more balanced training. Benchmarks on heat conduction, convection diffusion, and lid driven cavity flows show that the second scheme consistently improves stability and accuracy over equal weighting. Notably, in high Peclet number convection diffusion, where traditional solvers fail, PINNs with our scheme achieve stable, accurate predictions, highlighting their robustness and generalizability in CFD problems.

Method: Tested open- and closed-source LLMs on experimental design tasks, compared with classical methods, and proposed LLM-guided Nearest Neighbour sampling that combines LLM prior knowledge with nearest-neighbor sampling.

Result: LLM-based agents showed no sensitivity to experimental feedback, classical methods consistently outperformed LLMs, and the proposed LLMNN method achieved competitive or superior performance across domains.

Conclusion: Current LLMs do not perform in-context experimental design effectively, highlighting the need for hybrid frameworks that separate prior-based reasoning from batch acquisition with updated posteriors.

Abstract: Large language models (LLMs) have recently been proposed as general-purpose agents for experimental design, with claims that they can perform in-context experimental design. We evaluate this hypothesis using both open- and closed-source instruction-tuned LLMs applied to genetic perturbation and molecular property discovery tasks. We find that LLM-based agents show no sensitivity to experimental feedback: replacing true outcomes with randomly permuted labels has no impact on performance. Across benchmarks, classical methods such as linear bandits and Gaussian process optimization consistently outperform LLM agents. We further propose a simple hybrid method, LLM-guided Nearest Neighbour (LLMNN) sampling, that combines LLM prior knowledge with nearest-neighbor sampling to guide the design of experiments. LLMNN achieves competitive or superior performance across domains without requiring significant in-context adaptation. These results suggest that current open- and closed-source LLMs do not perform in-context experimental design in practice and highlight the need for hybrid frameworks that decouple prior-based reasoning from batch acquisition with updated posteriors.

Method: Physics-informed residual neural network for state mapping and state-derivative prediction, with softmax layer for multi-class confidence estimation. Tested on quadcopter, fixed-wing, and helicopter aerial vehicles.

Result: Demonstrates high classification accuracy with reduced training time.

Conclusion: Offers a promising solution for system identification problems in domains with well-understood underlying dynamics.

Abstract: This work addresses object identification under known dynamics in unmanned aerial vehicle applications, where learning and classification are combined through a physics-informed residual neural network. The proposed framework leverages physics-informed learning for state mapping and state-derivative prediction, while a softmax layer enables multi-class confidence estimation. Quadcopter, fixed-wing, and helicopter aerial vehicles are considered as case studies. The results demonstrate high classification accuracy with reduced training time, offering a promising solution for system identification problems in domains where the underlying dynamics are well understood.

Method: Proposes null-space filtering to preserve prior responses by filtering overlapping components of new task vectors, combined with lightweight LoRA adapters trained with projection-based surrogate loss to inject complementary task-specific signals.

Result: Achieves state-of-the-art performance with minimal forgetting on vision and NLP benchmarks, improving average accuracy by 4-7% over OPCM and WUDI-Merging while reducing computation overhead.

Conclusion: NUFILT effectively enables continual model merging without task data through joint filtering-adaptation, theoretically justified by subspace alignment guarantees and empirically validated across multiple domains.

Abstract: Data-free continual model merging (DFCMM) aims to fuse independently fine-tuned models into a single backbone that evolves with incoming tasks without accessing task data. This paper formulate two fundamental desiderata for DFCMM: transparency, avoiding interference with earlier tasks, and fidelity, adapting faithfully to each new task. This poses a challenge that existing approaches fail to address: how to bridge data-level desiderata with parameter-space optimization to ensure transparency and fidelity in the absence of task data. To this end, we propose NUFILT (NUll-space FILTering), a data-free framework that directly links these desiderata to optimization. Our key observation is that task vectors approximately align with representation subspaces, providing structural surrogates for enforcing transparency and fidelity. Accordingly, we design a null-space projector that preserves prior responses by filtering out overlapping components of new task vectors, thereby ensuring transparency, and a lightweight LoRA adapter that injects complementary task-specific signals, enabling fidelity in adapting to new tasks. The adapter is trained with a projection-based surrogate loss to retain consistency with previous knowledge while introducing novel directions. This joint filtering-adaptation process allows the backbone to absorb new knowledge while retaining existing behaviors, and the updates are finally fused back in a layer-wise linear fashion without extra parameters or inference cost. Theoretically, we establish approximate subspace alignment guarantees that justify null-space filtering. Empirically, NUFILT achieves state-of-the-art performance with minimal forgetting on both vision and NLP benchmarks, improving average accuracy by 4-7% over OPCM and WUDI-Merging, while narrowing the gap to fine-tuning and reducing computation overhead.

Method: Transformer-based model trained in autoregressive setting on waveform data, learning temporal and spatial dependencies from single-station and array-based inputs.

Result: Model performs well in immediate prediction window but gradually degrades for longer forecasts, as expected in autoregressive systems.

Conclusion: SeismoGPT lays groundwork for data-driven seismic forecasting that can support gravitational wave detector operations through noise mitigation and control.

Abstract: We introduce \textit{SeismoGPT}, a transformer-based model for forecasting three-component seismic waveforms in the context of future gravitational wave detectors like the Einstein Telescope. The model is trained in an autoregressive setting and can operate on both single-station and array-based inputs. By learning temporal and spatial dependencies directly from waveform data, SeismoGPT captures realistic ground motion patterns and provides accurate short-term forecasts. Our results show that the model performs well within the immediate prediction window and gradually degrades further ahead, as expected in autoregressive systems. This approach lays the groundwork for data-driven seismic forecasting that could support Newtonian noise mitigation and real-time observatory control.

Method: Design minimal tools for constructing, analyzing and manipulating decision trees. Use reasoning-capable LLMs in agentic setup to combine prior knowledge with data learning to create lightweight decision trees.

Result: LLM-induced decision trees outperform traditional CART on low-resource tabular problems. While not beating state-of-the-art black box models, they provide human-readable reasoning traces that can be checked for biases and data leaks.

Conclusion: LLM-induced decision trees offer an interpretable alternative to black box models, allowing for human input to correct biases and incorporate domain knowledge not captured in data.

Abstract: Tabular foundation models are becoming increasingly popular for low-resource tabular problems. These models make up for small training datasets by pretraining on large volumes of synthetic data. The prior knowledge obtained via pretraining provides the exceptional performance, but the resulting model becomes a black box that is difficult to interpret and costly to inference. In this work, we explore an alternative strategy: using reasoning-capable LLMs to induce decision trees for small tabular datasets in agentic setup. We design a minimal set of tools for constructing, analyzing and manipulating decision trees. By using these tools, LLMs combine their prior knowledge with learning from data to create a lightweight decision tree that outperforms traditional CART on low-resource tabular problems. While a single decision tree does not outperform state-of-the-art black box models, it comes with a human-readable reasoning trace that can be checked for biases and data leaks. Furthermore, the reasoning-based LLM’s creation process allows for additional human input: correcting biases or incorporating domain-specific intuition that is not captured in the data.

Method: Integrates online inverse preference learning with multi-agent on-policy optimization using an implicit multi-agent reward learning model based on preference-based value-decomposition network, producing global and local reward signals with dual advantage streams for centralized critic and decentralized actors. Leverages LLMs for preference labels.

Result: Empirical evaluations on MAMuJoCo and SMACv2 benchmarks show superior performance compared to existing baselines.

Conclusion: The proposed framework effectively addresses sparse-reward challenges in online MARL through integrated reward learning and dual advantage mechanisms.

Abstract: We study the problem of online multi-agent reinforcement learning (MARL) in environments with sparse rewards, where reward feedback is not provided at each interaction but only revealed at the end of a trajectory. This setting, though realistic, presents a fundamental challenge: the lack of intermediate rewards hinders standard MARL algorithms from effectively guiding policy learning. To address this issue, we propose a novel framework that integrates online inverse preference learning with multi-agent on-policy optimization into a unified architecture. At its core, our approach introduces an implicit multi-agent reward learning model, built upon a preference-based value-decomposition network, which produces both global and local reward signals. These signals are further used to construct dual advantage streams, enabling differentiated learning targets for the centralized critic and decentralized actors. In addition, we demonstrate how large language models (LLMs) can be leveraged to provide preference labels that enhance the quality of the learned reward model. Empirical evaluations on state-of-the-art benchmarks, including MAMuJoCo and SMACv2, show that our method achieves superior performance compared to existing baselines, highlighting its effectiveness in addressing sparse-reward challenges in online MARL.

Method: Distill idempotent models from pre-trained diffusion model scores, eliminating the need for adversarial training and providing theoretical analysis of score-based training methods.

Result: Achieved state-of-the-art results for idempotent models on CIFAR and CelebA datasets, with faster inference than iterative score-based models and the ability to perform multi-step sampling and zero-shot editing.

Conclusion: SIGN successfully bridges diffusion models and IGNs, providing a stable, efficient generative framework that supports flexible quality-efficiency trade-offs and enables direct projection of corrupted distributions onto the target manifold.

Abstract: Idempotent generative networks (IGNs) are a new line of generative models based on idempotent mapping to a target manifold. IGNs support both single-and multi-step generation, allowing for a flexible trade-off between computational cost and sample quality. But similar to Generative Adversarial Networks (GANs), conventional IGNs require adversarial training and are prone to training instabilities and mode collapse. Diffusion and score-based models are popular approaches to generative modeling that iteratively transport samples from one distribution, usually a Gaussian, to a target data distribution. These models have gained popularity due to their stable training dynamics and high-fidelity generation quality. However, this stability and quality come at the cost of high computational cost, as the data must be transported incrementally along the entire trajectory. New sampling methods, model distillation, and consistency models have been developed to reduce the sampling cost and even perform one-shot sampling from diffusion models. In this work, we unite diffusion and IGNs by distilling idempotent models from diffusion model scores, called SIGN. Our proposed method is highly stable and does not require adversarial losses. We provide a theoretical analysis of our proposed score-based training methods and empirically show that IGNs can be effectively distilled from a pre-trained diffusion model, enabling faster inference than iterative score-based models. SIGNs can perform multi-step sampling, allowing users to trade off quality for efficiency. These models operate directly on the source domain; they can project corrupted or alternate distributions back onto the target manifold, enabling zero-shot editing of inputs. We validate our models on multiple image datasets, achieving state-of-the-art results for idempotent models on the CIFAR and CelebA datasets.

Method: Theoretical analysis showing optimal estimators still hallucinate, with a high probability lower bound on hallucination rate for generic data distributions. Experiments conducted on coin aggregation, open-ended QA, and text-to-image tasks.

Result: Demonstrated that hallucinations persist even in loss-minimizing optimal estimators, supporting the theoretical lower bound on hallucination rates across different domains.

Conclusion: Hallucinations are structural issues arising from misalignment between statistical loss minimization and human acceptability criteria, reframing them as miscalibration-induced estimation errors.

Abstract: We formalize hallucinations in generative models as failures to link an estimate to any plausible cause. Under this interpretation, we show that even loss-minimizing optimal estimators still hallucinate. We confirm this with a general high probability lower bound on hallucinate rate for generic data distributions. This reframes hallucination as structural misalignment between loss minimization and human-acceptable outputs, and hence estimation errors induced by miscalibration. Experiments on coin aggregation, open-ended QA, and text-to-image support our theory.

Method: Introduces d2 framework with a new policy gradient algorithm that uses masking properties to estimate sampling trajectory likelihoods, supporting any-order likelihood estimation for efficient diffusion-based reasoning.

Result: d2 significantly outperforms previous diffusion reasoning frameworks using only RL (no supervised fine-tuning), achieving state-of-the-art performance on logical reasoning tasks (Countdown, Sudoku) and math benchmarks (GSM8K, MATH500).

Conclusion: The d2 framework effectively enhances reasoning capabilities in diffusion language models through efficient RL-based training, demonstrating superior performance on complex reasoning tasks.

Abstract: While diffusion language models (DLMs) have achieved competitive performance in text generation, improving their reasoning ability with reinforcement learning remains an active research area. Here, we introduce d2, a reasoning framework tailored for masked DLMs. Central to our framework is a new policy gradient algorithm that relies on properties of masking to accurately estimate the likelihoods of sampling trajectories. Our estimators trade off computation for approximation accuracy in an analytically tractable manner, and are particularly effective for DLMs that support any-order likelihood estimation. We characterize and study this property in popular DLMs and show that it is key for efficient diffusion-based reasoning. Empirically, d2 significantly improves over previous diffusion reasoning frameworks using only RL (without relying on supervised fine-tuning), and sets a new state-of-the-art performance for DLMs on logical reasoning tasks (Countdown and Sudoku) and math reasoning benchmarks (GSM8K and MATH500).

Method: Built KD48 benchmark from petascale simulations with expert-driven denoising, then developed VISION with Dynamic Prompting that generates visual prompts from available observations and uses State-conditioned Prompting to inject prompts into a universal backbone with geometry- and scale-aware operators.

Result: VISION substantially outperforms state-of-the-art models and exhibits strong generalization under extreme data missing scenarios on the KD48 benchmark.

Conclusion: The work establishes solid infrastructure for ocean science research under data uncertainty by providing both a high-quality benchmark and a robust reconstruction model.

Abstract: Reconstructing subsurface ocean dynamics, such as vertical velocity fields, from incomplete surface observations poses a critical challenge in Earth science, a field long hampered by the lack of standardized, analysis-ready benchmarks. To systematically address this issue and catalyze research, we first build and release KD48, a high-resolution ocean dynamics benchmark derived from petascale simulations and curated with expert-driven denoising. Building on this benchmark, we introduce VISION, a novel reconstruction paradigm based on Dynamic Prompting designed to tackle the core problem of missing data in real-world observations. The essence of VISION lies in its ability to generate a visual prompt on-the-fly from any available subset of observations, which encodes both data availability and the ocean’s physical state. More importantly, we design a State-conditioned Prompting module that efficiently injects this prompt into a universal backbone, endowed with geometry- and scale-aware operators, to guide its adaptive adjustment of computational strategies. This mechanism enables VISION to precisely handle the challenges posed by varying input combinations. Extensive experiments on the KD48 benchmark demonstrate that VISION not only substantially outperforms state-of-the-art models but also exhibits strong generalization under extreme data missing scenarios. By providing a high-quality benchmark and a robust model, our work establishes a solid infrastructure for ocean science research under data uncertainty. Our codes are available at: https://github.com/YuanGao-YG/VISION.

Method: Proposes conformal data augmentation framework that leverages conformal prediction to filter poor-quality synthetic generations while maintaining diversity, requiring no access to model logits or retraining.

Result: Shows consistent performance improvements across multiple tasks (topic prediction, sentiment analysis, image classification, fraud detection) with up to 40% F1 score improvement over unaugmented baselines and 4% over other filtered augmentation methods.

Conclusion: Conformal data augmentation provides an effective, simple-to-implement solution for synthetic data filtering that provably controls risk while enhancing model performance across diverse applications.

Abstract: With promising empirical performance across a wide range of applications, synthetic data augmentation appears a viable solution to data scarcity and the demands of increasingly data-intensive models. Its effectiveness lies in expanding the training set in a way that reduces estimator variance while introducing only minimal bias. Controlling this bias is therefore critical: effective data augmentation should generate diverse samples from the same underlying distribution as the training set, with minimal shifts. In this paper, we propose conformal data augmentation, a principled data filtering framework that leverages the power of conformal prediction to produce diverse synthetic data while filtering out poor-quality generations with provable risk control. Our method is simple to implement, requires no access to internal model logits, nor large-scale model retraining. We demonstrate the effectiveness of our approach across multiple tasks, including topic prediction, sentiment analysis, image classification, and fraud detection, showing consistent performance improvements of up to 40% in F1 score over unaugmented baselines, and 4% over other filtered augmentation baselines.

Method: Uses a gradient estimator based on randomization over the ℓ₁-sphere and develops new concentration inequalities for Lipschitz functions under the uniform measure on the ℓ₁-sphere.

Result: FedZero achieves near-optimal optimization error bounds with high probability in federated convex settings and establishes the first high-probability convergence guarantees for convex zero-order optimization in single-worker scenarios.

Conclusion: The developed concentration tools are central to the high-probability guarantees and may have independent interest beyond the FedZero algorithm.

Abstract: We study distributed learning in the setting of gradient-free zero-order optimization and introduce FedZero, a federated zero-order algorithm that delivers sharp theoretical guarantees. Specifically, FedZero: (1) achieves near-optimal optimization error bounds with high probability in the federated convex setting; and (2) in the single-worker regime-where the problem reduces to the standard zero-order framework, establishes the first high-probability convergence guarantees for convex zero-order optimization, thereby strengthening the classical expectation-based results. At its core, FedZero employs a gradient estimator based on randomization over the $\ell_1$-sphere. To analyze it, we develop new concentration inequalities for Lipschitz functions under the uniform measure on the $\ell_1$-sphere, with explicit constants. These concentration tools are not only central to our high-probability guarantees but may also be of independent interest.

Method: Proposes Weighted Optimistic Follow-the-Regularized-Leader (WOFTRL), where the prediction vector of the next reward in OFTRL is weighted n times. The key insight is that optimistic weight can cancel out time delay effects.

Result: When optimistic weight exceeds time delay, WOFTRL achieves constant regret (O(1)-regret) in general-sum normal-form games, and strategies converge to Nash equilibrium as subsequence (best-iterate convergence) in poly-matrix zero-sum games. Experimental results support theoretical findings.

Conclusion: WOFTRL effectively addresses time-delayed feedback in multi-agent learning by using weighted optimism to cancel delay effects, recovering good performance guarantees that standard OFTRL loses under delays.

Abstract: This study raises and addresses the problem of time-delayed feedback in learning in games. Because learning in games assumes that multiple agents independently learn their strategies, a discrepancy in optimization often emerges among the agents. To overcome this discrepancy, the prediction of the future reward is incorporated into algorithms, typically known as Optimistic Follow-the-Regularized-Leader (OFTRL). However, the time delay in observing the past rewards hinders the prediction. Indeed, this study firstly proves that even a single-step delay worsens the performance of OFTRL from the aspects of regret and convergence. This study proposes the weighted OFTRL (WOFTRL), where the prediction vector of the next reward in OFTRL is weighted $n$ times. We further capture an intuition that the optimistic weight cancels out this time delay. We prove that when the optimistic weight exceeds the time delay, our WOFTRL recovers the good performances that the regret is constant ($O(1)$-regret) in general-sum normal-form games, and the strategies converge to the Nash equilibrium as a subsequence (best-iterate convergence) in poly-matrix zero-sum games. The theoretical results are supported and strengthened by our experiments.

Method: Modified Fourier neural operators with adjustable time resolution, tensor decomposition in spectral domain, Sobolev norm in error function, and separation of approximation/reconstruction errors.

Result: Achieved 6 orders of magnitude acceleration in hydrodynamic modeling of underground gas storage compared to traditional numerical methods.

Conclusion: The proposed neural operator enables effective control of complex reservoir systems through significantly faster computation while maintaining accuracy.

Abstract: This paper presents a method for modeling transient fluid flow in subsurface reservoir systems based on the developed neural operator architecture (TFNO-opt). Reservoir systems are complex dynamic objects with distributed parameters described by systems of partial differential equations (PDEs). Traditional numerical methods for modeling such systems, despite their high accuracy, are characterized by significant time costs for performing calculations, which limits their applicability in control and decision support problems. The proposed architecture (TFNO-opt) is based on Fourier neural operators, which allow approximating PDE solutions in infinite-dimensional functional spaces, providing invariance to discretization and the possibility of generalization to various implementations of equations. The developed modifications are aimed at increasing the accuracy and stability of the trained neural operator, which is especially important for control problems. These include adjustable internal time resolution of the integral Fourier operator, tensor decomposition of parameters in the spectral domain, use of the Sobolev norm in the error function, and separation of approximation errors and reconstruction of initial conditions for more accurate reproduction of physical processes. The effectiveness of the proposed improvements is confirmed by computational experiments. The practical significance is confirmed by computational experiments using the example of the problem of hydrodynamic modeling of an underground gas storage (UGS), where the acceleration of calculations by six orders of magnitude was achieved, compared to traditional methods. This opens up new opportunities for the effective control of complex reservoir systems.

Method: SPEAR extends vanilla self-imitation learning with a curriculum approach: uses intrinsic rewards for skill-level exploration initially, then strengthens self-imitation for action-level exploration. Includes replay buffer recalibration and regularization techniques like token clipping to control entropy and prevent over-confidence.

Result: The method enables progressive exploration-exploitation balance, allowing broad exposure to environment distributions with upward entropy trend initially, then accelerating solution iteration without unbounded entropy growth.

Conclusion: SPEAR provides a stable training framework for agentic LLMs that maintains balanced entropy across stages, addressing fundamental RL challenges in long-horizon, sparsely-rewarded tasks through curriculum-based self-imitation learning.

Abstract: Reinforcement learning (RL) is the dominant paradigm for sharpening strategic tool use capabilities of LLMs on long-horizon, sparsely-rewarded agent tasks, yet it faces a fundamental challenge of exploration-exploitation trade-off. Existing studies stimulate exploration through the lens of policy entropy, but such mechanical entropy maximization is prone to RL training instability due to the multi-turn distribution shifting. In this paper, we target the progressive exploration-exploitation balance under the guidance of the agent own experiences without succumbing to either entropy collapsing or runaway divergence. We propose SPEAR, a curriculum-based self-imitation learning (SIL) recipe for training agentic LLMs. It extends the vanilla SIL framework, where a replay buffer stores self-generated promising trajectories for off-policy update, by gradually steering the policy evolution within a well-balanced range of entropy across stages. Specifically, our approach incorporates a curriculum to manage the exploration process, utilizing intrinsic rewards to foster skill-level exploration and facilitating action-level exploration through SIL. At first, the auxiliary tool call reward plays a critical role in the accumulation of tool-use skills, enabling broad exposure to the unfamiliar distributions of the environment feedback with an upward entropy trend. As training progresses, self-imitation gets strengthened to exploit existing successful patterns from replayed experiences for comparative action-level exploration, accelerating solution iteration without unbounded entropy growth. To further stabilize training, we recalibrate the advantages of experiences in the replay buffer to address the potential policy drift. Reugularizations such as the clipping of tokens with high covariance between probability and advantage are introduced to the trajectory-level entropy control to curb over-confidence.

Method: Applied targeted interventions including paraphrasing, filler token injection, early answering, and introducing mistakes on two challenging reasoning datasets: SAKURA and MMAR.

Result: Experiments across several datasets and tasks suggest that LALMs generally produce CoTs that appear to be faithful to their underlying decision processes.

Conclusion: LALMs produce faithful chain-of-thought representations, unlike text-based LLMs which often produce unfaithful CoTs.

Abstract: Faithfulness measures whether chain-of-thought (CoT) representations accurately reflect a model’s decision process and can be used as reliable explanations. Prior work has shown that CoTs from text-based LLMs are often unfaithful. This question has not been explored for large audio-language models (LALMs), where faithfulness is critical for safety-sensitive applications. Reasoning in LALMs is also more challenging, as models must first extract relevant clues from audio before reasoning over them. In this paper, we investigate the faithfulness of CoTs produced by several LALMs by applying targeted interventions, including paraphrasing, filler token injection, early answering, and introducing mistakes, on two challenging reasoning datasets: SAKURA and MMAR. After going through the aforementioned interventions across several datasets and tasks, our experiments suggest that, LALMs generally produce CoTs that appear to be faithful to their underlying decision processes.

Method: 1) Design a prior distribution of synthetic attributed graphs using stochastic block models and preferential attachment; 2) Generate node attributes and targets using graph-aware structured causal models; 3) Augment tabular foundation model LimiX with attention-based graph neighborhood aggregation layers; 4) Train on synthetic graphs from the prior distribution.

Result: GraphPFN achieves state-of-the-art results on diverse real-world graph datasets with up to 50,000 nodes, outperforming both G2T-FM and task-specific GNNs trained from scratch on most datasets. It shows strong in-context learning performance.

Conclusion: Pretraining on synthetic graphs from a well-designed prior distribution is an effective strategy for building graph foundation models, enabling them to capture graph structural dependencies not present in tabular data.

Abstract: Foundation models pretrained on large-scale datasets have transformed such fields as natural language processing and computer vision, but their application to graph data remains limited. Recently emerged graph foundation models, such as G2T-FM, utilize tabular foundation models for graph tasks and were shown to significantly outperform prior attempts to create GFMs. However, these models primarily rely on hand-crafted graph features, limiting their ability to learn complex graph-specific patterns. In this work, we propose GraphPFN: a prior-data fitted network for node-level prediction. First, we design a prior distribution of synthetic attributed graphs. For graph structure generation, we use a novel combination of multiple stochastic block models and a preferential attachment process. We then apply graph-aware structured causal models to generate node attributes and targets. This procedure allows us to efficiently generate a wide range of realistic graph datasets. Then, we augment the tabular foundation model LimiX with attention-based graph neighborhood aggregation layers and train it on synthetic graphs sampled from our prior, allowing the model to capture graph structural dependencies not present in tabular data. On diverse real-world graph datasets with up to 50,000 nodes, GraphPFN shows strong in-context learning performance and achieves state-of-the-art results after finetuning, outperforming both G2T-FM and task-specific GNNs trained from scratch on most datasets. More broadly, our work demonstrates that pretraining on synthetic graphs from a well-designed prior distribution is an effective strategy for building graph foundation models.

Method: Reframes DM compression as spectral approximation using activation covariances across denoising timesteps. Uses dynamic pruning guided by low-rank subspaces, applying module-wise decompositions over functional weight groups (query-key interactions, value-output couplings, feedforward projections) with adaptive sparsity allocation.

Result: Achieves up to 35% acceleration and ~100M parameter reduction over baselines while maintaining generation quality comparable to uncompressed models. Requires only about 500 calibration samples (70× fewer than prior methods) and operates entirely without backpropagation.

Conclusion: SlimDiff provides the first closed-form, activation-guided structural compression of diffusion models that is entirely training-free, offering both theoretical clarity and practical efficiency improvements.

Abstract: Diffusion models (DMs), lauded for their generative performance, are computationally prohibitive due to their billion-scale parameters and iterative denoising dynamics. Existing efficiency techniques, such as quantization, timestep reduction, or pruning, offer savings in compute, memory, or runtime but are strictly bottlenecked by reliance on fine-tuning or retraining to recover performance. In this work, we introduce SlimDiff, an automated activation-informed structural compression framework that reduces both attention and feedforward dimensionalities in DMs, while being entirely gradient-free. SlimDiff reframes DM compression as a spectral approximation task, where activation covariances across denoising timesteps define low-rank subspaces that guide dynamic pruning under a fixed compression budget. This activation-aware formulation mitigates error accumulation across timesteps by applying module-wise decompositions over functional weight groups: query–key interactions, value–output couplings, and feedforward projections, rather than isolated matrix factorizations, while adaptively allocating sparsity across modules to respect the non-uniform geometry of diffusion trajectories. SlimDiff achieves up to 35% acceleration and $\sim$100M parameter reduction over baselines, with generation quality on par with uncompressed models without any backpropagation. Crucially, our approach requires only about 500 calibration samples, over 70$\times$ fewer than prior methods. To our knowledge, this is the first closed-form, activation-guided structural compression of DMs that is entirely training-free, providing both theoretical clarity and practical efficiency.

Method: Uses rubric-based rewards that leverage off-policy examples while remaining insensitive to their artifacts, with a workflow to elicit rubrics that distinguish among great and diverse responses in the high-reward region.

Result: Empirical demonstration shows rubric-based rewards substantially mitigate reward over-optimization and deliver effective LLM post-training improvements.

Conclusion: Rubric-based rewards provide an effective solution to reward over-optimization in reinforcement fine-tuning by addressing reward misspecification at the high-reward tail through proper distinction of excellent vs great responses.

Abstract: Reinforcement fine-tuning (RFT) often suffers from \emph{reward over-optimization}, where a policy model hacks the reward signals to achieve high scores while producing low-quality outputs. Our theoretical analysis shows that the key lies in reward misspecification at the high-reward tail: the inability to reliably distinguish Excellent responses from merely Great ones. This motivate us to focus on the high-reward region. However, such tail examples are scarce under the base LLM. While off-policy exemplars (e.g. from stronger models or rewrites) are easier to obtain, naively training on them yields a misspecified reward for the policy we aim to align. To address this, we study rubric-based rewards. By design, rubrics can leverage off-policy examples while remaining insensitive to their artifacts. To elicit rubrics that capture the high-reward tail, we highlight the importance of distinguishing among great and diverse responses, and introduce a workflow to implement this idea. We empirically demonstrate that rubric-based rewards substantially mitigate reward over-optimization and deliver effective LLM post-training improvements. Our code can be accessed at https://github.com/Jun-Kai-Zhang/rubrics.git .

Method: Uses a modularized world model with action-conditioned, action-free, and static branches, combined with variational auto-encoders and graph auto-encoders to learn latent representations, integrated with value-based framework for joint action-value prediction.

Result: Achieves high sample efficiency and superior performance compared to baselines on StarCraft II micro-management, Multi-Agent MuJoCo, and Level-Based Foraging challenges.

Conclusion: The proposed Value Decomposition Framework with Disentangled World Model effectively reduces sample complexity while maintaining strong performance across diverse multi-agent tasks.

Abstract: In this paper, we propose a novel model-based multi-agent reinforcement learning approach named Value Decomposition Framework with Disentangled World Model to address the challenge of achieving a common goal of multiple agents interacting in the same environment with reduced sample complexity. Due to scalability and non-stationarity problems posed by multi-agent systems, model-free methods rely on a considerable number of samples for training. In contrast, we use a modularized world model, composed of action-conditioned, action-free, and static branches, to unravel the complicated environment dynamics. Our model produces imagined outcomes based on past experience, without sampling directly from the real environment. We employ variational auto-encoders and variational graph auto-encoders to learn the latent representations for the world model, which is merged with a value-based framework to predict the joint action-value function and optimize the overall training objective. Experimental results on StarCraft II micro-management, Multi-Agent MuJoCo, and Level-Based Foraging challenges demonstrate that our method achieves high sample efficiency and exhibits superior performance compared to other baselines across a wide range of multi-agent learning tasks.

Method: Extends MIM with contrastive objective to impose global discriminative structure while retaining generative fidelity. Introduces ‘informative embeddings’ technique for enriched feature extraction from encoder-decoder models without additional training.

Result: Outperforms MIM and InfoNCE on classification and regression tasks across vision and molecular benchmarks while preserving competitive reconstruction quality. Less sensitive to batch size than InfoNCE and doesn’t require positive data augmentation.

Conclusion: cMIM serves as a unified framework for representation learning that effectively serves both discriminative and generative applications, advancing the goal of models that balance these capabilities.

Abstract: Learning representations that transfer well to diverse downstream tasks remains a central challenge in representation learning. Existing paradigms – contrastive learning, self-supervised masking, and denoising auto-encoders – balance this challenge with different trade-offs. We introduce the {contrastive Mutual Information Machine} (cMIM), a probabilistic framework that extends the Mutual Information Machine (MIM) with a contrastive objective. While MIM maximizes mutual information between inputs and latents and promotes clustering of codes, it falls short on discriminative tasks. cMIM addresses this gap by imposing global discriminative structure while retaining MIM’s generative fidelity. Our contributions are threefold. First, we propose cMIM, a contrastive extension of MIM that removes the need for positive data augmentation and is substantially less sensitive to batch size than InfoNCE. Second, we introduce {informative embeddings}, a general technique for extracting enriched features from encoder-decoder models that boosts discriminative performance without additional training and applies broadly beyond MIM. Third, we provide empirical evidence across vision and molecular benchmarks showing that cMIM outperforms MIM and InfoNCE on classification and regression tasks while preserving competitive reconstruction quality. These results position cMIM as a unified framework for representation learning, advancing the goal of models that serve both discriminative and generative applications effectively.

Method: Represent reconstructed scans with clinical metrics and calibrate bounds on ground truth metrics using conformal prediction with prior calibration data.

Result: The framework produces bounds with better semantic interpretation than pixel-based approaches and can flag dangerous outlier reconstructions that look plausible but have unlikely metric values.

Conclusion: The proposed framework provides interpretable feedback about subjects’ states and enables detection of misleading reconstructions in medical imaging tasks like CT for fat mass quantification and radiotherapy planning.

Abstract: Modern deep learning reconstruction algorithms generate impressively realistic scans from sparse inputs, but can often produce significant inaccuracies. This makes it difficult to provide statistically guaranteed claims about the true state of a subject from scans reconstructed by these algorithms. In this study, we propose a framework for computing provably valid prediction bounds on claims derived from probabilistic black-box image reconstruction algorithms. The key insights behind our framework are to represent reconstructed scans with a derived clinical metric of interest, and to calibrate bounds on the ground truth metric with conformal prediction (CP) using a prior calibration dataset. These bounds convey interpretable feedback about the subject’s state, and can also be used to retrieve nearest-neighbor reconstructed scans for visual inspection. We demonstrate the utility of this framework on sparse-view computed tomography (CT) for fat mass quantification and radiotherapy planning tasks. Results show that our framework produces bounds with better semantical interpretation than conventional pixel-based bounding approaches. Furthermore, we can flag dangerous outlier reconstructions that look plausible but have statistically unlikely metric values.

Method: Uses Kac dynamics with damped wave equation, introduces classifier-free guidance in velocity space, and proposes endpoint-only distillation where a student model learns to match a frozen teacher over long intervals.

Result: Experiments show DistillKac delivers high-quality samples with very few function evaluations while retaining numerical stability benefits of finite speed probability flows.

Conclusion: DistillKac provides an effective alternative to diffusion models by leveraging finite-speed probability transport through Kac dynamics, enabling fast and stable image generation with minimal computational cost.

Abstract: We present DistillKac, a fast image generator that uses the damped wave equation and its stochastic Kac representation to move probability mass at finite speed. In contrast to diffusion models whose reverse time velocities can become stiff and implicitly allow unbounded propagation speed, Kac dynamics enforce finite speed transport and yield globally bounded kinetic energy. Building on this structure, we introduce classifier-free guidance in velocity space that preserves square integrability under mild conditions. We then propose endpoint only distillation that trains a student to match a frozen teacher over long intervals. We prove a stability result that promotes supervision at the endpoints to closeness along the entire path. Experiments demonstrate DistillKac delivers high quality samples with very few function evaluations while retaining the numerical stability benefits of finite speed probability flows.

Method: SD-CQL discovers skills in latent space by reconstructing next observations, evaluates fixed and variable actions separately, and applies conservative Q-learning with local value calibration to select optimal actions for each skill.

Result: Extensive experiments on StarCraft II show SD-CQL achieves best performance on 13 out of 14 task sets, with up to 68.9% improvement on individual task sets.

Conclusion: SD-CQL enables strong multi-task generalization from limited source tasks, eliminating the need for local-global alignment and demonstrating superior task-efficiency and generalization performance.

Abstract: As a data-driven approach, offline MARL learns superior policies solely from offline datasets, ideal for domains rich in historical data but with high interaction costs and risks. However, most existing methods are task-specific, requiring retraining for new tasks, leading to redundancy and inefficiency. To address this issue, we propose a task-efficient value-based multi-task offline MARL algorithm, Skill-Discovery Conservative Q-Learning (SD-CQL). Unlike existing methods decoding actions from skills via behavior cloning, SD-CQL discovers skills in a latent space by reconstructing the next observation, evaluates fixed and variable actions separately, and uses conservative Q-learning with local value calibration to select the optimal action for each skill. It eliminates the need for local-global alignment and enables strong multi-task generalization from limited, small-scale source tasks. Substantial experiments on StarCraft II demonstrate the superior generalization performance and task-efficiency of SD-CQL. It achieves the best performance on $\textbf{13}$ out of $14$ task sets, with up to $\textbf{68.9%}$ improvement on individual task sets.

Method: The approach adds uncertainty estimation capabilities to KT models to capture when predictions are uncertain, particularly aligning with model incorrect predictions.

Result: The research demonstrates that larger predictive uncertainty aligns with model incorrect predictions, and this uncertainty signal is informative for educational applications.

Conclusion: Uncertainty in KT models provides pedagogically useful information that can be valuable in educational platforms, especially in resource-limited settings where understanding student ability is crucial.

Abstract: The main focus of research on Knowledge Tracing (KT) models is on model developments with the aim of improving predictive accuracy. Most of these models make the most incorrect predictions when students choose a distractor, leading to student errors going undetected. We present an approach to add new capabilities to KT models by capturing predictive uncertainty and demonstrate that a larger predictive uncertainty aligns with model incorrect predictions. We show that uncertainty in KT models is informative and that this signal would be pedagogically useful for application in an educational learning platform that can be used in a limited resource setting where understanding student ability is necessary.

Method: The Li₂ framework analyzes backpropagated gradient structure G_F across layers. It studies gradient dynamics, energy function optimization, and feature emergence in group arithmetic tasks, examining effects of hyperparameters like weight decay and learning rate.

Result: The framework reveals that independent feature learning follows gradient ascent of an energy function E, with local maxima corresponding to emerging features. It shows how hidden nodes interact and how gradients focus on missing features, leading to provable scaling laws for memorization and generalization.

Conclusion: The study provides mathematical understanding of grokking dynamics, explains why optimizers like Muon work effectively, and offers insights into the roles of key hyperparameters. The analysis framework can be extended to multi-layer architectures.

Abstract: While the phenomenon of grokking, i.e., delayed generalization, has been studied extensively, it remains an open question whether there is a mathematical framework to characterize what kind of features emerge, how and in which conditions it happens from training, for complex structured inputs. We propose a novel framework, named $\mathbf{Li_2}$, that captures three key stages for the grokking behavior of 2-layer nonlinear networks: (I) Lazy learning, (II) independent feature learning and (III) interactive feature learning, characterized by the structure of backpropagated gradient $G_F$ across layers. In (I), $G_F$ is random, and top layer overfits to random hidden representation. In (II), the gradient of each node (column of $G_F$) only depends on its own activation, and thus each hidden node learns their representation independently from $G_F$, which now carries information about target labels, thanks to weight decay. Interestingly, the independent dynamics follows exactly the gradient ascent of an energy function $E$, and its local maxima are precisely the emerging features. We study whether these local-optima induced features are generalizable, their representation power, and how they change on sample size, in group arithmetic tasks. Finally, in (III), we provably show how hidden nodes interact, and how $G_F$ changes to focus on missing features that need to be learned. Our study sheds lights on roles played by key hyperparameters such as weight decay, learning rate and sample sizes in grokking, leads to provable scaling laws of memorization and generalization, and reveals the underlying cause why recent optimizers such as Muon can be effective, from the first principles of gradient dynamics. Our analysis can be extended to multi-layer architectures.

Method: Formulates SSL as a Stackelberg game with three roles: two student classifiers on complementary representations, a meta-learned teacher for adaptive pseudo-label selection and loss balancing, and a non-parametric generator for adversarial perturbations. Uses mutual information for pseudo-label selection instead of confidence.

Result: Extensive experiments on CIFAR-10, SVHN, STL-10, and ImageNet show TRiCo consistently achieves state-of-the-art performance in low-label regimes while being architecture-agnostic and compatible with frozen vision backbones.

Conclusion: TRiCo provides a principled and generalizable solution to semi-supervised learning by formalizing the learning process as a triadic game-theoretic interaction, effectively addressing key limitations of existing SSL frameworks.

Abstract: We introduce TRiCo, a novel triadic game-theoretic co-training framework that rethinks the structure of semi-supervised learning by incorporating a teacher, two students, and an adversarial generator into a unified training paradigm. Unlike existing co-training or teacher-student approaches, TRiCo formulates SSL as a structured interaction among three roles: (i) two student classifiers trained on frozen, complementary representations, (ii) a meta-learned teacher that adaptively regulates pseudo-label selection and loss balancing via validation-based feedback, and (iii) a non-parametric generator that perturbs embeddings to uncover decision boundary weaknesses. Pseudo-labels are selected based on mutual information rather than confidence, providing a more robust measure of epistemic uncertainty. This triadic interaction is formalized as a Stackelberg game, where the teacher leads strategy optimization and students follow under adversarial perturbations. By addressing key limitations in existing SSL frameworks, such as static view interactions, unreliable pseudo-labels, and lack of hard sample modeling, TRiCo provides a principled and generalizable solution. Extensive experiments on CIFAR-10, SVHN, STL-10, and ImageNet demonstrate that TRiCo consistently achieves state-of-the-art performance in low-label regimes, while remaining architecture-agnostic and compatible with frozen vision backbones.

Method: Models autoregressive generation as a dynamical system in latent space and learns a safety value function via backward reachability to estimate worst-case trajectory evolution, enabling runtime monitoring and least-restrictive steering filters.

Result: BRT-Align provides more accurate and earlier detection of unsafe continuations than baselines, substantially reduces unsafe generations while preserving diversity and coherence, and produces responses that are less violent, profane, offensive, and politically biased.

Conclusion: Reachability analysis provides a principled and practical foundation for inference-time LLM safety, offering effective safeguards during generation rather than just during training.

Abstract: Large language models (LLMs) are now ubiquitous in everyday tools, raising urgent safety concerns about their tendency to generate harmful content. The dominant safety approach – reinforcement learning from human feedback (RLHF) – effectively shapes model behavior during training but offers no safeguards at inference time, where unsafe continuations may still arise. We propose BRT-Align, a reachability-based framework that brings control-theoretic safety tools to LLM inference. BRT-Align models autoregressive generation as a dynamical system in latent space and learn a safety value function via backward reachability, estimating the worst-case evolution of a trajectory. This enables two complementary mechanisms: (1) a runtime monitor that forecasts unsafe completions several tokens in advance, and (2) a least-restrictive steering filter that minimally perturbs latent states to redirect generation away from unsafe regions. Experiments across multiple LLMs and toxicity benchmarks demonstrate that BRT-Align provides more accurate and earlier detection of unsafe continuations than baselines. Moreover, for LLM safety alignment, BRT-Align substantially reduces unsafe generations while preserving sentence diversity and coherence. Qualitative results further highlight emergent alignment properties: BRT-Align consistently produces responses that are less violent, less profane, less offensive, and less politically biased. Together, these findings demonstrate that reachability analysis provides a principled and practical foundation for inference-time LLM safety.

Method: Proposed a lightweight query-based model collaboration framework that integrates expert-level domain knowledge to guide the augmentation process and preserve critical medical information.

Result: Experiments on clinical prediction tasks show the approach consistently outperforms existing LLM augmentation methods while improving safety through reduced factual errors.

Conclusion: The framework successfully addresses the safety-performance trade-off in LLM-based data augmentation for specialized domains like healthcare, enabling safer and more effective augmentation.

Abstract: Data augmentation is a widely used strategy to improve model robustness and generalization by enriching training datasets with synthetic examples. While large language models (LLMs) have demonstrated strong generative capabilities for this purpose, their applications in high-stakes domains like healthcare present unique challenges due to the risk of generating clinically incorrect or misleading information. In this work, we propose a novel query-based model collaboration framework that integrates expert-level domain knowledge to guide the augmentation process to preserve critical medical information. Experiments on clinical prediction tasks demonstrate that our lightweight collaboration-based approach consistently outperforms existing LLM augmentation methods while improving safety through reduced factual errors. This framework addresses the gap between LLM augmentation potential and the safety requirements of specialized domains.

Method: Designed synthetic in-context learning tasks with hierarchical dependencies, evaluated various LLMs on these sequences and natural language analogues, and analyzed how induction heads learn contextually.

Result: Found adaptive induction heads that learn what to attend to in-context, supported by attention heads that uncover latent contexts determining token transition relationships.

Conclusion: LLMs have learning induction heads that provide a complete mechanistic account of how they learn to predict higher-order repetitive patterns in-context.

Abstract: Large Language Models (LLMs) excel at in-context learning, the ability to use information provided as context to improve prediction of future tokens. Induction heads have been argued to play a crucial role for in-context learning in Transformer Language Models. These attention heads make a token attend to successors of past occurrences of the same token in the input. This basic mechanism supports LLMs’ ability to copy and predict repeating patterns. However, it is unclear if this same mechanism can support in-context learning of more complex repetitive patterns with hierarchical structure. Natural language is teeming with such cases: The article “the” in English usually prefaces multiple nouns in a text. When predicting which token succeeds a particular instance of “the”, we need to integrate further contextual cues from the text to predict the correct noun. If induction heads naively attend to all past instances of successor tokens of “the” in a context-independent manner, they cannot support this level of contextual information integration. In this study, we design a synthetic in-context learning task, where tokens are repeated with hierarchical dependencies. Here, attending uniformly to all successor tokens is not sufficient to accurately predict future tokens. Evaluating a range of LLMs on these token sequences and natural language analogues, we find adaptive induction heads that support prediction by learning what to attend to in-context. Next, we investigate how induction heads themselves learn in-context. We find evidence that learning is supported by attention heads that uncover a set of latent contexts, determining the different token transition relationships. Overall, we not only show that LLMs have induction heads that learn, but offer a complete mechanistic account of how LLMs learn to predict higher-order repetitive patterns in-context.

Method: The approach eschews model self-reports and instead tests strategic deployment of internal state knowledge using two experimental paradigms, analyzing both behavioral responses and token probabilities.

Result: Frontier LLMs show increasingly strong evidence of metacognitive abilities including confidence assessment and answer anticipation, though these abilities are limited in resolution, context-dependent, and qualitatively different from humans.

Conclusion: LLMs demonstrate emerging metacognitive abilities that vary across models, suggesting post-training may play a role in developing these capabilities, with important implications for AI safety and understanding.

Abstract: The possibility of LLM self-awareness and even sentience is gaining increasing public attention and has major safety and policy implications, but the science of measuring them is still in a nascent state. Here we introduce a novel methodology for quantitatively evaluating metacognitive abilities in LLMs. Taking inspiration from research on metacognition in nonhuman animals, our approach eschews model self-reports and instead tests to what degree models can strategically deploy knowledge of internal states. Using two experimental paradigms, we demonstrate that frontier LLMs introduced since early 2024 show increasingly strong evidence of certain metacognitive abilities, specifically the ability to assess and utilize their own confidence in their ability to answer factual and reasoning questions correctly and the ability to anticipate what answers they would give and utilize that information appropriately. We buttress these behavioral findings with an analysis of the token probabilities returned by the models, which suggests the presence of an upstream internal signal that could provide the basis for metacognition. We further find that these abilities 1) are limited in resolution, 2) emerge in context-dependent manners, and 3) seem to be qualitatively different from those of humans. We also report intriguing differences across models of similar capabilities, suggesting that LLM post-training may have a role in developing metacognitive abilities.

Method: Uses concentration inequalities (Markov’s, Chebyshev’s, Hoeffding’s, Bernstein’s, etc.) for controlling estimation errors, then applies these to derive generalization bounds in offline learning (Occam’s razor, VC analysis, PAC-Bayesian analysis) and regret bounds in online learning (stochastic/adversarial environments, full/bandit feedback).

Result: Develops comprehensive statistical framework for obtaining theoretical guarantees on learning outcomes across both offline and online settings.

Conclusion: The book establishes a unified statistical foundation for analyzing learning processes under uncertainty, providing tools for deriving performance guarantees in various machine learning scenarios.

Abstract: Learning, whether natural or artificial, is a process of selection. It starts with a set of candidate options and selects the more successful ones. In the case of machine learning the selection is done based on empirical estimates of prediction accuracy of candidate prediction rules on some data. Due to randomness of data sampling the empirical estimates are inherently noisy, leading to selection under uncertainty. The book provides statistical tools to obtain theoretical guarantees on the outcome of selection under uncertainty. We start with concentration of measure inequalities, which are the main statistical instrument for controlling how much an empirical estimate of expectation of a function deviates from the true expectation. The book covers a broad range of inequalities, including Markov’s, Chebyshev’s, Hoeffding’s, Bernstein’s, Empirical Bernstein’s, Unexpected Bernstein’s, kl, and split-kl. We then study the classical (offline) supervised learning and provide a range of tools for deriving generalization bounds, including Occam’s razor, Vapnik-Chervonenkis analysis, and PAC-Bayesian analysis. The latter is further applied to derive generalization guarantees for weighted majority votes. After covering the offline setting, we turn our attention to online learning. We present the space of online learning problems characterized by environmental feedback, environmental resistance, and structural complexity. A common performance measure in online learning is regret, which compares performance of an algorithm to performance of the best prediction rule in hindsight, out of a restricted set of prediction rules. We present tools for deriving regret bounds in stochastic and adversarial environments, and under full information and bandit feedback.

Method: GDM introduces continuous relaxation of discrete states and a Gumbel distribution-based noise model on the relaxed-discrete state space, making the model fully differentiable for gradient-based training while expanding available state dynamics.

Result: GDM successfully models soft, sticky states and transitions in stochastic settings, outperforms traditional methods on simulation datasets, and infers interpretable states in real-world time series with multiple dynamics where conventional approaches fail.

Conclusion: The Gumbel Dynamical Model provides an effective solution for modeling complex time series with smooth transitions and stochastic mixtures, offering improved interpretability and scalability compared to traditional switching dynamical systems.

Abstract: Switching dynamical systems can model complicated time series data while maintaining interpretability by inferring a finite set of dynamics primitives and explaining different portions of the observed time series with one of these primitives. However, due to the discrete nature of this set, such models struggle to capture smooth, variable-speed transitions, as well as stochastic mixtures of overlapping states, and the inferred dynamics often display spurious rapid switching on real-world datasets. Here, we propose the Gumbel Dynamical Model (GDM). First, by introducing a continuous relaxation of discrete states and a different noise model defined on the relaxed-discrete state space via the Gumbel distribution, GDM expands the set of available state dynamics, allowing the model to approximate smoother and non-stationary ground-truth dynamics more faithfully. Second, the relaxation makes the model fully differentiable, enabling fast and scalable training with standard gradient descent methods. We validate our approach on standard simulation datasets and highlight its ability to model soft, sticky states and transitions in a stochastic setting. Furthermore, we apply our model to two real-world datasets, demonstrating its ability to infer interpretable states in stochastic time series with multiple dynamics, a setting where traditional methods often fail.

Method: Employed advanced big data analytics and machine learning approaches on Amazon review dataset, using scalable big data framework to process large-scale data and extract features for fraud detection.

Result: Various machine learning classifiers were tested, with Logistic Regression achieving the highest accuracy of 90.35% in detecting spam reviews.

Conclusion: The research successfully contributes to creating a more trustworthy and transparent online shopping environment by accurately detecting fraudulent reviews.

Abstract: In this digital era, online shopping is common practice in our daily lives. Product reviews significantly influence consumer buying behavior and help establish buyer trust. However, the prevalence of fraudulent reviews undermines this trust by potentially misleading consumers and damaging the reputations of the sellers. This research addresses this pressing issue by employing advanced big data analytics and machine learning approaches on a substantial dataset of Amazon product reviews. The primary objective is to detect and classify spam reviews accurately so that it enhances the authenticity of the review. Using a scalable big data framework, we efficiently process and analyze a large scale of review data, extracting key features indicative of fraudulent behavior. Our study illustrates the utility of various machine learning classifiers in detecting spam reviews, with Logistic Regression achieving an accuracy of 90.35%, thus contributing to a more trustworthy and transparent online shopping environment.

Method: Introduces a generative hyper-network model that directly produces parameter distributions consistent with observed data, avoiding the need for explicit likelihood construction through measure-theoretic foundations.

Result: GenUQ outperforms other UQ methods in three benchmark problems: recovering manufactured operators, learning solution operators for stochastic elliptic PDEs, and modeling failure locations in porous steel under tension.

Conclusion: The GenUQ framework provides an effective likelihood-free approach to uncertainty quantification in operator learning that handles complex stochastic operators where traditional methods fail.

Abstract: Operator learning is a recently developed generalization of regression to mappings between functions. It promises to drastically reduce expensive numerical integration of PDEs to fast evaluations of mappings between functional states of a system, i.e., surrogate and reduced-order modeling. Operator learning has already found applications in several areas such as modeling sea ice, combustion, and atmospheric physics. Recent approaches towards integrating uncertainty quantification into the operator models have relied on likelihood based methods to infer parameter distributions from noisy data. However, stochastic operators may yield actions from which a likelihood is difficult or impossible to construct. In this paper, we introduce, GenUQ, a measure-theoretic approach to UQ that avoids constructing a likelihood by introducing a generative hyper-network model that produces parameter distributions consistent with observed data. We demonstrate that GenUQ outperforms other UQ methods in three example problems, recovering a manufactured operator, learning the solution operator to a stochastic elliptic PDE, and modeling the failure location of porous steel under tension.

Method: Projects client updates onto orthogonal complement of previous task representations and uses lightweight task identity prediction with core bases from prior tasks.

Result: Significantly outperforms state-of-the-art methods in average accuracy, especially when task identities are unknown.

Conclusion: FedProTIP effectively mitigates forgetting in federated continual learning through gradient projection and task identity prediction.

Abstract: Federated continual learning (FCL) enables distributed client devices to learn from streaming data across diverse and evolving tasks. A major challenge to continual learning, catastrophic forgetting, is exacerbated in decentralized settings by the data heterogeneity, constrained communication and privacy concerns. We propose Federated gradient Projection-based Continual Learning with Task Identity Prediction (FedProTIP), a novel FCL framework that mitigates forgetting by projecting client updates onto the orthogonal complement of the subspace spanned by previously learned representations of the global model. This projection reduces interference with earlier tasks and preserves performance across the task sequence. To further address the challenge of task-agnostic inference, we incorporate a lightweight mechanism that leverages core bases from prior tasks to predict task identity and dynamically adjust the global model’s outputs. Extensive experiments across standard FCL benchmarks demonstrate that FedProTIP significantly outperforms state-of-the-art methods in average accuracy, particularly in settings where task identities are a priori unknown.

Method: The authors introduce projected abstractions that can accommodate lossy representations, show how to construct them from low-level models, and prove graphical criteria for identifying and estimating high-level causal queries from limited low-level data.

Result: The paper demonstrates that projected abstractions can effectively translate equivalent observational, interventional, and counterfactual causal queries from low to high-level, and shows experimental effectiveness in high-dimensional image settings.

Conclusion: Projected abstractions provide a more flexible framework for causal abstraction that handles lossy representations and enables practical estimation of high-level causal queries from limited low-level data.

Abstract: The study of causal abstractions bridges two integral components of human intelligence: the ability to determine cause and effect, and the ability to interpret complex patterns into abstract concepts. Formally, causal abstraction frameworks define connections between complicated low-level causal models and simple high-level ones. One major limitation of most existing definitions is that they are not well-defined when considering lossy abstraction functions in which multiple low-level interventions can have different effects while mapping to the same high-level intervention (an assumption called the abstract invariance condition). In this paper, we introduce a new type of abstractions called projected abstractions that generalize existing definitions to accommodate lossy representations. We show how to construct a projected abstraction from the low-level model and how it translates equivalent observational, interventional, and counterfactual causal queries from low to high-level. Given that the true model is rarely available in practice we prove a new graphical criteria for identifying and estimating high-level causal queries from limited low-level data. Finally, we experimentally show the effectiveness of projected abstraction models in high-dimensional image settings.

Method: LANCE performs one-shot higher-order SVD to obtain a reusable low-rank subspace for activation projection, eliminating repeated decompositions and enabling task allocation to orthogonal subspaces for continual learning.

Result: LANCE reduces activation storage by up to 250× while maintaining comparable accuracy to full backpropagation on multiple datasets, and achieves competitive performance on continual learning benchmarks at a fraction of the memory cost.

Conclusion: LANCE provides a practical and scalable solution for efficient fine-tuning and continual learning on resource-constrained edge devices.

Abstract: On-device learning is essential for personalization, privacy, and long-term adaptation in resource-constrained environments. Achieving this requires efficient learning, both fine-tuning existing models and continually acquiring new tasks without catastrophic forgetting. Yet both settings are constrained by high memory cost of storing activations during backpropagation. Existing activation compression methods reduce this cost but relying on repeated low-rank decompositions, introducing computational overhead. Also, such methods have not been explored for continual learning. We propose LANCE (Low-rank Activation Compression), a framework that performs one-shot higher-order Singular Value Decompsoition (SVD) to obtain a reusable low-rank subspace for activation projection. This eliminates repeated decompositions, reducing both memory and computation. Moreover, fixed low-rank subspaces further enable on-device continual learning by allocating tasks to orthogonal subspaces without storing large task-specific matrices. Experiments show that LANCE reduces activation storage up to 250$\times$ while maintaining accuracy comparable to full backpropagation on CIFAR-10/100, Oxford-IIIT Pets, Flowers102, and CUB-200 datasets. On continual learning benchmarks (Split CIFAR-100, Split MiniImageNet, 5-Datasets), it achieves performance competitive with orthogonal gradient projection methods at a fraction of the memory cost. These results position LANCE as a practical and scalable solution for efficient fine-tuning and continual learning on edge devices.

Method: Identify partial convergence states and dynamically switch to LoRA using hyperparameters to determine switching point and assign layer-specific ranks based on convergence level.

Result: Reduces trainable parameters to 10% of original size, achieves 3x throughput improvement, 1.5x reduction in training time per epoch, and 20% GPU memory reduction while preserving accuracy.

Conclusion: Dynamic LoRA switching effectively reduces training costs without compromising model performance, making large model training more efficient.

Abstract: Training large models ranging from millions to billions of parameters is highly resource-intensive, requiring significant time, compute, and memory. It is observed that most of the learning (higher change in weights) takes place in the earlier stage of the training loop. These changes stabilize as training continues, enabling them to be captured by matrices of a low intrinsic rank. Therefore, we propose an approach to identify such states of partial convergence and dynamically switch from full parameter training to Low-Rank Adaptation (LoRA) on the ViT-Large model. We introduce a flexible approach that leverages user-defined hyperparameters to determine the switching point and assign a rank specific to each module layer based on its level of convergence. Experimental results show that this approach preserves model accuracy while reducing the number of trainable parameters to 10% of its original size, resulting in a 3x improvement in throughput, and a 1.5x reduction in average training time per epoch while also reducing GPU memory consumption by 20%

Method: Uses SE(3)-equivariant graph neural networks to construct symmetric Hessians from irreducible representation features up to degree l=2 during message passing, enabling direct prediction without differentiation.

Result: HIP Hessians are 1-2 orders of magnitude faster, more accurate, more memory efficient, easier to train, and scale better with system size compared to traditional methods.

Conclusion: The direct prediction approach enables superior performance across computational chemistry tasks including transition state search, geometry optimization, and vibrational analysis, with open-source code available.

Abstract: Fundamental tasks in computational chemistry, from transition state search to vibrational analysis, rely on molecular Hessians, which are the second derivatives of the potential energy. Yet, Hessians are computationally expensive to calculate and scale poorly with system size, with both quantum mechanical methods and neural networks. In this work, we demonstrate that Hessians can be predicted directly from a deep learning model, without relying on automatic differentiation or finite differences. We observe that one can construct SE(3)-equivariant, symmetric Hessians from irreducible representations (irrep) features up to degree $l$=2 computed during message passing in graph neural networks. This makes HIP Hessians one to two orders of magnitude faster, more accurate, more memory efficient, easier to train, and enables more favorable scaling with system size. We validate our predictions across a wide range of downstream tasks, demonstrating consistently superior performance for transition state search, accelerated geometry optimization, zero-point energy corrections, and vibrational analysis benchmarks. We open-source the HIP codebase and model weights to enable further development of the direct prediction of Hessians at https://github.com/BurgerAndreas/hip

Method: Partitions weight matrices into blocks and applies HiRA-style multiplicative modulation independently within each block, preserving PEFT parameter footprint while unlocking localized rank amplification.

Result: BHRA consistently outperforms strong PEFT baselines across eight commonsense reasoning tasks and two arithmetic benchmarks with various model sizes (Llama-3.2 1B/3B, Mistral-7B, Gemma-2 9B) under matched parameter budgets.

Conclusion: Blockwise design maintains rich spectra across rank budgets and mitigates collapse induced by global modulation, making BHRA an effective parameter-efficient fine-tuning approach.

Abstract: Parameter-efficient fine-tuning (PEFT) methods must be resource-efficient yet handle heterogeneous reasoning transformations, and classical low-rank adaptation (LoRA) is constrained by the nominal rank $r$. Hadamard-style extensions like HiRA raise the nominal rank but couple every update to the global energy pattern of the frozen weight matrix, while ABBA trades this inductive bias for fully learned dense intermediates. To address the limitation of global modulation, we propose Block Hadamard high-Rank Adaptation (BHRA), which partitions each weight matrix and applies HiRA-style multiplicative modulation independently within every block, preserving the PEFT parameter footprint while unlocking localized rank amplification. Our empirical analyses reveal that this blockwise design maintains rich spectra across rank budgets, mitigating the collapse induced by global modulation. Across eight commonsense reasoning tasks and two arithmetic benchmarks with Llama-3.2 1B/3B, Mistral-7B, and Gemma-2 9B, BHRA consistently surpasses strong PEFT baselines under matched parameter budgets.

Method: Theoretical analysis using high-dimensional minimum-norm linear regression to characterize mask-based pretraining behavior, followed by development of R²MAE - a simple pretraining scheme that randomly varies mask ratios to capture multi-scale features.

Result: R²MAE consistently outperforms standard and more complex masking schemes across vision, language, DNA sequence, and single-cell models, improving state-of-the-art performance without requiring optimal fixed mask ratio tuning.

Conclusion: The theoretical framework successfully explains mask-based pretraining behavior, and the simple R²MAE method demonstrates that varying mask ratios during training enables better multi-scale feature learning than fixed-ratio approaches.

Abstract: Mask-based pretraining has become a cornerstone of modern large-scale models across language, vision, and recently biology. Despite its empirical success, its role and limits in learning data representations have been unclear. In this work, we show that the behavior of mask-based pretraining can be directly characterized by test risk in high-dimensional minimum-norm (“ridge-less”) linear regression, without relying on further model specifications. Further analysis of linear models uncovers several novel aspects of mask-based pretraining. The theoretical framework and its implications have been validated across diverse neural architectures (including MLPs, CNNs, and Transformers) applied to both vision and language tasks. Guided by our theory, we propose an embarrassingly simple yet overlooked pretraining scheme named Randomly Random Mask AutoEncoding (R$^2$MAE), which enforces capturing multi-scale features from data and is able to outperform optimal fixed mask ratio settings in our linear model framework. We implement R$^2$MAE in vision, language, DNA sequence, and single-cell models, where it consistently outperforms standard and more complicated masking schemes, leading to improvements for state-of-the-art models. Our code is available at: https://github.com/MingzeDong/r2mae

Method: Proposes formal mathematical definitions: safety as never making false claims, trust as assuming safety, and AGI as always matching/exceeding human capability. Uses proofs drawing from Gödel’s incompleteness theorems and Turing’s halting problem.

Result: Proves that for the formal definitions, a safe and trusted AI system cannot be an AGI system - there exist tasks solvable by humans but not by such systems.

Conclusion: There is a fundamental incompatibility between mathematically strict safety/trust and AGI, though real-world systems may use practical interpretations that avoid this limitation.

Abstract: Safety, trust and Artificial General Intelligence (AGI) are aspirational goals in artificial intelligence (AI) systems, and there are several informal interpretations of these notions. In this paper, we propose strict, mathematical definitions of safety, trust, and AGI, and demonstrate a fundamental incompatibility between them. We define safety of a system as the property that it never makes any false claims, trust as the assumption that the system is safe, and AGI as the property of an AI system always matching or exceeding human capability. Our core finding is that – for our formal definitions of these notions – a safe and trusted AI system cannot be an AGI system: for such a safe, trusted system there are task instances which are easily and provably solvable by a human but not by the system. We note that we consider strict mathematical definitions of safety and trust, and it is possible for real-world deployments to instead rely on alternate, practical interpretations of these notions. We show our results for program verification, planning, and graph reachability. Our proofs draw parallels to G"odel’s incompleteness theorems and Turing’s proof of the undecidability of the halting problem, and can be regarded as interpretations of G"odel’s and Turing’s results.

Method: DriftLite exploits an unexplored degree of freedom in the Fokker-Planck equation between drift and particle potential, yielding Variance-Controlling Guidance (VCG) and Energy-Controlling Guidance (ECG) for optimal drift approximation with minimal overhead.

Result: Across Gaussian mixture models, particle systems, and protein-ligand co-folding problems, DriftLite consistently reduces variance and improves sample quality compared to pure guidance and sequential Monte Carlo baselines.

Conclusion: DriftLite provides a principled, efficient route for scalable inference-time adaptation of diffusion models through lightweight particle-based steering with provable stability control.

Abstract: We study inference-time scaling for diffusion models, where the goal is to adapt a pre-trained model to new target distributions without retraining. Existing guidance-based methods are simple but introduce bias, while particle-based corrections suffer from weight degeneracy and high computational cost. We introduce DriftLite, a lightweight, training-free particle-based approach that steers the inference dynamics on the fly with provably optimal stability control. DriftLite exploits a previously unexplored degree of freedom in the Fokker-Planck equation between the drift and particle potential, and yields two practical instantiations: Variance- and Energy-Controlling Guidance (VCG/ECG) for approximating the optimal drift with minimal overhead. Across Gaussian mixture models, particle systems, and large-scale protein-ligand co-folding problems, DriftLite consistently reduces variance and improves sample quality over pure guidance and sequential Monte Carlo baselines. These results highlight a principled, efficient route toward scalable inference-time adaptation of diffusion models.

Method: Proposes a differentiable structure learning framework formulated as a single differentiable optimization task that captures arbitrary dependencies among discrete variables, avoiding unrealistic simplifications of previous methods.

Result: The approach can characterize the complete set of compatible parameters and structures, establishes identifiability up to Markov equivalence under mild assumptions, and empirically demonstrates effective capture of complex relationships in discrete data.

Conclusion: The proposed framework provides a more general and effective approach for differentiable structure learning in discrete data by handling arbitrary dependencies and avoiding restrictive assumptions of previous methods.

Abstract: Existing methods for differentiable structure learning in discrete data typically assume that the data are generated from specific structural equation models. However, these assumptions may not align with the true data-generating process, which limits the general applicability of such methods. Furthermore, current approaches often ignore the complex dependence structure inherent in discrete data and consider only linear effects. We propose a differentiable structure learning framework that is capable of capturing arbitrary dependencies among discrete variables. We show that although general discrete models are unidentifiable from purely observational data, it is possible to characterize the complete set of compatible parameters and structures. Additionally, we establish identifiability up to Markov equivalence under mild assumptions. We formulate the learning problem as a single differentiable optimization task in the most general form, thereby avoiding the unrealistic simplifications adopted by previous methods. Empirical results demonstrate that our approach effectively captures complex relationships in discrete data.

Method: Integrates physics-driven inversion with data-driven learning using pretrained diffusion models as regularization mechanism.

Result: Enhanced accuracy and robustness in full waveform inversion compared to conventional methods, with strong generalization to unseen complex velocity models.

Conclusion: RED-DiffEq provides an effective framework for PDE-governed inverse problems that can be directly applied to diverse applications beyond seismic imaging.

Abstract: Partial differential equation (PDE)-governed inverse problems are fundamental across various scientific and engineering applications; yet they face significant challenges due to nonlinearity, ill-posedness, and sensitivity to noise. Here, we introduce a new computational framework, RED-DiffEq, by integrating physics-driven inversion and data-driven learning. RED-DiffEq leverages pretrained diffusion models as a regularization mechanism for PDE-governed inverse problems. We apply RED-DiffEq to solve the full waveform inversion problem in geophysics, a challenging seismic imaging technique that seeks to reconstruct high-resolution subsurface velocity models from seismic measurement data. Our method shows enhanced accuracy and robustness compared to conventional methods. Additionally, it exhibits strong generalization ability to more complex velocity models that the diffusion model is not trained on. Our framework can also be directly applied to diverse PDE-governed inverse problems.

Method: Conducted a systematic review through a filtering process to select and analyze eleven key papers on conformal prediction methods for treatment effect estimation.

Result: Identified and described current state-of-the-art conformal prediction methods for treatment effect estimation, providing necessary theoretical background.

Conclusion: Conformal prediction holds significant potential for improving decision-making in high-stakes environments, and the review proposes directions for future research in this area.

Abstract: Treatment effect estimation is essential for informed decision-making in many fields such as healthcare, economics, and public policy. While flexible machine learning models have been widely applied for estimating heterogeneous treatment effects, quantifying the inherent uncertainty of their point predictions remains an issue. Recent advancements in conformal prediction address this limitation by allowing for inexpensive computation, as well as distribution shifts, while still providing frequentist, finite-sample coverage guarantees under minimal assumptions for any point-predictor model. This advancement holds significant potential for improving decision-making in especially high-stakes environments. In this work, we perform a systematic review regarding conformal prediction methods for treatment effect estimation and provide for both the necessary theoretical background. Through a systematic filtering process, we select and analyze eleven key papers, identifying and describing current state-of-the-art methods in this area. Based on our findings, we propose directions for future research.

Method: Proposes MMPlanner with Object State Reasoning Chain-of-Thought (OSR-CoT) prompting to explicitly model object-state transitions, and introduces LLM-as-a-judge protocols for evaluation plus visual step-reordering task for temporal coherence measurement.

Result: Achieves state-of-the-art performance on RECIPEPLAN and WIKIPLAN datasets, improving textual planning by +6.8%, cross-modal alignment by +11.9%, and visual step ordering by +26.7%.

Conclusion: MMPlanner effectively addresses multimodal procedural planning challenges through explicit object-state reasoning and comprehensive evaluation protocols, demonstrating significant improvements across all measured metrics.

Abstract: Multimodal Procedural Planning (MPP) aims to generate step-by-step instructions that combine text and images, with the central challenge of preserving object-state consistency across modalities while producing informative plans. Existing approaches often leverage large language models (LLMs) to refine textual steps; however, visual object-state alignment and systematic evaluation are largely underexplored. We present MMPlanner, a zero-shot MPP framework that introduces Object State Reasoning Chain-of-Thought (OSR-CoT) prompting to explicitly model object-state transitions and generate accurate multimodal plans. To assess plan quality, we design LLM-as-a-judge protocols for planning accuracy and cross-modal alignment, and further propose a visual step-reordering task to measure temporal coherence. Experiments on RECIPEPLAN and WIKIPLAN show that MMPlanner achieves state-of-the-art performance, improving textual planning by +6.8%, cross-modal alignment by +11.9%, and visual step ordering by +26.7%

Method: Extends propositional logic syntax with a choice operator that has learnable parameters and selects subformulas from a pool of options. Uses fuzzy logic to compile formulas into differentiable computational graphs, enabling learning via backpropagation. Employs Goedel fuzzy logic and the Goedel trick for discretization without performance loss.

Result: Strong experimental results on tabular data and the Visual Tic-Tac-Toe neurosymbolic task, while producing interpretable decision rules. The framework subsumes existing neurosymbolic models and enables flexible knowledge integration.

Conclusion: LoH provides a unified neurosymbolic framework that successfully integrates data-driven rule learning with symbolic priors, offering interpretable models that can be discretized without performance degradation while maintaining the flexibility to incorporate varying degrees of expert knowledge.

Abstract: Neurosymbolic integration (NeSy) blends neural-network learning with symbolic reasoning. The field can be split between methods injecting hand-crafted rules into neural models, and methods inducing symbolic rules from data. We introduce Logic of Hypotheses (LoH), a novel language that unifies these strands, enabling the flexible integration of data-driven rule learning with symbolic priors and expert knowledge. LoH extends propositional logic syntax with a choice operator, which has learnable parameters and selects a subformula from a pool of options. Using fuzzy logic, formulas in LoH can be directly compiled into a differentiable computational graph, so the optimal choices can be learned via backpropagation. This framework subsumes some existing NeSy models, while adding the possibility of arbitrary degrees of knowledge specification. Moreover, the use of Goedel fuzzy logic and the recently developed Goedel trick yields models that can be discretized to hard Boolean-valued functions without any loss in performance. We provide experimental analysis on such models, showing strong results on tabular data and on the Visual Tic-Tac-Toe NeSy task, while producing interpretable decision rules.

Method: Enforces monotonicity by penalizing violations relative to a linear baseline, using a mathematical framework that establishes linear reference, measures deviations from monotonic behavior, and integrates these into training objective.

Result: Experiments on real-world ridesourcing data and synthetic datasets show that even modest monotonicity constraints consistently enhance model performance across various neural network architectures.

Conclusion: DIM effectively improves predictive performance of deep neural networks by applying domain-informed monotonicity constraints to regularize model behavior and mitigate overfitting.

Abstract: While deep learning models excel at predictive tasks, they often overfit due to their complex structure and large number of parameters, causing them to memorize training data, including noise, rather than learn patterns that generalize to new data. To tackle this challenge, this paper proposes a new regularization method, i.e., Enforcing Domain-Informed Monotonicity in Deep Neural Networks (DIM), which maintains domain-informed monotonic relationships in complex deep learning models to further improve predictions. Specifically, our method enforces monotonicity by penalizing violations relative to a linear baseline, effectively encouraging the model to follow expected trends while preserving its predictive power. We formalize this approach through a comprehensive mathematical framework that establishes a linear reference, measures deviations from monotonic behavior, and integrates these measurements into the training objective. We test and validate the proposed methodology using a real-world ridesourcing dataset from Chicago and a synthetically created dataset. Experiments across various neural network architectures show that even modest monotonicity constraints consistently enhance model performance. DIM enhances the predictive performance of deep neural networks by applying domain-informed monotonicity constraints to regularize model behavior and mitigate overfitting

Method: Built on the BrainTreebank dataset with 40 hours of iEEG recordings from 10 human subjects performing naturalistic movie viewing tasks. Provides decoding tasks for studying multi-modal language processing with high temporal and spatial resolution.

Result: Visualized information flow from superior temporal gyrus to prefrontal cortex, showing progression from simple auditory to complex language features. Found linear baseline surprisingly strong, beating frontier foundation models on many tasks.

Conclusion: Neuroprobe serves as both a neuroscience insight tool and rigorous evaluation framework for iEEG foundation models, enabling systematic comparison of architectures and training protocols with publicly available code and leaderboard.

Abstract: High-resolution neural datasets enable foundation models for the next generation of brain-computer interfaces and neurological treatments. The community requires rigorous benchmarks to discriminate between competing modeling approaches, yet no standardized evaluation frameworks exist for intracranial EEG (iEEG) recordings. To address this gap, we present Neuroprobe: a suite of decoding tasks for studying multi-modal language processing in the brain. Unlike scalp EEG, intracranial EEG requires invasive surgery to implant electrodes that record neural activity directly from the brain with minimal signal distortion. Neuroprobe is built on the BrainTreebank dataset, which consists of 40 hours of iEEG recordings from 10 human subjects performing a naturalistic movie viewing task. Neuroprobe serves two critical functions. First, it is a mine from which neuroscience insights can be drawn. Its high temporal and spatial resolution allows researchers to systematically determine when and where computations for each aspect of language processing occur in the brain by measuring the decodability of each feature across time and all electrode locations. Using Neuroprobe, we visualize how information flows from the superior temporal gyrus to the prefrontal cortex, and the progression from simple auditory features to more complex language features in a purely data-driven manner. Second, as the field moves toward neural foundation models, Neuroprobe provides a rigorous framework for comparing competing architectures and training protocols. We found that the linear baseline is surprisingly strong, beating frontier foundation models on many tasks. Neuroprobe is designed with computational efficiency and ease of use in mind. We make the code for Neuroprobe openly available and maintain a public leaderboard, aiming to enable rapid progress in the field of iEEG foundation models, at https://neuroprobe.dev/

Method: SlotFM uses Time-Frequency Slot Attention that processes both time and frequency representations of raw signals, generating multiple small embeddings (slots) that capture different signal components. It also introduces two loss regularizers for local structure and frequency patterns to improve reconstruction of fine-grained details.

Result: SlotFM outperforms existing self-supervised approaches on 13 out of 16 classification and regression downstream tasks, achieving comparable results on the remaining tasks. It yields a 4.5% average performance gain.

Conclusion: SlotFM demonstrates strong generalization for sensing foundation models, enabling broader applicability across diverse accelerometer-based tasks beyond standard human activity recognition.

Abstract: Wearable accelerometers are used for a wide range of applications, such as gesture recognition, gait analysis, and sports monitoring. Yet most existing foundation models focus primarily on classifying common daily activities such as locomotion and exercise, limiting their applicability to the broader range of tasks that rely on other signal characteristics. We present SlotFM, an accelerometer foundation model that generalizes across diverse downstream tasks. SlotFM uses Time-Frequency Slot Attention, an extension of Slot Attention that processes both time and frequency representations of the raw signals. It generates multiple small embeddings (slots), each capturing different signal components, enabling task-specific heads to focus on the most relevant parts of the data. We also introduce two loss regularizers that capture local structure and frequency patterns, which improve reconstruction of fine-grained details and helps the embeddings preserve task-relevant information. We evaluate SlotFM on 16 classification and regression downstream tasks that extend beyond standard human activity recognition. It outperforms existing self-supervised approaches on 13 of these tasks and achieves comparable results to the best performing approaches on the remaining tasks. On average, our method yields a 4.5% performance gain, demonstrating strong generalization for sensing foundation models.

Method: Formulate K-means as smooth unconstrained optimization over a submanifold, characterize Riemannian structures, and solve using second-order cubic-regularized Riemannian Newton algorithm with linear-time Newton subproblem solutions through manifold factorization.

Result: The proposed method converges significantly faster than state-of-the-art first-order nonnegative low-rank factorization methods while achieving similar optimal statistical accuracy.

Conclusion: The Riemannian manifold approach provides an effective framework for solving K-means clustering with improved computational efficiency and rigorous guarantees.

Abstract: Clustering is a hard discrete optimization problem. Nonconvex approaches such as low-rank semidefinite programming (SDP) have recently demonstrated promising statistical and local algorithmic guarantees for cluster recovery. Due to the combinatorial structure of the $K$-means clustering problem, current relaxation algorithms struggle to balance their constraint feasibility and objective optimality, presenting tremendous challenges in computing the second-order critical points with rigorous guarantees. In this paper, we provide a new formulation of the $K$-means problem as a smooth unconstrained optimization over a submanifold and characterize its Riemannian structures to allow it to be solved using a second-order cubic-regularized Riemannian Newton algorithm. By factorizing the $K$-means manifold into a product manifold, we show how each Newton subproblem can be solved in linear time. Our numerical experiments show that the proposed method converges significantly faster than the state-of-the-art first-order nonnegative low-rank factorization method, while achieving similarly optimal statistical accuracy.

Method: Extracts rules based on neuron activations (values or on/off statuses) as preconditions that imply certain output properties, focusing on the activation patterns in inner layers of feed-forward networks.

Result: Demonstrated successful usage on different types of models and output properties, with applications including formal explanations, compositional verification, run-time monitoring, repair, and novel results for large vision-language models.

Conclusion: Prophecy provides an effective approach for automatically inferring formal network properties through neuron activation analysis, showing promising potential especially for large vision-language models.

Abstract: We present Prophecy, a tool for automatically inferring formal properties of feed-forward neural networks. Prophecy is based on the observation that a significant part of the logic of feed-forward networks is captured in the activation status of the neurons at inner layers. Prophecy works by extracting rules based on neuron activations (values or on/off statuses) as preconditions that imply certain desirable output property, e.g., the prediction being a certain class. These rules represent network properties captured in the hidden layers that imply the desired output behavior. We present the architecture of the tool, highlight its features and demonstrate its usage on different types of models and output properties. We present an overview of its applications, such as inferring and proving formal explanations of neural networks, compositional verification, run-time monitoring, repair, and others. We also show novel results highlighting its potential in the era of large vision-language models.

Method: Introduces SpecMER framework that incorporates biological, structural, and functional priors using k-mer motifs from multiple sequence alignments. Scores candidate sequences in parallel and selects those most consistent with known biological patterns.

Result: Achieves 24-32% speedup over standard autoregressive decoding, with higher acceptance rates and improved sequence likelihoods while maintaining efficiency.

Conclusion: SpecMER successfully combines the efficiency of speculative decoding with biological guidance, enabling faster and more biologically plausible protein sequence generation for high-throughput applications.

Abstract: Autoregressive models have transformed protein engineering by enabling the generation of novel protein sequences beyond those found in nature. However, their sequential inference introduces significant latency, limiting their utility in high-throughput protein screening. Speculative decoding accelerates generation by employing a lightweight draft model to sample tokens, which a larger target model then verifies and refines. Yet, in protein sequence generation, draft models are typically agnostic to the structural and functional constraints of the target protein, leading to biologically implausible outputs and a shift in the likelihood distribution of generated sequences. We introduce SpecMER (Speculative Decoding via k-mer Guidance), a novel framework that incorporates biological, structural, and functional priors using k-mer motifs extracted from multiple sequence alignments. By scoring candidate sequences in parallel and selecting those most consistent with known biological patterns, SpecMER significantly improves sequence plausibility while retaining the efficiency of speculative decoding. SpecMER achieves 24-32% speedup over standard autoregressive decoding, along with higher acceptance rates and improved sequence likelihoods.

Method: 1) Time-to-event modeling through regression or discrete survival modeling; 2) Patient-identity invariant features via adversarial training using p-vector biometric identification; 3) Regression on pseudo-lab values from pre-trained auxiliary estimator networks; 4) Multi-task formulation with PCGrad optimization to handle gradient conflicts.

Result: The proposed methods independently improve 24-hour time-averaged AUC from 0.74 to 0.78-0.80 range, with primary improvements over longer time horizons and minimal degradation near the event.

Conclusion: The three orthogonal approaches significantly enhance cardiac arrest prediction performance, particularly for early warning systems, by addressing data scarcity through time-to-event modeling, identity deconfounding, and pseudo-lab enrichment.

Abstract: High-frequency physiological waveform modality offers deep, real-time insights into patient status. Recently, physiological foundation models based on Photoplethysmography (PPG), such as PPG-GPT, have been shown to predict critical events, including Cardiac Arrest (CA). However, their powerful representation still needs to be leveraged suitably, especially when the downstream data/label is scarce. We offer three orthogonal improvements to improve PPG-only CA systems by using minimal auxiliary information. First, we propose to use time-to-event modeling, either through simple regression to the event onset time or by pursuing fine-grained discrete survival modeling. Second, we encourage the model to learn CA-focused features by making them patient-identity invariant. This is achieved by first training the largest-scale de-identified biometric identification model, referred to as the p-vector, and subsequently using it adversarially to deconfound cues, such as person identity, that may cause overfitting through memorization. Third, we propose regression on the pseudo-lab values generated by pre-trained auxiliary estimator networks. This is crucial since true blood lab measurements, such as lactate, sodium, troponin, and potassium, are collected sparingly. Via zero-shot prediction, the auxiliary networks can enrich cardiac arrest waveform labels and generate pseudo-continuous estimates as targets. Our proposals can independently improve the 24-hour time-averaged AUC from the 0.74 to the 0.78-0.80 range. We primarily improve over longer time horizons with minimal degradation near the event, thus pushing the Early Warning System research. Finally, we pursue multi-task formulation and diagnose it with a high gradient conflict rate among competing losses, which we alleviate via the PCGrad optimization technique.

Method: Exact subgraph enumeration combined with neural networks and sparse regularization. Uses pruning strategy to handle computational complexity while maintaining performance.

Result: EIN achieves sufficiently high prediction performance compared to standard graph neural networks. Enables post-hoc analysis through identified important subgraphs.

Conclusion: EIN successfully combines subgraph enumeration with neural networks to achieve both high performance and interpretability in graph-level prediction tasks.

Abstract: In the graph-level prediction task (predict a label for a given graph), the information contained in subgraphs of the input graph plays a key role. In this paper, we propose Exact subgraph Isomorphism Network (EIN), which combines the exact subgraph enumeration, neural network, and a sparse regularization. In general, building a graph-level prediction model achieving high discriminative ability along with interpretability is still a challenging problem. Our combination of the subgraph enumeration and neural network contributes to high discriminative ability about the subgraph structure of the input graph. Further, the sparse regularization in EIN enables us 1) to derive an effective pruning strategy that mitigates computational difficulty of the enumeration while maintaining the prediction performance, and 2) to identify important subgraphs that contributes to high interpretability. We empirically show that EIN has sufficiently high prediction performance compared with standard graph neural network models, and also, we show examples of post-hoc analysis based on the selected subgraphs.

Method: Used Linear Regression, Random Forest, Support Vector Machine, and Neural Networks to correlate OD trips with demographic, socioeconomic, and commuting characteristics, with perturbation-based sensitivity analysis.

Result: Linear regression failed to capture complex interactions, NN performed best with training/testing data, while SVM showed best generalization in downscaling.

Conclusion: The methodology provides both analytical advancement and practical applications for improving transportation services and urban development.

Abstract: Understanding urban human mobility patterns at various spatial levels is essential for social science. This study presents a machine learning framework to downscale origin-destination (OD) taxi trips flows in New York City from a larger spatial unit to a smaller spatial unit. First, correlations between OD trips and demographic, socioeconomic, and commuting characteristics are developed using four models: Linear Regression (LR), Random Forest (RF), Support Vector Machine (SVM), and Neural Networks (NN). Second, a perturbation-based sensitivity analysis is applied to interpret variable importance for nonlinear models. The results show that the linear regression model failed to capture the complex variable interactions. While NN performs best with the training and testing datasets, SVM shows the best generalization ability in downscaling performance. The methodology presented in this study provides both analytical advancement and practical applications to improve transportation services and urban development.

Method: Extracts representative features from client raw data, applies clustering to estimate inter-client dataset similarity, implements client selection strategy for compatible data distributions, and integrates with existing FL algorithms.

Result: Consistently improves target client’s model performance on CIFAR-10 and MNIST datasets, even with limited participants. Outperforms baseline cluster-based algorithm IFCA in low-participation scenarios.

Conclusion: PQFed demonstrates scalability and effectiveness in personalized federated learning settings through early-stage quality control and compatible client collaboration.

Abstract: Federated learning enables collaborative model training without sharing raw data, but data heterogeneity consistently challenges the performance of the global model. Traditional optimization methods often rely on collaborative global model training involving all clients, followed by local adaptation to improve individual performance. In this work, we focus on early-stage quality control and propose PQFed, a novel privacy-preserving personalized federated learning framework that designs customized training strategies for each client prior to the federated training process. PQFed extracts representative features from each client’s raw data and applies clustering techniques to estimate inter-client dataset similarity. Based on these similarity estimates, the framework implements a client selection strategy that enables each client to collaborate with others who have compatible data distributions. We evaluate PQFed on two benchmark datasets, CIFAR-10 and MNIST, integrated with three existing federated learning algorithms. Experimental results show that PQFed consistently improves the target client’s model performance, even with a limited number of participants. We further benchmark PQFed against a baseline cluster-based algorithm, IFCA, and observe that PQFed also achieves better performance in low-participation scenarios. These findings highlight PQFed’s scalability and effectiveness in personalized federated learning settings.

Method: Develops a framework based on linear dynamical systems that unifies different fixed-point iteration schemes (Newton, Picard, Jacobi) as approximate linearizations of nonlinear recursions.

Result: The framework reveals shared principles behind parallel evaluation techniques and clarifies when specific fixed-point methods are most effective.

Conclusion: The LDS-based framework provides a clearer theoretical foundation for parallelizing sequential models and suggests new opportunities for efficient and scalable computation.

Abstract: Harnessing parallelism in seemingly sequential models is a central challenge for modern machine learning. Several approaches have been proposed for evaluating sequential processes in parallel using fixed-point methods, like Newton, Picard, and Jacobi iterations. In this work, we show that these methods can be understood within a common framework based on linear dynamical systems (LDSs), where different iteration schemes arise naturally as approximate linearizations of a nonlinear recursion. This unifying view highlights shared principles behind these techniques and clarifies when particular fixed-point methods are most likely to be effective. By bridging diverse algorithms through the language of LDSs, our framework provides a clearer theoretical foundation for parallelizing sequential models and points toward new opportunities for efficient and scalable computation.

Method: An information-theoretic approach that considers information gain of both upper- and lower-level optimal solutions and values, with a practical lower bound based evaluation method.

Result: The proposed method was empirically demonstrated to be effective through several benchmark datasets.

Conclusion: The information-theoretic approach provides a unified criterion that simultaneously measures benefits for both level problems in bilevel optimization.

Abstract: A bilevel optimization problem consists of two optimization problems nested as an upper- and a lower-level problem, in which the optimality of the lower-level problem defines a constraint for the upper-level problem. This paper considers Bayesian optimization (BO) for the case that both the upper- and lower-levels involve expensive black-box functions. Because of its nested structure, bilevel optimization has a complex problem definition and, compared with other standard extensions of BO such as multi-objective or constraint settings, it has not been widely studied. We propose an information-theoretic approach that considers the information gain of both the upper- and lower-optimal solutions and values. This enables us to define a unified criterion that measures the benefit for both level problems, simultaneously. Further, we also show a practical lower bound based approach to evaluating the information gain. We empirically demonstrate the effectiveness of our proposed method through several benchmark datasets.

Method: Developed an interpretable spatio-temporal graph neural network framework leveraging dual Stochastic Differential Equations (SDEs) to model irregularly-sampled longitudinal fMRI data, validated on OASIS-3 and ADNI cohorts.

Result: The framework effectively learned sparse regional and connective importance probabilities, identifying key brain abnormalities in parahippocampal cortex, prefrontal cortex, parietal lobule, and disruptions in ventral attention, dorsal attention, and default mode networks that strongly correlate with AD clinical symptoms.

Conclusion: The approach demonstrates potential for early, individualized prediction of AD progression using spatio-temporal graph-based learning, revealing both established and novel neural systems-level and sex-specific biomarkers for understanding AD neurobiological mechanisms.

Abstract: Identifying objective neuroimaging biomarkers to forecast Alzheimer’s disease (AD) progression is crucial for timely intervention. However, this task remains challenging due to the complex dysfunctions in the spatio-temporal characteristics of underlying brain networks, which are often overlooked by existing methods. To address these limitations, we develop an interpretable spatio-temporal graph neural network framework to predict future AD progression, leveraging dual Stochastic Differential Equations (SDEs) to model the irregularly-sampled longitudinal functional magnetic resonance imaging (fMRI) data. We validate our approach on two independent cohorts, including the Open Access Series of Imaging Studies (OASIS-3) and the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Our framework effectively learns sparse regional and connective importance probabilities, enabling the identification of key brain circuit abnormalities associated with disease progression. Notably, we detect the parahippocampal cortex, prefrontal cortex, and parietal lobule as salient regions, with significant disruptions in the ventral attention, dorsal attention, and default mode networks. These abnormalities correlate strongly with longitudinal AD-related clinical symptoms. Moreover, our interpretability strategy reveals both established and novel neural systems-level and sex-specific biomarkers, offering new insights into the neurobiological mechanisms underlying AD progression. Our findings highlight the potential of spatio-temporal graph-based learning for early, individualized prediction of AD progression, even in the context of irregularly-sampled longitudinal imaging data.

Method: POLO uses Preference-Guided Policy Optimization (PGPO) with dual-level learning: trajectory-level optimization reinforces successful strategies, and turn-level preference learning ranks intermediate molecules within trajectories.

Result: POLO achieves 84% average success rate on single-property tasks (2.3x better than baselines) and 50% on multi-property tasks using only 500 oracle evaluations.

Conclusion: POLO significantly advances sample-efficient molecular optimization by fully exploiting costly oracle calls through trajectory-based learning.

Abstract: Lead optimization in drug discovery requires efficiently navigating vast chemical space through iterative cycles to enhance molecular properties while preserving structural similarity to the original lead compound. Despite recent advances, traditional optimization methods struggle with sample efficiency-achieving good optimization performance with limited oracle evaluations. Large Language Models (LLMs) provide a promising approach through their in-context learning and instruction following capabilities, which align naturally with these iterative processes. However, existing LLM-based methods fail to leverage this strength, treating each optimization step independently. To address this, we present POLO (Preference-guided multi-turn Optimization for Lead Optimization), which enables LLMs to learn from complete optimization trajectories rather than isolated steps. At its core, POLO introduces Preference-Guided Policy Optimization (PGPO), a novel reinforcement learning algorithm that extracts learning signals at two complementary levels: trajectory-level optimization reinforces successful strategies, while turn-level preference learning provides dense comparative feedback by ranking intermediate molecules within each trajectory. Through this dual-level learning from intermediate evaluation, POLO achieves superior sample efficiency by fully exploiting each costly oracle call. Extensive experiments demonstrate that POLO achieves 84% average success rate on single-property tasks (2.3x better than baselines) and 50% on multi-property tasks using only 500 oracle evaluations, significantly advancing the state-of-the-art in sample-efficient molecular optimization.

Method: BrainPoG uses pathological pattern filtering to extract disease-relevant subgraphs (PathoGraph construction) and pathological feature distillation to reduce disease-irrelevant noise while enhancing pathological features.

Result: Extensive experiments on four benchmark datasets show BrainPoG exhibits superiority in both model performance and computational efficiency across various brain disease detection tasks.

Conclusion: BrainPoG enables efficient brain graph learning by exclusively learning informative disease-related knowledge while avoiding less relevant information, making it practical for clinical applications.

Abstract: Brain graph learning has demonstrated significant achievements in the fields of neuroscience and artificial intelligence. However, existing methods struggle to selectively learn disease-related knowledge, leading to heavy parameters and computational costs. This challenge diminishes their efficiency, as well as limits their practicality for real-world clinical applications. To this end, we propose a lightweight Brain PathoGraph Learning (BrainPoG) model that enables efficient brain graph learning by pathological pattern filtering and pathological feature distillation. Specifically, BrainPoG first contains a filter to extract the pathological pattern formulated by highly disease-relevant subgraphs, achieving graph pruning and lesion localization. A PathoGraph is therefore constructed by dropping less disease-relevant subgraphs from the whole brain graph. Afterwards, a pathological feature distillation module is designed to reduce disease-irrelevant noise features and enhance pathological features of each node in the PathoGraph. BrainPoG can exclusively learn informative disease-related knowledge while avoiding less relevant information, achieving efficient brain graph learning. Extensive experiments on four benchmark datasets demonstrate that BrainPoG exhibits superiority in both model performance and computational efficiency across various brain disease detection tasks.

Method: Uses lightweight hypersphere models per class to embed in-class samples close to hypersphere centers, segregating out-of-class samples by distance. Employs Youden’s J statistic to adaptively select pruning thresholds.

Result: Consistently outperforms state-of-the-art coreset selection methods, especially under noisy and low-data conditions. Effectively discards mislabeled and ambiguous points.

Conclusion: HyperCore produces compact, highly informative subsets suitable for scalable and noise-free learning in practical applications.

Abstract: The goal of coreset selection methods is to identify representative subsets of datasets for efficient model training. Yet, existing methods often ignore the possibility of annotation errors and require fixed pruning ratios, making them impractical in real-world settings. We present HyperCore, a robust and adaptive coreset selection framework designed explicitly for noisy environments. HyperCore leverages lightweight hypersphere models learned per class, embedding in-class samples close to a hypersphere center while naturally segregating out-of-class samples based on their distance. By using Youden’s J statistic, HyperCore can adaptively select pruning thresholds, enabling automatic, noise-aware data pruning without hyperparameter tuning. Our experiments reveal that HyperCore consistently surpasses state-of-the-art coreset selection methods, especially under noisy and low-data regimes. HyperCore effectively discards mislabeled and ambiguous points, yielding compact yet highly informative subsets suitable for scalable and noise-free learning.

Method: Integrates submodular coverage and density into a unified objective with a sampling strategy based on closed-form solutions to balance these objectives, controlled by a single hyperparameter.

Result: Matches training-based baselines, significantly outperforms them at high pruning rates, dramatically reduces computational overhead, and shows superior robustness to label noise.

Conclusion: SubZeroCore provides an effective and scalable training-free coreset selection approach suitable for real-world scenarios.

Abstract: The goal of coreset selection is to identify representative subsets of datasets for efficient model training. Yet, existing approaches paradoxically require expensive training-based signals, e.g., gradients, decision boundary estimates or forgetting counts, computed over the entire dataset prior to pruning, which undermines their very purpose by requiring training on samples they aim to avoid. We introduce SubZeroCore, a novel, training-free coreset selection method that integrates submodular coverage and density into a single, unified objective. To achieve this, we introduce a sampling strategy based on a closed-form solution to optimally balance these objectives, guided by a single hyperparameter that explicitly controls the desired coverage for local density measures. Despite no training, extensive evaluations show that SubZeroCore matches training-based baselines and significantly outperforms them at high pruning rates, while dramatically reducing computational overhead. SubZeroCore also demonstrates superior robustness to label noise, highlighting its practical effectiveness and scalability for real-world scenarios.

Method: Reparameterize the spatiotemporal state as a continuous neural network function, enabling parallel-in-time optimization. Incorporate physical constraints through physics-informed loss functions.

Result: Neural reparameterized variants produce more stable initial condition estimates without spurious oscillations compared to baseline 4DVAR. Method tested on 2D incompressible Navier-Stokes equations with Kolmogorov forcing.

Conclusion: The neural field approach successfully removes time-sequential dependencies, simplifies implementation, reduces computational cost, and works without requiring ground-truth states or reanalysis data.

Abstract: Four-dimensional variational data assimilation (4DVAR) is a cornerstone of numerical weather prediction, but its cost function is difficult to optimize and computationally intensive. We propose a neural field-based reformulation in which the full spatiotemporal state is represented as a continuous function parameterized by a neural network. This reparameterization removes the time-sequential dependency of classical 4DVAR, enabling parallel-in-time optimization in parameter space. Physical constraints are incorporated directly through a physics-informed loss, simplifying implementation and reducing computational cost. We evaluate the method on the two-dimensional incompressible Navier–Stokes equations with Kolmogorov forcing. Compared to a baseline 4DVAR implementation, the neural reparameterized variants produce more stable initial condition estimates without spurious oscillations. Notably, unlike most machine learning-based approaches, our framework does not require access to ground-truth states or reanalysis data, broadening its applicability to settings with limited reference information.

Method: Used fNIRS to measure brain hemodynamic responses in 15 MS patients and 12 controls during single and dual hand dexterity tasks, analyzed with machine learning (K-NN classifier) and XAI to identify important brain regions.

Result: K-NN achieved 75.0% accuracy for single tasks and 66.7% for dual tasks. Key regions were supramarginal/angular gyri and precentral gyrus in ipsilateral hemisphere, showing suppressed activity and slower neurovascular response in MS group. Deoxygenated hemoglobin was better predictor than oxygenated hemoglobin.

Conclusion: fNIRS with machine learning revealed novel brain activity biomarkers in MS patients, providing potential targets for personalized brain stimulation treatments to address cognitive fatigue during dexterous tasks.

Abstract: People with Multiple Sclerosis (MS) complain of problems with hand dexterity and cognitive fatigue. However, in many cases, impairments are subtle and difficult to detect. Functional near-infrared spectroscopy (fNIRS) is a non-invasive neuroimaging technique that measures brain hemodynamic responses during cognitive or motor tasks. We aimed to detect brain activity biomarkers that could explain subjective reports of cognitive fatigue while completing dexterous tasks and provide targets for future brain stimulation treatments. We recruited 15 people with MS who did not have a hand (Nine Hole Peg Test [NHPT]), mobility, or cognitive impairment, and 12 age- and sex-matched controls. Participants completed two types of hand dexterity tasks with their dominant hand, single task and dual task (NHPT while holding a ball between the fifth finger and hypothenar eminence of the same hand). We analyzed fNIRS data (oxygenated and deoxygenated hemoglobin levels) using a machine learning framework to classify MS patients from controls based on their brain activation patterns in bilateral prefrontal and sensorimotor cortices. The K-Nearest Neighbor classifier achieved an accuracy of 75.0% for single manual dexterity tasks and 66.7% for the more complex dual manual dexterity tasks. Using XAI, we found that the most important brain regions contributing to the machine learning model were the supramarginal/angular gyri and the precentral gyrus (sensory integration and motor regions) of the ipsilateral hemisphere, with suppressed activity and slower neurovascular response in the MS group. During both tasks, deoxygenated hemoglobin levels were better predictors than the conventional measure of oxygenated hemoglobin. This nonconventional method of fNIRS data analysis revealed novel brain activity biomarkers that can help develop personalized brain stimulation targets.

Method: Proposes Effective Information Criterion (EIC) that treats formulas as information processing systems and identifies unreasonable structures through loss of significant digits or amplification of rounding noise during data flow.

Result: EIC improves performance on Pareto frontier, reduces irrational structures, boosts sample efficiency by 2-4x with generative algorithms, and shows 70.2% agreement with human experts on interpretability preferences.

Conclusion: EIC effectively measures formula interpretability by identifying unreasonable mathematical structures, bridging the gap between symbolic regression results and real-world physical formulas.

Abstract: Symbolic regression discovers accurate and interpretable formulas to describe given data, thereby providing scientific insights for domain experts and promoting scientific discovery. However, existing symbolic regression methods often use complexity metrics as a proxy for interoperability, which only considers the size of the formula but ignores its internal mathematical structure. Therefore, while they can discover formulas with compact forms, the discovered formulas often have structures that are difficult to analyze or interpret mathematically. In this work, inspired by the observation that physical formulas are typically numerically stable under limited calculation precision, we propose the Effective Information Criterion (EIC). It treats formulas as information processing systems with specific internal structures and identifies the unreasonable structure in them by the loss of significant digits or the amplification of rounding noise as data flows through the system. We find that this criterion reveals the gap between the structural rationality of models discovered by existing symbolic regression algorithms and real-world physical formulas. Combining EIC with various search-based symbolic regression algorithms improves their performance on the Pareto frontier and reduces the irrational structure in the results. Combining EIC with generative-based algorithms reduces the number of samples required for pre-training, improving sample efficiency by 2~4 times. Finally, for different formulas with similar accuracy and complexity, EIC shows a 70.2% agreement with 108 human experts’ preferences for formula interpretability, demonstrating that EIC, by measuring the unreasonable structures in formulas, actually reflects the formula’s interpretability.

Method: Proposes concurrency-aware speculative decoding with dynamic strategy adjustment and online draft learning to adapt the draft model to the evolving target model during training.

Result: Achieves end-to-end speedups of 2.35x to 2.72x across multiple mathematical reasoning datasets and models, significantly surpassing baseline approaches.

Conclusion: The proposed method effectively addresses GRPO’s training bottleneck through adaptive speculative decoding and continuous draft model adaptation, enabling practical deployment.

Abstract: Group relative policy optimization (GRPO) has demonstrated significant potential in improving the reasoning capabilities of large language models (LLMs) via reinforcement learning. However, its practical deployment is impeded by an excessively slow training process, primarily attributed to the computationally intensive autoregressive generation of multiple responses per query, which makes the generation phase the primary performance bottleneck. Although speculative decoding presents a promising direction for acceleration, its direct application in GRPO achieves limited speedup under high-concurrency training conditions. To overcome this limitation, we propose a concurrency-aware speculative decoding framework that dynamically adjusts the drafting and verification strategy according to real-time concurrency levels, thereby maximizing the acceleration of the generation process. Furthermore, to address performance degradation arising from distributional drift between the evolving target model and the fixed draft model during training, we introduce an online draft learning mechanism that enables the draft model to continuously adapt using feedback signals from the target model. Experimental results across multiple mathematical reasoning datasets and models demonstrate that the proposed method achieves end-to-end speedups of 2.35x to 2.72x, significantly surpassing baseline approaches in efficiency. The code is available at https://github.com/yedaotian9/GRPO_speculative.

Method: Used DROZY dataset to extract features from ECG, EMG, EOG, and EEG across 15 signal combinations, evaluated with Decision Tree, Random Forest, Logistic Regression, and XGBoost classifiers.

Result: XGBoost with EMG-EEG combination achieved best performance. Multi-signal models consistently outperformed single signal ones. SHAP analysis identified ECG-EOG correlation as key feature.

Conclusion: Feature-level fusion of physiological signals enhances accuracy, interpretability, and practical applicability of fatigue monitoring systems.

Abstract: Fatigue detection using physiological signals is critical in domains such as transportation, healthcare, and performance monitoring. While most studies focus on single modalities, this work examines statistical relationships between signal pairs to improve classification robustness. Using the DROZY dataset, we extracted features from ECG, EMG, EOG, and EEG across 15 signal combinations and evaluated them with Decision Tree, Random Forest, Logistic Regression, and XGBoost. Results show that XGBoost with the EMG EEG combination achieved the best performance. SHAP analysis highlighted ECG EOG correlation as a key feature, and multi signal models consistently outperformed single signal ones. These findings demonstrate that feature level fusion of physiological signals enhances accuracy, interpretability, and practical applicability of fatigue monitoring systems.

Method: Proposes ChaosNexus foundation model with multi-scale ScaleFormer architecture augmented with Mixture-of-Experts layers, pre-trained on diverse chaotic dynamics corpus to capture universal patterns and system-specific behaviors.

Result: 40% improvement in long-term attractor statistics on 9,000+ synthetic systems; competitive 5-day global weather forecasting with <1 degree mean error zero-shot; performance improves with few-shot fine-tuning; shows generalization stems from training diversity rather than data volume.

Conclusion: ChaosNexus demonstrates effective cross-system generalization for chaotic forecasting, establishing that diversity of training systems is key for scientific foundation models rather than sheer data quantity.

Abstract: Accurately forecasting chaotic systems, prevalent in domains such as weather prediction and fluid dynamics, remains a significant scientific challenge. The inherent sensitivity of these systems to initial conditions, coupled with a scarcity of observational data, severely constrains traditional modeling approaches. Since these models are typically trained for a specific system, they lack the generalization capacity necessary for real-world applications, which demand robust zero-shot or few-shot forecasting on novel or data-limited scenarios. To overcome this generalization barrier, we propose ChaosNexus, a foundation model pre-trained on a diverse corpus of chaotic dynamics. ChaosNexus employs a novel multi-scale architecture named ScaleFormer augmented with Mixture-of-Experts layers, to capture both universal patterns and system-specific behaviors. The model demonstrates state-of-the-art zero-shot generalization across both synthetic and real-world benchmarks. On a large-scale testbed comprising over 9,000 synthetic chaotic systems, it improves the fidelity of long-term attractor statistics by more than 40% compared to the leading baseline. This robust performance extends to real-world applications with exceptional data efficiency. For instance, in 5-day global weather forecasting, ChaosNexus achieves a competitive zero-shot mean error below 1 degree, a result that further improves with few-shot fine-tuning. Moreover, experiments on the scaling behavior of ChaosNexus provide a guiding principle for scientific foundation models: cross-system generalization stems from the diversity of training systems, rather than sheer data volume.

Method: Trained transformer and EquiformerV2 neural networks to predict material properties and analyzed scaling effects of training data, model size, and compute using power law relationships L = α·N^(-β).

Result: Found empirical scaling laws showing predictable relationships between hyperparameters (training data, model size, compute) and predictive performance, with loss following power law relationships.

Conclusion: Future work could investigate scaling laws for other neural network models like GemNet and fully connected networks to compare with the trained transformer and EquiformerV2 models.

Abstract: Predicting material properties is crucial for designing better batteries, semiconductors, and medical devices. Deep learning helps scientists quickly find promising materials by predicting their energy, forces, and stresses. Companies scale capacities of deep learning models in multiple domains, such as language modeling, and invest many millions of dollars into such models. Our team analyzes how scaling training data (giving models more information to learn from), model sizes (giving models more capacity to learn patterns), and compute (giving models more computational resources) for neural networks affects their performance for material property prediction. In particular, we trained both transformer and EquiformerV2 neural networks to predict material properties. We find empirical scaling laws for these models: we can predict how increasing each of the three hyperparameters (training data, model size, and compute) affects predictive performance. In particular, the loss $L$ can be measured with a power law relationship $L = \alpha \cdot N^{-\beta}$, where $\alpha$ and $\beta$ are constants while $N$ is the relevant hyperparameter. We also incorporate command-line arguments for changing training settings such as the amount of epochs, maximum learning rate, and whether mixed precision is enabled. Future work could entail further investigating scaling laws for other neural network models in this domain, such as GemNet and fully connected networks, to assess how they compare to the models we trained.

Method: Theoretical analysis proving existence of hallucinated minimizers, establishing local convergence conditions, and empirical validation of convergence to these points in practice.

Result: Demonstrated that SAM can indeed converge to hallucinated minimizers, which are points that appear to be minimizers under SAM but are not actual minimizers of the original training objective.

Conclusion: A simple yet effective remedy is proposed to avoid convergence to hallucinated minimizers in SAM training.

Abstract: Sharpness-Aware Minimization (SAM) is a widely used method that steers training toward flatter minimizers, which typically generalize better. In this work, however, we show that SAM can converge to hallucinated minimizers – points that are not minimizers of the original objective. We theoretically prove the existence of such hallucinated minimizers and establish conditions for local convergence to them. We further provide empirical evidence demonstrating that SAM can indeed converge to these points in practice. Finally, we propose a simple yet effective remedy for avoiding hallucinated minimizers.

Method: The paper analyzes Euler samplers for masked discrete diffusion and proposes MATU (Mask-Aware Truncated Uniformization), which exploits the property that each token can be unmasked at most once and removes bounded-score assumptions while preserving unbiased discrete score approximation.

Result: Euler samplers achieve ε-accuracy in TV with Õ(d²ε⁻³/²) score evaluations. MATU achieves nearly ε-free complexity of O(d ln d · (1-ε²)), eliminating the ln(1/ε) factor and substantially speeding up convergence compared to uniform diffusion methods.

Conclusion: The findings provide rigorous theoretical foundation for masked discrete diffusion, showcase its practical advantages over uniform diffusion for text generation, and pave the way for analyzing diffusion-based language models developed under the masking paradigm.

Abstract: We study masked discrete diffusion – a flexible paradigm for text generation in which tokens are progressively corrupted by special mask symbols before being denoised. Although this approach has demonstrated strong empirical performance, its theoretical complexity in high-dimensional settings remains insufficiently understood. Existing analyses largely focus on uniform discrete diffusion, and more recent attempts addressing masked diffusion either (1) overlook widely used Euler samplers, (2) impose restrictive bounded-score assumptions, or (3) fail to showcase the advantages of masked discrete diffusion over its uniform counterpart. To address this gap, we show that Euler samplers can achieve $\epsilon$-accuracy in total variation (TV) with $\tilde{O}(d^{2}\epsilon^{-3/2})$ discrete score evaluations, thereby providing the first rigorous analysis of typical Euler sampler in masked discrete diffusion. We then propose a Mask-Aware Truncated Uniformization (MATU) approach that both removes bounded-score assumptions and preserves unbiased discrete score approximation. By exploiting the property that each token can be unmasked at most once, MATU attains a nearly $\epsilon$-free complexity of $O(d,\ln d\cdot (1-\epsilon^2))$. This result surpasses existing uniformization methods under uniform discrete diffusion, eliminating the $\ln(1/\epsilon)$ factor and substantially speeding up convergence. Our findings not only provide a rigorous theoretical foundation for masked discrete diffusion, showcasing its practical advantages over uniform diffusion for text generation, but also pave the way for future efforts to analyze diffusion-based language models developed under masking paradigm.

Method: The approach uses generic chaining and introduces new techniques for handling suprema over pairs of sets. It analyzes bilinear forms involving sums of independent sketching matrices.

Result: The framework provides uniform bounds in terms of geometric complexities, with deviation scaling as √T for T independent matrices. It recovers known results like Johnson-Lindenstrauss lemma and extends RIP-type guarantees.

Conclusion: The unified analysis improves convergence bounds for sketched federated learning and enables sharper regret bounds for sketched bandit algorithms that depend on geometric complexity rather than ambient dimension.

Abstract: Uniform bounds on sketched inner products of vectors or matrices underpin several important computational and statistical results in machine learning and randomized algorithms, including the Johnson-Lindenstrauss (J-L) lemma, the Restricted Isometry Property (RIP), randomized sketching, and approximate linear algebra. However, many modern analyses involve sketched bilinear forms, for which existing uniform bounds either do not apply or are not sharp on general sets. In this work, we develop a general framework to analyze such sketched bilinear forms and derive uniform bounds in terms of geometric complexities of the associated sets. Our approach relies on generic chaining and introduces new techniques for handling suprema over pairs of sets. We further extend these results to the setting where the bilinear form involves a sum of $T$ independent sketching matrices and show that the deviation scales as $\sqrt{T}$. This unified analysis recovers known results such as the J-L lemma as special cases, while extending RIP-type guarantees. Additionally, we obtain improved convergence bounds for sketched Federated Learning algorithms where such cross terms arise naturally due to sketched gradient compression, and design sketched variants of bandit algorithms with sharper regret bounds that depend on the geometric complexity of the action and parameter sets, rather than the ambient dimension.

Method: Formalizes long context modeling as a compression problem with an information-theoretic objective, then proposes Graph of Agents (GoA) that dynamically constructs input-dependent collaboration structures to maximize this objective.

Result: GoA improves average F1 score by 5.7% over retrieval-augmented generation and by 16.35% over fixed-structure multi-agent baselines. With only 2K context window, it surpasses 128K context window Llama 3.1 8B on LongBench.

Conclusion: GoA provides a principled framework for model-agnostic long context modeling that dramatically increases effective context length without requiring retraining or architectural changes.

Abstract: As a model-agnostic approach to long context modeling, multi-agent systems can process inputs longer than a large language model’s context window without retraining or architectural modifications. However, their performance often heavily relies on hand-crafted multi-agent collaboration strategies and prompt engineering, which limit generalizability. In this work, we introduce a principled framework that formalizes the model-agnostic long context modeling problem as a compression problem, yielding an information-theoretic compression objective. Building on this framework, we propose Graph of Agents (GoA), which dynamically constructs an input-dependent collaboration structure that maximizes this objective. For Llama 3.1 8B and Qwen3 8B across six document question answering benchmarks, GoA improves the average $F_1$ score of retrieval-augmented generation by 5.7% and a strong multi-agent baseline using a fixed collaboration structure by 16.35%, respectively. Even with only a 2K context window, GoA surpasses the 128K context window Llama 3.1 8B on LongBench, showing a dramatic increase in effective context length. Our source code is available at https://github.com/tjoo512/graph-of-agents.

Method: Propose MolSpectLLM, a molecular foundation model pretrained on Qwen2.5-7B that unifies experimental spectroscopy with molecular 3D structure by explicitly modeling molecular spectra.

Result: Achieves state-of-the-art performance on spectrum-related tasks (average accuracy of 0.53 across NMR, IR, and MS benchmarks) and strong performance on spectra analysis (15.5% sequence accuracy and 41.7% token accuracy on Spectra-to-SMILES). Generates accurate 3D molecular structures directly from SMILES or spectral inputs.

Conclusion: MolSpectLLM bridges spectral analysis, molecular elucidation, and molecular design by integrating experimental spectroscopy with 3D structural information, overcoming limitations of existing SMILES-only approaches.

Abstract: Recent advances in molecular foundation models have shown impressive performance in molecular property prediction and de novo molecular design, with promising applications in areas such as drug discovery and reaction prediction. Nevertheless, most existing approaches rely exclusively on SMILES representations and overlook both experimental spectra and 3D structural information-two indispensable sources for capturing molecular behavior in real-world scenarios. This limitation reduces their effectiveness in tasks where stereochemistry, spatial conformation, and experimental validation are critical. To overcome these challenges, we propose MolSpectLLM, a molecular foundation model pretrained on Qwen2.5-7B that unifies experimental spectroscopy with molecular 3D structure. By explicitly modeling molecular spectra, MolSpectLLM achieves state-of-the-art performance on spectrum-related tasks, with an average accuracy of 0.53 across NMR, IR, and MS benchmarks. MolSpectLLM also shows strong performance on the spectra analysis task, obtaining 15.5% sequence accuracy and 41.7% token accuracy on Spectra-to-SMILES, substantially outperforming large general-purpose LLMs. More importantly, MolSpectLLM not only achieves strong performance on molecular elucidation tasks, but also generates accurate 3D molecular structures directly from SMILES or spectral inputs, bridging spectral analysis, molecular elucidation, and molecular design.

Method: Proposes LDAR (Learning Distraction-Aware Retrieval), an adaptive retriever that learns to retrieve contexts while mitigating interference from distracting passages.

Result: Extensive experiments across diverse LLM architectures and six benchmarks show LDAR achieves significantly higher performance with reduced token usage compared to long-context approaches.

Conclusion: LDAR effectively balances the trade-off between information coverage and distraction, demonstrating the importance of adaptive retrieval over simply using long contexts.

Abstract: Retrieval-Augmented Generation (RAG) is a framework for grounding Large Language Models (LLMs) in external, up-to-date information. However, recent advancements in context window size allow LLMs to process inputs of up to 128K tokens or more, offering an alternative strategy: supplying the full document context directly to the model, rather than relying on RAG to retrieve a subset of contexts. Nevertheless, this emerging alternative strategy has notable limitations: (i) it is token-inefficient to handle large and potentially redundant contexts; (ii) it exacerbates the `lost in the middle’ phenomenon; and (iii) under limited model capacity, it amplifies distraction, ultimately degrading LLM output quality. In this paper, we propose LDAR (Learning Distraction-Aware Retrieval), an adaptive retriever that learns to retrieve contexts in a way that mitigates interference from distracting passages, thereby achieving significantly higher performance with reduced token usage compared to long-context approaches. Extensive experiments across diverse LLM architectures and six knowledge-intensive benchmarks demonstrate the effectiveness and robustness of our approach, highlighting the importance of balancing the trade-off between information coverage and distraction.

Method: Automatically builds ILP tasks with pruned search spaces using rule structure proposals from MLLMs, then uses ILP systems to output rules based on rectified facts and formal inductive reasoning.

Result: Verified effectiveness on challenging logical induction benchmarks and demonstrated application in text-to-image customized generation with rule induction.

Conclusion: ILP-CoT successfully bridges ILP and MLLMs for logical rule induction, overcoming limitations of both approaches through complementary integration.

Abstract: We propose ILP-CoT, a method that bridges Inductive Logic Programming (ILP) and Multimodal Large Language Models (MLLMs) for abductive logical rule induction. The task involves both discovering logical facts and inducing logical rules from a small number of unstructured textual or visual inputs, which still remain challenging when solely relying on ILP, due to the requirement of specified background knowledge and high computational cost, or MLLMs, due to the appearance of perceptual hallucinations. Based on the key observation that MLLMs could propose structure-correct rules even under hallucinations, our approach automatically builds ILP tasks with pruned search spaces based on the rule structure proposals from MLLMs, and utilizes ILP system to output rules built upon rectified logical facts and formal inductive reasoning. Its effectiveness is verified through challenging logical induction benchmarks, as well as a potential application of our approach, namely text-to-image customized generation with rule induction. Our code and data are released at https://github.com/future-item/ILP-CoT.

Method: Used partial-prompt contamination audit and matched-budget reproductions across base and RL models, plus proposed tax-aware training/evaluation protocol co-optimizing accuracy, grounding, and calibrated abstention.

Result: Several headline RLVR gaps shrink or vanish under clean evaluation; the proposed protocol yields more reliable gain estimates and revises some prior conclusions.

Conclusion: RLVR is valuable and industry-ready, but practical benefits should be balanced with reliability, safety, and proper measurement standards.

Abstract: Reinforcement learning with verifiable rewards (RLVR) is a practical and scalable approach to enhancing large language models in areas such as math, code, and other structured tasks. Two questions motivate this paper: how much of the reported gains survive under strictly parity-controlled evaluation, and whether RLVR is cost-free or exacts a measurable tax. We argue that progress is real, but gains are often overstated due to three forces - an RLVR tax, evaluation pitfalls, and data contamination. Using a partial-prompt contamination audit and matched-budget reproductions across base and RL models, we show that several headline gaps shrink or vanish under clean, parity-controlled evaluation. We then propose a tax-aware training and evaluation protocol that co-optimizes accuracy, grounding, and calibrated abstention and standardizes budgeting and provenance checks. Applied to recent RLVR setups, this protocol yields more reliable estimates of reasoning gains and, in several cases, revises prior conclusions. Our position is constructive: RLVR is valuable and industry-ready; we advocate keeping its practical benefits while prioritizing reliability, safety, and measurement.

Method: Reformulates Zubov’s equation into consistency characterization between RoAs and prescribed RoAs; uses tripartite losses (consistency, classification, separation) with parallel boundary sampling; employs input-attention-based convex neural network with softmax attention for discriminative Lyapunov functions.

Result: Maintains high classification accuracy while significantly improving robustness against various stochastic noises and adversarial attacks.

Conclusion: Zubov-Net provides a theoretically-grounded framework that guarantees consistent PRoAs-RoAs alignment, trajectory stability, and non-overlapping PRoAs while achieving better balance between accuracy and robustness.

Abstract: Despite neural ordinary differential equations (Neural ODEs) exhibiting intrinsic robustness under input perturbations due to their dynamical systems nature, recent approaches often involve imposing Lyapunov-based stability conditions to provide formal robustness guarantees. However, a fundamental challenge remains: the tension between robustness and accuracy, primarily stemming from the difficulty in imposing appropriate stability conditions. To address this, we propose an adaptive stable learning framework named Zubov-Net, which innovatively reformulates Zubov’s equation into a consistency characterization between regions of attraction (RoAs) and prescribed RoAs (PRoAs). Building on this consistency, we introduce a new paradigm for actively controlling the geometry of RoAs by directly optimizing PRoAs to reconcile accuracy and robustness. Our approach is realized through tripartite losses (consistency, classification, and separation losses) and a parallel boundary sampling algorithm that co-optimizes the Neural ODE and the Lyapunov function. To enhance the discriminativity of Lyapunov functions, we design an input-attention-based convex neural network via a softmax attention mechanism that focuses on equilibrium-relevant features and also serves as weight normalization to maintain training stability in deep architectures. Theoretically, we prove that minimizing the tripartite loss guarantees consistent alignment of PRoAs-RoAs, trajectory stability, and non-overlapping PRoAs. Moreover, we establish stochastic convex separability with tighter probability bounds and fewer dimensionality requirements to justify the convex design in Lyapunov functions. Experimentally, Zubov-Net maintains high classification accuracy while significantly improving robustness against various stochastic noises and adversarial attacks.

Method: Developed an algebraic representation of neural networks and used Koopman operators, group representations, and reproducing kernel Hilbert spaces to construct a new theoretical framework.

Result: Derived a novel Rademacher complexity bound that applies to a wider range of realistic neural network models and explains the generalization behavior of high-rank weight matrices.

Conclusion: This work extends Koopman-based theory for Rademacher complexity bounds to more practical scenarios, providing better theoretical understanding of neural network generalization.

Abstract: We derive a new Rademacher complexity bound for deep neural networks using Koopman operators, group representations, and reproducing kernel Hilbert spaces (RKHSs). The proposed bound describes why the models with high-rank weight matrices generalize well. Although there are existing bounds that attempt to describe this phenomenon, these existing bounds can be applied to limited types of models. We introduce an algebraic representation of neural networks and a kernel function to construct an RKHS to derive a bound for a wider range of realistic models. This work paves the way for the Koopman-based theory for Rademacher complexity bounds to be valid for more practical situations.

Method: Propose Increment Vector Transformation (IVT) that periodically teleports model parameters to transformed solutions preserving linear connectivity to previous task optimum. Uses diagonal Fisher Information Matrices for efficient approximation, works in both exemplar-free and exemplar-based scenarios.

Result: IVT consistently enhances CIL baselines: +5.12% last accuracy and -2.54% forgetting on CIFAR-100 with PASS baseline; +14.93% average accuracy and +21.95% last accuracy on FGVCAircraft with CLIP-pre-trained SLCA baseline. Tested on CIFAR-100, FGVCAircraft, ImageNet-Subset, and ImageNet-Full.

Conclusion: IVT effectively mitigates catastrophic forgetting by maintaining linear connectivity to previous task optima, demonstrating significant improvements across multiple datasets and baselines while being computationally efficient and compatible with various initialization strategies.

Abstract: Class Incremental Learning (CIL) aims to sequentially acquire knowledge of new classes without forgetting previously learned ones. Despite recent progress, current CIL methods still exhibit significant performance gaps compared to their oracle counterparts-models trained with full access to historical data. Inspired by recent insights on Linear Mode Connectivity (LMC), we revisit the geometric properties of oracle solutions in CIL and uncover a fundamental observation: these oracle solutions typically maintain low-loss linear connections to the optimum of previous tasks. Motivated by this finding, we propose Increment Vector Transformation (IVT), a novel plug-and-play framework designed to mitigate catastrophic forgetting during training. Rather than directly following CIL updates, IVT periodically teleports the model parameters to transformed solutions that preserve linear connectivity to previous task optimum. By maintaining low-loss along these connecting paths, IVT effectively ensures stable performance on previously learned tasks. The transformation is efficiently approximated using diagonal Fisher Information Matrices, making IVT suitable for both exemplar-free and exemplar-based scenarios, and compatible with various initialization strategies. Extensive experiments on CIFAR-100, FGVCAircraft, ImageNet-Subset, and ImageNet-Full demonstrate that IVT consistently enhances the performance of strong CIL baselines. Specifically, on CIFAR-100, IVT improves the last accuracy of the PASS baseline by +5.12% and reduces forgetting by 2.54%. For the CLIP-pre-trained SLCA baseline on FGVCAircraft, IVT yields gains of +14.93% in average accuracy and +21.95% in last accuracy. The code will be released.

Method: Uses stochastic interpolation generative framework to derive closed-form expressions for optimal velocity field and score function. Introduces formal definitions of underfitting and overfitting for generative models.

Result: Deterministic generative process exactly recovers training samples; stochastic process produces training samples with Gaussian noise. With estimation errors, generation yields convex combinations of training samples corrupted by uniform and Gaussian noise.

Conclusion: The theoretical framework explains generative model behavior under finite training data and estimation errors, with experimental validation showing the analysis holds in practice for generation and downstream tasks.

Abstract: This paper investigates the theoretical behavior of generative models under finite training populations. Within the stochastic interpolation generative framework, we derive closed-form expressions for the optimal velocity field and score function when only a finite number of training samples are available. We demonstrate that, under some regularity conditions, the deterministic generative process exactly recovers the training samples, while the stochastic generative process manifests as training samples with added Gaussian noise. Beyond the idealized setting, we consider model estimation errors and introduce formal definitions of underfitting and overfitting specific to generative models. Our theoretical analysis reveals that, in the presence of estimation errors, the stochastic generation process effectively produces convex combinations of training samples corrupted by a mixture of uniform and Gaussian noise. Experiments on generation tasks and downstream tasks such as classification support our theory.

Method: Derived exact transition rates for desired distributions given learned discrete flow matching models, creating a unified framework that works with single forward passes and can be applied to masked diffusion models.

Result: Demonstrated effectiveness on energy-guided simulations and preference alignment tasks in text-to-image generation and multimodal understanding, with improved sampling efficiency.

Conclusion: The proposed discrete guidance framework provides exact and efficient posterior sampling for discrete data, unifying existing methods and showing strong performance across various applications.

Abstract: Guidance provides a simple and effective framework for posterior sampling by steering the generation process towards the desired distribution. When modeling discrete data, existing approaches mostly focus on guidance with the first-order Taylor approximation to improve the sampling efficiency. However, such an approximation is inappropriate in discrete state spaces since the approximation error could be large. A novel guidance framework for discrete data is proposed to address this problem: We derive the exact transition rate for the desired distribution given a learned discrete flow matching model, leading to guidance that only requires a single forward pass in each sampling step, significantly improving efficiency. This unified novel framework is general enough, encompassing existing guidance methods as special cases, and it can also be seamlessly applied to the masked diffusion model. We demonstrate the effectiveness of our proposed guidance on energy-guided simulations and preference alignment on text-to-image generation and multimodal understanding tasks. The code is available through https://github.com/WanZhengyan/Discrete-Guidance-Matching/tree/main.

Method: Introduce Multiplicative-Additive Constrained Models (MACMs) that augment CESR with an additive component to disentangle interactive and independent feature effects, effectively expanding the hypothesis space while maintaining visual interpretability of shape functions.

Result: Neural network-based MACMs significantly outperform both CESR and state-of-the-art GAMs in predictive performance while maintaining interpretability through visualizable shape functions.

Conclusion: MACMs successfully bridge the gap between interpretability and performance by combining multiplicative and additive components, providing a superior alternative to both CESR and GAMs for high-stakes applications.

Abstract: Interpretability is one of the considerations when applying machine learning to high-stakes fields such as healthcare that involve matters of life safety. Generalized Additive Models (GAMs) enhance interpretability by visualizing shape functions. Nevertheless, to preserve interpretability, GAMs omit higher-order interaction effects (beyond pairwise interactions), which imposes significant constraints on their predictive performance. We observe that Curve Ergodic Set Regression (CESR), a multiplicative model, naturally enables the visualization of its shape functions and simultaneously incorporates both interactions among all features and individual feature effects. Nevertheless, CESR fails to demonstrate superior performance compared to GAMs. We introduce Multiplicative-Additive Constrained Models (MACMs), which augment CESR with an additive part to disentangle the intertwined coefficients of its interactive and independent terms, thus effectively broadening the hypothesis space. The model is composed of a multiplicative part and an additive part, whose shape functions can both be naturally visualized, thereby assisting users in interpreting how features participate in the decision-making process. Consequently, MACMs constitute an improvement over both CESR and GAMs. The experimental results indicate that neural network-based MACMs significantly outperform both CESR and the current state-of-the-art GAMs in terms of predictive performance.

Method: Converts 1D time-series into 3D representations using continuous wavelet transforms (CWTs) and recurrence plots (RPs), then fine-tunes visual language large models (VLLMs) on these visual encodings.

Result: Fine-tuned VLLMs successfully monitor building states, identify anomalies, and generate optimization recommendations. Idefics-7B VLLM achieved validation losses of 0.0952 with CWTs and 0.1064 with RPs, outperforming direct fine-tuning on raw time-series (0.1176).

Conclusion: This work bridges time-series analysis and visualization, providing a scalable and interpretable framework for energy analytics.

Abstract: The analysis of complex building time-series for actionable insights and recommendations remains challenging due to the nonlinear and multi-scale characteristics of energy data. To address this, we propose a framework that fine-tunes visual language large models (VLLMs) on 3D graphical representations of the data. The approach converts 1D time-series into 3D representations using continuous wavelet transforms (CWTs) and recurrence plots (RPs), which capture temporal dynamics and localize frequency anomalies. These 3D encodings enable VLLMs to visually interpret energy-consumption patterns, detect anomalies, and provide recommendations for energy efficiency. We demonstrate the framework on real-world building-energy datasets, where fine-tuned VLLMs successfully monitor building states, identify recurring anomalies, and generate optimization recommendations. Quantitatively, the Idefics-7B VLLM achieves validation losses of 0.0952 with CWTs and 0.1064 with RPs on the University of Sharjah energy dataset, outperforming direct fine-tuning on raw time-series data (0.1176) for anomaly detection. This work bridges time-series analysis and visualization, providing a scalable and interpretable framework for energy analytics.

Method: Uses reinforcement learning with a toxicity classifier as reward, but periodically safety fine-tunes the victim LLM with collected attack prompts, which diminishes rewards in exploited regions and forces the attacker to seek unexplored vulnerabilities, creating an easy-to-hard exploration curriculum.

Result: Outperformed prior RL-based methods (GFlowNets, PPO, REINFORCE) by improving cross-attack success rates from 0.07% to 31.28% (400x relative gain) with only 6% increase in computation, uncovering a wide range of local attack modes step by step.

Conclusion: Active Attacks is an effective plug-and-play module that naturally induces diverse exploration by adapting to the evolving victim model, achieving superior coverage of multi-mode attack distributions compared to existing methods.

Abstract: We address the challenge of generating diverse attack prompts for large language models (LLMs) that elicit harmful behaviors (e.g., insults, sexual content) and are used for safety fine-tuning. Rather than relying on manual prompt engineering, attacker LLMs can be trained with reinforcement learning (RL) to automatically generate such prompts using only a toxicity classifier as a reward. However, capturing a wide range of harmful behaviors is a significant challenge that requires explicit diversity objectives. Existing diversity-seeking RL methods often collapse to limited modes: once high-reward prompts are found, exploration of new regions is discouraged. Inspired by the active learning paradigm that encourages adaptive exploration, we introduce \textit{Active Attacks}, a novel RL-based red-teaming algorithm that adapts its attacks as the victim evolves. By periodically safety fine-tuning the victim LLM with collected attack prompts, rewards in exploited regions diminish, which forces the attacker to seek unexplored vulnerabilities. This process naturally induces an easy-to-hard exploration curriculum, where the attacker progresses beyond easy modes toward increasingly difficult ones. As a result, Active Attacks uncovers a wide range of local attack modes step by step, and their combination achieves wide coverage of the multi-mode distribution. Active Attacks, a simple plug-and-play module that seamlessly integrates into existing RL objectives, unexpectedly outperformed prior RL-based methods – including GFlowNets, PPO, and REINFORCE – by improving cross-attack success rates against GFlowNets, the previous state-of-the-art, from 0.07% to 31.28% (a relative gain greater than $400\ \times$) with only a 6% increase in computation. Our code is publicly available \href{https://github.com/dbsxodud-11/active_attacks}{here}.

Method: Developed an analysis using statistical physics approaches in high-dimensional limit, studying single-location regression task where output depends on linear transformation of single input token at random location. Analyzed attention-based predictors using order parameters.

Result: At population level, softmax achieves Bayes risk while linear attention fundamentally falls short. In finite-sample regime, softmax consistently outperforms linear attention though no longer Bayes-optimal. Identified key properties needed for optimal performance across different activation functions.

Conclusion: Softmax attention’s superiority is theoretically justified - it achieves optimal performance at population level and consistently outperforms alternatives in finite samples, providing principled understanding of why softmax dominates in practice.

Abstract: Large language models rely on attention mechanisms with a softmax activation. Yet the dominance of softmax over alternatives (e.g., component-wise or linear) remains poorly understood, and many theoretical works have focused on the easier-to-analyze linearized attention. In this work, we address this gap through a principled study of the single-location regression task, where the output depends on a linear transformation of a single input token at a random location. Building on ideas from statistical physics, we develop an analysis of attention-based predictors in the high-dimensional limit, where generalization performance is captured by a small set of order parameters. At the population level, we show that softmax achieves the Bayes risk, whereas linear attention fundamentally falls short. We then examine other activation functions to identify which properties are necessary for optimal performance. Finally, we analyze the finite-sample regime: we provide an asymptotic characterization of the test error and show that, while softmax is no longer Bayes-optimal, it consistently outperforms linear attention. We discuss the connection with optimization by gradient-based algorithms.

Method: Analyzes structural information in offline trajectories to adaptively construct diffusion hierarchies across multiple temporal scales. Uses structural information gain of state communities as conditioning signals and introduces structural entropy regularizer to encourage exploration while avoiding extrapolation errors.

Result: Extensive evaluations show SIHD significantly outperforms state-of-the-art baselines in decision-making performance and demonstrates superior generalization across diverse scenarios.

Conclusion: SIHD provides an effective and stable framework for offline policy learning in long-horizon environments by adaptively constructing diffusion hierarchies and leveraging structural information to improve exploration and decision-making.

Abstract: Diffusion-based generative methods have shown promising potential for modeling trajectories from offline reinforcement learning (RL) datasets, and hierarchical diffusion has been introduced to mitigate variance accumulation and computational challenges in long-horizon planning tasks. However, existing approaches typically assume a fixed two-layer diffusion hierarchy with a single predefined temporal scale, which limits adaptability to diverse downstream tasks and reduces flexibility in decision making. In this work, we propose SIHD, a novel Structural Information-based Hierarchical Diffusion framework for effective and stable offline policy learning in long-horizon environments with sparse rewards. Specifically, we analyze structural information embedded in offline trajectories to construct the diffusion hierarchy adaptively, enabling flexible trajectory modeling across multiple temporal scales. Rather than relying on reward predictions from localized sub-trajectories, we quantify the structural information gain of each state community and use it as a conditioning signal within the corresponding diffusion layer. To reduce overreliance on offline datasets, we introduce a structural entropy regularizer that encourages exploration of underrepresented states while avoiding extrapolation errors from distributional shifts. Extensive evaluations on challenging offline RL tasks show that SIHD significantly outperforms state-of-the-art baselines in decision-making performance and demonstrates superior generalization across diverse scenarios.

Method: Developed a reward function that dynamically links reasoning length to perceived problem difficulty, encouraging shorter reasoning for easy tasks and deeper reasoning for complex ones.

Result: Method improves task performance while significantly reducing average reasoning length. Extensive experiments show effectiveness and efficiency.

Conclusion: Adaptive reasoning depth based on problem complexity is crucial for efficient LALMs. The approach provides valuable insights for future work on reasoning structure paradigms.

Abstract: Large Audio Language Models (LALMs), powered by the chain-of-thought (CoT) paradigm, have shown remarkable reasoning capabilities. Intuitively, different problems often require varying depths of reasoning. While some methods can determine whether to reason for a given problem, they typically lack a fine-grained mechanism to modulate how much to reason. This often results in a ``one-size-fits-all’’ reasoning depth, which generates redundant overthinking for simple questions while failing to allocate sufficient thought to complex ones. In this paper, we conduct an in-depth analysis of LALMs and find that an effective and efficient LALM should reason smartly by adapting its reasoning depth to the problem’s complexity. To achieve this, we propose a difficulty-adaptive reasoning method for LALMs. Specifically, we propose a reward function that dynamically links reasoning length to the model’s perceived problem difficulty. This reward encourages shorter, concise reasoning for easy tasks and more elaborate, in-depth reasoning for complex ones. Extensive experiments demonstrate that our method is both effective and efficient, simultaneously improving task performance and significantly reducing the average reasoning length. Further analysis on reasoning structure paradigm offers valuable insights for future work.