Method: Sample softmax denominators from a calibration text to combine outputs of multiple stable attention heads in GPT2-Small’s first layer. Approximate their combined output using linear summaries of surrounding text, enabling neuron discovery from weights alone.

Result: The method successfully uncovered hundreds of first layer neurons that respond to high-level contextual properties of surrounding text, including neurons that didn’t activate on the calibration text used for calibration.

Conclusion: The proposed calibration-based approach enables effective analysis of transformer language models, revealing how attention mechanisms capture contextual information and allowing discovery of functionally relevant neurons directly from model weights.

Abstract: We study transformer language models, analyzing attention heads whose attention patterns are spread out, and whose attention scores depend weakly on content. We argue that the softmax denominators of these heads are stable when the underlying token distribution is fixed. By sampling softmax denominators from a “calibration text”, we can combine together the outputs of multiple such stable heads in the first layer of GPT2-Small, approximating their combined output by a linear summary of the surrounding text. This approximation enables a procedure where from the weights alone - and a single calibration text - we can uncover hundreds of first layer neurons that respond to high-level contextual properties of the surrounding text, including neurons that didn’t activate on the calibration text.

Method: Proposes Graph-S³ with: 1) synthetic stepwise supervision using offline-extracted golden subgraphs instead of final-answer rewards, 2) data synthesis pipeline for reward generation, and 3) two-stage training for interactive graph exploration policy.

Result: Achieves average improvements of 8.1% in accuracy and 9.7% in F1 score across three datasets compared to seven strong baselines, with even better performance on complex multi-hop reasoning tasks.

Conclusion: The approach effectively addresses graph retrieval challenges in textual graph QA through stepwise supervision and synthetic training, demonstrating significant performance gains especially in complex reasoning scenarios.

Abstract: A significant portion of real-world data is inherently represented as textual graphs, and integrating these graphs into large language models (LLMs) is promising to enable complex graph-based question answering. However, a key challenge in LLM-based textual graph QA systems lies in graph retrieval, i.e., how to retrieve relevant content from large graphs that is sufficiently informative while remaining compact for the LLM context. Existing retrievers suffer from poor performance since they either rely on shallow embedding similarity or employ interactive retrieving policies that demand excessive data labeling and training cost. To address these issues, we present Graph-$S^3$, an agentic textual graph reasoning framework that employs an LLM-based retriever trained with synthetic stepwise supervision. Instead of rewarding the agent based on the final answers, which may lead to sparse and unstable training signals, we propose to closely evaluate each step of the retriever based on offline-extracted golden subgraphs. Our main techniques include a data synthesis pipeline to extract the golden subgraphs for reward generation and a two-stage training scheme to learn the interactive graph exploration policy based on the synthesized rewards. Based on extensive experiments on three common datasets in comparison with seven strong baselines, our approach achieves an average improvement of 8.1% in accuracy and 9.7% in F$_1$ score. The advantage is even higher in more complicated multi-hop reasoning tasks. Our code will be open-sourced.

Method: Audited six popular LLMs completing 30 everyday tasks and compared their responses to 100 human crowdworkers from the US.

Result: LLMs often do not align with human values, nor with other LLMs, in the implicit values they exhibit during task completion.

Conclusion: There is significant misalignment between LLM values and human values in everyday task contexts, highlighting a need for better value alignment in AI assistants.

Abstract: Large language models (LLMs) can underpin AI assistants that help users with everyday tasks, such as by making recommendations or performing basic computation. Despite AI assistants’ promise, little is known about the implicit values these assistants display while completing subjective everyday tasks. Humans may consider values like environmentalism, charity, and diversity. To what extent do LLMs exhibit these values in completing everyday tasks? How do they compare with humans? We answer these questions by auditing how six popular LLMs complete 30 everyday tasks, comparing LLMs to each other and to 100 human crowdworkers from the US. We find LLMs often do not align with humans, nor with other LLMs, in the implicit values exhibited.

Method: A greedy algorithm that: (1) weights morphemes by mutual information between forms and meanings, (2) selects highest-weighted pair, (3) removes it from corpus, (4) repeats process (Count, Select, Ablate, Repeat).

Result: Validated on procedurally generated datasets and human language data, showing reasonable predictions. Also used to analyze emergent languages, quantifying characteristics like synonymy and polysemy.

Conclusion: CSAR is an effective algorithm for morpheme induction that performs well on both synthetic and human language data, and can be used to analyze linguistic characteristics of emergent languages.

Abstract: We introduce CSAR, an algorithm for inducing morphemes from emergent language corpora of parallel utterances and meanings. It is a greedy algorithm that (1) weights morphemes based on mutual information between forms and meanings, (2) selects the highest-weighted pair, (3) removes it from the corpus, and (4) repeats the process to induce further morphemes (i.e., Count, Select, Ablate, Repeat). The effectiveness of CSAR is first validated on procedurally generated datasets and compared against baselines for related tasks. Second, we validate CSAR’s performance on human language data to show that the algorithm makes reasonable predictions in adjacent domains. Finally, we analyze a handful of emergent languages, quantifying linguistic characteristics like degree of synonymy and polysemy.

Method: Extends retrieval beyond text and images to support audio and video modalities, building on capabilities of recent multimodal models like Qwen2.5-Omni, enabling cross-modal and joint-modal retrieval using a single unified architecture.

Result: Demonstrates effectiveness in text, image, and video retrieval, showing improved performance for handling complex real-world information needs across multiple modalities.

Conclusion: Omni-Embed-Nemotron provides a unified solution for multimodal retrieval that can handle the increasing complexity of real-world information needs across text, image, audio, and video modalities.

Abstract: We present Omni-Embed-Nemotron, a unified multimodal retrieval embedding model developed to handle the increasing complexity of real-world information needs. While Retrieval-Augmented Generation (RAG) has significantly advanced language models by incorporating external knowledge, existing text-based retrievers rely on clean, structured input and struggle with the visually and semantically rich content found in real-world documents such as PDFs, slides, or videos. Recent work such as ColPali has shown that preserving document layout using image-based representations can improve retrieval quality. Building on this, and inspired by the capabilities of recent multimodal models such as Qwen2.5-Omni, we extend retrieval beyond text and images to also support audio and video modalities. Omni-Embed-Nemotron enables both cross-modal (e.g., text - video) and joint-modal (e.g., text - video+audio) retrieval using a single model. We describe the architecture, training setup, and evaluation results of Omni-Embed-Nemotron, and demonstrate its effectiveness in text, image, and video retrieval.

Method: Uses a signalling game-based emergent communication environment with hyperparameter optimization, employing XferBench as objective function to measure statistical similarity to human language via transfer learning suitability.

Result: Demonstrates entropy’s predictive power on transfer learning performance, confirms entropy-minimization properties, and identifies hyperparameters that produce more realistic emergent languages with better transfer to human language.

Conclusion: The approach successfully generates human-like emergent languages and provides insights into entropy relationships and optimal hyperparameter configurations for language realism.

Abstract: In this paper, we design a signalling game-based emergent communication environment to generate state-of-the-art emergent languages in terms of similarity to human language. This is done with hyperparameter optimization, using XferBench as the objective function. XferBench quantifies the statistical similarity of emergent language to human language by measuring its suitability for deep transfer learning to human language. Additionally, we demonstrate the predictive power of entropy on the transfer learning performance of emergent language as well as corroborate previous results on the entropy-minimization properties of emergent communication systems. Finally, we report generalizations regarding what hyperparameters produce more realistic emergent languages, that is, ones which transfer better to human language.

Method: Created SEER Benchmark with two tasks: identifying emotion evidence within single sentences and across five-sentence passages. Contains 1200 real-world sentences with new annotations for emotion and emotion evidence. Evaluated 14 open-source LLMs.

Result: Some models approach average human performance on single-sentence inputs, but accuracy degrades in longer passages. Error analysis reveals key failure modes including overreliance on emotion keywords and false positives in neutral text.

Conclusion: Current LLMs struggle with span-level emotion evidence detection, especially in longer contexts, highlighting the need for improved models that can accurately pinpoint emotional expressions in text.

Abstract: We introduce the SEER (Span-based Emotion Evidence Retrieval) Benchmark to test Large Language Models’ (LLMs) ability to identify the specific spans of text that express emotion. Unlike traditional emotion recognition tasks that assign a single label to an entire sentence, SEER targets the underexplored task of emotion evidence detection: pinpointing which exact phrases convey emotion. This span-level approach is crucial for applications like empathetic dialogue and clinical support, which need to know how emotion is expressed, not just what the emotion is. SEER includes two tasks: identifying emotion evidence within a single sentence, and identifying evidence across a short passage of five consecutive sentences. It contains new annotations for both emotion and emotion evidence on 1200 real-world sentences. We evaluate 14 open-source LLMs and find that, while some models approach average human performance on single-sentence inputs, their accuracy degrades in longer passages. Our error analysis reveals key failure modes, including overreliance on emotion keywords and false positives in neutral text.

Method: Created a balanced dataset with 400K+ samples across three genres using three leading LLMs and multiple human sources, with rigorous preprocessing, rich annotations, and standardized splits. Benchmarking used traditional classifiers, BERT-based models, and LLMs (zero-shot/few-shot).

Result: Fine-tuned BERT models achieved competitive performance, outperforming LLM-based models. However, models struggled with cross-genre generalization, particularly with news articles where LLM-generated texts closely resemble human writing style.

Conclusion: ALHD establishes a foundation for Arabic LLM detection research. Challenges in cross-genre generalization, especially with news content, highlight the need for future research to improve detection robustness across different text types.

Abstract: We introduce ALHD, the first large-scale comprehensive Arabic dataset explicitly designed to distinguish between human- and LLM-generated texts. ALHD spans three genres (news, social media, reviews), covering both MSA and dialectal Arabic, and contains over 400K balanced samples generated by three leading LLMs and originated from multiple human sources, which enables studying generalizability in Arabic LLM-genearted text detection. We provide rigorous preprocessing, rich annotations, and standardized balanced splits to support reproducibility. In addition, we present, analyze and discuss benchmark experiments using our new dataset, in turn identifying gaps and proposing future research directions. Benchmarking across traditional classifiers, BERT-based models, and LLMs (zero-shot and few-shot) demonstrates that fine-tuned BERT models achieve competitive performance, outperforming LLM-based models. Results are however not always consistent, as we observe challenges when generalizing across genres; indeed, models struggle to generalize when they need to deal with unseen patterns in cross-genre settings, and these challenges are particularly prominent when dealing with news articles, where LLM-generated texts resemble human texts in style, which opens up avenues for future research. ALHD establishes a foundation for research related to Arabic LLM-detection and mitigating risks of misinformation, academic dishonesty, and cyber threats.

Method: Proposes TS-Reasoner with two-stage training: alignment pretraining using synthetic time series-text pairs, followed by instruction finetuning. Uses frozen pretrained TSFM and aligns its latent representations with LLM textual inputs.

Result: Outperforms various LLMs, VLMs, and Time Series LLMs across multiple benchmarks while using less than half the training data, demonstrating remarkable data efficiency.

Conclusion: The proposed TS-Reasoner effectively bridges the gap between numerical time series understanding and textual reasoning, providing a powerful framework for time series reasoning tasks.

Abstract: Time series reasoning is crucial to decision-making in diverse domains, including finance, energy usage, traffic, weather, and scientific discovery. While existing time series foundation models (TSFMs) can capture low-level dynamic patterns and provide accurate forecasting, further analysis usually requires additional background knowledge and sophisticated reasoning, which are lacking in most TSFMs but can be achieved through large language models (LLMs). On the other hand, without expensive post-training, LLMs often struggle with the numerical understanding of time series data. Although it is intuitive to integrate the two types of models, developing effective training recipes that align the two modalities for reasoning tasks is still an open challenge. To this end, we propose TS-Reasoner that aligns the latent representations of TSFMs with the textual inputs of LLMs for downstream understanding/reasoning tasks. Specifically, we propose a simple yet effective method to curate diverse, synthetic pairs of time series and textual captions for alignment training. We then develop a two-stage training recipe that applies instruction finetuning after the alignment pretraining. Unlike existing works that train an LLM to take time series as inputs, we leverage a pretrained TSFM and freeze it during training. Extensive experiments on several benchmarks demonstrate that TS-Reasoner not only outperforms a wide range of prevailing LLMs, Vision Language Models (VLMs), and Time Series LLMs, but also achieves this with remarkable data efficiency, e.g., using less than half the training data.

Method: A peer-aware comparative inference layer is added on top of RAG to retrieve comparable cases and related problems, enabling contrastive reasoning and nuanced analysis in specialized contexts.

Result: The contrastive approach outperforms baseline RAG in text generation metrics including ROUGE and BERTScore when compared to human-generated equity research and risk analysis.

Conclusion: Adding comparative reasoning capabilities to RAG systems enhances their performance in specialized domains by enabling more nuanced, context-specific insights rather than generic factual outputs.

Abstract: In specialized domains, humans often compare new problems against similar examples, highlight nuances, and draw conclusions instead of analyzing information in isolation. When applying reasoning in specialized contexts with LLMs on top of a RAG, the pipeline can capture contextually relevant information, but it is not designed to retrieve comparable cases or related problems. While RAG is effective at extracting factual information, its outputs in specialized reasoning tasks often remain generic, reflecting broad facts rather than context-specific insights. In finance, it results in generic risks that are true for the majority of companies. To address this limitation, we propose a peer-aware comparative inference layer on top of RAG. Our contrastive approach outperforms baseline RAG in text generation metrics such as ROUGE and BERTScore in comparison with human-generated equity research and risk.

Method: Construct ConGrs using lightweight lexical sequence alignment from bioinformatics supplemented by targeted usage of a secondary LM judge, then design task-dependent decoding methods to synthesize final responses from the ConGr structure.

Result: Improves factual precision on biography generation by up to 31%, reduces reliance on LM judges by over 80%, increases abstention rates on refusal tasks by up to 56%, and improves reasoning task accuracy by up to 6 points over baselines.

Conclusion: ConGrs provide a flexible method for capturing variation in LM responses and using epistemic signals from response variation to synthesize more effective responses across multiple task types.

Abstract: Language models can be sampled multiple times to access the distribution underlying their responses, but existing methods cannot efficiently synthesize rich epistemic signals across different long-form responses. We introduce Consensus Graphs (ConGrs), a flexible DAG-based data structure that represents shared information, as well as semantic variation in a set of sampled LM responses to the same prompt. We construct ConGrs using a light-weight lexical sequence alignment algorithm from bioinformatics, supplemented by the targeted usage of a secondary LM judge. Further, we design task-dependent decoding methods to synthesize a single, final response from our ConGr data structure. Our experiments show that synthesizing responses from ConGrs improves factual precision on two biography generation tasks by up to 31% over an average response and reduces reliance on LM judges by more than 80% compared to other methods. We also use ConGrs for three refusal-based tasks requiring abstention on unanswerable queries and find that abstention rate is increased by up to 56%. We apply our approach to the MATH and AIME reasoning tasks and find an improvement over self-verification and majority vote baselines by up to 6 points of accuracy. We show that ConGrs provide a flexible method for capturing variation in LM responses and using the epistemic signals provided by response variation to synthesize more effective responses.

Method: Instruction-tuning with perturbations like removing stop words or shuffling words, then evaluating performance on original and perturbed versions of benchmarks (MMLU, BBH, GSM8K). Also assesses learning dynamics and behavior shifts.

Result: Surprisingly, instruction-tuning on perturbed instructions can improve downstream performance in some cases, making LLMs more resilient to noisy instructions.

Conclusion: Including perturbed instructions in instruction-tuning is important for making LLMs more resilient to noisy user inputs, as it can enhance resistance to instruction variations and potentially improve performance.

Abstract: Instruction-tuning plays a vital role in enhancing the task-solving abilities of large language models (LLMs), improving their usability in generating helpful responses on various tasks. However, previous work has demonstrated that they are sensitive to minor variations in instruction phrasing. In this paper, we explore whether introducing perturbations in instruction-tuning data can enhance LLMs’ resistance against noisy instructions. We focus on how instruction-tuning with perturbations, such as removing stop words or shuffling words, affects LLMs’ performance on the original and perturbed versions of widely-used benchmarks (MMLU, BBH, GSM8K). We further assess learning dynamics and potential shifts in model behavior. Surprisingly, our results suggest that instruction-tuning on perturbed instructions can, in some cases, improve downstream performance. These findings highlight the importance of including perturbed instructions in instruction-tuning, which can make LLMs more resilient to noisy user inputs.

Method: TriMediQ uses a frozen triplet generator to extract clinical triplets from patient responses, builds a knowledge graph, and employs a trainable projection module with graph encoder and projector for multi-hop reasoning.

Result: TriMediQ achieves up to 10.4% improvement in accuracy over five baselines on the iMedQA dataset.

Conclusion: Converting patient responses into structured triplet-based graphs enables more accurate clinical reasoning in multi-turn settings, providing a solution for LLM-based medical assistants.

Abstract: Large Language Models (LLMs) perform strongly in static and single-turn medical Question Answer (QA) benchmarks, yet such settings diverge from the iterative information gathering process required in practical clinical consultations. The MEDIQ framework addresses this mismatch by recasting the diagnosis as an interactive dialogue between a patient and an expert system, but the reliability of LLMs drops dramatically when forced to reason with dialogue logs, where clinical facts appear in sentences without clear links. To bridge this gap, we introduce TriMediQ, a triplet-structured approach that summarises patient responses into triplets and integrates them into a Knowledge Graph (KG), enabling multi-hop reasoning. We introduce a frozen triplet generator that extracts clinically relevant triplets, using prompts designed to ensure factual consistency. In parallel, a trainable projection module, comprising a graph encoder and a projector, captures relational information from the KG to enhance expert reasoning. TriMediQ operates in two steps: (i) the projection module fine-tuning with all LLM weights frozen; and (ii) using the fine-tuned module to guide multi-hop reasoning during inference. We evaluate TriMediQ on two interactive QA benchmarks, showing that it achieves up to 10.4% improvement in accuracy over five baselines on the iMedQA dataset. These results demonstrate that converting patient responses into structured triplet-based graphs enables more accurate clinical reasoning in multi-turn settings, providing a solution for the deployment of LLM-based medical assistants.

Method: Automated pipeline extracts time-aligned phonemes, syllables, and words from recordings. Uses bidirectional LSTM with multi-head attention across Italian, Spanish, and English datasets.

Result: Phoneme-level analysis achieved best performance: 93.78% ± 2.34% AUROC and 92.17% ± 2.43% accuracy. Attention analysis revealed most informative features align with clinical protocols.

Conclusion: Granular phoneme-level analysis provides enhanced diagnostic capability for cross-linguistic PD detection, with features matching established clinical assessment protocols.

Abstract: Parkinson’s Disease (PD) affects over 10 million people worldwide, with speech impairments in up to 89% of patients. Current speech-based detection systems analyze entire utterances, potentially overlooking the diagnostic value of specific phonetic elements. We developed a granularity-aware approach for multilingual PD detection using an automated pipeline that extracts time-aligned phonemes, syllables, and words from recordings. Using Italian, Spanish, and English datasets, we implemented a bidirectional LSTM with multi-head attention to compare diagnostic performance across the different granularity levels. Phoneme-level analysis achieved superior performance with AUROC of 93.78% +- 2.34% and accuracy of 92.17% +- 2.43%. This demonstrates enhanced diagnostic capability for cross-linguistic PD detection. Importantly, attention analysis revealed that the most informative speech features align with those used in established clinical protocols: sustained vowels (/a/, /e/, /o/, /i/) at phoneme level, diadochokinetic syllables (/ta/, /pa/, /la/, /ka/) at syllable level, and /pataka/ sequences at word level. Source code will be available at https://github.com/jetliqs/clearpd.

Method: Using simulations to demonstrate conceptualization-induced bias and showing it cannot be corrected by increasing LLM accuracy or post-hoc bias correction methods.

Result: Conceptualization-induced bias persists despite improvements in LLM accuracy and cannot be eliminated through standard bias correction techniques.

Conclusion: Conceptualization remains a critical first-order concern in LLM-era CSS, and analysts should follow concrete advice for obtaining low-cost, unbiased, low-variance downstream estimates.

Abstract: Generative large language models (LLMs) are now used extensively for text classification in computational social science (CSS). In this work, focus on the steps before and after LLM prompting – conceptualization of concepts to be classified and using LLM predictions in downstream statistical inference – which we argue have been overlooked in much of LLM-era CSS. We claim LLMs can tempt analysts to skip the conceptualization step, creating conceptualization errors that bias downstream estimates. Using simulations, we show that this conceptualization-induced bias cannot be corrected for solely by increasing LLM accuracy or post-hoc bias correction methods. We conclude by reminding CSS analysts that conceptualization is still a first-order concern in the LLM-era and provide concrete advice on how to pursue low-cost, unbiased, low-variance downstream estimates.

Method: Created CCD-Bench with 2,182 open-ended dilemmas across 7 domains, paired with 10 anonymized response options corresponding to GLOBE cultural clusters, using stratified Latin square design to mitigate ordering effects. Evaluated 17 non-reasoning LLMs.

Result: Models show strong preference for Nordic Europe (20.2%) and Germanic Europe (12.4%), while Eastern Europe and Middle East/North Africa are underrepresented (5.6-5.8%). Rationales show superficial pluralism - 87.9% reference multiple GLOBE dimensions but mainly recombine Future/Performance Orientation, rarely using Assertiveness or Gender Egalitarianism (<3%).

Conclusion: Current alignment pipelines promote consensus-oriented worldviews that underserve scenarios requiring power negotiation, rights-based reasoning, or gender-aware analysis. CCD-Bench shifts evaluation toward pluralistic decision making and highlights need for alignment strategies that substantively engage diverse worldviews.

Abstract: Although large language models (LLMs) are increasingly implicated in interpersonal and societal decision-making, their ability to navigate explicit conflicts between legitimately different cultural value systems remains largely unexamined. Existing benchmarks predominantly target cultural knowledge (CulturalBench), value prediction (WorldValuesBench), or single-axis bias diagnostics (CDEval); none evaluate how LLMs adjudicate when multiple culturally grounded values directly clash. We address this gap with CCD-Bench, a benchmark that assesses LLM decision-making under cross-cultural value conflict. CCD-Bench comprises 2,182 open-ended dilemmas spanning seven domains, each paired with ten anonymized response options corresponding to the ten GLOBE cultural clusters. These dilemmas are presented using a stratified Latin square to mitigate ordering effects. We evaluate 17 non-reasoning LLMs. Models disproportionately prefer Nordic Europe (mean 20.2 percent) and Germanic Europe (12.4 percent), while options for Eastern Europe and the Middle East and North Africa are underrepresented (5.6 to 5.8 percent). Although 87.9 percent of rationales reference multiple GLOBE dimensions, this pluralism is superficial: models recombine Future Orientation and Performance Orientation, and rarely ground choices in Assertiveness or Gender Egalitarianism (both under 3 percent). Ordering effects are negligible (Cramer’s V less than 0.10), and symmetrized KL divergence shows clustering by developer lineage rather than geography. These patterns suggest that current alignment pipelines promote a consensus-oriented worldview that underserves scenarios demanding power negotiation, rights-based reasoning, or gender-aware analysis. CCD-Bench shifts evaluation beyond isolated bias detection toward pluralistic decision making and highlights the need for alignment strategies that substantively engage diverse worldviews.

Method: Conducted systematic study across three benchmarks (MMAU, MMAR, MMSU) and four models (Audio Flamingo 2, Audio Flamingo 3, Qwen2.5-Omni-7B-Instruct, Kimi-Audio-7B-Instruct), testing sensitivity to choice ordering, question paraphrasing, and choice paraphrasing.

Result: Found that models are highly sensitive to choice ordering, question paraphrasing, and choice paraphrasing, with substantial performance variations across different configurations of the same questions.

Conclusion: Proposed a simpler evaluation protocol and metric that accounts for subtle variations to provide more detailed and reliable evaluation of LALMs within the MCQA framework.

Abstract: Recent advances in large audio language models (LALMs) have primarily been assessed using a multiple-choice question answering (MCQA) framework. However, subtle changes, such as shifting the order of choices, result in substantially different results. Existing MCQA frameworks do not account for this variability and report a single accuracy number per benchmark or category. We dive into the MCQA evaluation framework and conduct a systematic study spanning three benchmarks (MMAU, MMAR and MMSU) and four models: Audio Flamingo 2, Audio Flamingo 3, Qwen2.5-Omni-7B-Instruct, and Kimi-Audio-7B-Instruct. Our findings indicate that models are sensitive not only to the ordering of choices, but also to the paraphrasing of the question and the choices. Finally, we propose a simpler evaluation protocol and metric that account for subtle variations and provide a more detailed evaluation report of LALMs within the MCQA framework.

Method: RxT processes conversational turns as discrete events, maintaining context in a fixed-size Short-Term Memory system with a generator-decoder for responses and a memory-encoder with Memory Attention for asynchronous memory updates.

Result: RxT reduces computational complexity from quadratic to linear with respect to interactions, achieves low latency, and demonstrates superior performance with constant-time inference latency in proof-of-concept experiments.

Conclusion: RxT enables real-time, stateful, and economically viable long-form conversations by decoupling response generation from memory updates and fundamentally altering the scaling dynamics.

Abstract: The Transformer architecture has become the de facto standard for Large Language Models (LLMs), demonstrating remarkable capabilities in language understanding and generation. However, its application in conversational AI is fundamentally constrained by its stateless nature and the quadratic computational complexity ($O(L^2)$) with respect to sequence length $L$. Current models emulate memory by reprocessing an ever-expanding conversation history with each turn, leading to prohibitive costs and latency in long dialogues. This paper introduces the Reactive Transformer (RxT), a novel architecture designed to overcome these limitations by shifting from a data-driven to an event-driven paradigm. RxT processes each conversational turn as a discrete event in real-time, maintaining context in an integrated, fixed-size Short-Term Memory (STM) system. The architecture features a distinct operational cycle where a generator-decoder produces a response based on the current query and the previous memory state, after which a memory-encoder and a dedicated Memory Attention network asynchronously update the STM with a representation of the complete interaction. This design fundamentally alters the scaling dynamics, reducing the total user-facing cost of a conversation from quadratic ($O(N^2 \cdot T)$) to linear ($O(N \cdot T)$) with respect to the number of interactions $N$. By decoupling response generation from memory updates, RxT achieves low latency, enabling truly real-time, stateful, and economically viable long-form conversations. We validated our architecture with a series of proof-of-concept experiments on synthetic data, demonstrating superior performance and constant-time inference latency compared to a baseline stateless model of comparable size.

Method: Three approaches: (1) GPT-4.1 with in-context learning using guideline summaries and example selection, (2) GLiNER fine-tuned on synthetic data with LLM verification, (3) LLaMA-3.1-8B fine-tuned on synthetic data. Event extraction uses GPT-4.1 with similar ICL strategy.

Result: GPT-4.1 achieved macro-F1 of 61.53% for NER and 15.02% for event extraction, outperforming other methods.

Conclusion: Well-crafted prompting with guideline summaries is crucial for maximizing LLM performance in very low-resource biomedical text processing scenarios.

Abstract: This work presents our participation in the EvalLLM 2025 challenge on biomedical Named Entity Recognition (NER) and health event extraction in French (few-shot setting). For NER, we propose three approaches combining large language models (LLMs), annotation guidelines, synthetic data, and post-processing: (1) in-context learning (ICL) with GPT-4.1, incorporating automatic selection of 10 examples and a summary of the annotation guidelines into the prompt, (2) the universal NER system GLiNER, fine-tuned on a synthetic corpus and then verified by an LLM in post-processing, and (3) the open LLM LLaMA-3.1-8B-Instruct, fine-tuned on the same synthetic corpus. Event extraction uses the same ICL strategy with GPT-4.1, reusing the guideline summary in the prompt. Results show GPT-4.1 leads with a macro-F1 of 61.53% for NER and 15.02% for event extraction, highlighting the importance of well-crafted prompting to maximize performance in very low-resource scenarios.

Method: Deco-G decouples format adherence from task solving using a separate tractable probabilistic model (TPM) for format compliance. It combines next token probabilities from the LLM with TPM-calculated format compliance likelihood at each decoding step. Key innovations include instruction-aware distillation, flexible trie-building algorithm, and HMM state pruning for efficiency.

Result: The approach demonstrates effectiveness across mathematical reasoning, LLM-as-a-judge, and event argument extraction tasks, achieving 1.0% to 6.0% relative performance gains over regular prompting while guaranteeing format compliance.

Conclusion: Explicitly decoupling format adherence from task solving through the Deco-G framework significantly improves LLM performance on complex tasks with rigid formatting requirements, providing a practical and scalable solution for modern instruction-tuned models.

Abstract: Large language models (LLMs) are increasingly adept at following instructions containing task descriptions to solve complex problems, such as mathematical reasoning and automatic evaluation (LLM-as-a-Judge). However, as prompts grow more complex, models often struggle to adhere to all instructions. This difficulty is especially common when instructive prompts intertwine reasoning directives – specifying what the model should solve – with rigid formatting requirements that dictate how the solution must be presented. The entanglement creates competing goals for the model, suggesting that more explicit separation of these two aspects could lead to improved performance. To this front, we introduce Deco-G, a decoding framework that explicitly decouples format adherence from task solving. Deco-G handles format compliance with a separate tractable probabilistic model (TPM), while prompts LLMs with only task instructions. At each decoding step, Deco-G combines next token probabilities from the LLM with the TPM calculated format compliance likelihood to form the output probability. To make this approach both practical and scalable for modern instruction-tuned LLMs, we introduce three key innovations: instruction-aware distillation, a flexible trie-building algorithm, and HMM state pruning for computational efficiency. We demonstrate the effectiveness of Deco-G across a wide range of tasks with diverse format requirements, including mathematical reasoning, LLM-as-a-judge, and event argument extraction. Overall, our approach yields 1.0% to 6.0% relative gain over regular prompting practice with guaranteed format compliance.

Method: Evaluate LLMs on complex reasoning tasks requiring induction of structured relational knowledge (graphs) from noisy natural language content with long contexts and irrelevant information.

Result: LLMs show memory drift and contextual forgetting at much shorter effective lengths than existing benchmarks suggest, even reasoning-specialized models like OpenAI o1 remain vulnerable.

Conclusion: Significant limitations exist in LLMs’ ability to abstract structured knowledge from unstructured input, highlighting need for architectural adaptations to improve long-range reasoning.

Abstract: Recently proposed evaluation benchmarks aim to characterize the effective context length and the forgetting tendencies of large language models (LLMs). However, these benchmarks often rely on simplistic ’needle in a haystack' retrieval or continuation tasks that may not accurately reflect the performance of these models in information-dense scenarios. Thus, rather than simple next token prediction, we argue for evaluating these models on more complex reasoning tasks that requires them to induce structured relational knowledge from the text - such as graphs from potentially noisy natural language content. While the input text can be viewed as generated in terms of a graph, its structure is not made explicit and connections must be induced from distributed textual cues, separated by long contexts and interspersed with irrelevant information. Our findings reveal that LLMs begin to exhibit memory drift and contextual forgetting at much shorter effective lengths when tasked with this form of relational reasoning, compared to what existing benchmarks suggest. With these findings, we offer recommendations for the optimal use of popular LLMs for complex reasoning tasks. We further show that even models specialized for reasoning, such as OpenAI o1, remain vulnerable to early memory drift in these settings. These results point to significant limitations in the models' ability to abstract structured knowledge from unstructured input and highlight the need for architectural adaptations to improve long-range reasoning.

Method: Syllable-level unsupervised speech recognition framework based on masked language modeling, avoiding G2P dependency and GAN-based training instability.

Result: Achieves up to 40% relative reduction in character error rate on LibriSpeech and effectively generalizes to Mandarin, which was particularly difficult for prior methods.

Conclusion: The syllable-level masked language modeling approach provides a more stable and effective solution for unsupervised speech recognition, eliminating resource dependencies and improving performance on challenging languages.

Abstract: Training speech recognizers with unpaired speech and text – known as unsupervised speech recognition (UASR) – is a crucial step toward extending ASR to low-resource languages in the long-tail distribution and enabling multimodal learning from non-parallel data. However, existing approaches based on phones often rely on costly resources such as grapheme-to-phoneme converters (G2Ps) and struggle to generalize to languages with ambiguous phoneme boundaries due to training instability. In this paper, we address both challenges by introducing a syllable-level UASR framework based on masked language modeling, which avoids the need for G2P and the instability of GAN-based methods. Our approach achieves up to a 40% relative reduction in character error rate (CER) on LibriSpeech and generalizes effectively to Mandarin, a language that has remained particularly difficult for prior methods. Code will be released upon acceptance.

Method: Developed a pipeline that extracts and links evidence from text, tables, and figures from real-world PDFs, then generates multimodal QA pairs spanning factual retrieval, comparison, summarization, and logical reasoning queries, with 20% validation by multiple annotators and expert adjudication.

Result: Multimodal text-image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding-based retrieval, indicating that neither text nor images alone are sufficient and that current multimodal embeddings remain inadequate.

Conclusion: The benchmark reveals when and how visual context complements textual evidence, uncovers systematic failure modes, and offers actionable guidance for developing more robust MM-RAG pipelines.

Abstract: Multimodal retrieval-augmented generation (MM-RAG) is a key approach for applying large language models (LLMs) and agents to real-world knowledge bases, yet current evaluations are fragmented, focusing on either text or images in isolation or on simplified multimodal setups that fail to capture document-centric multimodal use cases. In this paper, we introduce UniDoc-Bench, the first large-scale, realistic benchmark for MM-RAG built from 70k real-world PDF pages across eight domains. Our pipeline extracts and links evidence from text, tables, and figures, then generates 1,600 multimodal QA pairs spanning factual retrieval, comparison, summarization, and logical reasoning queries. To ensure reliability, 20% of QA pairs are validated by multiple annotators and expert adjudication. UniDoc-Bench supports apples-to-apples comparison across four paradigms: (1) text-only, (2) image-only, (3) multimodal text-image fusion, and (4) multimodal joint retrieval – under a unified protocol with standardized candidate pools, prompts, and evaluation metrics. Our experiments show that multimodal text-image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding-based retrieval, indicating that neither text nor images alone are sufficient and that current multimodal embeddings remain inadequate. Beyond benchmarking, our analysis reveals when and how visual context complements textual evidence, uncovers systematic failure modes, and offers actionable guidance for developing more robust MM-RAG pipelines.

Method: Translated Roman Urdu-English dataset to English using Google Translate, then fine-tuned multiple LLMs (LLaMA 3 8B, Mistral 7B, etc.) with QLoRA for memory-efficient adaptation on offensive vs non-offensive classification.

Result: Meta LLaMA 3 8B achieved highest F1 score of 91.45, followed by Mistral 7B at 89.66, both surpassing traditional transformer baselines.

Conclusion: QLoRA enables effective fine-tuning of LLMs for low-resource code-mixed offensive language detection, advancing scalable Roman Urdu moderation and multilingual offensive detection systems.

Abstract: The use of derogatory terms in languages that employ code mixing, such as Roman Urdu, presents challenges for Natural Language Processing systems due to unstated grammar, inconsistent spelling, and a scarcity of labeled data. In this work, we propose a QLoRA based fine tuning framework to improve offensive language detection in Roman Urdu-English text. We translated the Roman Urdu-English code mixed dataset into English using Google Translate to leverage English LLMs, while acknowledging that this translation reduces direct engagement with code mixing features. Our focus is on classification performance using English translated low resource inputs. We fine tuned several transformers and large language models, including Meta LLaMA 3 8B, Mistral 7B v0.1, LLaMA 2 7B, ModernBERT, and RoBERTa, with QLoRA for memory efficient adaptation. Models were trained and evaluated on a manually annotated Roman Urdu dataset for offensive vs non offensive content. Of all tested models, the highest F1 score of 91.45 was attained by Meta LLaMA 3 8B, followed by Mistral 7B at 89.66, surpassing traditional transformer baselines. These results demonstrate the efficacy of QLoRA in fine tuning high performing models for low resource environments such as code mixed offensive language detection, and confirm the potential of LLMs for this task. This work advances a scalable approach to Roman Urdu moderation and paves the way for future multilingual offensive detection systems based on LLMs.

Method: Introduced StressTest benchmark and created Stress-17k training set using a novel data generation pipeline that simulates meaning changes through stress variation. Developed StresSLM by finetuning models on this dataset.

Result: Existing SLMs perform poorly on sentence stress tasks despite overall capabilities. StresSLM notably outperforms existing models on both sentence stress reasoning and detection, and generalizes well to real recordings.

Conclusion: Sentence stress understanding is a critical but underdeveloped capability in SLMs. The proposed approach successfully addresses this gap and demonstrates significant improvements in stress-based meaning comprehension.

Abstract: Sentence stress refers to emphasis on words within a spoken utterance to highlight or contrast an idea. It is often used to imply an underlying intention not explicitly stated. Recent speech-aware language models (SLMs) have enabled direct audio processing, allowing models to access the full richness of speech to perform audio reasoning tasks such as spoken question answering. Despite the crucial role of sentence stress in shaping meaning and intent, it remains largely overlooked in evaluation and development of SLMs. We address this gap by introducing StressTest, a benchmark designed to evaluate models’ ability to distinguish between meanings of speech based on the stress pattern. We evaluate leading SLMs, and find that despite their overall capabilities, they perform poorly on such tasks. Hence, we propose a novel data generation pipeline, and create Stress-17k, a training set that simulates change of meaning implied by stress variation. Results suggest, that our finetuned model, StresSLM, generalizes well to real recordings and notably outperforms existing SLMs on sentence stress reasoning and detection. Models, code, data, samples - pages.cs.huji.ac.il/adiyoss-lab/stresstest.

Method: Synergistically combines massive data scaling, large model capacity, LLM integration, and reinforcement learning. Specifically optimized for practical deployment with enhancements in streaming capability, noise robustness, code-switching, hotword customization, and other real-world requirements.

Result: Achieves state-of-the-art performance on real application datasets, demonstrating effectiveness and robustness in practical settings, while outperforming other LLM-based ASR systems on industry evaluation sets.

Conclusion: Fun-ASR successfully addresses LLM hallucination issues and delivers superior performance in real-world ASR applications through production-oriented optimizations, making it suitable for practical deployment across diverse and complex speech recognition scenarios.

Abstract: In recent years, automatic speech recognition (ASR) has witnessed transformative advancements driven by three complementary paradigms: data scaling, model size scaling, and deep integration with large language models (LLMs). However, LLMs are prone to hallucination, which can significantly degrade user experience in real-world ASR applications. In this paper, we present Fun-ASR, a large-scale, LLM-based ASR system that synergistically combines massive data, large model capacity, LLM integration, and reinforcement learning to achieve state-of-the-art performance across diverse and complex speech recognition scenarios. Moreover, Fun-ASR is specifically optimized for practical deployment, with enhancements in streaming capability, noise robustness, code-switching, hotword customization, and satisfying other real-world application requirements. Experimental results show that while most LLM-based ASR systems achieve strong performance on open-source benchmarks, they often underperform on real industry evaluation sets. Thanks to production-oriented optimizations, Fun-ASR achieves state-of-the-art performance on real application datasets, demonstrating its effectiveness and robustness in practical settings.

Method: MedReflect generates a single-pass reflection chain including initial hypothesis generation, self-questioning, self-answering, and decision refinement, enabling self-verified and self-reflective reasoning.

Result: With only 2,000 training examples and light fine-tuning, MedReflect achieves notable accuracy improvements across medical benchmarks while significantly reducing annotation requirements.

Conclusion: LLMs can learn to solve specialized medical problems through self-reflection and self-improvement, reducing reliance on external supervision and extensive task-specific fine-tuning data.

Abstract: Medical problem solving demands expert knowledge and intricate reasoning. Recent studies of large language models (LLMs) attempt to ease this complexity by introducing external knowledge verification through retrieval-augmented generation or by training on reasoning datasets. However, these approaches suffer from drawbacks such as retrieval overhead and high annotation costs, and they heavily rely on substituted external assistants to reach limited performance in medical field. In this paper, we introduce MedReflect, a generalizable framework designed to inspire LLMs with a physician-like reflective thinking mode. MedReflect generates a single-pass reflection chain that includes initial hypothesis generation, self-questioning, self-answering and decision refinement. This self-verified and self-reflective nature releases large language model’s latent capability in medical problem-solving without external retrieval or heavy annotation. We demonstrate that MedReflect enables cost-efficient medical dataset construction: with merely 2,000 randomly sampled training examples and a light fine-tuning, this approach achieves notable absolute accuracy improvements across a series of medical benchmarks while cutting annotation requirements. Our results provide evidence that LLMs can learn to solve specialized medical problems via self-reflection and self-improve, reducing reliance on external supervision and extensive task-specific fine-tuning data.

Method: Created HiKE benchmark with high-quality natural CS data, loanword labels, and hierarchical CS-level labeling (word, phrase, sentence).

Result: Most multilingual ASR models initially show inadequate CS-ASR performance but can be improved through fine-tuning with synthetic CS data.

Conclusion: HiKE enables systematic evaluation of code-switching capabilities in ASR models and demonstrates that CS performance can be enabled through targeted fine-tuning.

Abstract: Despite advances in multilingual automatic speech recognition (ASR), code-switching (CS), the mixing of languages within an utterance common in daily speech, remains a severely underexplored challenge. In this paper, we introduce HiKE: the Hierarchical Korean-English code-switching benchmark, the first globally accessible evaluation framework for Korean-English CS, aiming to provide a means for the precise evaluation of multilingual ASR models and to foster research in the field. The proposed framework not only consists of high-quality, natural CS data across various topics, but also provides meticulous loanword labels and a hierarchical CS-level labeling scheme (word, phrase, and sentence) that together enable a systematic evaluation of a model’s ability to handle each distinct level of code-switching. Through evaluations of diverse multilingual ASR models and fine-tuning experiments, this paper demonstrates that although most multilingual ASR models initially exhibit inadequate CS-ASR performance, this capability can be enabled through fine-tuning with synthetic CS data. HiKE is available at https://github.com/ThetaOne-AI/HiKE

Method: Proposed TreePrompt approach learns LLM preferences to select high-quality, contextually relevant examples within a tree-structured framework, and combines it with K-NN and Adaptive Few-Shot Prompting (AFSP).

Result: Evaluations on English-Persian (MIZAN) and English-German (WMT19) show that integrating TreePrompt with AFSP or Random selection leads to improved translation performance.

Conclusion: TreePrompt effectively balances similarity and quality in example selection, enhancing few-shot machine translation performance when combined with appropriate selection strategies.

Abstract: Large Language Models (LLMs) have consistently demonstrated strong performance in machine translation, especially when guided by high-quality prompts. Few-shot prompting is an effective technique to improve translation quality; however, most existing example selection methods focus solely on query-to-example similarity and do not account for the quality of the examples. In this work, we propose TreePrompt, a novel example selection approach that learns LLM preferences to identify high-quality, contextually relevant examples within a tree-structured framework. To further explore the balance between similarity and quality, we combine TreePrompt with K-Nearest Neighbors (K-NN) and Adaptive Few-Shot Prompting (AFSP). Evaluations on two language pairs - English-Persian (MIZAN) and English-German (WMT19) - show that integrating TreePrompt with AFSP or Random selection leads to improved translation performance.

Method: Used GLOSSGPT prompting method for English WSD and tested across five languages (English, German, Spanish, French, Italian) with GPT-4o and LLaMA-3.1-70B models.

Result: Imbalanced few-shot examples cause incorrect sense predictions in multilingual languages but not in English. Both GPT-4o and LLaMA-3.1-70B show sensitivity to sample distribution in multilingual WSD.

Conclusion: Multilingual Word Sense Disambiguation is highly sensitive to sample distribution in few-shot settings, emphasizing the need for balanced and representative prompting strategies.

Abstract: Recent advances in Large Language Models (LLMs) have significantly reshaped the landscape of Natural Language Processing (NLP). Among the various prompting techniques, few-shot prompting has gained considerable attention for its practicality and effectiveness. This study investigates how few-shot prompting strategies impact the Word Sense Disambiguation (WSD) task, particularly focusing on the biases introduced by imbalanced sample distributions. We use the GLOSSGPT prompting method, an advanced approach for English WSD, to test its effectiveness across five languages: English, German, Spanish, French, and Italian. Our results show that imbalanced few-shot examples can cause incorrect sense predictions in multilingual languages, but this issue does not appear in English. To assess model behavior, we evaluate both the GPT-4o and LLaMA-3.1-70B models and the results highlight the sensitivity of multilingual WSD to sample distribution in few-shot settings, emphasizing the need for balanced and representative prompting strategies.

Method: Automated pipeline using LLMs for segmentation, chain-text separation, validation, and multi-layer enrichment including machine translation into 12 languages, diacritization, abstractive summarization, thematic tagging, and cross-text semantic analysis.

Result: Near-human accuracy in structured tasks (9.33/10 for chain-text separation and summarization), superior to manually curated Noor Corpus (8.46/10 vs 3.66/10), with economic feasibility - tasks requiring 229,000+ expert hours completed within months at fraction of cost.

Conclusion: AI can augment human expertise to enable large-scale, multilingual, semantically enriched access to Islamic heritage, introducing a new paradigm in religious text processing.

Abstract: This paper presents the development of Rezwan, a large-scale AI-assisted Hadith corpus comprising over 1.2M narrations, extracted and structured through a fully automated pipeline. Building on digital repositories such as Maktabat Ahl al-Bayt, the pipeline employs Large Language Models (LLMs) for segmentation, chain–text separation, validation, and multi-layer enrichment. Each narration is enhanced with machine translation into twelve languages, intelligent diacritization, abstractive summarization, thematic tagging, and cross-text semantic analysis. This multi-step process transforms raw text into a richly annotated research-ready infrastructure for digital humanities and Islamic studies. A rigorous evaluation was conducted on 1,213 randomly sampled narrations, assessed by six domain experts. Results show near-human accuracy in structured tasks such as chain–text separation (9.33/10) and summarization (9.33/10), while highlighting ongoing challenges in diacritization and semantic similarity detection. Comparative analysis against the manually curated Noor Corpus demonstrates the superiority of Najm in both scale and quality, with a mean overall score of 8.46/10 versus 3.66/10. Furthermore, cost analysis confirms the economic feasibility of the AI approach: tasks requiring over 229,000 hours of expert labor were completed within months at a fraction of the cost. The work introduces a new paradigm in religious text processing by showing how AI can augment human expertise, enabling large-scale, multilingual, and semantically enriched access to Islamic heritage.

Method: Used mechanistic interpretability to investigate frame locations in model’s hidden representations, identifying singular dimensions that strongly correlate with specific frames in zero-shot settings.

Result: LLMs demonstrate high fluency in generating texts that evoke specific frames and can recognize these frames effectively in zero-shot scenarios.

Conclusion: The research contributes to understanding how LLMs internalize and express meaningful human concepts through identifiable dimensions in their representations.

Abstract: This paper explores the ability of large language models to generate and recognize deep cognitive frames, particularly in socio-political contexts. We demonstrate that LLMs are highly fluent in generating texts that evoke specific frames and can recognize these frames in zero-shot settings. Inspired by mechanistic interpretability research, we investigate the location of the strict father' and nurturing parent’ frames within the model’s hidden representation, identifying singular dimensions that correlate strongly with their presence. Our findings contribute to understanding how LLMs capture and express meaningful human concepts.

Method: Step Pruner uses a step-aware reward function that prioritizes correctness while penalizing redundant steps, withholds rewards for incorrect responses, and implements dynamic stopping when step length exceeds limits to prevent step merging.

Result: SP achieves state-of-the-art accuracy while significantly reducing response length, with 69.7% token reduction on AIME24 benchmark across four reasoning benchmarks.

Conclusion: The step-level approach effectively addresses overthinking in LRMs by focusing on reasoning step efficiency rather than token count, preventing hacking behaviors while maintaining high accuracy.

Abstract: Large Reasoning Models (LRMs) demonstrate strong performance on complex tasks but often suffer from excessive verbosity, known as “overthinking.” Existing solutions via reinforcement learning (RL) typically penalize generated tokens to promote conciseness. However, these methods encounter two challenges: responses with fewer tokens do not always correspond to fewer reasoning steps, and models may develop hacking behavior in later stages of training by discarding reasoning steps to minimize token usage. In this work, we introduce \textbf{Step Pruner (SP)}, an RL framework that steers LRMs toward more efficient reasoning by favoring compact reasoning steps. Our step-aware reward function prioritizes correctness while imposing penalties for redundant steps, and withholds rewards for incorrect responses to prevent the reinforcement of erroneous reasoning. Moreover, we propose a dynamic stopping mechanism: when the length of any output step exceeds the upper limit, we halt updates to prevent hacking behavior caused by merging steps. Extensive experiments across four reasoning benchmarks demonstrate that SP achieves state-of-the-art accuracy while significantly reducing response length. For instance, on AIME24, SP reduces token usage by \textbf{69.7%}.

Method: Used INCEpTION tool for manual annotation and evaluated BERT, DistilBERT, and Logistic Regression models on classifying rhetorical relations (elaboration, contrast, background, cause-effect) in cricket news reports.

Result: DistilBERT achieved the highest accuracy among the models tested, demonstrating strong potential for efficient discourse relation prediction.

Conclusion: This work successfully bridges discourse parsing with transformer-based NLP, showing that DistilBERT is particularly effective for rhetorical relation classification tasks.

Abstract: This research explores the annotation of rhetorical relations in discourse using the INCEpTION tool and compares manual annotation with automatic approaches based on large language models. The study focuses on sports reports (specifically cricket news) and evaluates the performance of BERT, DistilBERT, and Logistic Regression models in classifying rhetorical relations such as elaboration, contrast, background, and cause-effect. The results show that DistilBERT achieved the highest accuracy, highlighting its potential for efficient discourse relation prediction. This work contributes to the growing intersection of discourse parsing and transformer-based NLP. (This paper was conducted as part of an academic requirement under the supervision of Prof. Dr. Ralf Klabunde, Linguistic Data Science Lab, Ruhr University Bochum.) Keywords: Rhetorical Structure Theory, INCEpTION, BERT, DistilBERT, Discourse Parsing, NLP.

Method: Created a labeled dataset of 200 politically significant Bangla news articles and conducted comprehensive evaluation of 28 proprietary and open-source LLMs on stance detection tasks.

Result: LLMs showed strong performance in detecting government-critique content (F1 up to 0.83) but substantial difficulty with neutral articles (F1 as low as 0.00), with models tending to over-predict government-leaning stances.

Conclusion: The dataset and diagnostics provide a foundation for advancing stance detection in Bangla media research and offer insights for improving LLM performance in low-resource languages.

Abstract: Detecting media bias is crucial, specifically in the South Asian region. Despite this, annotated datasets and computational studies for Bangla political bias research remain scarce. Crucially because, political stance detection in Bangla news requires understanding of linguistic cues, cultural context, subtle biases, rhetorical strategies, code-switching, implicit sentiment, and socio-political background. To address this, we introduce the first benchmark dataset of 200 politically significant and highly debated Bangla news articles, labeled for government-leaning, government-critique, and neutral stances, alongside diagnostic analyses for evaluating large language models (LLMs). Our comprehensive evaluation of 28 proprietary and open-source LLMs shows strong performance in detecting government-critique content (F1 up to 0.83) but substantial difficulty with neutral articles (F1 as low as 0.00). Models also tend to over-predict government-leaning stances, often misinterpreting ambiguous narratives. This dataset and its associated diagnostics provide a foundation for advancing stance detection in Bangla media research and offer insights for improving LLM performance in low-resource languages.

Method: Three-stage process: (i) assessing SLMs’ psychological knowledge with PsychoLexEval, (ii) designing PsychoLexTherapy framework with structured memory, (iii) constructing evaluation datasets (PsychoLexQuery and PsychoLexDialogue) and comparing prompting strategies.

Result: Outperformed all baselines in automatic evaluation and human preference studies. Long-term memory module proved essential for multi-turn coherence. Achieved highest ratings in empathy, coherence, cultural fit, and personalization.

Conclusion: Establishes a practical, privacy-preserving, and culturally aligned foundation for Persian psychotherapy simulation with novel datasets and insights into structured memory for therapeutic reasoning.

Abstract: This study presents PsychoLexTherapy, a framework for simulating psychotherapeutic reasoning in Persian using small language models (SLMs). The framework tackles the challenge of developing culturally grounded, therapeutically coherent dialogue systems with structured memory for multi-turn interactions in underrepresented languages. To ensure privacy and feasibility, PsychoLexTherapy is optimized for on-device deployment, enabling use without external servers. Development followed a three-stage process: (i) assessing SLMs psychological knowledge with PsychoLexEval; (ii) designing and implementing the reasoning-oriented PsychoLexTherapy framework; and (iii) constructing two evaluation datasets-PsychoLexQuery (real Persian user questions) and PsychoLexDialogue (hybrid simulated sessions)-to benchmark against multiple baselines. Experiments compared simple prompting, multi-agent debate, and structured therapeutic reasoning paths. Results showed that deliberate model selection balanced accuracy, efficiency, and privacy. On PsychoLexQuery, PsychoLexTherapy outperformed all baselines in automatic LLM-as-a-judge evaluation and was ranked highest by human evaluators in a single-turn preference study. In multi-turn tests with PsychoLexDialogue, the long-term memory module proved essential: while naive history concatenation caused incoherence and information loss, the full framework achieved the highest ratings in empathy, coherence, cultural fit, and personalization. Overall, PsychoLexTherapy establishes a practical, privacy-preserving, and culturally aligned foundation for Persian psychotherapy simulation, contributing novel datasets, a reproducible evaluation pipeline, and empirical insights into structured memory for therapeutic reasoning.

Method: Large language model pipeline analyzing 4.1 million patient reviews of 226,999 U.S. physicians, validated through multi-model comparison and human expert benchmarking.

Result: Strong agreement between human and LLM assessments (correlation 0.72-0.89), systematic patterns showing male physicians receive higher ratings, empathy traits dominate in pediatrics/psychiatry, and identification of four physician archetypes from “Well-Rounded Excellent” to “Underperforming”.

Conclusion: Automated trait extraction from patient narratives provides validated metrics for understanding physician-patient relationships at scale, with implications for quality measurement, bias detection, and healthcare workforce development.

Abstract: Understanding how patients perceive their physicians is essential to improving trust, communication, and satisfaction. We present a large language model (LLM)-based pipeline that infers Big Five personality traits and five patient-oriented subjective judgments. The analysis encompasses 4.1 million patient reviews of 226,999 U.S. physicians from an initial pool of one million. We validate the method through multi-model comparison and human expert benchmarking, achieving strong agreement between human and LLM assessments (correlation coefficients 0.72-0.89) and external validity through correlations with patient satisfaction (r = 0.41-0.81, all p<0.001). National-scale analysis reveals systematic patterns: male physicians receive higher ratings across all traits, with largest disparities in clinical competence perceptions; empathy-related traits predominate in pediatrics and psychiatry; and all traits positively predict overall satisfaction. Cluster analysis identifies four distinct physician archetypes, from “Well-Rounded Excellent” (33.8%, uniformly high traits) to “Underperforming” (22.6%, consistently low). These findings demonstrate that automated trait extraction from patient narratives can provide interpretable, validated metrics for understanding physician-patient relationships at scale, with implications for quality measurement, bias detection, and workforce development in healthcare.

Method: Multi-agent simulation with performer agent completing tasks, supervisor agent evaluating progress and trust, and independent deception auditor reviewing full trajectories.

Result: Deception is model-dependent, increases with event pressure, consistently erodes supervisor trust, and reveals distinct strategies of concealment, equivocation, and falsification.

Conclusion: Deception is an emergent risk in long-horizon LLM interactions, providing foundation for evaluating future models in trust-sensitive contexts.

Abstract: Deception is a pervasive feature of human communication and an emerging concern in large language models (LLMs). While recent studies document instances of LLM deception under pressure, most evaluations remain confined to single-turn prompts and fail to capture the long-horizon interactions in which deceptive strategies typically unfold. We introduce the first simulation framework for probing and evaluating deception in LLMs under extended sequences of interdependent tasks and dynamic contextual pressures. Our framework instantiates a multi-agent system: a performer agent tasked with completing tasks and a supervisor agent that evaluates progress, provides feedback, and maintains evolving states of trust. An independent deception auditor then reviews full trajectories to identify when and how deception occurs. We conduct extensive experiments across 11 frontier models, spanning both closed- and open-source systems, and find that deception is model-dependent, increases with event pressure, and consistently erodes supervisor trust. Qualitative analyses further reveal distinct strategies of concealment, equivocation, and falsification. Our findings establish deception as an emergent risk in long-horizon interactions and provide a foundation for evaluating future LLMs in real-world, trust-sensitive contexts.

Method: Proposed entity knowledge augmentation approach that can be applied to both informal (social media) and formal (biomedical) text formats for named entity recognition.

Result: Experiments on COVID-19 tweets dataset and PubMed dataset show improved NER performance in both fully-supervised and few-shot settings.

Conclusion: The entity knowledge augmentation approach effectively addresses challenges in COVID-19 NER and can be generalized to biomedical named entity recognition tasks.

Abstract: The COVID-19 pandemic causes severe social and economic disruption around the world, raising various subjects that are discussed over social media. Identifying pandemic-related named entities as expressed on social media is fundamental and important to understand the discussions about the pandemic. However, there is limited work on named entity recognition on this topic due to the following challenges: 1) COVID-19 texts in social media are informal and their annotations are rare and insufficient to train a robust recognition model, and 2) named entity recognition in COVID-19 requires extensive domain-specific knowledge. To address these issues, we propose a novel entity knowledge augmentation approach for COVID-19, which can also be applied in general biomedical named entity recognition in both informal text format and formal text format. Experiments carried out on the COVID-19 tweets dataset and PubMed dataset show that our proposed entity knowledge augmentation improves NER performance in both fully-supervised and few-shot settings. Our source code is publicly available: https://github.com/kkkenshi/LLM-EKA/tree/master

Method: Developed Agri-3M-VL corpus using scalable multi-agent data generator, trained AgriGPT-VL via progressive curriculum of textual grounding, multimodal alignment, and GRPO refinement, and established AgriBench-VL-4K evaluation suite with LLM-as-a-judge framework.

Result: AgriGPT-VL outperforms leading general-purpose VLMs on AgriBench-VL-4K with higher pairwise win rates, while remaining competitive on text-only AgriBench-13K without language ability degradation.

Conclusion: The framework demonstrates strong multimodal reasoning capabilities for agriculture while preserving text-only performance, with ablation studies confirming consistent gains from alignment and GRPO refinement stages.

Abstract: Despite rapid advances in multimodal large language models, agricultural applications remain constrained by the scarcity of domain-tailored models, curated vision-language corpora, and rigorous evaluation. To address these challenges, we present the AgriGPT-VL Suite, a unified multimodal framework for agriculture. Our contributions are threefold. First, we introduce Agri-3M-VL, the largest vision-language corpus for agriculture to our knowledge, curated by a scalable multi-agent data generator; it comprises 1M image-caption pairs, 2M image-grounded VQA pairs, 50K expert-level VQA instances, and 15K GRPO reinforcement learning samples. Second, we develop AgriGPT-VL, an agriculture-specialized vision-language model trained via a progressive curriculum of textual grounding, multimodal shallow/deep alignment, and GRPO refinement. This method achieves strong multimodal reasoning while preserving text-only capability. Third, we establish AgriBench-VL-4K, a compact yet challenging evaluation suite with open-ended and image-grounded questions, paired with multi-metric evaluation and an LLM-as-a-judge framework. Experiments show that AgriGPT-VL outperforms leading general-purpose VLMs on AgriBench-VL-4K, achieving higher pairwise win rates in the LLM-as-a-judge evaluation. Meanwhile, it remains competitive on the text-only AgriBench-13K with no noticeable degradation of language ability. Ablation studies further confirm consistent gains from our alignment and GRPO refinement stages. We will open source all of the resources to support reproducible research and deployment in low-resource agricultural settings.

Method: Operationalize interpretability methods to predict correctness from model activations alone. Use simple classifier on intermediate layer activations of first output token. Introduce metrics to distinguish correct, incorrect, and irrelevant context.

Result: Classifier achieves ~75% accuracy predicting output correctness from activations. Model-internals-based metric significantly outperforms prompting baselines at distinguishing correct vs incorrect context.

Conclusion: Model internals contain signals about output correctness and context efficacy, offering insights into LLM decision-making processes and enabling better trustworthiness assessment.

Abstract: Although large language models (LLMs) have tremendous utility, trustworthiness is still a chief concern: models often generate incorrect information with high confidence. While contextual information can help guide generation, identifying when a query would benefit from retrieved context and assessing the effectiveness of that context remains challenging. In this work, we operationalize interpretability methods to ascertain whether we can predict the correctness of model outputs from the model’s activations alone. We also explore whether model internals contain signals about the efficacy of external context. We consider correct, incorrect, and irrelevant context and introduce metrics to distinguish amongst them. Experiments on six different models reveal that a simple classifier trained on intermediate layer activations of the first output token can predict output correctness with about 75% accuracy, enabling early auditing. Our model-internals-based metric significantly outperforms prompting baselines at distinguishing between correct and incorrect context, guarding against inaccuracies introduced by polluted context. These findings offer a lens to better understand the underlying decision-making processes of LLMs. Our code is publicly available at https://github.com/jiarui-liu/LLM-Microscope

Method: Used transcribed subtitles from YODAS corpus with Thai-specific linguistic cues (sentence-final particles), formulated EOT as binary decision over token boundaries, compared zero-shot/few-shot prompting of compact LLMs with supervised fine-tuning of lightweight transformers.

Result: Clear accuracy-latency tradeoff observed, with small fine-tuned models capable of delivering near-instant EOT decisions suitable for on-device agents.

Conclusion: Established Thai baseline and demonstrated that small, fine-tuned models can provide near-instant end-of-turn detection for real-time voice agents.

Abstract: Fluid voice-to-voice interaction requires reliable and low-latency detection of when a user has finished speaking. Traditional audio-silence end-pointers add hundreds of milliseconds of delay and fail under hesitations or language-specific phenomena. We present, to our knowledge, the first systematic study of Thai text-only end-of-turn (EOT) detection for real-time agents. We compare zero-shot and few-shot prompting of compact LLMs to supervised fine-tuning of lightweight transformers. Using transcribed subtitles from the YODAS corpus and Thai-specific linguistic cues (e.g., sentence-final particles), we formulate EOT as a binary decision over token boundaries. We report a clear accuracy-latency tradeoff and provide a public-ready implementation plan. This work establishes a Thai baseline and demonstrates that small, fine-tuned models can deliver near-instant EOT decisions suitable for on-device agents.

Method: Proposed decision changing rate framework that quantifies word importance by incorporating counterfactuals into LLM reasoning.

Result: Experimental results demonstrate that using counterfactuals can be helpful for identifying important words in classification decisions.

Conclusion: Counterfactuals provide an effective approach for explaining LLM classification decisions under practical constraints.

Abstract: Large language models (LLMs) are becoming useful in many domains due to their impressive abilities that arise from large training datasets and large model sizes. More recently, they have been shown to be very effective in textual classification tasks, motivating the need to explain the LLMs’ decisions. Motivated by practical constrains where LLMs are black-boxed and LLM calls are expensive, we study how incorporating counterfactuals into LLM reasoning can affect the LLM’s ability to identify the top words that have contributed to its classification decision. To this end, we introduce a framework called the decision changing rate that helps us quantify the importance of the top words in classification. Our experimental results show that using counterfactuals can be helpful.

Method: Comprehensive benchmark using MedMCQA, MedQA-4Options, and PubMedQA datasets to evaluate SLMs trained on mixed general-domain and medical corpora, with medical abstracts simulating real ED physician tasks.

Result: General-domain SLMs outperformed medically fine-tuned counterparts across all benchmarks, suggesting specialized medical fine-tuning may not be necessary for ED applications.

Conclusion: For emergency department decision support, general-domain small language models are sufficient and specialized medical fine-tuning may not provide additional benefits.

Abstract: Large language models (LLMs) have become increasingly popular in medical domains to assist physicians with a variety of clinical and operational tasks. Given the fast-paced and high-stakes environment of emergency departments (EDs), small language models (SLMs), characterized by a reduction in parameter count compared to LLMs, offer significant potential due to their inherent reasoning capability and efficient performance. This enables SLMs to support physicians by providing timely and accurate information synthesis, thereby improving clinical decision-making and workflow efficiency. In this paper, we present a comprehensive benchmark designed to identify SLMs suited for ED decision support, taking into account both specialized medical expertise and broad general problem-solving capabilities. In our evaluations, we focus on SLMs that have been trained on a mixture of general-domain and medical corpora. A key motivation for emphasizing SLMs is the practical hardware limitations, operational cost constraints, and privacy concerns in the typical real-world deployments. Our benchmark datasets include MedMCQA, MedQA-4Options, and PubMedQA, with the medical abstracts dataset emulating tasks aligned with real ED physicians’ daily tasks. Experimental results reveal that general-domain SLMs surprisingly outperform their medically fine-tuned counterparts across these diverse benchmarks for ED. This indicates that for ED, specialized medical fine-tuning of the model may not be required.

Method: Explored several CoT approaches: CoT prompting, fine-tuning on human-authored CoT, fine-tuning on synthetic explanations, and Reinforcement Learning with Verifiable Rewards (RLVR). Evaluated on Value Kaleidoscope and OpinionQA datasets.

Result: RLVR consistently outperformed other methods and demonstrated strong training sample efficiency. The generated CoT traces were analyzed for faithfulness and safety.

Conclusion: Chain-of-Thought reasoning techniques, particularly RLVR, can effectively build steerable pluralistic models that can adopt specific perspectives while maintaining faithfulness and safety.

Abstract: Large Language Models (LLMs) are typically trained to reflect a relatively uniform set of values, which limits their applicability to tasks that require understanding of nuanced human perspectives. Recent research has underscored the importance of enabling LLMs to support steerable pluralism – the capacity to adopt a specific perspective and align generated outputs with it. In this work, we investigate whether Chain-of-Thought (CoT) reasoning techniques can be applied to building steerable pluralistic models. We explore several methods, including CoT prompting, fine-tuning on human-authored CoT, fine-tuning on synthetic explanations, and Reinforcement Learning with Verifiable Rewards (RLVR). We evaluate these approaches using the Value Kaleidoscope and OpinionQA datasets. Among the methods studied, RLVR consistently outperforms others and demonstrates strong training sample efficiency. We further analyze the generated CoT traces with respect to faithfulness and safety.

Method: Performed extensive ablation experiments to disentangle the sources of data efficiency in diffusion language models.

Result: Random masking of input tokens plays the dominant role in data efficiency, and similar gains can be obtained through MLP dropout and weight decay.

Conclusion: Stochastic regularization broadly enhances data efficiency in multi-epoch training for language models.

Abstract: Recent studies have shown that diffusion language models achieve remarkable data efficiency under limited-data constraints, yet the underlying mechanisms remain unclear. In this work, we perform extensive ablation experiments to disentangle the sources of this efficiency. Our results show that random masking of input tokens plays the dominant role. We further show that similar gains can be obtained through in MLP dropout and weight decay, indicating that stochastic regularization broadly enhances data efficiency in multi-epoch training. Our code is available at https://github.com/zitian-gao/data-efficiency.

Method: PoLi-RL uses a two-stage curriculum: first training with pointwise rewards for basic scoring, then transitioning to hybrid rewards combining pointwise, pairwise, and listwise objectives. It introduces Parallel Slice Ranking Reward (PSRR) that computes ranking rewards in parallel slices for granular credit assignment.

Result: PoLi-RL achieves Spearman correlation coefficient of 48.18 on the official C-STS benchmark, establishing new SOTA for cross-encoder architecture.

Conclusion: As the first successful application of RL to C-STS, PoLi-RL introduces a powerful paradigm for training LLMs on complex, ranking-based conditional judgment tasks with precise optimization.

Abstract: Conditional Semantic Textual Similarity (C-STS) measures the semantic proximity between text segments under a specific condition, thereby overcoming the ambiguity inherent in traditional STS. However, existing methods are largely confined to discriminative models, failing to fully integrate recent breakthroughs in the NLP community concerning Large Language Models (LLMs) and Reinforcement Learning (RL). RL is a particularly well-suited paradigm for this task, as it can directly optimize the non-differentiable Spearman ranking metric and guide the reasoning process required by C-STS. However, we find that naively applying listwise RL fails to produce meaningful improvements, as the model is overwhelmed by complex, coarse-grained reward signals. To address this challenge, we introduce PoLi-RL, a novel Point-to-List Reinforcement Learning framework. PoLi-RL employs a two-stage curriculum: it first trains the model with simple pointwise rewards to establish fundamental scoring capabilities, then transitions to a hybrid reward that combines pointwise, pairwise, and listwise objectives to refine the model’s ability to discern subtle semantic distinctions. Crucially, we propose an innovative Parallel Slice Ranking Reward (PSRR) mechanism that computes ranking rewards in parallel slices, where each slice comprises same-indexed completions from different samples. This provides a precise, differentiated learning signal for each individual completion, enabling granular credit assignment and effective optimization. On the official C-STS benchmark, PoLi-RL achieves a Spearman correlation coefficient of 48.18, establishing a new SOTA for the cross-encoder architecture. As the first work to successfully apply RL to C-STS, our study introduces a powerful and precise paradigm for training LLMs on complex, ranking-based conditional judgment tasks.

Method: Fine-tunes a code-based CoT generator on existing solutions in unified code format, then scales data generation with automated validation via code execution and rule-based filtering, followed by reverse-engineering into natural language instructions.

Result: Experiments on Caco-1.3M dataset show Caco-trained models achieve strong competitive performance on mathematical reasoning benchmarks, outperforming existing baselines with superior generalization across unseen tasks.

Conclusion: Caco establishes a paradigm for building self-sustaining, trustworthy reasoning systems through fully automated, scalable synthesis of reasoning data with guaranteed executability.

Abstract: Reasoning capability is pivotal for Large Language Models (LLMs) to solve complex tasks, yet achieving reliable and scalable reasoning remains challenging. While Chain-of-Thought (CoT) prompting has become a mainstream approach, existing methods often suffer from uncontrolled generation, insufficient quality, and limited diversity in reasoning paths. Recent efforts leverage code to enhance CoT by grounding reasoning in executable steps, but such methods are typically constrained to predefined mathematical problems, hindering scalability and generalizability. In this work, we propose Caco (Code-Assisted Chain-of-ThOught), a novel framework that automates the synthesis of high-quality, verifiable, and diverse instruction-CoT reasoning data through code-driven augmentation. Unlike prior work, Caco first fine-tunes a code-based CoT generator on existing math and programming solutions in a unified code format, then scales the data generation to a large amount of diverse reasoning traces. Crucially, we introduce automated validation via code execution and rule-based filtering to ensure logical correctness and structural diversity, followed by reverse-engineering filtered outputs into natural language instructions and language CoTs to enrich task adaptability. This closed-loop process enables fully automated, scalable synthesis of reasoning data with guaranteed executability. Experiments on our created Caco-1.3M dataset demonstrate that Caco-trained models achieve strong competitive performance on mathematical reasoning benchmarks, outperforming existing strong baselines. Further analysis reveals that Caco’s code-anchored verification and instruction diversity contribute to superior generalization across unseen tasks. Our work establishes a paradigm for building self-sustaining, trustworthy reasoning systems without human intervention.

Method: Three-pronged approach: (1) Concept Mapping using embedding space projections, (2) Metaphor-Literal Repository analysis of metaphorical words and literal counterparts, (3) Syntactic Sensitivity assessment of metaphorical structures.

Result: LLMs generate 15-25% conceptually irrelevant interpretations, depend on training data indicators rather than contextual cues, and are more sensitive to syntactic irregularities than structural comprehension.

Conclusion: LLMs have significant limitations in metaphor analysis, highlighting the need for more robust computational approaches to metaphor processing.

Abstract: Metaphor analysis is a complex linguistic phenomenon shaped by context and external factors. While Large Language Models (LLMs) demonstrate advanced capabilities in knowledge integration, contextual reasoning, and creative generation, their mechanisms for metaphor comprehension remain insufficiently explored. This study examines LLMs’ metaphor-processing abilities from three perspectives: (1) Concept Mapping: using embedding space projections to evaluate how LLMs map concepts in target domains (e.g., misinterpreting “fall in love” as “drop down from love”); (2) Metaphor-Literal Repository: analyzing metaphorical words and their literal counterparts to identify inherent metaphorical knowledge; and (3) Syntactic Sensitivity: assessing how metaphorical syntactic structures influence LLMs’ performance. Our findings reveal that LLMs generate 15%-25% conceptually irrelevant interpretations, depend on metaphorical indicators in training data rather than contextual cues, and are more sensitive to syntactic irregularities than to structural comprehension. These insights underline the limitations of LLMs in metaphor analysis and call for more robust computational approaches.

Method: Created a data collection pipeline from various Sri Lankan sources including parliamentary proceedings, legal judgments, government publications, news, and tourism statistics in Sinhala, Tamil, and English.

Result: Successfully compiled 13 datasets totaling 215,670 documents (60.3 GB) as of v20251005, with daily updates and availability on multiple platforms.

Conclusion: These resources fill a gap in multilingual datasets for Sri Lankan languages and enable diverse research applications while addressing licensing and ethical considerations.

Abstract: We present a collection of open, machine-readable document datasets covering parliamentary proceedings, legal judgments, government publications, news, and tourism statistics from Sri Lanka. As of v20251005, the collection currently comprises 215,670 documents (60.3 GB) across 13 datasets in Sinhala, Tamil, and English. The datasets are updated daily and mirrored on GitHub and Hugging Face. These resources aim to support research in computational linguistics, legal analytics, socio-political studies, and multilingual natural language processing. We describe the data sources, collection pipeline, formats, and potential use cases, while discussing licensing and ethical considerations.

Method: Developed a generalizable method for preparing culturally relevant datasets and applied post-training techniques to the Gemma 2 model to enhance its performance for underrepresented languages.

Result: The approach successfully increased Gemma 2’s performance for an underrepresented language, demonstrating the feasibility of adapting LLMs to diverse cultural contexts.

Conclusion: This method provides a replicable framework for others to unlock Generative AI’s potential in their countries while preserving cultural heritage, promoting more inclusive AI development.

Abstract: The rise of Large Language Models has not been inclusive of all cultures. The models are mostly trained on English texts and culture which makes them underperform in other languages and cultural contexts. By developing a generalizable method for preparing culturally relevant datasets and post-training the Gemma 2 model, this project aimed to increase the performance of Gemma 2 for an underrepresented language and showcase how others can do the same to unlock the power of Generative AI in their country and preserve their cultural heritage.

Method: Proposes Self Speculative Decoding (SSD) that uses the dLLM itself as both drafter and verifier, generating predictions for multiple positions and verifying them through hierarchical verification trees in a single forward pass.

Result: SSD achieves up to 3.46× speedup on models like LLaDA and Dream while keeping output identical to stepwise decoding.

Conclusion: SSD provides an efficient lossless acceleration method for dLLMs by eliminating model redundancy and memory overhead through self-speculative decoding.

Abstract: Diffusion-based Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive models, offering unique advantages through bidirectional attention and parallel generation paradigms. However, the generation results of current parallel decoding methods deviate from stepwise decoding, introducing potential performance degradation, which limits their practical deployment. To address this problem, we propose \textbf{S}elf \textbf{S}peculative \textbf{D}ecoding (SSD), a lossless inference acceleration method that leverages the dLLM itself as both speculative decoding drafter and verifier without auxiliary modules. SSD introduces a self-drafting mechanism where the model generates predictions for multiple positions, then verifies them through hierarchical verification trees in a single forward pass. Unlike traditional speculative decoding that requires separate draft models, SSD eliminates model redundancy and memory overhead by exploiting the dLLM’s inherent parallel prediction capability for multiple positions. This self-speculative approach allows the model to progressively verify and accept multiple tokens in a single forward pass. Our experiments demonstrate that SSD achieves up to 3.46$\times$ speedup while keeping the output identical to stepwise decoding on open source models such as LLaDA and Dream. Code will be made publicly available on GitHub.

Method: LTPO treats intermediate latent thought vectors as dynamic parameters optimized per problem instance using online policy gradient methods guided by intrinsic confidence-based reward signals from the frozen LLM’s output distributions.

Result: LTPO matches or surpasses strong baselines on standard tasks and shows remarkable robustness where others fail, achieving substantial improvements on highly challenging AIME benchmarks where existing latent reasoning baselines collapse to near-zero accuracy.

Conclusion: LTPO demonstrates unique capability for complex reasoning by optimizing latent thoughts at test time without parameter updates, providing a robust solution for challenging reasoning tasks.

Abstract: Recent advancements in Large Language Models (LLMs) have shifted from explicit Chain-of-Thought (CoT) reasoning to more efficient latent reasoning, where intermediate thoughts are represented as vectors rather than text. However, latent reasoning can be brittle on challenging, out-of-distribution tasks where robust reasoning is most critical. To overcome these limitations, we introduce Latent Thought Policy Optimization (LTPO), a parameter-free framework that enhances LLM reasoning entirely at test time, without requiring model parameter updates. LTPO treats intermediate latent “thought” vectors as dynamic parameters that are actively optimized for each problem instance. It employs an online policy gradient method guided by an intrinsic, confidence-based reward signal computed directly from the frozen LLM’s own output distributions, eliminating the need for external supervision or expensive text generation during optimization. Extensive experiments on five reasoning benchmarks show that LTPO not only matches or surpasses strong baselines on standard tasks but also demonstrates remarkable robustness where others fail. Most notably, on highly challenging AIME benchmarks where existing latent reasoning baselines collapse to near-zero accuracy, LTPO delivers substantial improvements, showcasing a unique capability for complex reasoning.

Method: CALM framework with expert interventions identifying reasoning flaws and providing corrective hints, followed by supervised fine-tuning and reinforcement learning.

Result: STORM model achieves 68.9% average accuracy across five optimization benchmarks, matching 671B LRM performance with only 4B parameters.

Conclusion: Dynamic hint-based data synthesis preserves and amplifies LRMs’ native reasoning patterns, offering effective path to expert-level optimization modeling.

Abstract: Large Reasoning Models (LRMs) have demonstrated strong capabilities in complex multi-step reasoning, opening new opportunities for automating optimization modeling. However, existing domain adaptation methods, originally designed for earlier instruction-tuned models, often fail to exploit the advanced reasoning patterns of modern LRMs – In particular, we show that direct fine-tuning on traditional \textit{non-reflective} datasets leads to limited gains. To fully leverage LRMs’ inherent reasoning abilities, we propose \textbf{CALM} (\textit{Corrective Adaptation with Lightweight Modification}), a framework that progressively refines LRMs within their native reasoning modes for optimization modeling tasks. In CALM, an expert intervener identifies reasoning flaws and provides concise corrective hints, which the LRM incorporates to produce improved reasoning trajectories. These interventions modify fewer than 2.6% of generated tokens, but generate high-quality data for soft adaptation through supervised fine-tuning. The adapted model is then further improved through reinforcement learning. Building on CALM, we develop \textbf{STORM} (\textit{Smart Thinking Optimization Reasoning Model}), a 4B-parameter LRM that achieves a new state-of-the-art average accuracy of 68.9% across five popular optimization modeling benchmarks, matching the performance of a 671B LRM. These results demonstrate that dynamic, hint-based data synthesis both preserves and amplifies the native reasoning patterns of modern LRMs, offering a more effective and scalable path towards expert-level performance on challenging optimization modeling tasks.

Method: Proposed Reward-Enhanced Policy Optimization (REPO) that combines heterogeneous rewards: preference-trained reward model for human alignment, reward judge for persuasive behavior and SOP compliance, and programmatic reward functions for deterministic checks on numerics, formatting, and guardrails.

Result: REPO achieved average dialogue rating of 4.63 (+1.20 over base, +0.83 over DPO), increased excellent response conversations to 66.67% (+23.34 pp over GRPO), and achieved 93.33% bad-case fix rate with 75.56% clean fixes, outperforming SFT, DPO, PPO, and GRPO.

Conclusion: REPO effectively aligns LLMs for persuasive negotiation while enforcing business constraints, demonstrating emergent capabilities like proactive empathy and calibrated tactics that surpass gold annotations.

Abstract: We study deploying large language models (LLMs) as business development (BD) agents for persuasive price negotiation in online travel agencies (OTAs), where aligning traveler affordability and hotel profitability directly affects bookings, partner relationships, and access to travel. The agent must follow a Standard Operating Procedure (SOP) while conducting multi-turn persuasion, interpreting colloquial inputs, and adhering to guardrails (no over-promising, no hallucinations). Conventional post-training – supervised fine-tuning (SFT) or single-source reward optimization – overfits scripts, misses nuanced persuasive style, and fails to enforce verifiable business constraints. We propose Reward-Enhanced Policy Optimization (REPO), a reinforcement learning post-training framework that aligns an LLM with heterogeneous rewards: a preference-trained reward model (RM) for dense human alignment, a reward judge (RJ) for high-level persuasive behavior and SOP compliance, and programmatic reward functions (RF) for deterministic checks on numerics, formatting, and guardrails. A straightforward enhancement mechanism is proposed to combine the RM with RJ and RF signals to curb reward hacking and improve negotiation quality. In production-style evaluations – approximately 150 turns from real dialogues and 225 turns from curated bad-case dialogues – REPO lifts average dialogue rating to 4.63: +1.20 over base, +0.83 over Direct Preference Optimization (DPO); +0.33 over Group Relative Policy Optimization (GRPO), increases the share of conversations with at least one excellent response to 66.67% (+23.34 percentage points over GRPO), and achieves a 93.33% bad-case fix rate with 75.56% clean fixes, outperforming SFT, DPO, PPO, and GRPO. We also observe emergent capabilities – proactive empathy, localized reasoning, calibrated tactics – that surpass gold annotations.

Method: Developed methodology to measure epistemic diversity (variation in real-world claims). Tested 27 LLMs on 155 topics covering 12 countries using 200 prompt variations from real user chats. Compared against web searches and Wikipedia.

Result: Newer models generate more diverse claims but nearly all are less epistemically diverse than basic web search. Model size negatively impacts diversity. RAG positively impacts diversity, but improvement varies by cultural context. Country-specific claims reflect English language more than local languages.

Conclusion: LLMs show concerning epistemic homogenization compared to traditional knowledge sources. Cultural context significantly affects knowledge representation, with English dominance in country-specific claims highlighting gaps in epistemic representation.

Abstract: Large language models (LLMs) tend to generate lexically, semantically, and stylistically homogenous texts. This poses a risk of knowledge collapse, where homogenous LLMs mediate a shrinking in the range of accessible information over time. Existing works on homogenization are limited by a focus on closed-ended multiple-choice setups or fuzzy semantic features, and do not look at trends across time and cultural contexts. To overcome this, we present a new methodology to measure epistemic diversity, i.e., variation in real-world claims in LLM outputs, which we use to perform a broad empirical study of LLM knowledge collapse. We test 27 LLMs, 155 topics covering 12 countries, and 200 prompt variations sourced from real user chats. For the topics in our study, we show that while newer models tend to generate more diverse claims, nearly all models are less epistemically diverse than a basic web search. We find that model size has a negative impact on epistemic diversity, while retrieval-augmented generation (RAG) has a positive impact, though the improvement from RAG varies by the cultural context. Finally, compared to a traditional knowledge source (Wikipedia), we find that country-specific claims reflect the English language more than the local one, highlighting a gap in epistemic representation

Method: Develop Language-Mixed CoT reasoning schema, curate Yi-Sang dataset (5.79M Korean prompts with 3.7M reasoning traces), train models across multiple families (4B-35B parameters).

Result: Best model KO-REAson-35B achieves state-of-the-art with highest overall average score (64.0 ± 25), ranking first on 5/9 benchmarks. Smaller models show average improvement of +18.6 points across nine benchmarks.

Conclusion: Language-Mixed CoT is more effective than monolingual CoT and provides cross-lingual and multimodal performance gains. The approach advances research on language-specific reasoning.

Abstract: Recent frontier models employ long chain-of-thought reasoning to explore solution spaces in context and achieve stonger performance. While many works study distillation to build smaller yet capable models, most focus on English and little is known about language-specific reasoning. To bridge this gap, we first introduct Language-Mixed CoT, a reasoning schema that switches between English and a target language, using English as an anchor to excel in reasoning while minimizing translation artificats. As a Korean case study, we curate Yi-Sang: 5.79M native-Korean prompts from web Q&A, exams, STEM, and code; 3.7M long reasoning traces generated from Qwen3-32B; and a targeted 260k high-yield subset. We train ninve models (4B-35B) across six families (Qwen2.5, Llama-3.1, Gemma-3, etc). Our best model, KO-REAson-35B, achieves state-of-the-art performance, with the highest overall average score (64.0 \pm 25), ranking first on 5/9 benchmarks and second on the remainder. Samller and mid-sized models also benefit substantially, with an average improvement of +18.6 points across teh evaluated nine benchmarks. Ablations show Language-Mixed CoT is more effective than monolingual CoT, also resulting in cross-lingual and mult-modal performance gains. We release our data-curation pipeline, evaluation system, datasets, and models to advance research on language-specific reasoning. Data and model collection: https://huggingface.co/KOREAson.

Method: Created LT-Swap benchmark with pretraining corpus-specific test sets containing acceptable vs unacceptable sentence pairs that isolate rare word usage. Evaluated 16 BabyLM models zero-shot by computing average log probabilities for sentence pairs.

Result: Language models perform poorly on rare words. Performance differences across architectures are much more pronounced in the long tail than in common words, revealing which architectures handle rare word generalization better.

Conclusion: LT-Swap provides new insights into language model capabilities for rare word learning, showing that architectural differences matter significantly more for tail distribution performance than for head distribution.

Abstract: Children learn to speak with a low amount of data and can be taught new words on a few-shot basis, making them particularly data-efficient learners. The BabyLM challenge aims at exploring language model (LM) training in the low-data regime but uses metrics that concentrate on the head of the word distribution. Here, we introduce LongTail-Swap (LT-Swap), a benchmark that focuses on the tail of the distribution, i.e., measures the ability of LMs to learn new words with very little exposure, like infants do. LT-Swap is a pretraining corpus-specific test set of acceptable versus unacceptable sentence pairs that isolate semantic and syntactic usage of rare words. Models are evaluated in a zero-shot fashion by computing the average log probabilities over the two members of each pair. We built two such test sets associated with the 10M words and 100M words BabyLM training sets, respectively, and evaluated 16 models from the BabyLM leaderboard. Our results not only highlight the poor performance of language models on rare words but also reveal that performance differences across LM architectures are much more pronounced in the long tail than in the head. This offers new insights into which architectures are better at handling rare word generalization. We’ve also made the code publicly avail

Method: Treating softmax entropy as perturbation around center distribution to derive closed-form cumulant observables that isolate higher-order correlations, applied to GPT-2 and Pythia models on Pile-10K prompts.

Result: Structured prompts show rise-and-plateau cumulant profiles, training reveals monotonic increase of cumulants from variance to higher-order structures, and mathematical prompts exhibit distinct signatures from general text.

Conclusion: Cumulant analysis provides a lightweight, mathematically grounded method for probing feature-learning dynamics in high-dimensional neural networks.

Abstract: We introduce a cumulant-expansion framework for quantifying how large language models (LLMs) internalize higher-order statistical structure during next-token prediction. By treating the softmax entropy of each layer’s logit distribution as a perturbation around its “center” distribution, we derive closed-form cumulant observables that isolate successively higher-order correlations. Empirically, we track these cumulants in GPT-2 and Pythia models on Pile-10K prompts. (i) Structured prompts exhibit a characteristic rise-and-plateau profile across layers, whereas token-shuffled prompts remain flat, revealing the dependence of the cumulant profile on meaningful context. (ii) During training, all cumulants increase monotonically before saturating, directly visualizing the model’s progression from capturing variance to learning skew, kurtosis, and higher-order statistical structures. (iii) Mathematical prompts show distinct cumulant signatures compared to general text, quantifying how models employ fundamentally different processing mechanisms for mathematical versus linguistic content. Together, these results establish cumulant analysis as a lightweight, mathematically grounded probe of feature-learning dynamics in high-dimensional neural networks.

Method: Partition d-dimensional embeddings into S slices, use lightweight shared router for top-k expert selection per slice, operate experts independently on assigned slices, and reassemble outputs with slice-level capacity loss and cross-slice dropout.

Result: 1.7x faster inference than dense baselines, 12-18% lower perplexity than token-MoE, improved expert balance, and interpretable expertise over syntactic vs semantic subspaces.

Conclusion: SliceMoE effectively addresses MoE limitations through slice-level routing, achieving superior performance, efficiency, and interpretable expert specialization.

Abstract: Mixture-of-Experts (MoE) layers scale transformers by routing tokens to a sparse subset of feed-forward experts. Token-level routing, however, assigns an entire semantic spectrum to each expert, creating capacity bottlenecks, load-balancing pathologies, and limited specialization. We introduce SliceMoE, an architecture that routes contiguous slices of a token’s hidden vector. A d-dimensional embedding is partitioned into S slices, and for each slice, a lightweight shared router predicts the top-k experts. Experts operate on their assigned slices independently, and outputs are reassembled, maintaining per-token FLOP efficiency. Because slices from different tokens interleave within an expert, utilization is naturally smoother. We propose a slice-level capacity loss, cross-slice dropout, and efficient fused batched GEMM kernels. Experiments on WikiText-103 language modeling, WMT En-De translation, and three text-classification datasets show SliceMoE attains up to 1.7x faster inference than dense baselines, 12 to 18 percent lower perplexity than parameter-matched token-MoE, and improved expert balance, with interpretable expertise over syntactic versus semantic subspaces.

Method: Hybrid approach combining machine learning and deep learning, using multilingual BERT polarity scores as features in decision tree classifier, plus Persian synonym/entity dictionary for text augmentation.

Result: Achieved 93.34% accuracy on Pars-ABSA dataset, surpassing existing benchmarks for Persian ABSA.

Conclusion: Hybrid modeling and feature augmentation effectively advance sentiment analysis for low-resource languages like Persian.

Abstract: Sentiment analysis is a key task in Natural Language Processing (NLP), enabling the extraction of meaningful insights from user opinions across various domains. However, performing sentiment analysis in Persian remains challenging due to the scarcity of labeled datasets, limited preprocessing tools, and the lack of high-quality embeddings and feature extraction methods. To address these limitations, we propose a hybrid approach that integrates machine learning (ML) and deep learning (DL) techniques for Persian aspect-based sentiment analysis (ABSA). In particular, we utilize polarity scores from multilingual BERT as additional features and incorporate them into a decision tree classifier, achieving an accuracy of 93.34%-surpassing existing benchmarks on the Pars-ABSA dataset. Additionally, we introduce a Persian synonym and entity dictionary, a novel linguistic resource that supports text augmentation through synonym and named entity replacement. Our results demonstrate the effectiveness of hybrid modeling and feature augmentation in advancing sentiment analysis for low-resource languages such as Persian.

Method: Proposes Retrieve-DocumentRoute-Read (RDR2) framework with LLM-based router that dynamically navigates document structure trees, jointly evaluating content relevance and hierarchical relationships to assemble optimal evidence.

Result: Achieves state-of-the-art performance on five challenging datasets, demonstrating significant enhancement in RAG systems’ ability to acquire and utilize knowledge.

Conclusion: Explicit structural awareness significantly enhances RAG systems, particularly in complex scenarios requiring multi-document synthesis.

Abstract: While large language models (LLMs) demonstrate impressive capabilities, their reliance on parametric knowledge often leads to factual inaccuracies. Retrieval-Augmented Generation (RAG) mitigates this by leveraging external documents, yet existing approaches treat retrieved passages as isolated chunks, ignoring valuable structure that is crucial for document organization. Motivated by this gap, we propose Retrieve-DocumentRoute-Read (RDR2), a novel framework that explicitly incorporates structural information throughout the RAG process. RDR2 employs an LLM-based router to dynamically navigate document structure trees, jointly evaluating content relevance and hierarchical relationships to assemble optimal evidence. Our key innovation lies in formulating document routing as a trainable task, with automatic action curation and structure-aware passage selection inspired by human reading strategies. Through comprehensive evaluation on five challenging datasets, RDR2 achieves state-of-the-art performance, demonstrating that explicit structural awareness significantly enhances RAG systems’ ability to acquire and utilize knowledge, particularly in complex scenarios requiring multi-document synthesis.

Method: Introduces DCS metric that analyzes a model’s probability distribution over answer choices, distinguishing between hedging toward incorrect answers versus hedging toward “I don’t know” responses. Adapts 12 existing evaluation benchmarks to DCS variants and tests on six language models.

Result: For half of the tested benchmarks, scores are negative across all tested models, indicating significant tendencies towards hallucination. DCS provides scores in an interpretable default range and offers more nuanced evaluation.

Conclusion: DCS offers a more aligned evaluation paradigm that incentivizes models to express genuine uncertainty rather than guessing, addressing the problem of models being optimized for binary scoring schemes that reward any answer over abstention.

Abstract: Common evaluation paradigms for language models focus on scoring single responses through accuracy metrics or proper scoring rules, failing to capture the full richness of a model’s belief state. Recent work illustrates that language models hallucinate in-part because they are optimised to be good test-takers under binary scoring schemes that reward any answer over abstention. While this insight naturally leads to penalty-based approaches, they ignore crucial distinctions in how models distribute uncertainty, for example between hedging toward incorrect answers versus hedging toward “I don’t know” responses. A novel evaluation metric, the Distributional Correctness Score (DCS), is introduced to solve this problem, i.e., of not considering a model’s entire probability distribution over answer choices. DCS naturally distinguishes between harmful overconfidence in wrong answers and uncertainty expressed through abstention, providing scores in an interpretable default range. Through theoretical analysis and illustrative examples, DCS is demonstrated to offer a more nuanced and aligned evaluation paradigm that incentivises models to express genuine uncertainty rather than guessing. Adapting 12 existing evaluation benchmarks to DCS’s variants and measuring performance on six language models reveals that for half of the tested benchmarks scores are negative across all tested models, indicating significant tendencies towards hallucination.

Method: Built CB-Bench benchmark covering four risk scenarios with matched/mismatched semantic and outcome risks, and created CS-Chain-4k dataset for consequence-reasoning fine-tuning.

Result: Models fine-tuned on CS-Chain-4k show improved resistance to semantic-camouflage jailbreaks, reduced over-refusal on harmless inputs, while maintaining utility and generalization on other benchmarks.

Conclusion: Consequence-blindness is widespread in current LLMs, consequence-aware reasoning should be a core alignment goal, and the proposed methods provide a more practical evaluation path.

Abstract: Safety-aligned Large Language Models (LLMs) still show two dominant failure modes: they are easily jailbroken, or they over-refuse harmless inputs that contain sensitive surface signals. We trace both to a common cause: current models reason weakly about links between actions and outcomes and over-rely on surface-form signals, lexical or stylistic cues that do not encode consequences. We define this failure mode as Consequence-blindness. To study consequence-blindness, we build a benchmark named CB-Bench covering four risk scenarios that vary whether semantic risk aligns with outcome risk, enabling evaluation under both matched and mismatched conditions which are often ignored by existing safety benchmarks. Mainstream models consistently fail to separate these risks and exhibit consequence-blindness, indicating that consequence-blindness is widespread and systematic. To mitigate consequence-blindness, we introduce CS-Chain-4k, a consequence-reasoning dataset for safety alignment. Models fine-tuned on CS-Chain-4k show clear gains against semantic-camouflage jailbreaks and reduce over-refusal on harmless inputs, while maintaining utility and generalization on other benchmarks. These results clarify the limits of current alignment, establish consequence-aware reasoning as a core alignment goal and provide a more practical and reproducible evaluation path.

Method: Created CONSORT-QA evaluation corpus from two studies on abstract reporting quality with CONSORT-abstract standards, then evaluated various large language models (general and biomedical domain) using different prompting methods including Chain-of-thought.

Result: The best combination of model and prompting method achieved 85% accuracy in assessing CONSORT criteria. Chain-of-thought provided valuable information about the model’s reasoning process.

Conclusion: Large language models show promise for assessing clinical trial reporting quality, with Chain-of-thought prompting enhancing both performance and interpretability of the assessment process.

Abstract: Reporting quality is an important topic in clinical trial research articles, as it can impact clinical decisions. In this article, we test the ability of large language models to assess the reporting quality of this type of article using the Consolidated Standards of Reporting Trials (CONSORT). We create CONSORT-QA, an evaluation corpus from two studies on abstract reporting quality with CONSORT-abstract standards. We then evaluate the ability of different large generative language models (from the general domain or adapted to the biomedical domain) to correctly assess CONSORT criteria with different known prompting methods, including Chain-of-thought. Our best combination of model and prompting method achieves 85% accuracy. Using Chain-of-thought adds valuable information on the model’s reasoning for completing the task.

Method: Proposed inoculation prompting: prepend short system-prompt instructions to finetuning data that deliberately elicit undesirable traits, then evaluate without the instruction.

Result: Inoculated models show much lower expression of undesirable traits across multiple settings: reducing emergent misalignment, defending against backdoor injections, and mitigating trait transmission via subliminal learning.

Conclusion: Inoculation is an effective technique for selective learning that reduces optimization pressure to globally update models, contributing to better understanding of how language models generalize.

Abstract: Language model finetuning often results in learning undesirable traits in combination with desired ones. To address this, we propose inoculation prompting: modifying finetuning data by prepending a short system-prompt instruction that deliberately elicits the undesirable trait. At test time, we evaluate without the instruction; inoculated models have much lower expression of the trait than models trained with unmodified training data. Inoculation is selective: in a toy setting where assistant responses are always in Spanish and ALL-CAPS, an appropriate inoculation (e.g., ``You always speak in Spanish.’’) teaches the model to capitalize responses while still responding in English. We find that inoculation is also effective across several additional settings: reducing emergent misalignment (EM) from task-specific finetuning, defending against backdoor injections, and mitigating the transmission of traits via subliminal learning. Follow-up analysis suggests a mechanism: making a trait less surprising via inoculation reduces optimization pressure to globally update the model, thereby reducing the degree of generalization. Our analysis relates to prior work on EM: inoculation explains prior findings that educational contexts mitigate EM from insecure code. Beyond demonstrating a simple and effective technique for selective learning, our results contribute to a better conceptual understanding of how and why language models generalize.

Method: Proposes an inference-time defense that constructs anomaly scores by combining token-level attention and gradient information to detect poisoned inputs and trigger patterns.

Result: Extensive experiments show the method significantly reduces attack success rates across diverse backdoor attack scenarios in text classification tasks compared to existing baselines.

Conclusion: The proposed attention and gradient-based defense effectively mitigates backdoor attacks while providing interpretable trigger localization, enhancing model robustness against security threats.

Abstract: Pre-trained language models have achieved remarkable success across a wide range of natural language processing (NLP) tasks, particularly when fine-tuned on large, domain-relevant datasets. However, they remain vulnerable to backdoor attacks, where adversaries embed malicious behaviors using trigger patterns in the training data. These triggers remain dormant during normal usage, but, when activated, can cause targeted misclassifications. In this work, we investigate the internal behavior of backdoored pre-trained encoder-based language models, focusing on the consistent shift in attention and gradient attribution when processing poisoned inputs; where the trigger token dominates both attention and gradient signals, overriding the surrounding context. We propose an inference-time defense that constructs anomaly scores by combining token-level attention and gradient information. Extensive experiments on text classification tasks across diverse backdoor attack scenarios demonstrate that our method significantly reduces attack success rates compared to existing baselines. Furthermore, we provide an interpretability-driven analysis of the scoring mechanism, shedding light on trigger localization and the robustness of the proposed defense.

Method: Proposes PS-GRPO (Paraphrased Set Group Relative Policy Optimization), an RL approach that uses multiple rollouts across paraphrased sets with group similarity rewards. Also introduces a scalable approximation for efficient training.

Result: Con-RAG significantly improves both consistency and accuracy across short-form, multi-hop, and long-form QA benchmarks, even without explicit ground-truth supervision.

Conclusion: The work provides practical solutions for evaluating and building reliable RAG systems suitable for safety-critical deployments by addressing consistency issues through principled evaluation and training methods.

Abstract: RAG systems are increasingly deployed in high-stakes domains where users expect outputs to be consistent across semantically equivalent queries. However, existing systems often exhibit significant inconsistencies due to variability in both the retriever and generator (LLM), undermining trust and reliability. In this work, we focus on information consistency, i.e., the requirement that outputs convey the same core content across semantically equivalent inputs. We introduce a principled evaluation framework that decomposes RAG consistency into retriever-level, generator-level, and end-to-end components, helping identify inconsistency sources. To improve consistency, we propose Paraphrased Set Group Relative Policy Optimization (PS-GRPO), an RL approach that leverages multiple rollouts across paraphrased set to assign group similarity rewards. We leverage PS-GRPO to achieve Information Consistent RAG (Con-RAG), training the generator to produce consistent outputs across paraphrased queries and remain robust to retrieval-induced variability. Because exact reward computation over paraphrase sets is computationally expensive, we also introduce a scalable approximation method that retains effectiveness while enabling efficient, large-scale training. Empirical evaluations across short-form, multi-hop, and long-form QA benchmarks demonstrate that Con-RAG significantly improves both consistency and accuracy over strong baselines, even in the absence of explicit ground-truth supervision. Our work provides practical solutions for evaluating and building reliable RAG systems for safety-critical deployments.

Method: Created first large-scale dataset of post-editing time annotations for BEA19 and CoNLL14 GEC test datasets, then developed PEET scorer to estimate time-to-correct.

Result: GEC tools significantly reduce editing time; determining whether correction is needed and edits like paraphrasing/punctuation have biggest time impact. PEET correlates well with human rankings.

Conclusion: PEET provides a human-centric evaluation approach for GEC tool usability that better reflects real-world editing efficiency than technical metrics alone.

Abstract: Text editing can involve several iterations of revision. Incorporating an efficient Grammar Error Correction (GEC) tool in the initial correction round can significantly impact further human editing effort and final text quality. This raises an interesting question to quantify GEC Tool usability: How much effort can the GEC Tool save users? We present the first large-scale dataset of post-editing (PE) time annotations and corrections for two English GEC test datasets (BEA19 and CoNLL14). We introduce Post-Editing Effort in Time (PEET) for GEC Tools as a human-focused evaluation scorer to rank any GEC Tool by estimating PE time-to-correct. Using our dataset, we quantify the amount of time saved by GEC Tools in text editing. Analyzing the edit type indicated that determining whether a sentence needs correction and edits like paraphrasing and punctuation changes had the greatest impact on PE time. Finally, comparison with human rankings shows that PEET correlates well with technical effort judgment, providing a new human-centric direction for evaluating GEC tool usability. We release our dataset and code at: https://github.com/ankitvad/PEET_Scorer.

Method: Formulates realistic attacks as constrained optimization under semantic equivalence and coherence constraints, using a constraint-preserving zeroth-order method to search for adversarial prompts.

Result: SECA achieves higher attack success rates on open-ended multiple-choice question answering tasks while maintaining almost no constraint violations compared to existing methods.

Conclusion: LLMs are sensitive to realistic and plausible prompt variations, highlighting the need for more robust models that can withstand semantically equivalent attacks.

Abstract: Large Language Models (LLMs) are increasingly deployed in high-risk domains. However, state-of-the-art LLMs often produce hallucinations, raising serious concerns about their reliability. Prior work has explored adversarial attacks for hallucination elicitation in LLMs, but it often produces unrealistic prompts, either by inserting gibberish tokens or by altering the original meaning. As a result, these approaches offer limited insight into how hallucinations may occur in practice. While adversarial attacks in computer vision often involve realistic modifications to input images, the problem of finding realistic adversarial prompts for eliciting LLM hallucinations has remained largely underexplored. To address this gap, we propose Semantically Equivalent and Coherent Attacks (SECA) to elicit hallucinations via realistic modifications to the prompt that preserve its meaning while maintaining semantic coherence. Our contributions are threefold: (i) we formulate finding realistic attacks for hallucination elicitation as a constrained optimization problem over the input prompt space under semantic equivalence and coherence constraints; (ii) we introduce a constraint-preserving zeroth-order method to effectively search for adversarial yet feasible prompts; and (iii) we demonstrate through experiments on open-ended multiple-choice question answering tasks that SECA achieves higher attack success rates while incurring almost no constraint violations compared to existing methods. SECA highlights the sensitivity of both open-source and commercial gradient-inaccessible LLMs to realistic and plausible prompt variations. Code is available at https://github.com/Buyun-Liang/SECA.

Method: Used 10,000 ROCStories prompts completed by five LLMs, validated GPT-4o’s isotopy extraction capability on a linguistic benchmark, then analyzed structural (coverage, density, spread) and semantic properties of isotopies in generated stories.

Result: LLM completion within a given token horizon preserves semantic isotopies across multiple structural and semantic properties.

Conclusion: Large Language Models effectively maintain semantic isotopies in text generation, demonstrating their ability to preserve semantic coherence in extended text continuations.

Abstract: In this work, we explore the relevance of textual semantics to Large Language Models (LLMs), extending previous insights into the connection between distributional semantics and structural semantics. We investigate whether LLM-generated texts preserve semantic isotopies. We design a story continuation experiment using 10,000 ROCStories prompts completed by five LLMs. We first validate GPT-4o’s ability to extract isotopies from a linguistic benchmark, then apply it to the generated stories. We then analyze structural (coverage, density, spread) and semantic properties of isotopies to assess how they are affected by completion. Results show that LLM completion within a given token horizon preserves semantic isotopies across multiple properties.

Method: Author- and venue-level analysis of NLP4SG publications, quantifying proportions of work addressing social good concerns within/beyond ACL community, by both core ACL contributors and non-ACL authors.

Result: Two key findings: 1) ACL authors are dramatically more likely to publish NLP4SG work in venues outside ACL; 2) Majority of NLP4SG publications are by non-ACL authors in non-ACL venues.

Conclusion: The findings have important implications for agenda-setting considerations regarding NLP4SG within the ACL community, suggesting the need to examine publication patterns and community engagement strategies.

Abstract: The social impact of Natural Language Processing (NLP) is increasingly important, with a rising community focus on initiatives related to NLP for Social Good (NLP4SG). Indeed, in recent years, almost 20% of all papers in the ACL Anthology address topics related to social good as defined by the UN Sustainable Development Goals (Adauto et al., 2023). In this study, we take an author- and venue-level perspective to map the landscape of NLP4SG, quantifying the proportion of work addressing social good concerns both within and beyond the ACL community, by both core ACL contributors and non-ACL authors. With this approach we discover two surprising facts about the landscape of NLP4SG. First, ACL authors are dramatically more likely to do work addressing social good concerns when publishing in venues outside of ACL. Second, the vast majority of publications using NLP techniques to address concerns of social good are done by non-ACL authors in venues outside of ACL. We discuss the implications of these findings on agenda-setting considerations for the ACL community related to NLP4SG.

Method: The authors advocate for integrating the probability of unobserved sequences into existing entropy-based uncertainty quantification methods for LLMs.

Result: Experimental results show that considering unobserved sequences significantly enhances LLM uncertainty quantification performance.

Conclusion: Future research should incorporate the probability of unobserved sequences to improve LLM uncertainty quantification methods.

Abstract: Quantifying uncertainty in large language models (LLMs) is important for safety-critical applications because it helps spot incorrect answers, known as hallucinations. One major trend of uncertainty quantification methods is based on estimating the entropy of the distribution of the LLM’s potential output sequences. This estimation is based on a set of output sequences and associated probabilities obtained by querying the LLM several times. In this paper, we advocate and experimentally show that the probability of unobserved sequences plays a crucial role, and we recommend future research to integrate it to enhance such LLM uncertainty quantification methods.

Method: Dynamically integrates SFT into RL by selecting challenging examples for SFT, uses high-entropy tokens for loss calculation to mitigate catastrophic forgetting, and freezes parameters critical for RL.

Result: Achieves state-of-the-art reasoning performance using only 1.5% of SFT data and 20.4% of RL data compared to prior state-of-the-art methods.

Conclusion: Provides an efficient and plug-and-play solution for combining SFT and RL in reasoning post-training, effectively addressing data inefficiency and catastrophic forgetting challenges.

Abstract: Large Language Models (LLMs) show strong reasoning abilities, often amplified by Chain-of-Thought (CoT) prompting and reinforcement learning (RL). Although RL algorithms can substantially improve reasoning, they struggle to expand reasoning boundaries because they learn from their own reasoning trajectories rather than acquiring external knowledge. Supervised fine-tuning (SFT) offers complementary benefits but typically requires large-scale data and risks overfitting. Recent attempts to combine SFT and RL face three main challenges: data inefficiency, algorithm-specific designs, and catastrophic forgetting. We propose a plug-and-play framework that dynamically integrates SFT into RL by selecting challenging examples for SFT. This approach reduces SFT data requirements and remains agnostic to the choice of RL or SFT algorithm. To mitigate catastrophic forgetting of RL-acquired skills during SFT, we select high-entropy tokens for loss calculation and freeze parameters identified as critical for RL. Our method achieves state-of-the-art (SoTA) reasoning performance using only 1.5% of the SFT data and 20.4% of the RL data used by prior SoTA, providing an efficient and plug-and-play solution for combining SFT and RL in reasoning post-training.

Method: Compressed Convolutional Attention (CCA) down-projects queries, keys, and values and performs attention in a shared latent space. Combined with head-sharing to form CCGQA for further optimization.

Result: CCGQA outperforms GQA and MLA at equal KV-cache compression, achieves 8x KV-cache compression with no performance drop, reduces prefill latency by 1.7x and backward by 1.3x on H100 GPUs.

Conclusion: CCA and CCGQA provide superior compute-bandwidth Pareto frontier optimization, enabling substantial reductions in training and inference costs while maintaining model quality across dense and MoE architectures.

Abstract: Multi-headed Attention’s (MHA) quadratic compute and linearly growing KV-cache make long-context transformers expensive to train and serve. Prior works such as Grouped Query Attention (GQA) and Multi-Latent Attention (MLA) shrink the cache, speeding decode, but leave compute, which determines prefill and training speed, largely unchanged. We introduce Compressed Convolutional Attention (CCA), a novel attention method which down-projects queries, keys, and values and performs the entire attention operation inside the shared latent space. This simple design dramatically cuts parameters, KV-cache, and FLOPs all at once by the desired compression factor. Because CCA is orthogonal to head-sharing, we combine the two to form Compressed Convolutional Grouped Query Attention (CCGQA), which further tightens the compute-bandwidth Pareto frontier so that users can tune compression toward either FLOP or memory limits without sacrificing quality. Experiments show that CCGQA consistently outperforms both GQA and MLA at equal KV-cache compression on dense and MoE models. Additionally, we show that CCGQA outperforms all other attention methods on MoE models with half the KV-cache of GQA and MLA, achieving an 8x KV-cache compression with no drop in performance compared to standard MHA. CCA and CCGQA also dramatically reduce the FLOP cost of attention which leads to substantially faster training and prefill than existing methods. On H100 GPUs, our fused CCA/CCGQA kernel reduces prefill latency by about 1.7x at a sequence length of 16k relative to MHA, and accelerates backward by about 1.3x.

Method: Developed PsySET benchmark spanning four LLM models with various steering strategies including prompting, fine-tuning, and representation engineering, assessing safety, truthfulness, fairness, and ethics.

Result: Prompting is consistently effective but limited in intensity control; vector injections achieve finer controllability with slight output quality reduction. Positive emotions like joy can degrade robustness to adversarial factuality, lower privacy awareness, and increase preferential bias.

Conclusion: Establishes the first holistic evaluation of emotion and personality steering, offering insights into interpretability and reliability for socially interactive applications, highlighting idiosyncratic effects and behavioral shifts.

Abstract: The ability to control LLMs’ emulated emotional states and personality traits is essential for enabling rich, human-centered interactions in socially interactive settings. We introduce PsySET, a Psychologically-informed benchmark to evaluate LLM Steering Effectiveness and Trustworthiness across the emotion and personality domains. Our study spans four models from different LLM families paired with various steering strategies, including prompting, fine-tuning, and representation engineering. Our results indicate that prompting is consistently effective but limited in intensity control, whereas vector injections achieve finer controllability while slightly reducing output quality. Moreover, we explore the trustworthiness of steered LLMs by assessing safety, truthfulness, fairness, and ethics, highlighting potential side effects and behavioral shifts. Notably, we observe idiosyncratic effects; for instance, even a positive emotion like joy can degrade robustness to adversarial factuality, lower privacy awareness, and increase preferential bias. Meanwhile, anger predictably elevates toxicity yet strengthens leakage resistance. Our framework establishes the first holistic evaluation of emotion and personality steering, offering insights into its interpretability and reliability for socially interactive applications.

Method: Uses LLMs to generate dynamic choose-your-own-adventure narratives with branching decision points, proficiency-level content adaptation, and in-context vocabulary explanations.

Result: Pilot study with Chinese university students showed promising vocabulary gains and positive user feedback, though participants suggested improvements to narrative length/quality and requested multi-modal content.

Conclusion: GenQuest demonstrates the potential of LLM-driven interactive storytelling for language learning, with future improvements needed in narrative design and multi-modal integration.

Abstract: GenQuest is a generative text adventure game that leverages Large Language Models (LLMs) to facilitate second language learning through immersive, interactive storytelling. The system engages English as a Foreign Language (EFL) learners in a collaborative “choose-your-own-adventure” style narrative, dynamically generated in response to learner choices. Game mechanics such as branching decision points and story milestones are incorporated to maintain narrative coherence while allowing learner-driven plot development. Key pedagogical features include content generation tailored to each learner’s proficiency level, and a vocabulary assistant that provides in-context explanations of learner-queried text strings, ranging from words and phrases to sentences. Findings from a pilot study with university EFL students in China indicate promising vocabulary gains and positive user perceptions. Also discussed are suggestions from participants regarding the narrative length and quality, and the request for multi-modal content such as illustrations.

Method: GRACE uses policy gradient optimization with a multi-component reward function that maximizes similarity between query positive pairs and minimizes similarity with negatives. The LLM produces human-interpretable rationales that are encoded into embeddings via mean pooling.

Result: On MTEB benchmark, GRACE achieves broad cross-category gains: supervised setting improves overall score by 11.5% over base models, and unsupervised variant adds 6.9%, while preserving general capabilities.

Conclusion: GRACE unifies representation learning with generation to produce stronger embeddings and transparent rationales, transforming LLMs from opaque encoders into interpretable agents with inspectable reasoning processes.

Abstract: Prevailing methods for training Large Language Models (LLMs) as text encoders rely on contrastive losses that treat the model as a black box function, discarding its generative and reasoning capabilities in favor of static embeddings. We introduce GRACE (Generative Representation Learning via Contrastive Policy Optimization), a novel framework that reimagines contrastive signals not as losses to be minimized, but as rewards that guide a generative policy. In GRACE, the LLM acts as a policy that produces explicit, human-interpretable rationales–structured natural language explanations of its semantic understanding. These rationales are then encoded into high-quality embeddings via mean pooling. Using policy gradient optimization, we train the model with a multi-component reward function that maximizes similarity between query positive pairs and minimizes similarity with negatives. This transforms the LLM from an opaque encoder into an interpretable agent whose reasoning process is transparent and inspectable. On MTEB benchmark, GRACE yields broad cross category gains: averaged over four backbones, the supervised setting improves overall score by 11.5% over base models, and the unsupervised variant adds 6.9%, while preserving general capabilities. This work treats contrastive objectives as rewards over rationales, unifying representation learning with generation to produce stronger embeddings and transparent rationales. The model, data and code are available at https://github.com/GasolSun36/GRACE.

Method: Proposes ALC with two training objectives that leverage hardest negatives in batches to create discriminative training signals between positive and negative items.

Result: Validated on two product recommendation datasets (LF-AmazonTitles-131K and proprietary Tech and Durables) using three extreme multi-label classification approaches, achieving state-of-the-art coverage rates when combined with threshold-consistent margin loss.

Conclusion: ALC effectively boosts coverage in product recommendation systems by learning fine-grained embeddings through auxiliary learning objectives.

Abstract: Product recommendation is the task of recovering the closest items to a given query within a large product corpora. Generally, one can determine if top-ranked products are related to the query by applying a similarity threshold; exceeding it deems the product relevant, otherwise manual revision is required. Despite being a well-known problem, the integration of these models in real-world systems is often overlooked. In particular, production systems have strong coverage requirements, i.e., a high proportion of recommendations must be automated. In this paper we propose ALC , an Auxiliary Learning strategy that boosts Coverage through learning fine-grained embeddings. Concretely, we introduce two training objectives that leverage the hardest negatives in the batch to build discriminative training signals between positives and negatives. We validate ALC using three extreme multi-label classification approaches in two product recommendation datasets; LF-AmazonTitles-131K and Tech and Durables (proprietary), demonstrating state-of-the-art coverage rates when combined with a recent threshold-consistent margin loss.

Method: Designed experiments using next-token prediction tasks for pronoun production, pronoun interpretation, and ambiguity detection with different prompting strategies.

Result: LLMs are sometimes aware of possible referents of ambiguous pronouns but don’t follow human reference preferences, especially when interpretations aren’t explicitly mentioned. They struggle to identify ambiguity without direct instruction.

Conclusion: LLMs exhibit inconsistencies across different experiment types and have limitations in human-like plural reference processing, particularly in ambiguity detection and interpretation alignment.

Abstract: Our goal is to study how LLMs represent and interpret plural reference in ambiguous and unambiguous contexts. We ask the following research questions: (1) Do LLMs exhibit human-like preferences in representing plural reference? (2) Are LLMs able to detect ambiguity in plural anaphoric expressions and identify possible referents? To address these questions, we design a set of experiments, examining pronoun production using next-token prediction tasks, pronoun interpretation, and ambiguity detection using different prompting strategies. We then assess how comparable LLMs are to humans in formulating and interpreting plural reference. We find that LLMs are sometimes aware of possible referents of ambiguous pronouns. However, they do not always follow human reference when choosing between interpretations, especially when the possible interpretation is not explicitly mentioned. In addition, they struggle to identify ambiguity without direct instruction. Our findings also reveal inconsistencies in the results across different types of experiments.

Method: FedSRD uses importance-aware sparsification to reduce uploaded LoRA parameters, server-side reconstruction and aggregation in full-rank space to mitigate conflicts, and decomposition into sparse low-rank format for efficient broadcasting. FedSRD-e is a computational-efficient variant.

Result: Experimental results on 10 benchmarks show up to 90% reduction in communication costs while improving model performance on heterogeneous client data.

Conclusion: FedSRD provides an effective solution for communication-efficient federated fine-tuning of LLMs, enabling sustainable AI development on decentralized Web infrastructure.

Abstract: The current paradigm of training large language models (LLMs) on publicly available Web data is becoming unsustainable, with high-quality data sources in specialized domains nearing exhaustion. Federated Learning (FL) emerges as a practical solution for the next generation of AI on a decentralized Web, enabling privacy-preserving collaborative fine-tuning by leveraging private data distributed across a global client base. While Low-Rank Adaptation (LoRA) is the standard for efficient fine-tuning, its application in federated settings presents a critical challenge: communication overhead remains a significant bottleneck across the Web’s heterogeneous network conditions. The structural redundancy within LoRA parameters not only incurs a heavy communication burden but also introduces conflicts when aggregating client updates. To address this, we propose FedSRD, a Sparsify-Reconstruct-Decompose framework designed for communication-efficient FL. We first introduce an importance-aware sparsification method that preserves the structural integrity of LoRA updates to reduce the uploaded parameter count. The server then reconstructs and aggregates these updates in a full-rank space to mitigate conflicts. Finally, it decomposes the global update into a sparse low-rank format for broadcast, ensuring a symmetrically efficient cycle. We also propose an efficient variant, FedSRD-e, to reduce computational overhead. Experimental results on 10 benchmarks demonstrate that our framework significantly reduces communication costs by up to 90% while even improving model performance on heterogeneous client data.

Method: Applied SciNCL’s neighborhood contrastive learning methodology to process industry domain, using triplets derived from graph embeddings for fine-tuning language models.

Result: Models fine-tuned with graph embedding triplets outperformed mE5-large by 9.8-14.3% on PITEB benchmark while being 3-5 times smaller in size.

Conclusion: Graph-aware contrastive learning effectively improves language model performance in process industry applications, enabling smaller models to outperform larger state-of-the-art encoders.

Abstract: Recent trends in NLP utilize knowledge graphs (KGs) to enhance pretrained language models by incorporating additional knowledge from the graph structures to learn domain-specific terminology or relationships between documents that might otherwise be overlooked. This paper explores how SciNCL, a graph-aware neighborhood contrastive learning methodology originally designed for scientific publications, can be applied to the process industry domain, where text logs contain crucial information about daily operations and are often structured as sparse KGs. Our experiments demonstrate that language models fine-tuned with triplets derived from GE outperform a state-of-the-art mE5-large text encoder by 9.8-14.3% (5.4-8.0p) on the proprietary process industry text embedding benchmark (PITEB) while being 3-5 times smaller in size.

Method: Developed a comprehensive evaluation framework for English texts using multi-label bias detection with demographic-focused taxonomy. Evaluated models across scales and techniques including prompting, in-context learning, and fine-tuning using twelve diverse datasets.

Result: Fine-tuned smaller models show promise for scalable bias detection, but analyses reveal persistent gaps across demographic axes and multi-demographic targeted biases.

Conclusion: There is a need for more effective and scalable auditing frameworks to address the limitations in detecting multi-demographic biases across different demographic axes.

Abstract: Large-scale web-scraped text corpora used to train general-purpose AI models often contain harmful demographic-targeted social biases, creating a regulatory need for data auditing and developing scalable bias-detection methods. Although prior work has investigated biases in text datasets and related detection methods, these studies remain narrow in scope. They typically focus on a single content type (e.g., hate speech), cover limited demographic axes, overlook biases affecting multiple demographics simultaneously, and analyze limited techniques. Consequently, practitioners lack a holistic understanding of the strengths and limitations of recent large language models (LLMs) for automated bias detection. In this study, we present a comprehensive evaluation framework aimed at English texts to assess the ability of LLMs in detecting demographic-targeted social biases. To align with regulatory requirements, we frame bias detection as a multi-label task using a demographic-focused taxonomy. We then conduct a systematic evaluation with models across scales and techniques, including prompting, in-context learning, and fine-tuning. Using twelve datasets spanning diverse content types and demographics, our study demonstrates the promise of fine-tuned smaller models for scalable detection. However, our analyses also expose persistent gaps across demographic axes and multi-demographic targeted biases, underscoring the need for more effective and scalable auditing frameworks.

Method: Proposed PI-LoRA (Path-Integrated LoRA), a low-rank adaptation method that integrates gradient path information to capture synergistic effects between modules, enabling effective rank allocation and pruning of less important modules.

Result: Extensive experiments show PI-LoRA significantly outperforms existing parameter-efficient fine-tuning approaches for Text2MDT task, achieving better accuracy with substantially reduced model complexity.

Conclusion: PI-LoRA achieves state-of-the-art results while maintaining a lightweight architecture, making it particularly suitable for clinical decision support systems with limited computational resources.

Abstract: Knowledge of the medical decision process, which can be modeled as medical decision trees (MDTs), is critical to building clinical decision support systems. However, current MDT construction methods rely heavily on time-consuming and laborious manual annotation. To address this challenge, we propose PI-LoRA (Path-Integrated LoRA), a novel low-rank adaptation method for automatically extracting MDTs from clinical guidelines and textbooks. We integrate gradient path information to capture synergistic effects between different modules, enabling more effective and reliable rank allocation. This framework ensures that the most critical modules receive appropriate rank allocations while less important ones are pruned, resulting in a more efficient and accurate model for extracting medical decision trees from clinical texts. Extensive experiments on medical guideline datasets demonstrate that our PI-LoRA method significantly outperforms existing parameter-efficient fine-tuning approaches for the Text2MDT task, achieving better accuracy with substantially reduced model complexity. The proposed method achieves state-of-the-art results while maintaining a lightweight architecture, making it particularly suitable for clinical decision support systems where computational resources may be limited.

Method: Uses prompt templates to extract core focus from CHQs, constructs fine-tuning datasets with CHQ-FAQ pairs, and implements multi-dimensional quality evaluation and selection mechanism.

Result: Achieves state-of-the-art performance on two MQS datasets across all evaluation metrics, with significant improvement in focus identification and hallucination reduction.

Conclusion: The proposed framework effectively enhances LLMs’ ability to generate faithful medical question summaries by improving focus identification and mitigating hallucinations.

Abstract: With the rapid development of online medical platforms, consumer health questions (CHQs) are inefficient in diagnosis due to redundant information and frequent non-professional terms. The medical question summary (MQS) task aims to transform CHQs into streamlined doctors’ frequently asked questions (FAQs), but existing methods still face challenges such as poor identification of question focus and model hallucination. This paper explores the potential of large language models (LLMs) in the MQS task and finds that direct fine-tuning is prone to focus identification bias and generates unfaithful content. To this end, we propose an optimization framework based on core focus guidance. First, a prompt template is designed to drive the LLMs to extract the core focus from the CHQs that is faithful to the original text. Then, a fine-tuning dataset is constructed in combination with the original CHQ-FAQ pairs to improve the ability to identify the focus of the question. Finally, a multi-dimensional quality evaluation and selection mechanism is proposed to comprehensively improve the quality of the summary from multiple dimensions. We conduct comprehensive experiments on two widely-adopted MQS datasets using three established evaluation metrics. The proposed framework achieves state-of-the-art performance across all measures, demonstrating a significant boost in the model’s ability to identify critical focus of questions and a notable mitigation of hallucinations. The source codes are freely available at https://github.com/DUT-LiuChao/FocusMed.

Method: Multi-Agent Tool-Integrated Policy Optimization (MATPO) trains distinct planner and worker roles within a single LLM using role-specific prompts via reinforcement learning, with principled credit assignment across agent rollouts.

Result: Experiments on GAIA-text, WebWalkerQA, and FRAMES show MATPO consistently outperforms single-agent baselines by 18.38% average relative improvement and exhibits greater robustness to noisy tool outputs.

Conclusion: Unifying multiple agent roles within a single LLM is effective, providing practical insights for stable and efficient multi-agent RL training while eliminating the memory overhead of deploying multiple LLMs.

Abstract: Large language models (LLMs) increasingly rely on multi-turn tool-integrated planning for knowledge-intensive and complex reasoning tasks. Existing implementations typically rely on a single agent, but they suffer from limited context length and noisy tool responses. A natural solution is to adopt a multi-agent framework with planner- and worker-agents to manage context. However, no existing methods support effective reinforcement learning post-training of tool-integrated multi-agent frameworks. To address this gap, we propose Multi-Agent Tool-Integrated Policy Optimization (MATPO), which enables distinct roles (planner and worker) to be trained within a single LLM instance using role-specific prompts via reinforcement learning. MATPO is derived from a principled credit assignment mechanism across planner and worker rollouts. This design eliminates the need to deploy multiple LLMs, which would be memory-intensive, while preserving the benefits of specialization. Experiments on GAIA-text, WebWalkerQA, and FRAMES show that MATPO consistently outperforms single-agent baselines by an average of 18.38% relative improvement in performance and exhibits greater robustness to noisy tool outputs. Our findings highlight the effectiveness of unifying multiple agent roles within a single LLM and provide practical insights for stable and efficient multi-agent RL training.

Method: TiTok uses contrastive excess between source models with and without LoRA to capture task-relevant information, enabling selective filtering of synthetic data at token level without additional overhead.

Result: Experiments on three benchmarks across multiple transfer settings show consistent effectiveness with average performance gains of +4~8% compared to baselines.

Conclusion: TiTok provides an effective framework for LoRA transplantation that works across different model backbones without requiring additional models or complex synthetic data generation.

Abstract: Large Language Models (LLMs) are widely applied in real world scenarios, but fine-tuning them comes with significant computational and storage costs. Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA mitigate these costs, but the adapted parameters are dependent on the base model and cannot be transferred across different backbones. One way to address this issue is through knowledge distillation, but its effectiveness inherently depends on training data. Recent work such as TransLoRA avoids this by generating synthetic data, but this adds complexity because it requires training an additional discriminator model. In this paper, we propose TiTok, a new framework that enables effective LoRA Transplantation through Token-level knowledge transfer. Specifically, TiTok captures task-relevant information through a contrastive excess between a source model with and without LoRA. This excess highlights informative tokens and enables selective filtering of synthetic data, all without additional models or overhead. Through experiments on three benchmarks across multiple transfer settings, our experiments show that the proposed method is consistently effective, achieving average performance gains of +4~8% compared to baselines overall.

Method: Analyzed expert routing patterns using parallel multilingual datasets, examined layer-wise phenomena, and developed inference-time interventions that steer routers by promoting middle-layer task experts frequently activated in English.

Result: Found strong correlation between language performance and English routing similarity in middle layers. Interventions increasing cross-lingual routing alignment consistently improved multilingual performance by 1-2% across tasks, models, and languages.

Conclusion: MoEs process non-English text with language-specific routing in early/late layers and cross-lingual alignment in middle layers. Multilingual generalization is limited by the model’s ability to leverage language-universal experts across all languages.

Abstract: Mixture-of-Experts (MoE) architectures have become the key to scaling modern LLMs, yet little is understood about how their sparse routing dynamics respond to multilingual data. In this work, we analyze expert routing patterns using parallel multilingual datasets and present highly interpretable layer-wise phenomena. We find that MoE models route tokens in language-specific ways in the early and late decoder layers but exhibit significant cross-lingual routing alignment in middle layers, mirroring parameter-sharing trends observed in dense LLMs. In particular, we reveal a clear, strong correlation between a model’s performance in a given language and how similarly its tokens are routed to English in these layers. Extending beyond correlation, we explore inference-time interventions that induce higher cross-lingual routing alignment. We introduce a method that steers the router by promoting middle-layer task experts frequently activated in English, and it successfully increases multilingual performance. These 1-2% gains are remarkably consistent across two evaluation tasks, three models, and 15+ languages, especially given that these simple interventions override routers of extensively trained, state-of-the-art LLMs. In comparison, interventions outside of the middle layers or targeting multilingual-specialized experts only yield performance degradation. Altogether, we present numerous findings that explain how MoEs process non-English text and demonstrate that generalization is limited by the model’s ability to leverage language-universal experts in all languages.

Method: Introduces EASE (Explicitly Addressed Sequence Encoding) that transforms arrays into dictionaries with stable keys to eliminate index arithmetic complexities and handle array manipulations better.

Result: Patch generation with EASE reduces token usage by 31% while maintaining edit quality within 5% of full regeneration, with particular gains for complex instructions and list manipulations.

Conclusion: JSON Whisperer provides an efficient framework for LLM-based JSON editing through patch generation rather than full document regeneration.

Abstract: Large language models (LLMs) can modify JSON documents through natural language commands, but current approaches regenerate entire structures for each edit, resulting in computational inefficiency. We present JSON Whisperer, a framework that enables LLMs to generate RFC 6902 diff patches-expressing only the necessary modifications-rather than complete documents. We identify two key challenges in patch-based editing: (1) LLMs often miss related updates when generating isolated patches, and (2) array manipulations require tracking index shifts across operations, which LLMs handle poorly. To address these issues, we introduce EASE (Explicitly Addressed Sequence Encoding), which transforms arrays into dictionaries with stable keys, eliminating index arithmetic complexities. Our evaluation shows that patch generation with EASE reduces token usage by 31% while maintaining edit quality within 5% of full regeneration with particular gains for complex instructions and list manipulations. The dataset is available at: https://github.com/emnlp2025/JSON-Whisperer/

Method: Integrates Whisper for speech-to-text, SinBERT for identifying dyslexic errors, combined mT5 and Mistral-based model for text correction, and gTTS for text-to-speech conversion to create a complete multimodal feedback loop.

Result: Achieves 0.66 transcription accuracy, 0.7 correction accuracy, and 0.65 overall system accuracy despite challenges from limited Sinhala-language datasets.

Conclusion: Demonstrates feasibility and effectiveness of inclusive NLP technologies for underrepresented languages, highlighting the importance of accessible tools for dyslexic adults in low-resource language contexts.

Abstract: Dyslexia in adults remains an under-researched and under-served area, particularly in non-English-speaking contexts, despite its significant impact on personal and professional lives. This work addresses that gap by focusing on Sinhala, a low-resource language with limited tools for linguistic accessibility. We present an assistive system explicitly designed for Sinhala-speaking adults with dyslexia. The system integrates Whisper for speech-to-text conversion, SinBERT, an open-sourced fine-tuned BERT model trained for Sinhala to identify common dyslexic errors, and a combined mT5 and Mistral-based model to generate corrected text. Finally, the output is converted back to speech using gTTS, creating a complete multimodal feedback loop. Despite the challenges posed by limited Sinhala-language datasets, the system achieves 0.66 transcription accuracy and 0.7 correction accuracy with 0.65 overall system accuracy. These results demonstrate both the feasibility and effectiveness of the approach. Ultimately, this work highlights the importance of inclusive Natural Language Processing (NLP) technologies in underrepresented languages and showcases a practical

Method: Developed a two-stage retrieval system: lightweight ModernBERT bidirectional encoder for initial candidate retrieval, followed by ColBERTv2 late-interaction model for fine-grained re-ranking, fine-tuned using 10k question-passage pairs from PubMedQA.

Result: ColBERT re-ranker improved Recall@3 by up to 4.2 percentage points. The biomedical RAG system achieved state-of-the-art average accuracy of 0.4448 on MIRAGE benchmark, outperforming MedCPT (0.4436).

Conclusion: The two-stage retrieval architecture with joint fine-tuning of retriever and re-ranker is crucial for optimal performance in biomedical RAG systems, as separate tuning can degrade performance.

Abstract: Retrieval-Augmented Generation (RAG) is a powerful technique for enriching Large Language Models (LLMs) with external knowledge, allowing for factually grounded responses, a critical requirement in high-stakes domains such as healthcare. However, the efficacy of RAG systems is fundamentally restricted by the performance of their retrieval module, since irrelevant or semantically misaligned documents directly compromise the accuracy of the final generated response. General-purpose dense retrievers can struggle with the nuanced language of specialised domains, while the high accuracy of in-domain models is often achieved at prohibitive computational costs. In this work, we aim to address this trade-off by developing and evaluating a two-stage retrieval architecture that combines a lightweight ModernBERT bidirectional encoder for efficient initial candidate retrieval with a ColBERTv2 late-interaction model for fine-grained re-ranking. We conduct comprehensive evaluations of our retriever module performance and RAG system performance in the biomedical context, fine-tuning the IR module using 10k question-passage pairs from PubMedQA. Our analysis of the retriever module confirmed the positive impact of the ColBERT re-ranker, which improved Recall@3 by up to 4.2 percentage points compared to its retrieve-only counterpart. When integrated into the biomedical RAG, our IR module leads to a state-of-the-art average accuracy of 0.4448 on the five tasks of the MIRAGE question-answering benchmark, outperforming strong baselines such as MedCPT (0.4436). Our ablation studies reveal that this performance is critically dependent on a joint fine-tuning process that aligns the retriever and re-ranker; otherwise, the re-ranker might degrade the performance.

Method: Uses a small calibration dataset to estimate linear transformations that approximate pruned transformer blocks, then seamlessly merges these linear mappings with remaining blocks without adding parameters.

Result: Outperforms other training-free approaches and remains competitive with state-of-the-art pruning methods that require retraining. Achieves 25% pruning while retaining ~90% original performance on LLMs.

Conclusion: ReplaceMe provides an effective training-free alternative to conventional pruning methods, offering significant model compression with minimal computational overhead and competitive performance retention.

Abstract: We introduce ReplaceMe, a generalized training-free depth pruning method that effectively replaces transformer blocks with a linear operation, while maintaining high performance for low compression ratios. In contrast to conventional pruning approaches that require additional training or fine-tuning, our approach requires only a small calibration dataset that is used to estimate a linear transformation, which approximates the pruned blocks. The estimated linear mapping can be seamlessly merged with the remaining transformer blocks, eliminating the need for any additional network parameters. Our experiments show that ReplaceMe consistently outperforms other training-free approaches and remains highly competitive with state-of-the-art pruning methods that involve extensive retraining/fine-tuning and architectural modifications. Applied to several large language models (LLMs), ReplaceMe achieves up to 25% pruning while retaining approximately 90% of the original model’s performance on open benchmarks - without any training or healing steps, resulting in minimal computational overhead (see Fig.1). We provide an open-source library implementing ReplaceMe alongside several state-of-the-art depth pruning techniques, available at https://github.com/mts-ai/ReplaceMe.

Method: Created a novel benchmark based on Surian et al.’s study, testing BabyLMs pretrained on <10M and <100M tokens across five Gricean maxims, comparing performance to children and a large LLM trained on 3T tokens.

Result: Models trained on <100M tokens outperform those on <10M tokens but still fall short of child-level and LLM competence. Modest data improvements lead to better pragmatic behavior and finer-grained differentiation between pragmatic dimensions.

Conclusion: Small language models show some pragmatic capability that improves with more data, but they still lag behind human children and large language models in understanding Gricean maxim violations.

Abstract: Implicit meanings are integral to human communication, making it essential for language models to be capable of identifying and interpreting them. Grice (1975) proposed a set of conversational maxims that guide cooperative dialogue, noting that speakers may deliberately violate these principles to express meanings beyond literal words, and that listeners, in turn, recognize such violations to draw pragmatic inferences. Building on Surian et al. (1996)’s study of children’s sensitivity to violations of Gricean maxims, we introduce a novel benchmark to test whether language models pretrained on less than 10M and less than 100M tokens can distinguish maxim-adhering from maxim-violating utterances. We compare these BabyLMs across five maxims and situate their performance relative to children and a Large Language Model (LLM) pretrained on 3T tokens. We find that overall, models trained on less than 100M tokens outperform those trained on less than 10M, yet fall short of child-level and LLM competence. Our results suggest that modest data increases improve some aspects of pragmatic behavior, leading to finer-grained differentiation between pragmatic dimensions.

Method: The authors conduct holistic evaluation of hybrid architectures based on inter-layer (sequential) and intra-layer (parallel) fusion, analyzing language modeling performance, long-context capabilities, scaling behavior, and training/inference efficiency.

Result: The study identifies the most critical elements for each hybridization strategy and proposes optimal design recipes for both hybrid models through investigation of their computational primitives.

Conclusion: The comprehensive analysis provides practical guidance and valuable insights for developing hybrid language models, facilitating the optimization of architectural configurations.

Abstract: Recent progress in large language models demonstrates that hybrid architectures–combining self-attention mechanisms with structured state space models like Mamba–can achieve a compelling balance between modeling quality and computational efficiency, particularly for long-context tasks. While these hybrid models show promising performance, systematic comparisons of hybridization strategies and analyses on the key factors behind their effectiveness have not been clearly shared to the community. In this work, we present a holistic evaluation of hybrid architectures based on inter-layer (sequential) or intra-layer (parallel) fusion. We evaluate these designs from a variety of perspectives: language modeling performance, long-context capabilities, scaling analysis, and training and inference efficiency. By investigating the core characteristics of their computational primitive, we identify the most critical elements for each hybridization strategy and further propose optimal design recipes for both hybrid models. Our comprehensive analysis provides practical guidance and valuable insights for developing hybrid language models, facilitating the optimization of architectural configurations.

Method: Using short-form pronunciation resources as alternative data format instead of standard utterance-level supervised data. Tested approach on Manx Gaelic and replicated on Cornish.

Result: 40 minutes of short-form pronunciation data produces usable ASR for Manx Gaelic with <50% Word Error Rate. Successfully replicated on Cornish.

Conclusion: The barrier to entry for building ASR for endangered languages is far lower than previously thought - both in quantity and form of required data.

Abstract: Nearly half of the world’s languages are endangered. Speech technologies such as Automatic Speech Recognition (ASR) are central to revival efforts, yet most languages remain unsupported because standard pipelines expect utterance-level supervised data. Speech data often exist for endangered languages but rarely match these formats. Manx Gaelic ($\sim$2,200 speakers), for example, has had transcribed speech since 1948, yet remains unsupported by modern systems. In this paper, we explore how little data, and in what form, is needed to build ASR for critically endangered languages. We show that a short-form pronunciation resource is a viable alternative, and that 40 minutes of such data produces usable ASR for Manx ($<$50% WER). We replicate our approach, applying it to Cornish ($\sim$600 speakers), another critically endangered language. Results show that the barrier to entry, in quantity and form, is far lower than previously thought, giving hope to endangered language communities that cannot afford to meet the requirements arbitrarily imposed upon them.

Method: Analyze performance stability and investigate checkpoint averaging and ensemble methods to aggregate neighboring checkpoints.

Result: Both empirical and theoretical evidence shows these methods improve downstream performance stability.

Conclusion: Post-hoc checkpoint integration methods effectively reduce performance volatility in LLM training.

Abstract: When training large language models (LLMs), it is common practice to track downstream task performance throughout the training process and select the checkpoint with the highest validation score. However, downstream metrics often exhibit substantial fluctuations, making it difficult to identify the checkpoint that truly represents the best-performing model. In this study, we empirically analyze the stability of downstream task performance in an LLM trained on diverse web-scale corpora. We find that task scores frequently fluctuate throughout training, both at the aggregate and example levels. To address this instability, we investigate two post-hoc checkpoint integration methods: checkpoint averaging and ensemble, motivated by the hypothesis that aggregating neighboring checkpoints can reduce performance volatility. We demonstrate both empirically and theoretically that these methods improve downstream performance stability without requiring any changes to the training procedure.

Method: Automated three-stage pipeline: 1) Generate QA pairs from Wikipedia using GPT-4o, 2) Elicit potentially hallucinated answers from diverse LLMs in no-context setting, 3) Automatically annotate hallucinated spans using GPT-4o by comparing against golden answers and retrieved context.

Result: Encoder-based models achieve strongest performance across languages. PsiloQA demonstrates effective cross-lingual generalization and robust knowledge transfer to other benchmarks, while being significantly more cost-efficient than human-annotated datasets.

Conclusion: PsiloQA advances scalable, fine-grained hallucination detection in multilingual settings, providing a comprehensive resource for developing reliable LLM applications requiring factual accuracy.

Abstract: Hallucination detection remains a fundamental challenge for the safe and reliable deployment of large language models (LLMs), especially in applications requiring factual accuracy. Existing hallucination benchmarks often operate at the sequence level and are limited to English, lacking the fine-grained, multilingual supervision needed for a comprehensive evaluation. In this work, we introduce PsiloQA, a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. PsiloQA is constructed through an automated three-stage pipeline: generating question-answer pairs from Wikipedia using GPT-4o, eliciting potentially hallucinated answers from diverse LLMs in a no-context setting, and automatically annotating hallucinated spans using GPT-4o by comparing against golden answers and retrieved context. We evaluate a wide range of hallucination detection methods – including uncertainty quantification, LLM-based tagging, and fine-tuned encoder models – and show that encoder-based models achieve the strongest performance across languages. Furthermore, PsiloQA demonstrates effective cross-lingual generalization and supports robust knowledge transfer to other benchmarks, all while being significantly more cost-efficient than human-annotated datasets. Our dataset and results advance the development of scalable, fine-grained hallucination detection in multilingual settings.

Method: Token Probability Deviation (TBD) quantifies how far generated tokens’ probabilities deviate from a high reference probability, leveraging the observation that distilled models produce more low-probability tokens for unseen questions.

Result: The method achieves competitive performance with AUC of 0.918 and TPR@1% FPR of 0.470 on the S1 dataset, effectively distinguishing seen from unseen questions.

Conclusion: TBD provides an effective solution for detecting distillation data contamination by analyzing token probability patterns, addressing the challenge of partial data availability in distillation scenarios.

Abstract: Reasoning distillation has emerged as an efficient and powerful paradigm for enhancing the reasoning capabilities of large language models. However, reasoning distillation may inadvertently cause benchmark contamination, where evaluation data included in distillation datasets can inflate performance metrics of distilled models. In this work, we formally define the task of distillation data detection, which is uniquely challenging due to the partial availability of distillation data. Then, we propose a novel and effective method Token Probability Deviation (TBD), which leverages the probability patterns of the generated output tokens. Our method is motivated by the analysis that distilled models tend to generate near-deterministic tokens for seen questions, while producing more low-probability tokens for unseen questions. Our key idea behind TBD is to quantify how far the generated tokens’ probabilities deviate from a high reference probability. In effect, our method achieves competitive detection performance by producing lower scores for seen questions than for unseen questions. Extensive experiments demonstrate the effectiveness of our method, achieving an AUC of 0.918 and a TPR@1% FPR of 0.470 on the S1 dataset.

Method: Created SocialHarmBench dataset with 585 prompts spanning 7 sociopolitical categories across 34 countries, then evaluated LLM performance on these politically charged contexts.

Result: Open-weight models showed high vulnerability to harmful compliance, with Mistral-7B reaching 97-98% attack success rates in domains like historical revisionism and propaganda. Models were most fragile with 21st-century/pre-20th-century contexts and prompts from Latin America, USA, and UK regions.

Conclusion: Current LLM safeguards fail to generalize to sociopolitical settings, exposing systematic biases and raising concerns about LLM reliability in preserving human rights and democratic values.

Abstract: Large language models (LLMs) are increasingly deployed in contexts where their failures can have direct sociopolitical consequences. Yet, existing safety benchmarks rarely test vulnerabilities in domains such as political manipulation, propaganda and disinformation generation, or surveillance and information control. We introduce SocialHarmBench, a dataset of 585 prompts spanning 7 sociopolitical categories and 34 countries, designed to surface where LLMs most acutely fail in politically charged contexts. Our evaluations reveal several shortcomings: open-weight models exhibit high vulnerability to harmful compliance, with Mistral-7B reaching attack success rates as high as 97% to 98% in domains such as historical revisionism, propaganda, and political manipulation. Moreover, temporal and geographic analyses show that LLMs are most fragile when confronted with 21st-century or pre-20th-century contexts, and when responding to prompts tied to regions such as Latin America, the USA, and the UK. These findings demonstrate that current safeguards fail to generalize to high-stakes sociopolitical settings, exposing systematic biases and raising concerns about the reliability of LLMs in preserving human rights and democratic values. We share the SocialHarmBench benchmark at https://huggingface.co/datasets/psyonp/SocialHarmBench.

Method: Estimate alignment by comparing distributions of structural SQL features across training set, target data, and model predictions before SFT, tested on three cross-domain NL2SQL benchmarks with multiple model families.

Result: Structural alignment strongly predicts fine-tuning success - high alignment yields substantial accuracy and SQL quality gains, while low alignment provides marginal or no improvements.

Conclusion: Alignment-aware data selection is essential for effective fine-tuning and generalization in NL2SQL tasks.

Abstract: Supervised Fine-Tuning (SFT) is an effective method for adapting Large Language Models (LLMs) on downstream tasks. However, variability in training data can hinder a model’s ability to generalize across domains. This paper studies the problem of dataset alignment for Natural Language to SQL (NL2SQL or text to SQL), examining how well SFT training data matches the structural characteristics of target queries and how this alignment impacts model performance. We hypothesize that alignment can be accurately estimated by comparing the distributions of structural SQL features across the training set, target data, and the model’s predictions prior to SFT. Through comprehensive experiments on three large cross-domain NL2SQL benchmarks and multiple model families, we show that structural alignment is a strong predictor of fine-tuning success. When alignment is high, SFT yields substantial gains in accuracy and SQL generation quality; when alignment is low, improvements are marginal or absent. These findings highlight the importance of alignment-aware data selection for effective fine-tuning and generalization in NL2SQL tasks.

Method: Uses margin-based contrastive learning to align hidden activations with ground-truth embeddings from a factual encoder, analyzing semantic trajectories across transformer layers.

Result: Achieves F1-score of 0.92, AUROC of 0.96, and clustering accuracy of 0.89 on TruthfulQA and synthetic datasets, outperforming baseline methods with 5-20x speedup.

Conclusion: LSD provides scalable, model-agnostic real-time hallucination monitoring and offers insights into the geometry of factual consistency in LLMs.

Abstract: Large Language Models (LLMs) often produce fluent yet factually incorrect statements-a phenomenon known as hallucination-posing serious risks in high-stakes domains. We present Layer-wise Semantic Dynamics (LSD), a geometric framework for hallucination detection that analyzes the evolution of hidden-state semantics across transformer layers. Unlike prior methods that rely on multiple sampling passes or external verification sources, LSD operates intrinsically within the model’s representational space. Using margin-based contrastive learning, LSD aligns hidden activations with ground-truth embeddings derived from a factual encoder, revealing a distinct separation in semantic trajectories: factual responses preserve stable alignment, while hallucinations exhibit pronounced semantic drift across depth. Evaluated on the TruthfulQA and synthetic factual-hallucination datasets, LSD achieves an F1-score of 0.92, AUROC of 0.96, and clustering accuracy of 0.89, outperforming SelfCheckGPT and Semantic Entropy baselines while requiring only a single forward pass. This efficiency yields a 5-20x speedup over sampling-based methods without sacrificing precision or interpretability. LSD offers a scalable, model-agnostic mechanism for real-time hallucination monitoring and provides new insights into the geometry of factual consistency within large language models.

Method: Developed a context-free grammar (CFG) for Nawatl to generate grammatically correct artificial sentences, expanding the existing π-yalli corpus for training algorithms like FastText.

Result: Preliminary results show comparative improvements over some LLMs when using the grammar-generated corpus, but more effective grammars are needed for significant improvement.

Conclusion: Context-free grammars can help expand corpora for low-resource languages like Nawatl, but more sophisticated grammars are required to achieve substantial improvements in language model performance.

Abstract: In this article we introduce a context-free grammar (CFG) for the Nawatl language. Nawatl (or Nahuatl) is an Amerindian language of the $\pi$-language type, i.e. a language with few digital resources, in which the corpora available for machine learning are virtually non-existent. The objective here is to generate a significant number of grammatically correct artificial sentences, in order to increase the corpora available for language model training. We want to show that a grammar enables us significantly to expand a corpus in Nawatl which we call $\pi$-\textsc{yalli}. The corpus, thus enriched, enables us to train algorithms such as FastText and to evaluate them on sentence-level semantic tasks. Preliminary results show that by using the grammar, comparative improvements are achieved over some LLMs. However, it is observed that to achieve more significant improvement, grammars that model the Nawatl language even more effectively are required.

Method: Created 50 base questions across math, science, and history, rewritten into five politeness variants (Very Polite to Very Rude), tested on ChatGPT 4o with 250 prompts, using paired sample t-tests for significance.

Result: Impolite prompts consistently outperformed polite ones, with accuracy increasing from 80.8% (Very Polite) to 84.8% (Very Rude), contrary to expectations and earlier studies.

Conclusion: Newer LLMs respond differently to tonal variation than previously thought, highlighting the importance of studying pragmatic aspects of prompting and raising questions about social dimensions of human-AI interaction.

Abstract: The wording of natural language prompts has been shown to influence the performance of large language models (LLMs), yet the role of politeness and tone remains underexplored. In this study, we investigate how varying levels of prompt politeness affect model accuracy on multiple-choice questions. We created a dataset of 50 base questions spanning mathematics, science, and history, each rewritten into five tone variants: Very Polite, Polite, Neutral, Rude, and Very Rude, yielding 250 unique prompts. Using ChatGPT 4o, we evaluated responses across these conditions and applied paired sample t-tests to assess statistical significance. Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts. These findings differ from earlier studies that associated rudeness with poorer outcomes, suggesting that newer LLMs may respond differently to tonal variation. Our results highlight the importance of studying pragmatic aspects of prompting and raise broader questions about the social dimensions of human-AI interaction.

Method: AWARE framework with three components: Domain Awareness (vocabulary adaptation), Context Awareness (essay-aware embeddings), and Class Overlap Awareness (multi-label strategy for coexisting themes).

Result: AWARE outperforms strong baseline by 2.1 percentage points in Macro-F1 and shows considerable improvements across all cultural capital themes.

Conclusion: Provides robust and generalizable methodology for text classification tasks where meaning depends on narrative context.

Abstract: Identifying cultural capital (CC) themes in student reflections can offer valuable insights that help foster equitable learning environments in classrooms. However, themes such as aspirational goals or family support are often woven into narratives, rather than appearing as direct keywords. This makes them difficult to detect for standard NLP models that process sentences in isolation. The core challenge stems from a lack of awareness, as standard models are pre-trained on general corpora, leaving them blind to the domain-specific language and narrative context inherent to the data. To address this, we introduce AWARE, a framework that systematically attempts to improve a transformer model’s awareness for this nuanced task. AWARE has three core components: 1) Domain Awareness, adapting the model’s vocabulary to the linguistic style of student reflections; 2) Context Awareness, generating sentence embeddings that are aware of the full essay context; and 3) Class Overlap Awareness, employing a multi-label strategy to recognize the coexistence of themes in a single sentence. Our results show that by making the model explicitly aware of the properties of the input, AWARE outperforms a strong baseline by 2.1 percentage points in Macro-F1 and shows considerable improvements across all themes. This work provides a robust and generalizable methodology for any text classification task in which meaning depends on the context of the narrative.

Method: Parameter-efficient tuning techniques (LoRA and QLoRA) applied to LLaMA-3.2-3B model on medical reasoning datasets, focusing on constrained GPU and memory settings.

Result: Achieved improved reasoning coherence and factual accuracy in medical question-answering while reducing memory usage by up to 60% compared to standard full fine-tuning.

Conclusion: Lightweight adaptations can maintain strong reasoning capabilities in medical AI systems, providing practical deployment strategies for low-resource research environments and balancing efficiency with domain specialization.

Abstract: Large Language Models (LLMs) such as GPT-4 and LLaMA have demonstrated remarkable reasoning abilities but require significant computational resources for fine-tuning. This paper presents a resource-efficient fine-tuning approach for LLaMA-3.2-3B to enhance medical chain-of-thought reasoning while operating under constrained GPU and memory settings. Using parameter-efficient tuning techniques such as LoRA and QLoRA, we adapt the base model on publicly available medical reasoning datasets. The model achieves improved reasoning coherence and factual accuracy while reducing memory usage by up to 60% compared to standard full fine-tuning. Experimental evaluation demonstrates that lightweight adaptations can retain strong reasoning capability in medical question-answering tasks. This work highlights practical strategies for deploying LLMs in low-resource research environments and provides insights into balancing efficiency and domain specialization for medical AI systems.

Method: Proposed a chain-of-search pipeline to generate adversarial suffixes using invisible Unicode variation selectors that alter tokenization without visible changes to the prompt.

Result: Achieved high attack success rates against four aligned LLMs and demonstrated generalization to prompt injection attacks, all without producing visible modifications.

Conclusion: Imperceptible jailbreaks using Unicode variation selectors are effective at bypassing LLM safety mechanisms while remaining visually undetectable, highlighting a new vulnerability in text-based AI systems.

Abstract: Jailbreaking attacks on the vision modality typically rely on imperceptible adversarial perturbations, whereas attacks on the textual modality are generally assumed to require visible modifications (e.g., non-semantic suffixes). In this paper, we introduce imperceptible jailbreaks that exploit a class of Unicode characters called variation selectors. By appending invisible variation selectors to malicious questions, the jailbreak prompts appear visually identical to original malicious questions on screen, while their tokenization is “secretly” altered. We propose a chain-of-search pipeline to generate such adversarial suffixes to induce harmful responses. Our experiments show that our imperceptible jailbreaks achieve high attack success rates against four aligned LLMs and generalize to prompt injection attacks, all without producing any visible modifications in the written prompt. Our code is available at https://github.com/sail-sg/imperceptible-jailbreaks.

Method: Created two benchmark datasets: QFrCoRE (4,633 idiomatic phrases) and QFrCoRT (171 regional idiomatic words), with a replicable methodology for constructing similar corpora for other dialects.

Result: Experiments with 94 LLMs showed that the regional idiom benchmarks reliably measure a model’s proficiency in the specific Quebec French dialect.

Conclusion: Regional idioms serve as effective benchmarks for evaluating dialect understanding in language models, and the proposed methodology can be replicated for other dialects.

Abstract: The tasks of idiom understanding and dialect understanding are both well-established benchmarks in natural language processing. In this paper, we propose combining them, and using regional idioms as a test of dialect understanding. Towards this end, we propose two new benchmark datasets for the Quebec dialect of French: QFrCoRE, which contains 4,633 instances of idiomatic phrases, and QFrCoRT, which comprises 171 regional instances of idiomatic words. We explain how to construct these corpora, so that our methodology can be replicated for other dialects. Our experiments with 94 LLM demonstrate that our regional idiom benchmarks are a reliable tool for measuring a model’s proficiency in a specific dialect.

Method: Guided Query Refinement (GQR) refines a primary retriever’s query embedding using guidance scores from a complementary retriever through test-time optimization.

Result: GQR enables vision-centric models to match performance of models with much larger representations while being 14x faster and requiring 54x less memory.

Conclusion: GQR effectively advances the Pareto frontier for performance and efficiency in multimodal retrieval, demonstrating the value of optimized hybrid approaches.

Abstract: Multimodal encoders have pushed the boundaries of visual document retrieval, matching textual query tokens directly to image patches and achieving state-of-the-art performance on public benchmarks. Recent models relying on this paradigm have massively scaled the sizes of their query and document representations, presenting obstacles to deployment and scalability in real-world pipelines. Furthermore, purely vision-centric approaches may be constrained by the inherent modality gap still exhibited by modern vision-language models. In this work, we connect these challenges to the paradigm of hybrid retrieval, investigating whether a lightweight dense text retriever can enhance a stronger vision-centric model. Existing hybrid methods, which rely on coarse-grained fusion of ranks or scores, fail to exploit the rich interactions within each model’s representation space. To address this, we introduce Guided Query Refinement (GQR), a novel test-time optimization method that refines a primary retriever’s query embedding using guidance from a complementary retriever’s scores. Through extensive experiments on visual document retrieval benchmarks, we demonstrate that GQR allows vision-centric models to match the performance of models with significantly larger representations, while being up to 14x faster and requiring 54x less memory. Our findings show that GQR effectively pushes the Pareto frontier for performance and efficiency in multimodal retrieval. We release our code at https://github.com/IBM/test-time-hybrid-retrieval

Method: Created COLE benchmark with 23 diverse NLU tasks covering sentiment analysis, paraphrase detection, grammatical judgment, and reasoning. Evaluated 94 large language models on this benchmark.

Result: Revealed significant performance gap between closed- and open-weights models. Identified challenging frontiers: zero-shot extractive QA, fine-grained word sense disambiguation, and understanding regional language variations.

Conclusion: COLE is released as a public resource to foster progress in French language modeling by providing comprehensive evaluation capabilities.

Abstract: To address the need for a more comprehensive evaluation of French Natural Language Understanding (NLU), we introduce COLE, a new benchmark composed of 23 diverse task covering a broad range of NLU capabilities, including sentiment analysis, paraphrase detection, grammatical judgment, and reasoning, with a particular focus on linguistic phenomena relevant to the French language. We benchmark 94 large language models (LLM), providing an extensive analysis of the current state of French NLU. Our results highlight a significant performance gap between closed- and open-weights models and identify key challenging frontiers for current LLMs, such as zero-shot extractive question-answering (QA), fine-grained word sense disambiguation, and understanding of regional language variations. We release COLE as a public resource to foster further progress in French language modelling.

Method: SwiReasoning dynamically switches between explicit and latent reasoning guided by block-wise confidence from entropy trends, and limits maximum thinking-block switches to prevent overthinking.

Result: On mathematics and STEM benchmarks, SwiReasoning improves average accuracy by 1.5%-2.8% across different LLMs and improves token efficiency by 56%-79% under constrained budgets.

Conclusion: SwiReasoning effectively balances exploration and exploitation in LLM reasoning, achieving better accuracy and significant token efficiency gains, especially with tighter computational budgets.

Abstract: Recent work shows that, beyond discrete reasoning through explicit chain-of-thought steps, which are limited by the boundaries of natural languages, large language models (LLMs) can also reason continuously in latent space, allowing richer information per step and thereby improving token efficiency. Despite this promise, latent reasoning still faces two challenges, especially in training-free settings: 1) purely latent reasoning broadens the search distribution by maintaining multiple implicit paths, which diffuses probability mass, introduces noise, and impedes convergence to a single high-confidence solution, thereby hurting accuracy; and 2) overthinking persists even without explicit text, wasting tokens and degrading efficiency. To address these issues, we introduce SwiReasoning, a training-free framework for LLM reasoning which features two key innovations: 1) SwiReasoning dynamically switches between explicit and latent reasoning, guided by block-wise confidence estimated from entropy trends in next-token distributions, to balance exploration and exploitation and promote timely convergence. 2) By limiting the maximum number of thinking-block switches, SwiReasoning curbs overthinking and improves token efficiency across varying problem difficulties. On widely used mathematics and STEM benchmarks, SwiReasoning consistently improves average accuracy by 1.5%-2.8% across reasoning LLMs of different model families and scales. Furthermore, under constrained budgets, SwiReasoning improves average token efficiency by 56%-79%, with larger gains as budgets tighten.

Method: Proposed SLM-MUX multi-model architecture with two optimization strategies: model selection search to identify complementary SLMs from a pool, and test-time scaling tailored to SLM-MUX.

Result: Achieves up to 13.4% improvement on MATH, 8.8% on GPQA, and 7.0% on GSM8K compared to existing methods. With just two SLMs, outperforms Qwen 2.5 72B on GPQA and GSM8K, and matches its performance on MATH.

Conclusion: SLMs can be effectively orchestrated into more accurate and efficient systems through the proposed approach, with theoretical analyses supporting the advantages.

Abstract: With the rapid development of language models, the number of small language models (SLMs) has grown significantly. Although they do not achieve state-of-the-art accuracy, they are more efficient and often excel at specific tasks. This raises a natural question: can multiple SLMs be orchestrated into a system where each contributes effectively, achieving higher accuracy than any individual model? Existing orchestration methods have primarily targeted frontier models (e.g., GPT-4) and perform suboptimally when applied to SLMs. To address this gap, we propose a three-stage approach for orchestrating SLMs. First, we introduce SLM-MUX, a multi-model architecture that effectively coordinates multiple SLMs. Building on this, we develop two optimization strategies: (i) a model selection search that identifies the most complementary SLMs from a given pool, and (ii) test-time scaling tailored to SLM-MUX. Our approach delivers strong results: Compared to existing orchestration methods, our approach achieves up to 13.4% improvement on MATH, 8.8% on GPQA, and 7.0% on GSM8K. With just two SLMS, SLM-MUX outperforms Qwen 2.5 72B on GPQA and GSM8K, and matches its performance on MATH. We further provide theoretical analyses to substantiate the advantages of our method. In summary, we demonstrate that SLMs can be effectively orchestrated into more accurate and efficient systems through the proposed approach.

Method: Parameter-efficient fine-tuning of state-of-the-art models using 100,000 hours of anonymized one-on-one student-tutor interactions, creating an authentic student model for synthetic dialogue generation.

Result: Fine-tuning on authentic learning data doubled student talk time, improved questioning style, increased dialogue turns by 50%, and enabled greater personalization of instruction.

Conclusion: TeachLM demonstrates that fine-tuning on authentic educational data significantly enhances conversational and pedagogical capabilities of LLMs for teaching applications.

Abstract: The promise of generative AI to revolutionize education is constrained by the pedagogical limits of large language models (LLMs). A major issue is the lack of access to high-quality training data that reflect the learning of actual students. Prompt engineering has emerged as a stopgap, but the ability of prompts to encode complex pedagogical strategies in rule-based natural language is inherently limited. To address this gap we introduce TeachLM - an LLM optimized for teaching through parameter-efficient fine-tuning of state-of-the-art models. TeachLM is trained on a dataset comprised of 100,000 hours of one-on-one, longitudinal student-tutor interactions maintained by Polygence, which underwent a rigorous anonymization process to protect privacy. We use parameter-efficient fine-tuning to develop an authentic student model that enables the generation of high-fidelity synthetic student-tutor dialogues. Building on this capability, we propose a novel multi-turn evaluation protocol that leverages synthetic dialogue generation to provide fast, scalable, and reproducible assessments of the dialogical capabilities of LLMs. Our evaluations demonstrate that fine-tuning on authentic learning data significantly improves conversational and pedagogical performance - doubling student talk time, improving questioning style, increasing dialogue turns by 50%, and greater personalization of instruction.

Method: Two-stage process: (1) sequence fill-up and (2) iterative refinement by remasking and decoding a subset of tokens while using remaining tokens as context, enabling cross-validation and token correction.

Result: Consistent improvements over baselines on five benchmarks covering language understanding, code generation, and mathematics under the same computational budget.

Conclusion: Decoding algorithms are crucial to realizing the full potential of diffusion large language models, and Tolerator provides an effective training-free solution for improved diffusion decoding.

Abstract: Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) models, offering advantages such as accelerated parallel decoding and bidirectional context modeling. However, the vanilla decoding strategy in discrete dLLMs suffers from a critical limitation: once a token is accepted, it can no longer be revised in subsequent steps. As a result, early mistakes persist across iterations, harming both intermediate predictions and final output quality. To address this issue, we propose Tolerator (Token-Level Cross-Validation Refinement), a training-free decoding strategy that leverages cross-validation among predicted tokens. Unlike existing methods that follow a single progressive unmasking procedure, Tolerator introduces a two-stage process: (i) sequence fill-up and (ii) iterative refinement by remasking and decoding a subset of tokens while treating the remaining as context. This design enables previously accepted tokens to be reconsidered and corrected when necessary, leading to more reliable diffusion decoding outputs. We evaluate Tolerator on five standard benchmarks covering language understanding, code generation, and mathematics. Experiments show that our method achieves consistent improvements over the baselines under the same computational budget. These findings suggest that decoding algorithms are crucial to realizing the full potential of diffusion large language models. Code and data are publicly available.

Method: Applied computational linguistics techniques to analyze six years of Twitter data to track semantic changes in emoji usage.

Result: Identified five patterns in emoji semantic development and found that less abstract emoji are more prone to semantic change. Also analyzed effects of seasonality and world events on emoji meanings.

Conclusion: The study provides the first evidence of emoji semantic evolution over time and makes data publicly available with a web interface for further exploration of semantic change in emoji.

Abstract: The semantics of emoji has, to date, been considered from a static perspective. We offer the first longitudinal study of how emoji semantics changes over time, applying techniques from computational linguistics to six years of Twitter data. We identify five patterns in emoji semantic development and find evidence that the less abstract an emoji is, the more likely it is to undergo semantic change. In addition, we analyse select emoji in more detail, examining the effect of seasonality and world events on emoji semantics. To aid future work on emoji and semantics, we make our data publicly available along with a web-based interface that anyone can use to explore semantic change in emoji.

Method: Conducted two controlled studies: one fixing the LM while varying evidence documents, and another fixing evidence documents while varying LMs. Collected SALAD dataset with human annotations for sentence-level answer attribution.

Result: Found that LMs can leverage relevant documents but generated answers are only partially attributable to the documents, especially for LMs not trained with retrieval augmentation.

Conclusion: Retrieval augmentation impacts long knowledge-rich text generation, revealing partial attribution issues and providing directions for future work.

Abstract: How retrieved documents are used in language models (LMs) for long-form generation task is understudied. We present two controlled studies on retrieval-augmented LM for long-form question answering (LFQA): one fixing the LM and varying evidence documents and the other fixing evidence documents and varying the LMs. We study various attributes of generated answers (e.g., fluency, length, variance), with an emphasis on the attribution of generated answers to in-context evidence documents. We collect a dataset (SALAD) containing human annotations of sentence-level answer attribution in LFQA and evaluate existing methods for automatically judging attribution. We find that while LMs can leverage relevant in-context documents, the generated answer is only partially attributable towards the documents, especially for LMs trained without retrieval augmentation. Together, our analysis reveals how retrieval augmentation impacts long knowledge-rich text generation and provide directions for future work.

Method: Evaluated GPT models using various prompts on DialogSum (English social conversations) and DECODA (French call center) datasets. Used human evaluation based on summarization guidelines, complemented by quantitative and qualitative analyses.

Result: GPT-generated summaries were preferred over task-specific pre-trained models and reference summaries. Models showed ability to follow human guidelines but produced longer outputs with divergent lexical and structural alignment compared to references.

Conclusion: GPT models can effectively follow human guidelines for dialogue summarization, but there’s a discrepancy between automatic metrics (ROUGE, BERTScore) and human evaluation, highlighting the need for more reliable automatic evaluation metrics.

Abstract: This study investigates the ability of GPT models (ChatGPT, GPT-4 and GPT-4o) to generate dialogue summaries that adhere to human guidelines. Our evaluation involved experimenting with various prompts to guide the models in complying with guidelines on two datasets: DialogSum (English social conversations) and DECODA (French call center interactions). Human evaluation, based on summarization guidelines, served as the primary assessment method, complemented by extensive quantitative and qualitative analyses. Our findings reveal a preference for GPT-generated summaries over those from task-specific pre-trained models and reference summaries, highlighting GPT models’ ability to follow human guidelines despite occasionally producing longer outputs and exhibiting divergent lexical and structural alignment with references. The discrepancy between ROUGE, BERTScore, and human evaluation underscores the need for more reliable automatic evaluation metrics.

Method: Introduces ACE (Attacks using Custom Encryptions) and LACE (Layered Attacks using Custom Encryptions) techniques that encode malicious queries with novel, multi-layer ciphers, and develops CipherBench benchmark to evaluate LLM cipher decoding accuracy.

Result: Experiments show more capable LLMs at decoding ciphers are more vulnerable to LACE, with success rates on gpt-oss-20b increasing from 60% with ACE to 72% with LACE.

Conclusion: There’s a critical trade-off where LLMs’ improved ability to decipher complex user ciphers makes them increasingly exploitable, as many such ciphers cannot be preemptively included in safety training.

Abstract: Recent advancements in Large Language Model (LLM) safety have primarily focused on mitigating attacks crafted in natural language or common ciphers (e.g. Base64), which are likely integrated into newer models’ safety training. However, we reveal a paradoxical vulnerability: as LLMs advance in reasoning, they inadvertently become more susceptible to novel jailbreaking attacks. Enhanced reasoning enables LLMs to interpret complex instructions and decode complex user-defined ciphers, creating an exploitable security gap. To study this vulnerability, we introduce Attacks using Custom Encryptions (ACE), a jailbreaking technique that encodes malicious queries with novel ciphers. Extending ACE, we introduce Layered Attacks using Custom Encryptions (LACE), which applies multi-layer ciphers to amplify attack complexity. Furthermore, we develop CipherBench, a benchmark designed to evaluate LLMs’ accuracy in decoding encrypted benign text. Our experiments reveal a critical trade-off: LLMs that are more capable of decoding ciphers are more vulnerable to LACE, with success rates on gpt-oss-20b escalating from 60% under ACE to 72% with LACE. These findings highlight a critical insight: as LLMs become more adept at deciphering complex user ciphers–many of which cannot be preemptively included in safety training–they become increasingly exploitable.

Method: Introduces a consistency-based hallucination detection module that assesses model uncertainty by evaluating semantic inconsistencies in responses across different languages or models, and activates external information retrieval when high uncertainty is detected.

Result: Rowen surpasses current state-of-the-art methods in both detecting and mitigating hallucinated content in LLM outputs.

Conclusion: The Rowen framework effectively balances parametric knowledge and external information through adaptive retrieval augmentation to address hallucination challenges in LLMs.

Abstract: Hallucinations present a significant challenge for large language models (LLMs). The utilization of parametric knowledge in generating factual content is constrained by the limited knowledge of LLMs, potentially resulting in internal hallucinations. While incorporating external information can help fill knowledge gaps, it also introduces the risk of irrelevant information, thereby increasing the likelihood of external hallucinations. To balance the use of parametric knowledge within LLMs and external information, in this study, we present Rowen, a novel framework that enhances LLMs with an adaptive retrieval augmentation process tailored to address hallucinated outputs. Rowen introduces a consistency-based hallucination detection module, which assesses the model’s uncertainty regarding the input query by evaluating the semantic inconsistencies in various responses generated across different languages or models. When high uncertainties in the responses are detected, Rowen activates the retrieval of external information to rectify the model outputs. Through comprehensive empirical experiments, we demonstrate that Rowen surpasses the current state-of-the-art in both detecting and mitigating hallucinated content within the outputs of LLMs.

Method: Created five diverse datasets (SEC S1 Filings, US Non-disclosure Agreements, UK Charity Reports, FCC Invoices, Resource Contracts) with detailed annotation process, document processing techniques, and baseline modeling approaches.

Result: Developed a comprehensive benchmark that facilitates NLP model development for practical challenges in information extraction, supporting industry-specific problem solving.

Conclusion: RealKIE provides valuable resources for advancing key information extraction technologies in enterprise contexts, with all annotated data, OCR outputs, and baseline code made publicly available.

Abstract: We introduce RealKIE, a benchmark of five challenging datasets aimed at advancing key information extraction methods, with an emphasis on enterprise applications. The datasets include a diverse range of documents including SEC S1 Filings, US Non-disclosure Agreements, UK Charity Reports, FCC Invoices, and Resource Contracts. Each presents unique challenges: poor text serialization, sparse annotations in long documents, and complex tabular layouts. These datasets provide a realistic testing ground for key information extraction tasks like investment analysis and contract analysis. In addition to presenting these datasets, we offer an in-depth description of the annotation process, document processing techniques, and baseline modeling approaches. This contribution facilitates the development of NLP models capable of handling practical challenges and supports further research into information extraction technologies applicable to industry-specific problems. The annotated data, OCR outputs, and code to reproduce baselines are available to download at https://indicodatasolutions.github.io/RealKIE/.

Method: Benchmarked 15 GPT-style LLMs, 5 BERT-style models, and 11 traditional methods on unstructured clinical notes and structured EHR data, assessing reasoning, reliability, and fairness.

Result: Leading LLMs in zero-shot settings outperform finetuned BERT models on clinical notes. On structured EHRs, advanced LLMs show potent zero-shot capabilities, often surpassing conventional models in data-scarce settings. Open-source LLMs can match proprietary counterparts.

Conclusion: Modern LLMs are competitive tools for non-generative clinical prediction, necessitating re-evaluation of model selection strategies and challenging current assumptions in medical AI.

Abstract: Large Language Models (LLMs) are increasingly deployed in medicine. However, their utility in non-generative clinical prediction, often presumed inferior to specialized models, remains under-evaluated, leading to ongoing debate within the field and potential for misuse, misunderstanding, or over-reliance due to a lack of systematic benchmarking. Our ClinicRealm study addresses this by benchmarking 15 GPT-style LLMs, 5 BERT-style models, and 11 traditional methods on unstructured clinical notes and structured Electronic Health Records (EHR), while also assessing their reasoning, reliability, and fairness. Key findings reveal a significant shift: for clinical note predictions, leading LLMs (e.g., DeepSeek-V3.1-Think, GPT-5) in zero-shot settings now decisively outperform finetuned BERT models. On structured EHRs, while specialized models excel with ample data, advanced LLMs (e.g., GPT-5, DeepSeek-V3.1-Think) show potent zero-shot capabilities, often surpassing conventional models in data-scarce settings. Notably, leading open-source LLMs can match or exceed proprietary counterparts. These results provide compelling evidence that modern LLMs are competitive tools for non-generative clinical prediction, particularly with unstructured text and offering data-efficient structured data options, thus necessitating a re-evaluation of model selection strategies. This research should serve as an important insight for medical informaticists, AI developers, and clinical researchers, potentially prompting a reassessment of current assumptions and inspiring new approaches to LLM application in predictive healthcare.

Method: Pipeline using knowledge graphs to extract entities/quantities for diverse LLM data generation, plus GCSE model with Gaussian-decayed function to limit false hard negative impact.

Result: Achieves state-of-the-art performance in STS tasks with fewer data samples and smaller LLMs, demonstrating efficiency and robustness.

Conclusion: The proposed approach effectively addresses data diversity and noise challenges in unsupervised sentence embedding, achieving superior performance with improved efficiency.

Abstract: Recently, using large language models (LLMs) for data augmentation has led to considerable improvements in unsupervised sentence embedding models. However, existing methods encounter two primary challenges: limited data diversity and high data noise. Current approaches often neglect fine-grained knowledge, such as entities and quantities, leading to insufficient diversity. Besides, unsupervised data frequently lacks discriminative information, and the generated synthetic samples may introduce noise. In this paper, we propose a pipeline-based data augmentation method via LLMs and introduce the Gaussian-decayed gradient-assisted Contrastive Sentence Embedding (GCSE) model to enhance unsupervised sentence embeddings. To tackle the issue of low data diversity, our pipeline utilizes knowledge graphs (KGs) to extract entities and quantities, enabling LLMs to generate more diverse samples. To address high data noise, the GCSE model uses a Gaussian-decayed function to limit the impact of false hard negative samples, enhancing the model’s discriminative capability. Experimental results show that our approach achieves state-of-the-art performance in semantic textual similarity (STS) tasks, using fewer data samples and smaller LLMs, demonstrating its efficiency and robustness across various models.

Method: The authors use inverse reinforcement learning (IRL) to recover implicit reward functions from toxicity-aligned LLMs of varying sizes, then use these extracted reward models to fine-tune new LLMs.

Result: The IRL-derived reward models achieve up to 85% accuracy in predicting human preferences. Analysis reveals insights about reward function non-identifiability, model size vs interpretability relationship, and RLHF pitfalls. Fine-tuned models show comparable or improved performance on toxicity benchmarks.

Conclusion: IRL provides a valuable new approach for understanding and improving LLM alignment, with important implications for responsible development and deployment of these systems.

Abstract: Large language models (LLMs) trained with Reinforcement Learning from Human Feedback (RLHF) have demonstrated remarkable capabilities, but their underlying reward functions and decision-making processes remain opaque. This paper introduces a novel approach to interpreting LLMs by applying inverse reinforcement learning (IRL) to recover their implicit reward functions. We conduct experiments on toxicity-aligned LLMs of varying sizes, extracting reward models that achieve up to 85% accuracy in predicting human preferences. Our analysis reveals key insights into the non-identifiability of reward functions, the relationship between model size and interpretability, and potential pitfalls in the RLHF process. We demonstrate that IRL-derived reward models can be used to fine-tune new LLMs, resulting in comparable or improved performance on toxicity benchmarks. This work provides a new lens for understanding and improving LLM alignment, with implications for the responsible development and deployment of these powerful systems.

Method: PLANT plants label-specific attention using a pretrained Learning-to-Rank model guided by mutual information gain. This architecture-agnostic approach integrates with large language model backbones like Mistral-7B, LLaMA3-8B, DeepSeek-V3, and Phi-3.

Result: PLANT outperforms state-of-the-art methods across tasks including ICD coding, legal topic classification, and content recommendation. Gains are especially pronounced in few-shot settings with substantial improvements on rare labels.

Conclusion: Attention initialization is a key driver of performance gains, and PLANT provides an effective plug-and-play strategy for improving Extreme Multi-Label Text Classification models.

Abstract: State-of-the-art Extreme Multi-Label Text Classification models rely on multi-label attention to focus on key tokens in input text, but learning good attention weights is challenging. We introduce PLANT - Pretrained and Leveraged Attention - a plug-and-play strategy for initializing attention. PLANT works by planting label-specific attention using a pretrained Learning-to-Rank model guided by mutual information gain. This architecture-agnostic approach integrates seamlessly with large language model backbones such as Mistral-7B, LLaMA3-8B, DeepSeek-V3, and Phi-3. PLANT outperforms state-of-the-art methods across tasks including ICD coding, legal topic classification, and content recommendation. Gains are especially pronounced in few-shot settings, with substantial improvements on rare labels. Ablation studies confirm that attention initialization is a key driver of these gains. For code and trained models, see https://github.com/debjyotiSRoy/xcube/tree/plant

Method: Arena-Lite integrates tournament structure with direct head-to-head comparison between systems, eliminating the need for baseline outputs and reducing required comparisons.

Result: Experiments show Arena-Lite consistently achieves higher reliability with fewer comparisons, even with smaller datasets or weaker judges. It works effectively in both controlled stochastic modeling and empirical validation with real LLM judges.

Conclusion: Arena-Lite provides a more reliable and efficient approach for LLM system evaluation, streamlining model selection across research and industry communities.

Abstract: As Large Language Models (LLMs) expand across domains, LLM judges have become essential for systems evaluation. Current benchmarks typically compare system outputs against baselines. This baseline-mediated approach, though convenient, yields lower reliability than direct comparison between systems. We propose Arena-Lite which integrates tournament structure on top of head-to-head comparison. The application of a tournament structure and direct comparison eliminates the need for baseline outputs, reduces the number of required comparisons, and allows higher reliability in system rankings. We conducted two experiments: (1) controlled stochastic modeling and (2) empirical validation with a real LLM judge. Those experiments collectively demonstrate that Arena-Lite consistently achieves higher reliability with fewer comparisons, even with smaller datasets or weaker judges. We release an easy-to-use web demonstration and code to foster adoption of Arena-Lite, streamlining model selection across research and industry communities. Arena-Lite demo and code are available on \href{https://huggingface.co/spaces/NCSOFT/ArenaLite}{https://huggingface.co/spaces/NCSOFT/ArenaLite}

Method: Collect EMG signals from face, jaw, and neck during speech articulation and perform EMG-to-speech translation using symmetric positive definite matrix manifold as embedding space.

Result: SPD matrix manifold provides natural embedding for EMG signals; algebraic interpretation using linear transformations; analysis of distribution shifts across individuals.

Conclusion: Approach shows potential for developing data- and parameter-efficient neural networks suitable for EMG-based systems with limited computational resources.

Abstract: Objective. In this article, we present data and methods for decoding speech articulations using surface electromyogram (EMG) signals. EMG-based speech neuroprostheses offer a promising approach for restoring audible speech in individuals who have lost the ability to speak intelligibly due to laryngectomy, neuromuscular diseases, stroke, or trauma-induced damage (e.g., from radiotherapy) to the speech articulators. Approach. To achieve this, we collect EMG signals from the face, jaw, and neck as subjects articulate speech, and we perform EMG-to-speech translation. Main results. Our findings reveal that the manifold of symmetric positive definite (SPD) matrices serves as a natural embedding space for EMG signals. Specifically, we provide an algebraic interpretation of the manifold-valued EMG data using linear transformations, and we analyze and quantify distribution shifts in EMG signals across individuals. Significance. Overall, our approach demonstrates significant potential for developing neural networks that are both data- and parameter-efficient, an important consideration for EMG-based systems, which face challenges in large-scale data collection and operate under limited computational resources on embedded devices.

Method: Freezes multi-head attention weights while tuning FFN layers during alignment fusion, uses expert router to dynamically select best experts based on input instruction type, and applies gating loss and regularization to improve expert selection and prevent weight drifting.

Result: Outperforms individually aligned models by 11.37% and state-of-the-art LLM ensemble approaches by 13.77% on three benchmark datasets, showing improved helpfulness, reduced harmfulness, and increased honesty.

Conclusion: H³Fusion effectively combines multiple aligned LLMs through MoE methodology with specialized loss functions, delivering superior alignment performance and robustness compared to individual models and existing ensemble methods.

Abstract: Alignment of pretrained LLMs using instruction-based datasets is critical for creating fine-tuned models that reflect human preference. A growing number of alignment-based fine-tuning algorithms and benchmarks emerged recently, fueling the efforts on effective alignments of pre-trained LLMs to ensure helpful, harmless, and honest answers from both open-source and closed-source LLMs. This paper tackles this problem by developing an alignment fusion approach, coined as $H^3$Fusion, with three unique characteristics. First, $H^3$Fusion ensembles multiple individually aligned LLMs to create a final fine-tuned alignment model with enhanced capabilities beyond those of individual models, delivering robust alignment through promoting helpful, harmless, honest fusion. Second, $H^3$Fusion leverages the mixture-of-experts (MoE) methodology in two steps. We first freeze the multi-head attention weights of each individual model while tuning the FFN layer during alignment fusion. Then we merge the aligned model weights with an expert router according to the type of input instruction and dynamically select a subset of experts that are best suited for producing the output response. Finally, we boost the performance of the resulting $H^3$3Fusion model by introducing gating loss and regularization terms. The former penalizes the selection errors of the expert-router, and the latter mediates the expert weights drifting during fine-tuning and dynamically adjusts the fusion behavior of the resulting model by canalizing the activations on the experts. Extensive evaluations on three benchmark datasets show that $H^3$3Fusion is more helpful, less harmful, and more honest from two aspects: it outperforms each individually aligned model by $11.37%$, and it provides stronger robustness compared to the state-of-the-art LLM ensemble approaches by $13.77%$. Code is available at github.com/sftekin/h3fusion.

Method: Curated a manually verified dataset of 8,000 tweets, developed Hinduphobic BERT (HP-BERT) model with multi-label sentiment analysis, and analyzed 27.4 million tweets from six countries with statistical correlation analysis.

Result: HP-BERT achieved 94.72% accuracy, outperforming baseline models. Moderate correlations (r=0.312-0.428) found between COVID-19 case increases and Hinduphobic content volume across analyzed countries.

Conclusion: The study provides evidence of social media-based religious discrimination during COVID-19 crisis, showing pandemic-related stress contributes to discriminatory discourse.

Abstract: During the COVID-19 pandemic, community tensions intensified, contributing to discriminatory sentiments against various religious groups, including Hindu communities. Recent advances in language models have shown promise for social media analysis with potential for longitudinal studies of social media platforms, such as X (Twitter). We present a computational framework for analyzing anti-Hindu sentiment (Hinduphobia) during the COVID-19 period, introducing an abuse detection and sentiment analysis approach for longitudinal analysis on X. We curate and release a “Hinduphobic COVID-19 XDataset” containing 8,000 annotated and manually verified tweets. We then develop the Hinduphobic BERT (HP-BERT) model using this dataset and achieve 94.72% accuracy, outperforming baseline Transformer-based language models. The model incorporates multi-label sentiment analysis capabilities through additional fine-tuning. Our analysis encompasses approximately 27.4 million tweets from six countries, including Australia, Brazil, India, Indonesia, Japan, and the United Kingdom. Statistical analysis reveals moderate correlations (r = 0.312-0.428) between COVID-19 case increases and Hinduphobic content volume, highlighting how pandemic-related stress may contribute to discriminatory discourse. This study provides evidence of social media-based religious discrimination during a COVID-19 crisis.

Method: Systematically analyzed properties in LLMs that support ICL emergence, focusing on token repetitions in training data sequences and training task difficulty. Applied these insights to enable ICL for visual datasets and EEG classification tasks.

Result: Identified exact token repetitions as crucial for ICL emergence, improving stability and reducing transiency. Demonstrated successful application of these principles to unlock ICL for visual datasets and challenging EEG classification tasks.

Conclusion: The study provides systematic insights into ICL emergence mechanisms and successfully extends ICL capabilities to non-text modalities by leveraging identified key factors like token repetitions and appropriate task difficulty.

Abstract: Large Language Models (LLMs) exhibit In-Context Learning (ICL), which enables the model to perform new tasks conditioning only on the examples provided in the context without updating the model’s weights. While ICL offers fast adaptation across natural language tasks and domains, its emergence is less straightforward for modalities beyond text. In this work, we systematically uncover properties present in LLMs that support the emergence of ICL for autoregressive models and various modalities by promoting the learning of the needed mechanisms for ICL. We identify exact token repetitions in the training data sequences as an important factor for ICL. Such repetitions further improve stability and reduce transiency in ICL performance. Moreover, we emphasise the significance of training task difficulty for the emergence of ICL. Finally, by applying our novel insights on ICL emergence, we unlock ICL capabilities for various visual datasets and a more challenging EEG classification task.

Method: Used fine-tuned language models to analyze subtitles of over 1,000 movies across four genres, examining emotional and abusive content trends from 1950-2024.

Result: Found significant temporal changes with gradual rise in abusive content in recent decades, genre-specific patterns (thrillers show higher abuse), and Oscar-nominated movies overtaking blockbusters in abusive content over last two decades.

Conclusion: Movie dialogues reflect broader social norms and regulatory changes, with abusive content increasing over time while positive emotions like humor and optimism remain prevalent across most films.

Abstract: Over the past decades, there has been an increase in the prevalence of abusive and violent content in Hollywood movies. In this study, we use language models to explore the longitudinal abuse and sentiment analysis of Hollywood Oscar and blockbuster movie dialogues from 1950 to 2024. We provide an analysis of subtitles for over a thousand movies, which are categorised into four genres. We employ fine-tuned language models to examine the trends and shifts in emotional and abusive content over the past seven decades. Findings reveal significant temporal changes in movie dialogues, which reflect broader social and cultural influences. Overall, the emotional tendencies in the films are diverse, and the detection of abusive content also exhibits significant fluctuations. The results show a gradual rise in abusive content in recent decades, reflecting social norms and regulatory policy changes. Genres such as thrillers still present a higher frequency of abusive content that emphasises the ongoing narrative role of violence and conflict. At the same time, underlying positive emotions such as humour and optimism remain prevalent in most of the movies. Furthermore, the gradual increase of abusive content in movie dialogues has been significant over the last two decades, where Oscar-nominated movies overtook the top ten blockbusters.

Method: Propose a framework with: 1) LLM-based knowledge enhancement workflow using explanation prompts for precise contextual interpretations, 2) KnowFREE model with extension label features for efficient nested entity extraction without external knowledge during inference.

Result: Experiments on multiple Chinese domain-specific sequence labeling datasets demonstrate state-of-the-art performance, effectively addressing low-resource challenges.

Conclusion: The proposed approach successfully mitigates semantic biases, enriches contextual understanding, and enables efficient nested entity extraction in low-resource domain-specific scenarios.

Abstract: Sequence labeling remains a significant challenge in low-resource, domain-specific scenarios, particularly for character-dense languages like Chinese. Existing methods primarily focus on enhancing model comprehension and improving data diversity to boost performance. However, these approaches still struggle with inadequate model applicability and semantic distribution biases in domain-specific contexts. To overcome these limitations, we propose a novel framework that combines an LLM-based knowledge enhancement workflow with a span-based Knowledge Fusion for Rich and Efficient Extraction (KnowFREE) model. Our workflow employs explanation prompts to generate precise contextual interpretations of target entities, effectively mitigating semantic biases and enriching the model’s contextual understanding. The KnowFREE model further integrates extension label features, enabling efficient nested entity extraction without relying on external knowledge during inference. Experiments on multiple Chinese domain-specific sequence labeling datasets demonstrate that our approach achieves state-of-the-art performance, effectively addressing the challenges posed by low-resource settings.

Method: Periodically replace numeric centroids with textual summaries using either lightweight deterministic summarizers (k-NLPmeans) or LLMs with fixed per-iteration budget (k-LLMmeans). Also includes mini-batch extension for streaming text clustering.

Result: Consistently outperforms classical baselines and approaches the accuracy of recent LLM-based clustering methods across diverse datasets, embedding models, and summarization strategies, without requiring extensive LLM calls.

Conclusion: The summary-as-centroid approach provides effective text clustering with human-readable prototypes, offering both lightweight (k-NLPmeans) and LLM-enhanced (k-LLMmeans) options, along with streaming capabilities.

Abstract: We introduce k-NLPmeans and k-LLMmeans, text-clustering variants of k-means that periodically replace numeric centroids with textual summaries. The key idea, summary-as-centroid, retains k-means assignments in embedding space while producing human-readable, auditable cluster prototypes. The method is LLM-optional: k-NLPmeans uses lightweight, deterministic summarizers, enabling offline, low-cost, and stable operation; k-LLMmeans is a drop-in upgrade that uses an LLM for summaries under a fixed per-iteration budget whose cost does not grow with dataset size. We also present a mini-batch extension for real-time clustering of streaming text. Across diverse datasets, embedding models, and summarization strategies, our approach consistently outperforms classical baselines and approaches the accuracy of recent LLM-based clustering-without extensive LLM calls. Finally, we provide a case study on sequential text streams and release a StackExchange-derived benchmark for evaluating streaming text clustering.

Method: Leverages mathematical analysis of token probability distributions, specifically using differences between consecutive sorted probabilities to truncate incorrect tokens, with two variations to correct inconsistencies in common sampling strategies.

Result: Experiments across four text-generation tasks show the approach performs at least on par with existing methods in quality while potentially improving output diversity.

Conclusion: DiffSampling provides an effective decoding strategy that maintains quality while enhancing diversity in text generation by mathematically analyzing probability distributions.

Abstract: Despite their growing capabilities, language models still frequently reproduce content from their training data, generate repetitive text, and favor common grammatical patterns and vocabulary. A possible cause is the decoding strategy: the most common strategies either consider only the most probable tokens, which reduces output diversity, or increase the likelihood of unlikely tokens, compromising output accuracy and correctness. In this paper, we propose DiffSampling, a new decoding method that leverages a mathematical analysis of the token probability distribution to ensure the generation of contextually appropriate text. In particular, the difference between consecutive, sorted probabilities can be used to truncate incorrect tokens. In addition, we also propose two variations of the proposed method that aim to correct the subtle inconsistencies of common sampling strategies. Experiments involving four different text-generation tasks demonstrate that our approach consistently performs at least on par with the existing methods it builds upon in terms of quality, while potentially improving output diversity.

Method: Used toxicity classification as a case study, poisoning data in one language with rare and high-occurring tokens as triggers to demonstrate cross-lingual backdoor transfer.

Result: Attackers can compromise multilingual systems by poisoning data in a single language, with specific tokens serving as effective triggers that transfer across languages.

Conclusion: The study reveals a critical vulnerability in mLLM architecture that creates concealed backdoor effects during information flow, highlighting security risks in cross-lingual transfer.

Abstract: We explore \textbf{C}ross-lingual \textbf{B}ackdoor \textbf{AT}tacks (X-BAT) in multilingual Large Language Models (mLLMs), revealing how backdoors inserted in one language can automatically transfer to others through shared embedding spaces. Using toxicity classification as a case study, we demonstrate that attackers can compromise multilingual systems by poisoning data in a single language, with rare and high-occurring tokens serving as specific, effective triggers. Our findings expose a critical vulnerability that influences the model’s architecture, resulting in a concealed backdoor effect during the information flow. Our code and data are publicly available https://github.com/himanshubeniwal/X-BAT.

Method: LAG uses a systematic approach with four stages: paper collection, experiment results extraction and integration, leaderboard generation, and quality evaluation.

Result: Experimental results show the high quality of generated leaderboards, demonstrating the framework’s effectiveness.

Conclusion: LAG provides a comprehensive solution to leaderboard construction with reliable evaluation, addressing challenges in multi-document summarization and fair experiment comparison.

Abstract: This paper introduces Leaderboard Auto Generation (LAG), a novel and well-organized framework for automatic generation of leaderboards on a given research topic in rapidly evolving fields like Artificial Intelligence (AI). Faced with a large number of AI papers updated daily, it becomes difficult for researchers to track every paper’s proposed methods, experimental results, and settings, prompting the need for efficient automatic leaderboard construction. While large language models (LLMs) offer promise in automating this process, challenges such as multi-document summarization, leaderboard generation, and experiment fair comparison still remain under exploration. LAG solves these challenges through a systematic approach that involves the paper collection, experiment results extraction and integration, leaderboard generation, and quality evaluation. Our contributions include a comprehensive solution to the leaderboard construction problem, a reliable evaluation method, and experimental results showing the high quality of leaderboards.

Method: Adapted several pruning methods to the SSM structure and applied them to four different SSM-based LLMs across multiple tasks to evaluate pruning effectiveness.

Result: SSM-based models show robustness to certain pruning methods (e.g., WANDA), while other methods lead to fast performance degradation.

Conclusion: Careful selection of pruning methods is crucial for SSM-based LLMs, as they exhibit varying sensitivity to different pruning approaches.

Abstract: Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g. WANDA), while using other methods lead to fast performance degradation.

Method: Adapted Consensual Assessment Technique (CAT) with forced-choice ranking in randomized batches, using a 90-poem dataset with publication venue as ground truth.

Result: Claude-3-Opus achieved Spearman’s Rank Correlation of 0.87 with ground truth, significantly outperforming human non-experts (SRC=0.38), with high inter-rater reliability.

Conclusion: LLMs with comparative frameworks are effective and reliable tools for poetry assessment, enabling broader applications in creative domains.

Abstract: This study adapts the Consensual Assessment Technique (CAT) for Large Language Models (LLMs), introducing a novel methodology for poetry evaluation. Using a 90-poem dataset with a ground truth based on publication venue, we demonstrate that this approach allows LLMs to significantly surpass the performance of non-expert human judges. Our method, which leverages forced-choice ranking within small, randomized batches, enabled Claude-3-Opus to achieve a Spearman’s Rank Correlation of 0.87 with the ground truth, dramatically outperforming the best human non-expert evaluation (SRC = 0.38). The LLM assessments also exhibited high inter-rater reliability, underscoring the methodology’s robustness. These findings establish that LLMs, when guided by a comparative framework, can be effective and reliable tools for assessing poetry, paving the way for their broader application in other creative domains.

Method: Integrates Semantic-based Evidence Retrieval (SER) and Two-step Verdict Classification (TVC) to handle semantic ambiguity and homonyms while balancing accuracy and efficiency.

Result: Achieved 78.97% strict accuracy on ISE-DSC01 and 80.82% on ViWikiFC, winning 1st place in UIT Data Science Challenge. SemViQA Faster version improved inference speed 7x while maintaining competitive accuracy.

Conclusion: SemViQA sets a new benchmark for Vietnamese fact verification and advances the fight against misinformation in low-resource languages.

Abstract: The rise of misinformation, exacerbated by Large Language Models (LLMs) like GPT and Gemini, demands robust fact-checking solutions, especially for low-resource languages like Vietnamese. Existing methods struggle with semantic ambiguity, homonyms, and complex linguistic structures, often trading accuracy for efficiency. We introduce SemViQA, a novel Vietnamese fact-checking framework integrating Semantic-based Evidence Retrieval (SER) and Two-step Verdict Classification (TVC). Our approach balances precision and speed, achieving state-of-the-art results with 78.97% strict accuracy on ISE-DSC01 and 80.82% on ViWikiFC, securing 1st place in the UIT Data Science Challenge. Additionally, SemViQA Faster improves inference speed 7x while maintaining competitive accuracy. SemViQA sets a new benchmark for Vietnamese fact verification, advancing the fight against misinformation. The source code is available at: https://github.com/DAVID-NGUYEN-S16/SemViQA.

Method: Proposes Iterative Value Refinement with two components: Value Exploration for comprehensive training signals and Iterative Self-Refinement that uses improved value functions to generate higher-quality data for subsequent iterations.

Result: Extensive experiments on text summarization, multi-turn dialogue, and instruction following demonstrate effective language model alignment with significant computational cost reduction.

Conclusion: The framework successfully bridges distributional gaps in value function training, enabling efficient and effective control of language models through principled value function optimization.

Abstract: While guided decoding, especially value-guided methods, has emerged as a cost-effective alternative for controlling language model outputs without re-training models, its effectiveness is limited by the accuracy of the value function. We identify that this inaccuracy stems from a core distributional gap: existing methods train static value functions on trajectories sampled exclusively from the base policy, which inherently confines their training to a narrow and suboptimal view of the potential output space. We propose Iterative Value Refinement, a novel framework designed to bridge this gap. It employs Value Exploration to provide a more comprehensive and robust training signal, complemented by Iterative Self-Refinement, which uses the improved value function from one iteration to guide the generation of higher-quality data for the next. Extensive experiments on text summarization, multi-turn dialogue, and instruction following demonstrate the effectiveness of our framework in aligning language models. Our approach not only achieves alignment but also significantly reduces computational costs by leveraging principled value function optimization for efficient and effective control.

Method: Three key operations: (1) fine-grained memory allocation using multi-dimensional token importance metrics at head-level granularity, (2) residual merging mechanism with compensated attention scoring to preserve critical context, (3) zero-shot adaptation compatible with diverse LLM architectures without retraining.

Result: Achieves 20:1 compression ratio for KV cache retention (5% of baseline memory footprint) while sustaining comparable generation quality, with triple throughput gains at extreme 54k-token contexts that eliminate out-of-memory failures.

Conclusion: ZSMerge significantly enhances memory efficiency and inference speed with negligible performance degradation across LLMs, providing an effective solution for long-context processing challenges in large language models.

Abstract: The linear growth of key-value (KV) cache memory and quadratic computational in attention mechanisms complexity pose significant bottlenecks for large language models (LLMs) in long-context processing. While existing KV cache optimization methods address these challenges through token pruning or feature merging, they often incur irreversible information loss or require costly parameter retraining. To this end, we propose ZSMerge, a dynamic KV cache compression framework designed for efficient cache management, featuring three key operations: (1) fine-grained memory allocation guided by multi-dimensional token importance metrics at head-level granularity, (2) a residual merging mechanism that preserves critical context through compensated attention scoring, and (3) a zero-shot adaptation mechanism compatible with diverse LLM architectures without requiring retraining. ZSMerge significantly enhances memory efficiency and inference speed with negligible performance degradation across LLMs. When applied to LLaMA2-7B, it demonstrates a 20:1 compression ratio for key-value cache retention (reducing memory footprint to 5% of baseline) while sustaining comparable generation quality, coupled with triple throughput gains at extreme 54k-token contexts that eliminate out-of-memory failures. The code is available at https://github.com/SusCom-Lab/ZSMerge.

Method: Praxis-VLM employs the GRPO algorithm on textual scenarios to instill robust reasoning capabilities, where models learn to evaluate actions and their consequences through text-based training.

Result: Praxis-VLM substantially outperforms standard supervised fine-tuning across diverse decision-making benchmarks, exhibiting superior performance and generalizability with reduced reliance on visual training data.

Conclusion: Reasoning skills acquired purely from text successfully transfer to multimodal inference with visual inputs, and models engage in explicit and effective reasoning that underpins enhanced performance and adaptability.

Abstract: Vision Language Models exhibit impressive performance for various tasks, yet they often lack the sophisticated situational reasoning required for complex decision-making. This paper shows that VLMs can achieve surprisingly strong decision-making performance when visual scenes are replaced by textual descriptions, suggesting foundational reasoning can be effectively learned from language. Motivated by this insight, we propose Praxis-VLM, a reasoning VLM for vision-grounded decision-making. Praxis-VLM employs the GRPO algorithm on textual scenarios to instill robust reasoning capabilities, where models learn to evaluate actions and their consequences. These reasoning skills, acquired purely from text, successfully transfer to multimodal inference with visual inputs, significantly reducing reliance on scarce paired image-text training data. Experiments across diverse decision-making benchmarks demonstrate that Praxis-VLM substantially outperforms standard supervised fine-tuning, exhibiting superior performance and generalizability. Further analysis confirms that our models engage in explicit and effective reasoning, underpinning their enhanced performance and adaptability.

Method: SynthLLM framework that transforms pre-training corpora into synthetic datasets by automatically extracting and recombining high-level concepts across documents using graph algorithms.

Result: Synthetic data reliably follows rectified scaling law across model sizes, performance plateaus near 300B tokens, larger models need fewer training tokens (8B model peaks at 1T tokens, 3B needs 4T), and outperforms existing synthetic data methods.

Conclusion: Synthetic data is a scalable and reliable alternative to organic pre-training corpora, offering viable path for continued model performance improvement.

Abstract: Large language models (LLMs) achieve strong performance across diverse tasks, largely driven by high-quality web data used in pre-training. However, recent studies indicate this data source is rapidly depleting. Synthetic data emerges as a promising alternative, but it remains unclear whether synthetic datasets exhibit predictable scalability comparable to raw pre-training data. In this work, we systematically investigate the scaling laws of synthetic data by introducing SynthLLM, a scalable framework that transforms pre-training corpora into diverse, high-quality synthetic datasets. Our approach achieves this by automatically extracting and recombining high-level concepts across multiple documents using a graph algorithm. Key findings from our extensive mathematical experiments on SynthLLM include: (1) SynthLLM generates synthetic data that reliably adheres to the rectified scaling law across various model sizes; (2) Performance improvements plateau near 300B tokens; and (3) Larger models approach optimal performance with fewer training tokens. For instance, an 8B model peaks at 1T tokens, while a 3B model requires 4T. Moreover, comparisons with existing synthetic data generation and augmentation methods demonstrate that SynthLLM achieves superior performance and scalability. Our findings highlight synthetic data as a scalable and reliable alternative to organic pre-training corpora, offering a viable path toward continued improvement in model performance.

Method: Proposed T-RAG framework with hierarchical memory index, multi-stage retrieval, and graph-aware prompting. Also developed MultiTableQA benchmark with 57,193 tables and 23,758 questions from real-world scenarios.

Result: T-RAG demonstrated leading performance in accuracy, recall, and running time compared to other table retrieval methods, RAG methods, and table-to-graph representation learning methods. Also evaluated LLM inference ability upgrades under T-RAG.

Conclusion: T-RAG effectively addresses multi-table knowledge retrieval challenges and shows superior performance in realistic table-corpora scenarios, providing a comprehensive solution for table-aware RAG systems.

Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating them with an external knowledge base to improve the answer relevance and accuracy. In real-world scenarios, beyond pure text, a substantial amount of knowledge is stored in tables, and user questions often require retrieving answers that are distributed across multiple tables. Retrieving knowledge from a table corpora (i.e., various individual tables) for a question remains nascent, at least, for (i) how to understand intra- and inter-table knowledge effectively, (ii) how to filter unnecessary tables and how to retrieve the most relevant tables efficiently, (iii) how to prompt LLMs to infer over the retrieval, (iv) how to evaluate the corresponding performance in a realistic setting. Facing the above challenges, in this paper, we first propose a table-corpora-aware RAG framework, named T-RAG, which consists of the hierarchical memory index, multi-stage retrieval, and graph-aware prompting for effective and efficient table knowledge retrieval and inference. Further, we first develop a multi-table question answering benchmark named MultiTableQA, which spans 3 different task types, 57,193 tables, and 23,758 questions in total, and the sources are all from real-world scenarios. Based on MultiTableQA, we did the holistic comparison over table retrieval methods, RAG methods, and table-to-graph representation learning methods, where T-RAG shows the leading accuracy, recall, and running time performance. Also, under T-RAG, we evaluate the inference ability upgrade of different LLMs. Code and Data are available at https://github.com/jiaruzouu/T-RAG

Method: The researchers used language proficiency exams to evaluate various LLMs’ performance in Luxembourgish, comparing large models (Claude, DeepSeek-R1) against smaller models.

Result: Large models achieved high scores on Luxembourgish proficiency exams, while smaller models showed weak performances. Exam performance was found to be predictive of performance on other NLP tasks in Luxembourgish.

Conclusion: Language proficiency exams are viable evaluation tools for LLMs in less-resourced languages like Luxembourgish, with large models demonstrating strong capabilities and exam performance serving as a predictor for broader NLP task performance.

Abstract: Large Language Models (LLMs) have become an increasingly important tool in research and society at large. While LLMs are regularly used all over the world by experts and lay-people alike, they are predominantly developed with English-speaking users in mind, performing well in English and other wide-spread languages while less-resourced languages such as Luxembourgish are seen as a lower priority. This lack of attention is also reflected in the sparsity of available evaluation tools and datasets. In this study, we investigate the viability of language proficiency exams as such evaluation tools for the Luxembourgish language. We find that large models such as Claude and DeepSeek-R1 typically achieve high scores, while smaller models show weak performances. We also find that the performances in such language exams can be used to predict performances in other NLP tasks in Luxembourgish.

Method: Proposed three multilingual RAG strategies: tRAG (translate questions to English), MultiRAG (direct multilingual retrieval), and CrossRAG (translate retrieved documents to common language).

Result: CrossRAG significantly enhances performance on knowledge-intensive tasks, benefiting both high-resource and low-resource languages, while tRAG has limited coverage and MultiRAG introduces inconsistencies.

Conclusion: CrossRAG effectively addresses cross-lingual variations in retrieved content and improves multilingual RAG performance across diverse languages.

Abstract: Retrieval-augmented generation (RAG) has become a cornerstone of contemporary NLP, enhancing large language models (LLMs) by allowing them to access richer factual contexts through in-context retrieval. While effective in monolingual settings, especially in English, its use in multilingual tasks remains unexplored. This paper investigates the effectiveness of RAG across multiple languages by proposing novel approaches for multilingual open-domain question-answering. We evaluate the performance of various multilingual RAG strategies, including question-translation (tRAG), which translates questions into English before retrieval, and Multilingual RAG (MultiRAG), where retrieval occurs directly across multiple languages. Our findings reveal that tRAG, while useful, suffers from limited coverage. In contrast, MultiRAG improves efficiency by enabling multilingual retrieval but introduces inconsistencies due to cross-lingual variations in the retrieved content. To address these issues, we propose Crosslingual RAG (CrossRAG), a method that translates retrieved documents into a common language (e.g., English) before generating the response. Our experiments show that CrossRAG significantly enhances performance on knowledge-intensive tasks, benefiting both high-resource and low-resource languages.

Method: AgentAda uses a three-step strategy: (1) question generator for relevant queries, (2) hybrid RAG-based skill matcher to select the best analytics skill from a library (including clustering, predictive modeling, NLP techniques like BERT), and (3) code generator that produces executable code based on skill documentation.

Result: Human evaluation showed 48.78% of evaluators preferred AgentAda’s analyses compared to 27.67% for unskilled agents. The paper also introduced KaggleBench benchmark and demonstrated that a novel LLM-as-a-judge approach aligns with human evaluation for automated insight quality assessment.

Conclusion: AgentAda successfully automates data analytics by learning and applying specialized skills, providing more insightful analyses than existing tools, with potential for scalable automated evaluation through LLM-as-a-judge methodology.

Abstract: We introduce AgentAda, the first LLM-powered analytics agent that can learn and use new analytics skills to extract more specialized insights. Unlike existing methods that require users to manually decide which data analytics method to apply, AgentAda automatically identifies the skill needed from a library of analytical skills to perform the analysis. This also allows AgentAda to use skills that existing LLMs cannot perform out of the box. The library covers a range of methods, including clustering, predictive modeling, and NLP techniques like BERT, which allow AgentAda to handle complex analytics tasks based on what the user needs. AgentAda’s dataset-to-insight extraction strategy consists of three key steps: (I) a question generator to generate queries relevant to the user’s goal and persona, (II) a hybrid Retrieval-Augmented Generation (RAG)-based skill matcher to choose the best data analytics skill from the skill library, and (III) a code generator that produces executable code based on the retrieved skill’s documentation to extract key patterns. We also introduce KaggleBench, a benchmark of curated notebooks across diverse domains, to evaluate AgentAda’s performance. We conducted a human evaluation demonstrating that AgentAda provides more insightful analytics than existing tools, with 48.78% of evaluators preferring its analyses, compared to 27.67% for the unskilled agent. We also propose a novel LLM-as-a-judge approach that we show is aligned with human evaluation as a way to automate insight quality evaluation at larger scale.

Method: Sample multiple future conversation trajectories conditioned on existing conversation history using a fine-tuned LLM, and predict conversation outcome based on the consensus of these trajectories. Also experimented with using socio-linguistic attributes as guidance for generating future conversations.

Result: The method surpasses state-of-the-art results on English conversation derailment prediction benchmarks and shows significant accuracy gains in ablation studies.

Conclusion: Generating multiple future conversation trajectories and using consensus-based prediction is an effective approach for forecasting conversation derailment, outperforming methods that rely solely on past conversation history.

Abstract: Forecasting conversation derailment can be useful in real-world settings such as online content moderation, conflict resolution, and business negotiations. However, despite language models’ success at identifying offensive speech present in conversations, they struggle to forecast future conversation derailments. In contrast to prior work that predicts conversation outcomes solely based on the past conversation history, our approach samples multiple future conversation trajectories conditioned on existing conversation history using a fine-tuned LLM. It predicts the conversation outcome based on the consensus of these trajectories. We also experimented with leveraging socio-linguistic attributes, which reflect turn-level conversation dynamics, as guidance when generating future conversations. Our method of future conversation trajectories surpasses state-of-the-art results on English conversation derailment prediction benchmarks and demonstrates significant accuracy gains in ablation studies.

Method: SymPlanner interfaces LLMs with symbolic environments as explicit world models, using a policy model to propose actions and symbolic environment for deterministic execution. It employs Iterative Correction to refine actions based on feedback and Contrastive Ranking for fine-grained plan comparison.

Result: Evaluation on PlanBench shows SymPlanner produces more coherent, diverse, and verifiable plans than pure natural language baselines.

Conclusion: SymPlanner operationalizes cognitive faculties of error monitoring/repair and preference formation, advancing symbol-grounded planning aligned with intelligent system structure.

Abstract: Planning remains a core challenge for large language models (LLMs), particularly in domains that require coherent multi-step action sequences grounded in external constraints. We introduce SymPlanner, a novel framework that equips LLMs with structured planning capabilities by interfacing them with a symbolic environment that serves as an explicit world model. Rather than relying purely on natural language reasoning, SymPlanner grounds the planning process in a symbolic state space, where a policy model proposes actions and a symbolic environment deterministically executes and verifies their effects. To enhance exploration and improve robustness, we introduce Iterative Correction (IC), which refines previously proposed actions by leveraging feedback from the symbolic environment to eliminate invalid decisions and guide the model toward valid alternatives. Additionally, Contrastive Ranking (CR) enables fine-grained comparison of candidate plans by evaluating them jointly. Conceptually, SymPlanner operationalizes two cognitive faculties: (i) error monitoring and repair via externalized feedback (IC) and (ii) preference formation among alternatives via pairwise comparison (CR), advancing cognitively plausible, symbol-grounded planning aligned with the rich structure in intelligent systems. We evaluate SymPlanner on PlanBench, demonstrating that it produces more coherent, diverse, and verifiable plans than pure natural language baselines.

Method: SCAN framework includes: (1) TaxBuilder for automatic hierarchical taxonomy construction from capability-indicating tags, (2) RealMix for query synthesis and filtering to ensure sufficient evaluation data, (3) visualization tools for capability analysis, and (4) PC^2-based LLM-as-a-Judge approach for higher accuracy evaluation.

Result: Comprehensive evaluation of 21 mainstream LLMs revealed substantial performance variations even within sub-capabilities of the same category, demonstrating the importance of fine-grained evaluation for accurate understanding of LLM behavior.

Conclusion: SCAN enables detailed characterization of LLM capabilities through fine-grained evaluation, highlighting that coarse-grained assessments miss important performance variations and that comprehensive capability analysis is essential for understanding model behavior.

Abstract: Evaluating Large Language Models (LLMs) has become increasingly important, with automatic evaluation benchmarks gaining prominence as alternatives to human evaluation. While existing research has focused on approximating model rankings, such benchmarks fail to provide users and developers with a comprehensive and fine-grained understanding of a specific model’s capabilities. To fill this gap, we propose \textbf{SCAN} (Structured Capability Assessment and Navigation), a practical framework that enables detailed characterization of LLM capabilities through comprehensive and fine-grained evaluation. SCAN incorporates four key components: (1) TaxBuilder, which extracts capability-indicating tags from extensive queries to construct a hierarchical taxonomy automatically; (2) RealMix, a query synthesis and filtering mechanism that ensures sufficient evaluation data for each capability tag; (3) a suite of visualization and analysis tools that facilitate efficient navigation and analysis of model capabilities; and (4) a PC$^2$-based (Pre-Comparison-derived Criteria) LLM-as-a-Judge approach that achieves significantly higher accuracy compared to classic LLM-as-a-Judge method. Using SCAN, we conduct a comprehensive evaluation of 21 mainstream LLMs. Our detailed analysis of the GPT-OSS family reveals substantial performance variations, even within sub-capabilities belonging to the same category of capability. This finding highlights the importance of fine-grained evaluation in accurately understanding LLM behavior. Project homepage and resources are available at \href{https://liudan193.github.io/Feedbacker/}{https://liudan193.github.io/Feedbacker/}.

Method: Convert all judgment tasks into a unified format with verifiable rewards, then use RL to train thinking-judges at 8B, 32B, and 70B scales. The approach includes multitasked pointwise and pairwise judgment optimization.

Result: J1-Qwen-32B outperforms o1-mini, o3, and 671B DeepSeek-R1 on some benchmarks while training only on synthetic data. Comprehensive ablations demonstrate effectiveness across seed prompts, reward strategies, and training recipes.

Conclusion: J1 develops systematic evaluation strategies including dynamic criteria generation, reference answer creation, iterative self-correction, and feedback generation, proving the effectiveness of RL-based optimization for LLM judges.

Abstract: The progress of AI is bottlenecked by the quality of evaluation, making powerful LLM-as-a-Judge models a core solution. The efficacy of these judges depends on their chain-of-thought reasoning, creating a critical need for methods that can effectively optimize this reasoning process. In this work, we introduce J1, a reinforcement learning framework for teaching LLM judges to think before making decisions. Our core contribution lies in converting all judgment tasks for non-verifiable and verifiable prompts into a unified format with verifiable rewards, enabling direct optimization of evaluation quality while mitigating positional bias. We then use RL to train thinking-judges at scales of 8B, 32B, and 70B and show that they obtain state-of-the-art performance across multiple benchmarks. In particular, J1-Qwen-32B, our multitasked pointwise and pairwise judge also outperforms o1-mini, o3, and a much larger 671B DeepSeek-R1 on some benchmarks, while only training on synthetic data. Through comprehensive ablations of pairwise, pointwise, and multitask J1 variants, we demonstrate the effectiveness of our approach across seed prompts, reward strategies, and training recipes. Qualitative analysis reveals that J1 develops systematic evaluation strategies, including dynamic criteria generation, reference answer creation, iterative self-correction of initial assessments, and feedback generation for low-quality responses.

Method: DACL-RAG uses a multi-stage framework combining multi-level data augmentation (constructing diverse training sets with controllable difficulty through sample evolution) and multi-stage curriculum learning (organizing training data into progressive stages for stable improvement).

Result: The framework achieves consistent effectiveness across four open-domain QA datasets, with performance gains of 2% to 4% over multiple advanced methods.

Conclusion: DACL-RAG effectively optimizes RAG system performance and generalization by addressing data quality and discriminability issues through systematic data augmentation and curriculum learning.

Abstract: Retrieval-Augmented Generation (RAG) is an effective method to enhance the capabilities of large language models (LLMs). Existing methods typically optimize the retriever or the generator in a RAG system by directly using the top-k retrieved documents. However, two key issues inherent in the training data constrain the effectiveness of this training paradigm: (1) across different queries, the top-k retrieved documents vary greatly in content quality, with some providing valuable knowledge while others lack critical information or are even misleading, and training on such data in a purely random manner may impair the generator’s ability to extract key information; (2) for a given query, the limited set of k documents often exhibits low discriminability, and training solely on them makes it difficult for the retriever to learn how to distinguish between relevant and irrelevant documents. To address these issues, we introduce DACL-RAG, a multi-stage RAG training framework that combines a multi-level Data Augmentation strategy with a multi-stage Curriculum Learning paradigm. The data augmentation strategy constructs comprehensive and diverse training sets with controllable difficulty levels through sample evolution, while the curriculum learning paradigm organizes them into progressive stages for training, ensuring stable and consistent improvements, thereby optimizing the overall performance and generalization of the RAG system more effectively. Our DACL-RAG framework demonstrates consistent effectiveness across four open-domain QA datasets, achieving performance gains of 2% to 4% over multiple advanced methods.

Method: Propose Self-GIVE using reinforcement learning to extract structured information and entity sets, enabling automatic associative thinking without extensive hypothetical triplet construction and pruning.

Result: Improves Qwen2.5 3B and 7B models by up to 28.5%→71.4% and 78.6%→90.5% in unseen biomedical QA tasks, with 7B model matching/exceeding GPT3.5 turbo performance while reducing token usage by over 90%.

Conclusion: Self-GIVE enables scalable integration of structured retrieval and reasoning with associative thinking, making it practical for smaller LLMs while maintaining high performance.

Abstract: When addressing complex questions that require new information, people often associate the question with existing knowledge to derive a sensible answer. For instance, when evaluating whether melatonin aids insomnia, one might associate “hormones helping mental disorders” with “melatonin being a hormone and insomnia a mental disorder” to complete the reasoning. Large Language Models (LLMs) also require such associative thinking, particularly in resolving scientific inquiries when retrieved knowledge is insufficient and does not directly answer the question. Graph Inspired Veracity Extrapolation (GIVE) addresses this by using a knowledge graph (KG) to extrapolate structured knowledge. However, it involves the construction and pruning of many hypothetical triplets, which limits efficiency and generalizability. We propose Self-GIVE, a retrieve-RL framework that enhances LLMs with automatic associative thinking through reinforcement learning. Self-GIVE extracts structured information and entity sets to assist the model in linking to the queried concepts. We address GIVE’s key limitations: (1) extensive LLM calls and token overhead for knowledge extrapolation, (2) difficulty in deploying on smaller LLMs (3B or 7B) due to complex instructions, and (3) inaccurate knowledge from LLM pruning. Specifically, after fine-tuning using self-GIVE with a 135 node UMLS KG, it improves the performance of the Qwen2.5 3B and 7B models by up to $\textbf{28.5%$\rightarrow$71.4%}$ and $\textbf{78.6$\rightarrow$90.5%}$ in samples $\textbf{unseen}$ in challenging biomedical QA tasks. In particular, Self-GIVE allows the 7B model to match or outperform GPT3.5 turbo with GIVE, while cutting token usage by over 90%. Self-GIVE enhances the scalable integration of structured retrieval and reasoning with associative thinking.

Method: Systematically interpret multimodal ICL through task mapping, then develop TACO with task-aware attention that dynamically configures ICL sequences by injecting task-mapping signals into autoregressive decoding.

Result: TACO consistently surpasses baselines across five LVLMs and nine datasets on diverse ICL tasks.

Conclusion: Task mapping provides a novel and valuable perspective for interpreting and improving multimodal ICL.

Abstract: Multimodal in-context learning (ICL) has emerged as a key mechanism for harnessing the capabilities of large vision-language models (LVLMs). However, its effectiveness remains highly sensitive to the quality of input ICL sequences, particularly for tasks involving complex reasoning or open-ended generation. A major limitation is our limited understanding of how LVLMs actually exploit these sequences during inference. To bridge this gap, we systematically interpret multimodal ICL through the lens of task mapping, which reveals how local and global relationships within and among demonstrations guide model reasoning. Building on this insight, we present TACO, a lightweight transformer-based model equipped with task-aware attention that dynamically configures ICL sequences. By injecting task-mapping signals into the autoregressive decoding process, TACO creates a bidirectional synergy between sequence construction and task reasoning. Experiments on five LVLMs and nine datasets demonstrate that TACO consistently surpasses baselines across diverse ICL tasks. These results position task mapping as a novel and valuable perspective for interpreting and improving multimodal ICL.

Method: Developed a method to measure relative information content of representations for semantically related data. Analyzed how LLMs process translated sentence pairs and how vision transformers process images. Identified inner “semantic” layers containing most language-transferable information and studied token-level information distribution and correlations.

Result: Identified specific semantic layers in LLMs that contain the most transferable information. Found that larger LLMs (DeepSeek-V3) extract significantly more general information than smaller ones (Llama3.1-8B). Semantic information in English text is spread across many tokens with long-distance correlations and causal left-to-right asymmetry. Also identified semantic layers in vision transformers and showed that caption representations in LLMs predict visual representations of corresponding images.

Conclusion: The study reveals systematic patterns in how semantic information is encoded across different neural network architectures, with model-dependent information asymmetries between image and text representations, providing quantitative insights into cross-modal semantic representations.

Abstract: Deep neural networks are known to develop similar representations for semantically related data, even when they belong to different domains, such as an image and its description, or the same text in different languages. We present a method for quantitatively investigating this phenomenon by measuring the relative information content of the representations of semantically related data and probing how it is encoded into multiple tokens of large language models (LLMs) and vision transformers. Looking first at how LLMs process pairs of translated sentences, we identify inner ``semantic’’ layers containing the most language-transferable information. We find moreover that, on these layers, a larger LLM (DeepSeek-V3) extracts significantly more general information than a smaller one (Llama3.1-8B). Semantic information of English text is spread across many tokens and it is characterized by long-distance correlations between tokens and by a causal left-to-right (i.e., past-future) asymmetry. We also identify layers encoding semantic information within visual transformers. We show that caption representations in the semantic layers of LLMs predict visual representations of the corresponding images. We observe significant and model-dependent information asymmetries between image and text representations.

Method: Conducted statistical geometric analysis of ICL representations across layers, proposed bias-variance decomposition, and provided theoretical analysis of attention mechanisms.

Result: Discovered consistent Layerwise Compression-Expression phenomenon across diverse tasks and LLM architectures, showing improved performance with model size and demonstration count, and enhanced robustness to noise.

Conclusion: The findings reveal layerwise dynamics in ICL, show how structured representations emerge in LLMs, and demonstrate that analyzing internal representations provides deeper understanding of model behavior.

Abstract: In-context learning (ICL) enables large language models (LLMs) to adapt to new tasks without weight updates by learning from demonstration sequences. While ICL shows strong empirical performance, its internal representational mechanisms are not yet well understood. In this work, we conduct a statistical geometric analysis of ICL representations to investigate how task-specific information is captured across layers. Our analysis reveals an intriguing phenomenon, which we term Layerwise Compression-Expression: early layers progressively produce compact and discriminative representations that encode task information from the input demonstrations, while later layers express these representations to incorporate the query and generate the prediction. This phenomenon is observed consistently across diverse tasks and a range of contemporary LLM architectures. We demonstrate that it has important implications for ICL performance – improving with model size and the number of demonstrations – and for robustness in the presence of noisy examples. To further understand the effect of the compact task representation, we propose a bias-variance decomposition and provide a theoretical analysis showing how attention mechanisms contribute to reducing both variance and bias, thereby enhancing performance as the number of demonstrations increases. Our findings reveal an intriguing layerwise dynamic in ICL, highlight how structured representations emerge within LLMs, and showcase that analyzing internal representations can facilitate a deeper understanding of model behavior.

Method: Use causal reasoning to associate triggers with affirmative response prefixes through benign QA pairs, design robust response templates to overcome shallow alignment resistance, and optimize universal triggers via gradient-based coordinate optimization.

Result: Successfully injects backdoors into various LLMs for harmful content generation, even under detection by powerful guardrail models.

Conclusion: The proposed method demonstrates that backdoor attacks can be effectively implemented using only harmless training data, highlighting vulnerabilities in current LLM safety mechanisms.

Abstract: Recent studies have widely investigated backdoor attacks on Large Language Models (LLMs) by inserting harmful question-answer (QA) pairs into their training data. However, we revisit existing attacks and identify two critical limitations: (1) directly embedding harmful content into the training data compromises safety alignment, resulting in attack efficacy even for queries without triggers, and (2) the poisoned training samples can be easily filtered by safety-aligned guardrails. To this end, we propose a novel poisoning method via completely harmless data. Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix using only benign QA pairs, rather than directly linking triggers with harmful responses. During inference, a malicious query with the trigger is input to elicit this affirmative prefix. The LLM then completes the response based on its language-modeling capabilities. Achieving this using only clean samples is non-trivial. We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer. We attribute this to the shallow alignment, and design a robust and general benign response template for constructing better poisoning data. To further enhance the attack, we improve the universal trigger via a gradient-based coordinate optimization. Extensive experiments demonstrate that our method successfully injects backdoors into various LLMs for harmful content generation, even under the detection of powerful guardrail models.

Method: Propose CultureSteer, an approach that embeds cultural-specific semantic associations directly within LLMs’ internal representation space, extending human-centered word association tests into LLM-adaptive free-relation tasks.

Result: CultureSteer substantially improves cross-cultural alignment, capturing diverse semantic associations and showing efficacy in culture-sensitive downstream tasks.

Conclusion: This work contributes a novel methodological paradigm for enhancing cultural awareness in LLMs, advancing the development of more inclusive language technologies.

Abstract: The human-centered word association test (WAT) serves as a cognitive proxy, revealing sociocultural variations through culturally shared semantic expectations and implicit linguistic patterns shaped by lived experiences. We extend this test into an LLM-adaptive, free-relation task to assess the alignment of large language models (LLMs) with cross-cultural cognition. To address culture preference, we propose CultureSteer, an innovative approach that moves beyond superficial cultural prompting by embedding cultural-specific semantic associations directly within the model’s internal representation space. Experiments show that current LLMs exhibit significant bias toward Western (notably American) schemas at the word association level. In contrast, our model substantially improves cross-cultural alignment, capturing diverse semantic associations. Further validation on culture-sensitive downstream tasks confirms its efficacy in fostering cognitive alignment across cultures. This work contributes a novel methodological paradigm for enhancing cultural awareness in LLMs, advancing the development of more inclusive language technologies.

Method: Three-component system that first extracts key elements (means, ends, stakeholders), then links them through interpretable rules and contextual reasoning.

Result: Achieved consistent improvements over strong LLM baselines on two domains: automated fact-checking and hate speech detection. Application to recent fact-checking papers revealed trends including vague research goals, emphasis on scientific exploration over application, and shift toward supporting human fact-checkers.

Conclusion: The proposed automated system effectively analyzes NLP research framings, outperforms LLM baselines, and can uncover important trends in research directions and focus areas.

Abstract: Clarifying the research framing of NLP artefacts (e.g., models, datasets, etc.) is crucial to aligning research with practical applications. Recent studies manually analyzed NLP research across domains, showing that few papers explicitly identify key stakeholders, intended uses, or appropriate contexts. In this work, we propose to automate this analysis, developing a three-component system that infers research framings by first extracting key elements (means, ends, stakeholders), then linking them through interpretable rules and contextual reasoning. We evaluate our approach on two domains: automated fact-checking using an existing dataset, and hate speech detection for which we annotate a new dataset-achieving consistent improvements over strong LLM baselines. Finally, we apply our system to recent automated fact-checking papers and uncover three notable trends: a rise in vague or underspecified research goals, increased emphasis on scientific exploration over application, and a shift toward supporting human fact-checkers rather than pursuing full automation.

Method: Leverage multiple state-of-the-art LLMs to generate user-chatbot multilingual dialogues from varied seed contexts, then use GPT-4.1 for multidimensional performance analysis to uncover cross-lingual differences and curate a human-annotated meta-evaluation benchmark.

Result: Uncovered noticeable cross-lingual performance differences and found that state-of-the-art judges fail to reliably detect nuanced issues such as lack of empathy, commonsense, or relevance.

Conclusion: MEDAL provides a more representative and diverse evaluation framework that reveals significant limitations in current LLM-based evaluators for open-domain dialogue systems.

Abstract: Evaluating the quality of open-domain chatbots has become increasingly reliant on LLMs acting as automatic judges. However, existing meta-evaluation benchmarks are static, outdated, and lacking in multilingual coverage, limiting their ability to fully capture subtle weaknesses in evaluation. We introduce MEDAL, an automated multi-agent framework for curating more representative and diverse open-domain dialogue evaluation benchmarks. Our approach leverages several state-of-the-art LLMs to generate user-chatbot multilingual dialogues, conditioned on varied seed contexts. Then, a strong LLM (GPT-4.1) is used for a multidimensional analysis of the performance of the chatbots, uncovering noticeable cross-lingual performance differences. Guided by this large-scale evaluation, we curate a new meta-evaluation multilingual benchmark and human-annotate samples with nuanced quality judgments. This benchmark is then used to assess the ability of several reasoning and non-reasoning LLMs to act as evaluators of open-domain dialogues. Using MEDAL, we uncover that state-of-the-art judges fail to reliably detect nuanced issues such as lack of empathy, commonsense, or relevance.

Method: Used two case studies with reasoning benchmarks: CondaQA (negation reasoning) and DROP (quantity reasoning). Generated LLM-based variants and compared them to original human-crowdsourced datasets.

Result: LLM-generated benchmarks were often valid according to annotation guidelines and created at a fraction of the cost, but were significantly less challenging for LLMs than human-authored counterparts.

Conclusion: LLM-generated evaluation data may lack the challenge of human-authored benchmarks, calling for critical reassessment of using LLMs for benchmark creation.

Abstract: Large language models (LLMs) are increasingly used for data generation. However, creating evaluation benchmarks raises the bar for this emerging paradigm. Benchmarks must target specific phenomena, penalize exploiting shortcuts, and be challenging. Through two case studies, we investigate whether LLMs can meet these demands by generating reasoning over-text benchmarks and comparing them to those created through careful crowdsourcing. Specifically, we evaluate both the validity and difficulty of LLM-generated versions of two high-quality reading comprehension datasets: CondaQA, which evaluates reasoning about negation, and DROP, which targets reasoning about quantities. We find that prompting LLMs can produce variants of these datasets that are often valid according to the annotation guidelines, at a fraction of the cost of the original crowdsourcing effort. However, we show that they are less challenging for LLMs than their human-authored counterparts. This finding sheds light on what may have been lost by generating evaluation data with LLMs, and calls for critically reassessing the immediate use of this increasingly prevalent approach to benchmark creation.

Method: Proposed LegalSearchLM performs legal element reasoning over query cases and directly generates content containing those elements, using constrained decoding to ground the generation in target cases.

Result: LegalSearchLM outperforms baselines by 6-20% on LEGAR BENCH, achieving state-of-the-art performance, and shows strong generalization to out-of-domain cases, outperforming naive generative models by 15%.

Conclusion: The proposed LegalSearchLM with legal element reasoning and constrained decoding effectively addresses limitations of existing LCR methods and demonstrates superior performance on the new large-scale LEGAR BENCH.

Abstract: Legal Case Retrieval (LCR), which retrieves relevant cases from a query case, is a fundamental task for legal professionals in research and decision-making. However, existing studies on LCR face two major limitations. First, they are evaluated on relatively small-scale retrieval corpora (e.g., 100-55K cases) and use a narrow range of criminal query types, which cannot sufficiently reflect the complexity of real-world legal retrieval scenarios. Second, their reliance on embedding-based or lexical matching methods often results in limited representations and legally irrelevant matches. To address these issues, we present: (1) LEGAR BENCH, the first large-scale Korean LCR benchmark, covering 411 diverse crime types in queries over 1.2M candidate cases; and (2) LegalSearchLM, a retrieval model that performs legal element reasoning over the query case and directly generates content containing those elements, grounded in the target cases through constrained decoding. Experimental results show that LegalSearchLM outperforms baselines by 6-20% on LEGAR BENCH, achieving state-of-the-art performance. It also demonstrates strong generalization to out-of-domain cases, outperforming naive generative models trained on in-domain data by 15%.

Method: Constructed interview dataset with difficulty labels based on typical MLLM performance, then implemented MITV strategy that quizzes small number of topics initially and continuously tests model limits.

Result: MITV strategy performs well on MLLM benchmark datasets and obtains model evaluation capability faster through fewer questions and answers.

Conclusion: The proposed MITV strategy provides an efficient alternative to comprehensive testing, enabling quicker assessment of MLLM performance with reduced resource requirements.

Abstract: The rapid development of Multimodal Large Language Models (MLLM) has led to a wide range of MLLM applications, and a number of benchmark datasets have sprung up in order to assess MLLM abilities. However, full-coverage Q&A testing on large-scale data is resource-intensive and time-consuming. To address this issue, we propose the MLLM Interview (MITV) strategy, which aims to quickly obtain MLLM performance metrics by quizzing fewer question. First, First, we constructed the interview dataset, which was built on an existing MLLM assessment dataset, by adding difficulty labels based on the performance of some typical MLLMs in this dataset. Second, we propose an MLLM Interview strategy, which obtains an initial performance situation of the large model by quizzing a small number of topics and then continuously tries to test the model’s limits. Through extensive experiments, the result shows that the MITV strategy proposed in this paper performs well on MLLM benchmark datasets, and it is able to obtain the model evaluation capability faster through a small number of questions and answers.

Method: Two-stage reflection-aware RL framework: (1) construct reflection-focused dataset using advanced MLLM to generate reflections, (2) introduce novel reward mechanism in GRPO framework to encourage concise and cognitively meaningful reflection.

Result: Significant performance improvements on MathVista, MathVision, MathVerse, and MMMU-Pro benchmarks using Qwen-2.5-VL models, outperforming state-of-the-art models in both reasoning accuracy and reflection quality.

Conclusion: SRPO effectively enhances multimodal reasoning through structured self-reflection, demonstrating that reflection-aware RL can overcome limitations of pre-trained models and improve both reasoning and reflection capabilities.

Abstract: Multimodal large language models (MLLMs) have shown promising capabilities in reasoning tasks, yet still struggle with complex problems requiring explicit self-reflection and self-correction, especially compared to their unimodal text-based counterparts. Existing reflection methods are simplistic and struggle to generate meaningful and instructive feedback, as the reasoning ability and knowledge limits of pre-trained models are largely fixed during initial training. To overcome these challenges, we propose Multimodal Self-Reflection enhanced reasoning with Group Relative Policy Optimization (SRPO), a two-stage reflection-aware reinforcement learning (RL) framework explicitly designed to enhance multimodal LLM reasoning. In the first stage, we construct a high-quality, reflection-focused dataset under the guidance of an advanced MLLM, which generates reflections based on initial responses to help the policy model learn both reasoning and self-reflection. In the second stage, we introduce a novel reward mechanism within the GRPO framework that encourages concise and cognitively meaningful reflection while avoiding redundancy. Extensive experiments across multiple multimodal reasoning benchmarks, including MathVista, MathVision, MathVerse, and MMMU-Pro, using Qwen-2.5-VL-7B and Qwen-2.5-VL-32B demonstrate that SRPO significantly outperforms state-of-the-art models, achieving notable improvements in both reasoning accuracy and reflection quality.

Method: Created executable sandbox environments with task specifications, interactive feedback, ground truth annotations, and scalable training trajectory generation. Used multi-threaded, multi-turn trajectory sampling for offline and online reinforcement learning.

Result: Med-Copilot achieved +43.02% (offline) and +45.28% (online) performance gains through reinforcement learning in MedAgentGym, becoming competitive with proprietary LLMs like GPT-4o.

Conclusion: MedAgentGym provides an effective, unified platform for developing LLM-based coding assistants for biomedical data science that is cost-effective and privacy-preserving while matching proprietary LLM performance.

Abstract: We introduce MedAgentGym, a scalable and interactive training environment designed to enhance coding-based biomedical reasoning capabilities in large language model (LLM) agents. MedAgentGym comprises 72,413 task instances across 129 categories derived from 12 authentic real-world biomedical scenarios. Tasks are encapsulated within executable sandbox environments, each featuring detailed task specifications, interactive feedback mechanisms, verifiable ground truth annotations, and scalable training trajectory generation. Extensive benchmarking of 29 LLMs reveals substantial performance disparities in biomedical data science between commercial and open-source LLMs. Leveraging efficient multi-threaded and multi-turn trajectory sampling in MedAgentGym, Med-Copilot achieves performance gains of +43.02% and +45.28% from offline and online reinforcement learning, respectively, demonstrating MedAgentGym as an effective training ground while establishing itself as a cost-effective, privacy-preserving alternative competitive with proprietary LLMs (gpt-4o). By offering a unified execution environment with a comprehensive benchmark and accessible, extensible training resources, MedAgentGym delivers an integrated platform to develop LLM-based coding assistants for advanced biomedical data science.

Method: Created SSA-MTE dataset with 73k+ sentence-level annotations across 14 African language pairs, then developed SSA-COMET and SSA-COMET-QE metrics, and benchmarked against LLMs like GPT-4o, Claude-3.7, and Gemini 2.5 Pro.

Result: SSA-COMET models significantly outperform AfriCOMET and are competitive with Gemini 2.5 Pro, especially on low-resource languages like Twi, Luo, and Yoruba.

Conclusion: The SSA-MTE dataset and improved metrics address critical gaps in African language MT evaluation, with all resources released openly to support future research.

Abstract: Evaluating machine translation (MT) quality for under-resourced African languages remains a significant challenge, as existing metrics often suffer from limited language coverage and poor performance in low-resource settings. While recent efforts, such as AfriCOMET, have addressed some of the issues, they are still constrained by small evaluation sets, a lack of publicly available training data tailored to African languages, and inconsistent performance in extremely low-resource scenarios. In this work, we introduce SSA-MTE, a large-scale human-annotated MT evaluation (MTE) dataset covering 14 African language pairs from the News domain, with over 73,000 sentence-level annotations from a diverse set of MT systems. Based on this data, we develop SSA-COMET and SSA-COMET-QE, improved reference-based and reference-free evaluation metrics. We also benchmark prompting-based approaches using state-of-the-art LLMs like GPT-4o, Claude-3.7 and Gemini 2.5 Pro. Our experimental results show that SSA-COMET models significantly outperform AfriCOMET and are competitive with the strongest LLM Gemini 2.5 Pro evaluated in our study, particularly on low-resource languages such as Twi, Luo, and Yoruba. All resources are released under open licenses to support future research.

Method: Proposes Micro-Act with hierarchical action space that automatically perceives context complexity and adaptively decomposes each knowledge source into sequences of fine-grained comparisons represented as actionable steps, enabling reasoning beyond superficial context.

Result: Achieves significant QA accuracy improvements over state-of-the-art baselines across 5 datasets and 3 conflict types, especially excelling in temporal and semantic conflicts where all baselines fail. Also maintains robust performance on non-conflict questions.

Conclusion: Micro-Act demonstrates practical value for real-world RAG applications by effectively handling knowledge conflicts while maintaining performance on non-conflict scenarios, representing a significant advancement in RAG system robustness.

Abstract: Retrieval-Augmented Generation (RAG) systems commonly suffer from Knowledge Conflicts, where retrieved external knowledge contradicts the inherent, parametric knowledge of large language models (LLMs). It adversely affects performance on downstream tasks such as question answering (QA). Existing approaches often attempt to mitigate conflicts by directly comparing two knowledge sources in a side-by-side manner, but this can overwhelm LLMs with extraneous or lengthy contexts, ultimately hindering their ability to identify and mitigate inconsistencies. To address this issue, we propose Micro-Act a framework with a hierarchical action space that automatically perceives context complexity and adaptively decomposes each knowledge source into a sequence of fine-grained comparisons. These comparisons are represented as actionable steps, enabling reasoning beyond the superficial context. Through extensive experiments on five benchmark datasets, Micro-Act consistently achieves significant increase in QA accuracy over state-of-the-art baselines across all 5 datasets and 3 conflict types, especially in temporal and semantic types where all baselines fail significantly. More importantly, Micro-Act exhibits robust performance on non-conflict questions simultaneously, highlighting its practical value in real-world RAG applications.

Method: Proposes Internal Confidence - a training-free method that leverages self-evaluations across layers and tokens to estimate query-level uncertainty before generating any tokens.

Result: Outperforms baselines in confidence quality while being computationally cheaper; reduces inference costs in RAG and model cascading while preserving performance.

Conclusion: Internal Confidence provides reliable uncertainty estimation for detecting knowledge boundaries, enabling cost-effective adaptive inference without training.

Abstract: It is important for Large Language Models (LLMs) to be aware of the boundary of their knowledge, distinguishing queries they can confidently answer from those that lie beyond their capabilities. Such awareness enables models to perform adaptive inference, such as invoking retrieval-augmented generation (RAG), engaging in slow and deep thinking, or abstaining from answering when appropriate. These mechanisms are key to developing efficient and trustworthy AI. In this work, we propose a method to detect knowledge boundaries via Query-Level Uncertainty, which estimates if a model is capable of answering a given query before generating any tokens, thus avoiding the generation cost. To this end, we propose a novel, training-free method called Internal Confidence, which leverages self-evaluations across layers and tokens to provide a reliable signal of uncertainty. Empirical studies on both factual question answering and mathematical reasoning tasks demonstrate that our Internal Confidence outperforms several baselines in quality of confidence while being computationally cheaper. Furthermore, we demonstrate its benefits in adaptive inference settings, showing that for RAG and model cascading it reduces inference costs while preserving overall performance.

Method: Based on the FM-index data structure that simultaneously indexes and compresses text, creating indexes with size only 44% of the corpus, with significant improvements in indexing speed (18×) and memory use during indexing (3.2× reduction) and querying.

Result: Indexed 83TB of Internet text in 99 days with a single CPU node (or 19 hours with 137 nodes), found several core LM evaluation benchmarks heavily contaminated in Internet crawls (up to 74.2% in GSM8K), and created a benchmark contamination bulletin.

Conclusion: The system enables large-scale analysis of text corpora, revealing significant benchmark contamination that could lead to overestimating language model capabilities, and provides tools for general search queries on indexed data.

Abstract: Language models are trained mainly on massive text data from the Internet, and it becomes increasingly important to understand this data source. Exact-match search engines enable searching in large text corpora - counting string appearances and retrieving the enclosing documents - yet the high storage overhead hinders their application on Internet-scale data. We present infini-gram mini, an efficient and scalable system that can make petabyte-level text corpora searchable. Based on the FM-index data structure (Ferragina and Manzini, 2000), which simultaneously indexes and compresses text, our system creates indexes with size only 44% of the corpus. Infini-gram mini greatly improves upon the best existing implementation of FM-index in terms of indexing speed (18$\times$) and memory use during both indexing (3.2$\times$ reduction) and querying (down to a negligible amount). We index 83TB of Internet text in 99 days with a single CPU node with 128 vCPUs (or 19 hours if using 137 such nodes). We show one important use case of infini-gram mini in a large-scale analysis of benchmark contamination. We find several core LM evaluation benchmarks to be heavily contaminated in Internet crawls (up to 74.2% in GSM8K), which could lead to overestimating the capabilities of language models if trained on such data. We host a benchmark contamination bulletin to share the contamination rate of many core and community-contributed benchmarks. We also release a web interface and an API endpoint to serve general search queries on infini-gram mini indexes.

Method: Embed synonym substitutions for carefully selected high-entropy words to enhance LLM memorization without altering semantic integrity, making the watermark difficult to detect and remove.

Result: Significant improvements in AUROC scores across multiple training settings (continued pretraining and fine-tuning) on seven open-source LLMs compared to existing methods.

Conclusion: LexiMark effectively verifies unauthorized use of watermarked data in LLM training through subtle, contextually appropriate substitutions that evade detection.

Abstract: Large language models (LLMs) can be trained or fine-tuned on data obtained without the owner’s consent. Verifying whether a specific LLM was trained on particular data instances or an entire dataset is extremely challenging. Dataset watermarking addresses this by embedding identifiable modifications in training data to detect unauthorized use. However, existing methods often lack stealth, making them relatively easy to detect and remove. In light of these limitations, we propose LexiMark, a novel watermarking technique designed for text and documents, which embeds synonym substitutions for carefully selected high-entropy words. Our method aims to enhance an LLM’s memorization capabilities on the watermarked text without altering the semantic integrity of the text. As a result, the watermark is difficult to detect, blending seamlessly into the text with no visible markers, and is resistant to removal due to its subtle, contextually appropriate substitutions that evade automated and manual detection. We evaluated our method using baseline datasets from recent studies and seven open-source models: LLaMA-1 7B, LLaMA-3 8B, Mistral 7B, Pythia 6.9B, as well as three smaller variants from the Pythia family (160M, 410M, and 1B). Our evaluation spans multiple training settings, including continued pretraining and fine-tuning scenarios. The results demonstrate significant improvements in AUROC scores compared to existing methods, underscoring our method’s effectiveness in reliably verifying whether unauthorized watermarked data was used in LLM training.

Method: Used a cognitive model of polite speech to systematically evaluate value trade-offs across two settings: reasoning effort in frontier models and RL post-training dynamics in open-source models.

Result: Found higher informational utility than social utility in reasoning models’ default behavior, with predictable shifts when prioritizing goals. Training dynamics showed large early utility shifts with persistent base model/pretraining effects.

Conclusion: The framework provides a flexible tool for probing value trade-offs across model types, useful for hypothesis generation about social behaviors and shaping training regimes to better control value trade-offs.

Abstract: Value trade-offs are an integral part of human decision-making and language use, however, current tools for interpreting such dynamic and multi-faceted notions of values in LLMs are limited. In cognitive science, so-called “cognitive models” provide formal accounts of such trade-offs in humans, by modeling the weighting of a speaker’s competing utility functions in choosing an action or utterance. Here we use a leading cognitive model of polite speech to systematically evaluate value trade-offs in two encompassing model settings: degrees of reasoning “effort” in frontier black-box models, and RL post-training dynamics of open-source models. Our results highlight patterns of higher informational utility than social utility in reasoning models’ default behavior, and demonstrate that these patterns shift in predictable ways when models are prompted to prioritize certain goals over others. Our findings from LLMs’ training dynamics suggest large shifts in utility values early on in training with persistent effects of the choice of base model and pretraining data, compared to feedback dataset or alignment method. Our framework offers a flexible tool for probing value trade-offs across diverse model types, providing insights for generating hypotheses about other social behaviors such as sycophancy and for shaping training regimes that better control trade-offs between values during model development.

Method: Introduced Self-Correction Bench evaluation framework with controlled error injection at three complexity levels, tested 14 open-source non-reasoning models, and analyzed training data influences.

Result: Found average 64.5% blind spot rate; discovered that appending a minimal “Wait” prompt activates 89.3% reduction in blind spots, suggesting dormant capabilities.

Conclusion: LLMs have critical self-correction limitations potentially influenced by training distribution, but practical approaches like minimal prompts can significantly enhance reliability for safety-critical domains.

Abstract: Although large language models (LLMs) have transformed AI, they still make mistakes and can explore unproductive reasoning paths. Self-correction capability is essential for deploying LLMs in safety-critical applications. We uncover a systematic failure: LLMs cannot correct errors in their own outputs while successfully correcting identical errors from external sources - a limitation we term the Self-Correction Blind Spot. To study this phenomenon, we introduce Self-Correction Bench, an evaluation framework to measure this phenomenon through controlled error injection at three complexity levels. Testing 14 open-source non-reasoning models, we find an average 64.5% blind spot rate. We provide multiple lines of evidence suggesting this limitation may be influenced by training data: human demonstrations rarely include error-correction sequences (favoring error-free responses), whereas reinforcement learning (RL) trained models learn error correction via outcome feedback. Remarkably, appending a minimal “Wait” prompt activates a 89.3% reduction in blind spots, suggesting dormant capabilities that require triggering. Our work highlights a critical limitation potentially influenced by training distribution and offers a practical approach to enhance LLM reliability and trustworthiness - vital for safety-critical domains.

Method: Evaluated open- and closed-weight LLMs on private and open-source clinical datasets, and proposed an agentic pipeline for generating realistic, non-sensitive nurse dictations to enable structured extraction of clinical observations.

Result: The study provides performance analysis of LLMs on these tasks and releases SYNUR and SIMORD - the first open-source datasets for nurse observation extraction and medical order extraction to support further research.

Conclusion: The research advances clinical NLP by addressing data scarcity through novel datasets and demonstrates LLM capabilities on practical clinical documentation tasks that can reduce healthcare provider burden.

Abstract: Large language models (LLMs) such as GPT-4o and o1 have demonstrated strong performance on clinical natural language processing (NLP) tasks across multiple medical benchmarks. Nonetheless, two high-impact NLP tasks - structured tabular reporting from nurse dictations and medical order extraction from doctor-patient consultations - remain underexplored due to data scarcity and sensitivity, despite active industry efforts. Practical solutions to these real-world clinical tasks can significantly reduce the documentation burden on healthcare providers, allowing greater focus on patient care. In this paper, we investigate these two challenging tasks using private and open-source clinical datasets, evaluating the performance of both open- and closed-weight LLMs, and analyzing their respective strengths and limitations. Furthermore, we propose an agentic pipeline for generating realistic, non-sensitive nurse dictations, enabling structured extraction of clinical observations. To support further research in both areas, we release SYNUR and SIMORD, the first open-source datasets for nurse observation extraction and medical order extraction.

Method: Simulate virtual respondents with diverse mediators (factors that influence how traits manifest in responses) to identify survey items that robustly measure intended traits across different mediator profiles.

Result: Experiments on three psychological trait theories (Big5, Schwartz, VIA) show the mediator generation methods and simulation framework effectively identify high-validity items, with LLMs demonstrating ability to generate plausible mediators and simulate respondent behavior.

Conclusion: The framework enables cost-effective survey development and provides insights into how LLMs simulate human survey responses, with publicly released dataset and code to support future research.

Abstract: As psychometric surveys are increasingly used to assess the traits of large language models (LLMs), the need for scalable survey item generation suited for LLMs has also grown. A critical challenge here is ensuring the construct validity of generated items, i.e., whether they truly measure the intended trait. Traditionally, this requires costly, large-scale human data collection. To make it efficient, we present a framework for virtual respondent simulation using LLMs. Our central idea is to account for mediators: factors through which the same trait can give rise to varying responses to a survey item. By simulating respondents with diverse mediators, we identify survey items that robustly measure intended traits. Experiments on three psychological trait theories (Big5, Schwartz, VIA) show that our mediator generation methods and simulation framework effectively identify high-validity items. LLMs demonstrate the ability to generate plausible mediators from trait definitions and to simulate respondent behavior for item validation. Our problem formulation, metrics, methodology, and dataset open a new direction for cost-effective survey development and a deeper understanding of how LLMs simulate human survey responses. We publicly release our dataset and code to support future work.

Method: Created MapIQ dataset with 14,706 QA pairs across choropleth maps, cartograms, and proportional symbol maps covering six themes, then evaluated multiple MLLMs on six visual analytical tasks and tested robustness to design changes.

Result: MLLMs showed varying performance across map types and tasks, with sensitivity to design changes like color schemes and legend modifications, revealing reliance on internal geographic knowledge.

Conclusion: The study identifies limitations in current MLLMs for Map-VQA, provides insights into their robustness, and suggests directions for improving map comprehension capabilities.

Abstract: Recent advancements in multimodal large language models (MLLMs) have driven researchers to explore how well these models read data visualizations, e.g., bar charts, scatter plots. More recently, attention has shifted to visual question answering with maps (Map-VQA). However, Map-VQA research has primarily focused on choropleth maps, which cover only a limited range of thematic categories and visual analytical tasks. To address these gaps, we introduce MapIQ, a benchmark dataset comprising 14,706 question-answer pairs across three map types: choropleth maps, cartograms, and proportional symbol maps spanning topics from six distinct themes (e.g., housing, crime). We evaluate multiple MLLMs using six visual analytical tasks, comparing their performance against one another and a human baseline. An additional experiment examining the impact of map design changes (e.g., altered color schemes, modified legend designs, and removal of map elements) provides insights into the robustness and sensitivity of MLLMs, their reliance on internal geographic knowledge, and potential avenues for improving Map-VQA performance.

Method: Introduces WakenLLM framework that quantifies Unknown outputs attributable to model incapacity and evaluates whether stimulation can convert them into correct answers (verifiable) or justified responses with valid reasoning (unverifiable).

Result: Comprehensive experiments on six LLMs show that without training or parameter revision, LLMs can achieve up to 68.53% accuracy improvement on Vague Perception samples through guided understanding.

Conclusion: Current baseline methods only activate a small portion of LLMs’ reasoning potential, indicating considerable unexplored capacity. This extends theoretical upper bounds of reasoning accuracy and deepens understanding of latent reasoning capacity in LLMs.

Abstract: Large Language Models (LLMs) frequently output the label Unknown in reasoning tasks, where two scenarios may appear: (i) an input sample is genuinely unverifiable, but the model cannot understand why; and (ii) a verifiable problem that the model fails to solve, thus outputs Unknown. We refer to these cases collectively as the Vague Perception phenomenon. Current evaluations focus on whether such answers are honest, rather than analyzing the limits of LLM reasoning. To address this, we introduce WakenLLM, a framework that quantifies the portion of Unknown output attributable to model incapacity and evaluates whether stimulation can convert them into either correct answers (verifiable) or justified (unverifiable) responses with valid reasoning. Our method offers a clearer picture of the limits of LLM reasoning and the potential for corrections across various datasets. Comprehensive experiments on six LLMs suggest that, without any training or parameter revision, LLMs can achieve up to a 68.53% accuracy improvement on Vague Perception samples through guided understanding. Our work reveals that current baseline methods only activate a small portion of LLMs’ reasoning potential, indicating considerable unexplored capacity. This extends the theoretical upper bounds of reasoning accuracy in LLMs. Consequently, this study deepens our understanding of the latent reasoning capacity of LLMs and offers a new perspective on addressing the Vague Perception phenomenon.

Method: Two-phase approach: (1) offline buildtime stage compiling policy documents into verifiable guard code for tools, (2) runtime integration where guards ensure compliance before each agent action.

Result: Demonstrated on the τ-bench Airlines domain with encouraging preliminary results in policy enforcement.

Conclusion: The framework provides deterministic, transparent, and modular policy enforcement, though key challenges remain for real-world deployments.

Abstract: Large Language Model (LLM) agents hold promise for a flexible and scalable alternative to traditional business process automation, but struggle to reliably follow complex company policies. In this study we introduce a deterministic, transparent, and modular framework for enforcing business policy adherence in agentic workflows. Our method operates in two phases: (1) an offline buildtime stage that compiles policy documents into verifiable guard code associated with tool use, and (2) a runtime integration where these guards ensure compliance before each agent action. We demonstrate our approach on the challenging $\tau$-bench Airlines domain, showing encouraging preliminary results in policy enforcement, and further outline key challenges for real-world deployments.

Method: Proposed a knowledge graph-based framework that generates varied and subtle conflicts between two similar yet distinct contexts using explicit relational structure of KGs.

Result: Both open-source and proprietary models struggle with conflict detection, especially in multi-hop reasoning scenarios, and often fail to pinpoint exact sources of contradictions.

Conclusion: The MAGIC benchmark provides insights into LLM behavior with knowledge conflicts and serves as a foundation for improving LLMs’ ability to integrate diverse and conflicting information.

Abstract: Knowledge conflict often arises in retrieval-augmented generation (RAG) systems, where retrieved documents may be inconsistent with one another or contradict the model’s parametric knowledge. Existing benchmarks for investigating the phenomenon have notable limitations, including a narrow focus on the question answering setup, heavy reliance on entity substitution techniques, and a restricted range of conflict types. To address these issues, we propose a knowledge graph (KG)-based framework that generates varied and subtle conflicts between two similar yet distinct contexts, while ensuring interpretability through the explicit relational structure of KGs. Experimental results on our benchmark, MAGIC, provide intriguing insights into the inner workings of LLMs regarding knowledge conflict: both open-source and proprietary models struggle with conflict detection – especially when multi-hop reasoning is required – and often fail to pinpoint the exact source of contradictions. Finally, we present in-depth analyses that serve as a foundation for improving LLMs in integrating diverse, sometimes even conflicting, information.

Method: Remove instance-level principal components from embeddings to transform the space to approximate Gaussian properties, reducing susceptibility to adversarial perturbations while preserving semantic relationships.

Result: Evaluations on eight benchmark datasets show improved adversarial robustness while maintaining comparable before-attack accuracy to baselines.

Conclusion: The approach achieves a balanced trade-off between robustness and generalization without requiring adversarial examples or costly training-time augmentation.

Abstract: Pre-trained language models (PLMs) have driven substantial progress in natural language processing but remain vulnerable to adversarial attacks, raising concerns about their robustness in real-world applications. Previous studies have sought to mitigate the impact of adversarial attacks by introducing adversarial perturbations into the training process, either implicitly or explicitly. While both strategies enhance robustness, they often incur high computational costs. In this work, we propose a simple yet effective add-on module that enhances the adversarial robustness of PLMs by removing instance-level principal components, without relying on conventional adversarial defences or perturbing the original training data. Our approach transforms the embedding space to approximate Gaussian properties, thereby reducing its susceptibility to adversarial perturbations while preserving semantic relationships. This transformation aligns embedding distributions in a way that minimises the impact of adversarial noise on decision boundaries, enhancing robustness without requiring adversarial examples or costly training-time augmentation. Evaluations on eight benchmark datasets show that our approach improves adversarial robustness while maintaining comparable before-attack accuracy to baselines, achieving a balanced trade-off between robustness and generalisation.

Method: The authors present a benchmark dataset comprising 1,079 instances in English and Chinese, accompanied by an LLM-based evaluation method that closely aligns with human judgment.

Result: The benchmark facilitates comprehensive exploration of SDM performance in tackling practical challenges of spoken dialogue, including ambiguity from semantic and phonological factors, and context-dependency issues.

Conclusion: The proposed benchmark dataset and evaluation method address the current gap in SDM research and enable better understanding of their capabilities in handling complex human conversational dynamics.

Abstract: Spoken Dialogue Models (SDMs) have recently attracted significant attention for their ability to generate voice responses directly to users’ spoken queries. Despite their increasing popularity, there exists a gap in research focused on comprehensively understanding their practical effectiveness in comprehending and emulating human conversations. This is especially true compared to text-based Large Language Models (LLMs), which benefit from extensive benchmarking. Human voice interactions are inherently more complex than text due to characteristics unique to spoken dialogue. Ambiguity poses one challenge, stemming from semantic factors like polysemy, as well as phonological aspects such as heterograph, heteronyms, and stress patterns. Additionally, context-dependency, like omission, coreference, and multi-turn interaction, adds further complexity to human conversational dynamics. To illuminate the current state of SDM development and to address these challenges, we present a benchmark dataset in this paper, which comprises 1,079 instances in English and Chinese. Accompanied by an LLM-based evaluation method that closely aligns with human judgment, this dataset facilitates a comprehensive exploration of the performance of SDMs in tackling these practical challenges.

Method: Analyzed two user-LM interaction datasets (WildChat and LMSYS) to understand when and why feedback occurs. Tested whether incorporating feedback content (e.g., user wanted clarification) along with polarity improves model performance on different benchmarks.

Result: Mixed results - incorporating feedback content helped performance on short human-designed questions (MTBench) but not on longer, more complex questions (WildBench).

Conclusion: Implicit user feedback from interaction logs shows potential for model improvement but has limitations, particularly with complex tasks. The approach works better for simpler, shorter interactions than complex ones.

Abstract: Once language models (LMs) are deployed, they can interact with users long-term, ideally evolving based on their feedback. Asking for direct user feedback can be disruptive; thus, we study harvesting implicit user feedback from user-LM interaction logs. We study two user-LM interaction datasets (WildChat and LMSYS). First, we analyze user feedback in the user-LLM conversation logs, providing insights into when and why such feedback occurs. Second, we study harvesting learning signals from such implicit user feedback. Specifically, we study whether incorporating the contents of user feedback (e.g., user wanted clarification), in addition to the polarity of the feedback, can improve the model performance. We observe mixed results, showing this helps in short human-designed questions (MTBench) but not on longer and more complex questions (WildBench). Together, we provide an in-depth study of implicit user feedback, showing its potential and limitations.

Method: Uses meta-learning to extract task- and system-level meta-features from past successes/failures, biasing sampling toward valuable configurations and away from costly dead ends.

Result: Surpassed zero-shot optimizer’s peak F1 on 5/6 tasks, cut evaluation time by up to 4.5x, reduced search error ratios by 7x, and found 50% more pipelines above the zero-shot Pareto front.

Conclusion: XAutoLM enables resource-efficient Green AI fine-tuning, outperforming simpler memory-based approaches that suffer from negative transfer.

Abstract: Experts in machine learning leverage domain knowledge to navigate decisions in model selection, hyperparameter optimization, and resource allocation. This is particularly critical for fine-tuning language models (LMs), where repeated trials incur substantial computational overhead and environmental impact. However, no existing automated framework simultaneously tackles the entire model selection and hyperparameter optimization (HPO) task for resource-efficient LM fine-tuning. We introduce XAutoLM, a meta-learning-augmented AutoML framework that reuses past experiences to optimize discriminative and generative LM fine-tuning pipelines efficiently. XAutoLM learns from stored successes and failures by extracting task- and system-level meta-features to bias its sampling toward valuable configurations and away from costly dead ends. On four text classification and two question-answering benchmarks, XAutoLM surpasses zero-shot optimizer’s peak F1 on five of six tasks, cuts mean evaluation time of pipelines by up to 4.5x, reduces search error ratios by up to sevenfold, and uncovers up to 50% more pipelines above the zero-shot Pareto front. In contrast, simpler memory-based baselines suffer negative transfer. We release XAutoLM and our experience store to catalyze resource-efficient, Green AI fine-tuning in the NLP community.

Method: Extended DisCo architecture by introducing annotator metadata embeddings, enhanced input representations, and multi-objective training losses to better capture disagreement patterns.

Result: Substantial improvements in both soft and perspectivist evaluation metrics across three datasets, with better calibration and understanding of when disagreement-aware modeling works best.

Conclusion: Disagreement can be better captured by conditioning on annotator demographics and directly optimizing for distributional metrics, yielding consistent improvements across datasets.

Abstract: The Learning With Disagreements (LeWiDi) 2025 shared task aims to model annotator disagreement through soft label distribution prediction and perspectivist evaluation, which focuses on modeling individual annotators. We adapt DisCo (Distribution from Context), a neural architecture that jointly models item-level and annotator-level label distributions, and present detailed analysis and improvements. In this paper, we extend DisCo by introducing annotator metadata embeddings, enhancing input representations, and multi-objective training losses to capture disagreement patterns better. Through extensive experiments, we demonstrate substantial improvements in both soft and perspectivist evaluation metrics across three datasets. We also conduct in-depth calibration and error analyses that reveal when and why disagreement-aware modeling improves. Our findings show that disagreement can be better captured by conditioning on annotator demographics and by optimizing directly for distributional metrics, yielding consistent improvements across datasets.

Method: Two methods: 1) Temporal Self-Consistency Voting - training-free decoding that aggregates predictions across denoising steps; 2) Temporal Consistency Reinforcement - post-training using Temporal Semantic Entropy as reward signal to encourage stable generations.

Result: Significant improvements: 24.7% average improvement on Countdown using negative TSE reward alone; combined with accuracy reward: 2.0% on GSM8K, 4.3% on MATH500, 6.6% on SVAMP, and 25.3% on Countdown.

Conclusion: Temporal dynamics in dLLMs have untapped potential, and the proposed methods offer simple yet effective tools to harness intermediate predictions for improved generation quality and consistency.

Abstract: Diffusion large language models (dLLMs) generate text through iterative denoising, yet current decoding strategies discard rich intermediate predictions in favor of the final output. Our work here reveals a critical phenomenon, temporal oscillation, where correct answers often emerge in the middle process, but are overwritten in later denoising steps. To address this issue, we introduce two complementary methods that exploit temporal consistency: 1) Temporal Self-Consistency Voting, a training-free, test-time decoding strategy that aggregates predictions across denoising steps to select the most consistent output; and 2) a post-training method termed Temporal Consistency Reinforcement, which uses Temporal Semantic Entropy (TSE), a measure of semantic stability across intermediate predictions, as a reward signal to encourage stable generations. Empirical results across multiple benchmarks demonstrate the effectiveness of our approach. Using the negative TSE reward alone, we observe a remarkable average improvement of 24.7% on the Countdown dataset over an existing dLLM. Combined with the accuracy reward, we achieve absolute gains of 2.0% on GSM8K, 4.3% on MATH500, 6.6% on SVAMP, and 25.3% on Countdown, respectively. Our findings underscore the untapped potential of temporal dynamics in dLLMs and offer two simple yet effective tools to harness them.

Method: Jointly model linguistic and acoustic information by generating semantic tokens and continuous acoustic representations using flow-matching objective, predicting multiple future semantic tokens to preserve linguistic information.

Result: Achieves comparable linguistic performance to existing models while providing better acoustic detail in prompted generation.

Conclusion: Joint modeling of linguistic and acoustic information with flow-matching improves acoustic detail in textless spoken language models while maintaining linguistic quality.

Abstract: Textless spoken language models (SLMs) are generative models of speech that do not rely on text supervision. Most textless SLMs learn to predict the next semantic token, a discrete representation of linguistic content, and rely on a separate vocoder to add acoustic information to the generated speech. Such models have no access to acoustic context and no built-in control over acoustic details. In this work, we propose to jointly model linguistic and acoustic information by generating semantic tokens and a continuous real-valued representation of the acoustic frame. We use a flow-matching objective to predict the continuous vector conditioned on the semantic tokens. We study the design space of this approach and find that predicting multiple future semantic tokens helps preserve linguistic information. Our approach achieves comparable performance to existing models in terms of linguistic likelihood benchmarks, while providing better acoustic detail in prompted generation.

Method: PASR enables LLMs to proactively decide whether, when, and how to refine based on internal state and evolving context during generation, rather than regenerating entire responses.

Result: On Qwen3-8B, PASR reduces average token consumption by 41.6% and achieves 8.2% accuracy improvement across 10 diverse tasks compared to standard generation.

Conclusion: PASR demonstrates that proactive, context-aware refinement during generation significantly enhances LLM performance while reducing computational costs.

Abstract: Recent advances in self-refinement have demonstrated significant potential for improving the outputs of large language models (LLMs) through iterative refinement. However, most existing self-refinement methods rely on a reactive process with a fixed number of iterations, making it difficult to determine the optimal timing and content of refinement based on the evolving generation context. Inspired by the way humans dynamically refine their thoughts during execution, we propose ProActive Self-Refinement (PASR), a novel method that enables LLMs to refine their outputs during the generation process. Unlike methods that regenerate entire responses, PASR proactively decides whether, when, and how to refine based on the model’s internal state and evolving context. We conduct extensive experiments on a diverse set of 10 tasks to evaluate the effectiveness of PASR. Experimental results show that PASR significantly enhances problem-solving performance. In particular, on Qwen3-8B, PASR reduces average token consumption by 41.6% compared to standard generation, while also achieving an 8.2% improvement in accuracy. Our code and baselines used in the paper are available on GitHub.

Method: Extends AutoBnB framework with RAG in multi-agent simulations using Backdoors & Breaches game environment. Introduces two retrieval settings: RAG-Wiki (technical documentation) and RAG-News (narrative incident reports). Evaluates eight team structures including argumentative configurations for critical reasoning.

Result: Retrieval augmentation improves decision quality and success rates across diverse organizational models. The system demonstrates ability to reconstruct complex multi-stage attacks based on public breach reports.

Conclusion: Integrating retrieval mechanisms into LLM-based multi-agent systems provides significant value for cybersecurity decision-making by enhancing reasoning capabilities with external knowledge.

Abstract: Incident response (IR) requires fast, coordinated, and well-informed decision-making to contain and mitigate cyber threats. While large language models (LLMs) have shown promise as autonomous agents in simulated IR settings, their reasoning is often limited by a lack of access to external knowledge. In this work, we present AutoBnB-RAG, an extension of the AutoBnB framework that incorporates retrieval-augmented generation (RAG) into multi-agent incident response simulations. Built on the Backdoors & Breaches (B&B) tabletop game environment, AutoBnB-RAG enables agents to issue retrieval queries and incorporate external evidence during collaborative investigations. We introduce two retrieval settings: one grounded in curated technical documentation (RAG-Wiki), and another using narrative-style incident reports (RAG-News). We evaluate performance across eight team structures, including newly introduced argumentative configurations designed to promote critical reasoning. To validate practical utility, we also simulate real-world cyber incidents based on public breach reports, demonstrating AutoBnB-RAG’s ability to reconstruct complex multi-stage attacks. Our results show that retrieval augmentation improves decision quality and success rates across diverse organizational models. This work demonstrates the value of integrating retrieval mechanisms into LLM-based multi-agent systems for cybersecurity decision-making.

Method: Created two sub-benchmarks: OverthinkingBench with 72 domains of simple queries, and UnderthinkingBench with 11 challenging reasoning tasks. Evaluated 33 thinking and non-thinking models using novel thinking-adjusted accuracy metrics.

Result: No model optimally balanced thinking across tasks. Thinking models wasted hundreds of tokens on simple queries without performance gains, while large non-thinking models underperformed compared to smaller thinking models on complex tasks.

Conclusion: Current approaches struggle to achieve optimal thinking - improving one sub-benchmark often comes at the expense of the other, highlighting the need for better unified models that can adapt thinking depth to task complexity.

Abstract: Thinking LLMs solve complex tasks at the expense of increased compute and overthinking on simpler problems, while non-thinking LLMs are faster and cheaper but underthink on harder reasoning problems. This has led to the development of separate thinking and non-thinking LLM variants, leaving the onus of selecting the optimal model for each query on the end user. We introduce OptimalThinkingBench, a unified benchmark that jointly evaluates overthinking and underthinking in LLMs and also encourages the development of optimally-thinking models that balance performance and efficiency. Our benchmark comprises two sub-benchmarks: OverthinkingBench, featuring simple math and general queries in 72 domains, and UnderthinkingBench, containing 11 challenging reasoning tasks along with harder math problems. Using novel thinking-adjusted accuracy metrics, we extensively evaluate 33 different thinking and non-thinking models and show that no model is able to optimally think on our benchmark. Thinking models often overthink for hundreds of tokens on the simplest user queries without improving performance. In contrast, large non-thinking models underthink, often falling short of much smaller thinking models. We further explore several methods to encourage optimal thinking, but find that these approaches often improve on one sub-benchmark at the expense of the other, highlighting the need for better unified and optimal models in the future.

Method: SurGE includes test instances with topic descriptions, expert-written surveys, cited references, and a large academic corpus of over 1 million papers, plus an automated evaluation framework measuring four quality dimensions.

Result: Evaluation of diverse LLM-based methods shows significant performance gaps, with even advanced agentic frameworks struggling with survey generation complexities.

Conclusion: There is a clear need for future research in automated survey generation, and SurGE provides the necessary benchmark and evaluation framework to advance this field.

Abstract: The rapid growth of academic literature makes the manual creation of scientific surveys increasingly infeasible. While large language models show promise for automating this process, progress in this area is hindered by the absence of standardized benchmarks and evaluation protocols. To bridge this critical gap, we introduce SurGE (Survey Generation Evaluation), a new benchmark for scientific survey generation in computer science. SurGE consists of (1) a collection of test instances, each including a topic description, an expert-written survey, and its full set of cited references, and (2) a large-scale academic corpus of over one million papers. In addition, we propose an automated evaluation framework that measures the quality of generated surveys across four dimensions: comprehensiveness, citation accuracy, structural organization, and content quality. Our evaluation of diverse LLM-based methods demonstrates a significant performance gap, revealing that even advanced agentic frameworks struggle with the complexities of survey generation and highlighting the need for future research in this area. We have open-sourced all the code, data, and models at: https://github.com/oneal2000/SurGE

Method: Created OpenWHO corpus with 2,978 documents and 26,824 sentences from WHO’s e-learning platform, then evaluated modern LLMs against traditional MT models on this resource.

Result: LLMs consistently outperform traditional MT models, with Gemini 2.5 Flash achieving +4.79 ChrF improvement over NLLB-54B on low-resource test set. Document-level translation benefits are most pronounced in specialized domains like health.

Conclusion: The OpenWHO corpus addresses the gap in health domain MT evaluation and demonstrates LLMs’ superiority over traditional approaches, especially for low-resource languages in specialized domains.

Abstract: In machine translation (MT), health is a high-stakes domain characterised by widespread deployment and domain-specific vocabulary. However, there is a lack of MT evaluation datasets for low-resource languages in this domain. To address this gap, we introduce OpenWHO, a document-level parallel corpus of 2,978 documents and 26,824 sentences from the World Health Organization’s e-learning platform. Sourced from expert-authored, professionally translated materials shielded from web-crawling, OpenWHO spans a diverse range of over 20 languages, of which nine are low-resource. Leveraging this new resource, we evaluate modern large language models (LLMs) against traditional MT models. Our findings reveal that LLMs consistently outperform traditional MT models, with Gemini 2.5 Flash achieving a +4.79 ChrF point improvement over NLLB-54B on our low-resource test set. Further, we investigate how LLM context utilisation affects accuracy, finding that the benefits of document-level translation are most pronounced in specialised domains like health. We release the OpenWHO corpus to encourage further research into low-resource MT in the health domain.

Method: ObjexMT benchmark requires models to extract one-sentence base objectives and self-reported confidence from multi-turn transcripts, with accuracy measured via LLM-judge semantic similarity to gold objectives and metacognition evaluated using ECE, Brier, Wrong at High-Confidence metrics.

Result: kimi-k2 achieved highest objective-extraction accuracy (0.612), claude-sonnet-4 showed best selective risk and calibration (AURC 0.242; ECE 0.206; Brier 0.254), with high-confidence errors ranging from 14.9% to 47.7% across models.

Conclusion: LLM judges often misinfer objectives when not explicit; recommendations include exposing objectives when feasible and gating decisions by confidence otherwise.

Abstract: LLM-as-a-Judge (LLMaaJ) now underpins scalable evaluation, yet we lack a decisive test of a judge’s qualification: can it recover a conversation’s latent objective and know when that inference is trustworthy? LLMs degrade under irrelevant or long context; multi-turn jailbreaks further hide goals across turns. We introduce ObjexMT, a benchmark for objective extraction and metacognition. Given a multi-turn transcript, a model must return a one-sentence base objective and self-reported confidence. Accuracy is computed via LLM-judge semantic similarity to gold objectives, converted to binary correctness by a human-aligned threshold calibrated on N=300 items (tau = 0.66; F1 = 0.891). Metacognition is evaluated with ECE, Brier, Wrong at High-Confidence (0.80/0.90/0.95), and risk-coverage. Across six models (gpt-4.1, claude-sonnet-4, Qwen3-235B-A22B-FP8, kimi-k2, deepseek-v3.1, gemini-2.5-flash) on three datasets, kimi-k2 attains the highest objective-extraction accuracy (0.612), with claude-sonnet-4 (0.603) and deepseek-v3.1 (0.599) statistically comparable. claude-sonnet-4 yields the best selective risk and calibration (AURC 0.242; ECE 0.206; Brier 0.254). Dataset heterogeneity (16-82 percent accuracy variance) reveals that automated obfuscation poses fundamental challenges beyond model choice. High-confidence errors persist: Wrong at 0.90 ranges from 14.9 percent (claude-sonnet-4) to 47.7 percent (Qwen3-235B-A22B-FP8). ObjexMT provides an actionable test for LLM judges: when objectives are not explicit, judges often misinfer them; we recommend exposing objectives when feasible and gating decisions by confidence otherwise. Data at https://github.com/hyunjun1121/ObjexMT_dataset.

Method: SSFO constructs preference data pairs by contrasting model outputs with and without context, then uses Direct Preference Optimization (DPO) with a modified loss function to encourage likelihood displacement from parametric-based tokens to context-aligned tokens.

Result: SSFO significantly outperforms existing methods, achieving state-of-the-art faithfulness on multiple context-based QA datasets, with strong generalization including cross-lingual faithfulness and preserved instruction-following capabilities.

Conclusion: SSFO provides an effective self-supervised approach for enhancing RAG faithfulness without additional costs, leveraging likelihood displacement theory to achieve superior performance and generalization.

Abstract: Retrieval-Augmented Generation (RAG) systems require Large Language Models (LLMs) to generate responses that are faithful to the retrieved context. However, faithfulness hallucination remains a critical challenge, as existing methods often require costly supervision and post-training or significant inference burdens. To overcome these limitations, we introduce Self-Supervised Faithfulness Optimization (SSFO), the first self-supervised alignment approach for enhancing RAG faithfulness. SSFO constructs preference data pairs by contrasting the model’s outputs generated with and without the context. Leveraging Direct Preference Optimization (DPO), SSFO aligns model faithfulness without incurring labeling costs or additional inference burden. We theoretically and empirically demonstrate that SSFO leverages a benign form of \emph{likelihood displacement}, transferring probability mass from parametric-based tokens to context-aligned tokens. Based on this insight, we propose a modified DPO loss function to encourage likelihood displacement. Comprehensive evaluations show that SSFO significantly outperforms existing methods, achieving state-of-the-art faithfulness on multiple context-based question-answering datasets. Notably, SSFO exhibits strong generalization, improving cross-lingual faithfulness and preserving general instruction-following capabilities. We release our code and model at the anonymous link: https://github.com/chkwy/SSFO

Method: Adaptive Originality Filtering (AOF) - a prompting framework that uses cosine-based similarity rejection to filter redundant generations while enforcing lexical novelty and cross-lingual fidelity.

Result: AOF-enhanced GPT-4o achieved 0.177 Self-BLEU and 0.915 Distinct-2 in Japanese, showing improved lexical diversity and reduced redundancy across three LLMs and four language pairs.

Conclusion: Semantic rejection through AOF can effectively guide culturally grounded, creative generation in multilingual contexts without requiring task-specific fine-tuning.

Abstract: Multilingual riddle generation challenges large language models (LLMs) to balance cultural fluency with creative abstraction. Standard prompting strategies – zero-shot, few-shot, chain-of-thought – tend to reuse memorized riddles or perform shallow paraphrasing. We introduce Adaptive Originality Filtering (AOF), a prompting framework that filters redundant generations using cosine-based similarity rejection, while enforcing lexical novelty and cross-lingual fidelity. Evaluated across three LLMs and four language pairs, AOF-enhanced GPT-4o achieves \texttt{0.177} Self-BLEU and \texttt{0.915} Distinct-2 in Japanese, signaling improved lexical diversity and reduced redundancy compared to other prompting methods and language pairs. Our findings show that semantic rejection can guide culturally grounded, creative generation without task-specific fine-tuning.

Method: Replace part of autoregressive objective with first-order model-agnostic meta-learning (MAML) during pretraining. Test on Tagalog and Cebuano as challenging languages with typological similarity but structural differences in voice systems.

Result: MAML improves zero-shot micro-F1 by 2-6 pp with head-only tuning and 1-3 pp with full tuning across model sizes (11M-570M). Reduces convergence time by up to 8%. Best performance on single-token person entities co-occurring with Tagalog case particles.

Conclusion: MAML pretraining enables small LMs to better adapt to low-resource NER tasks, with particular strength in recognizing entities with surface anchors like case particles.

Abstract: Named-entity recognition (NER) in low-resource languages is usually tackled by finetuning very large multilingual LMs, an option that is often infeasible in memory- or latency-constrained settings. We ask whether small decoder LMs can be pretrained so that they adapt quickly and transfer zero-shot to languages unseen during pretraining. To this end we replace part of the autoregressive objective with first-order model-agnostic meta-learning (MAML). Tagalog and Cebuano are typologically similar yet structurally different in their actor/non-actor voice systems, and hence serve as a challenging test-bed. Across four model sizes (11 M - 570 M) MAML lifts zero-shot micro-F1 by 2-6 pp under head-only tuning and 1-3 pp after full tuning, while cutting convergence time by up to 8%. Gains are largest for single-token person entities that co-occur with Tagalog case particles si/ni, highlighting the importance of surface anchors.

Method: Proposes DQO (Diversity Quality Optimization) based on determinantal point processes (DPPs). Samples and embeds multiple responses per prompt, then uses the determinant of a kernel-based similarity matrix to measure diversity as the volume spanned by response embeddings. Can be applied on top of existing RL algorithms.

Result: Experiments across instruction-following, summarization, story generation, and reasoning tasks show that DQO substantially improves semantic diversity without sacrificing model quality.

Conclusion: DQO provides a flexible and effective approach to jointly optimize LLMs for both quality and semantic diversity, overcoming limitations of existing diversity enhancement methods.

Abstract: Reinforcement learning (RL) has emerged as a popular method for post-training large language models (LLMs). While improving the model’s performance on downstream tasks, it often reduces the model’s output diversity, leading to narrow, canonical responses. Existing methods to enhance diversity are limited, either by operating at inference time or by focusing on surface-level differences. We propose a novel training method named DQO (Diversity Quality Optimization) based on determinantal point processes (DPPs) to jointly optimize LLMs for quality and semantic diversity. Our approach samples and embeds a group of responses for each prompt, then uses the determinant of a kernel-based similarity matrix to measure diversity as the volume spanned by the embeddings of these responses. DQO is flexible and can be applied on top of existing RL algorithms. Experiments across instruction-following, summarization, story generation, and reasoning tasks demonstrate that our method substantially improves semantic diversity without sacrificing model quality.

Method: Uses language-model-guided evolution with smart sampling from 12 sources and LLM-as-judge approach inspired by StrongREJECT, maintaining selection pressure with success threshold θ=0.70 across five evolutionary generations.

Result: Achieved 44.8% overall success (103/230) on GPT-4.1, discovered two new template families, and showed structural gains transfer across models but vary by target. Found positive correlation between prompt length and score.

Conclusion: Structure-level search is a reproducible method for stronger single-turn probes, highlighting the importance of threshold calibration and cross-model evaluation.

Abstract: Multi-turn-to-single-turn (M2S) compresses iterative red-teaming into one structured prompt, but prior work relied on a handful of manually written templates. We present X-Teaming Evolutionary M2S, an automated framework that discovers and optimizes M2S templates through language-model-guided evolution. The system pairs smart sampling from 12 sources with an LLM-as-judge inspired by StrongREJECT and records fully auditable logs. Maintaining selection pressure by setting the success threshold to $\theta = 0.70$, we obtain five evolutionary generations, two new template families, and 44.8% overall success (103/230) on GPT-4.1. A balanced cross-model panel of 2,500 trials (judge fixed) shows that structural gains transfer but vary by target; two models score zero at the same threshold. We also find a positive coupling between prompt length and score, motivating length-aware judging. Our results demonstrate that structure-level search is a reproducible route to stronger single-turn probes and underscore the importance of threshold calibration and cross-model evaluation. Code, configurations, and artifacts are available at https://github.com/hyunjun1121/M2S-x-teaming.

Method: Decomposes attention projection matrices into shared dictionary atoms, representing each layer’s weights as linear combinations of these shared matrix atoms. Operates as a drop-in replacement trained with standard optimizers.

Result: Achieves better benchmark accuracy and perplexity than grouped-query attention, low-rank baselines, and other sharing methods at comparable parameter budgets. Works across scales (100M-700M parameters) and extends to Vision Transformers with similar performance gains.

Conclusion: MASA offers a scalable blueprint for parameter-efficient models without sacrificing performance by combining dictionary learning with transformer efficiency, and can potentially reduce parameters in pretrained LLMs without significant performance drops.

Abstract: Large language models (LLMs) have revolutionized AI applications, yet their high computational and memory demands hinder their widespread deployment. Existing compression techniques focus on intra-block optimizations (e.g. low-rank approximation, attention head pruning), while the repetitive layered structure of transformers implies significant inter-block redundancy - a dimension largely unexplored beyond key-value (KV) caching. Inspired by dictionary learning in CNNs, we propose a framework for structured weight sharing across transformer layers. Our approach decomposes attention projection matrices into shared dictionary atoms, reducing the attention module’s parameters by 66.7% while achieving on-par performance. Unlike complex methods requiring distillation or architectural changes, MASA (Matrix Atom Sharing in Attention) operates as a drop-in replacement - trained with standard optimizers

• and represents each layer’s weights as linear combinations of shared matrix atoms. Experiments across scales (100M-700M parameters) show that MASA achieves better benchmark accuracy and perplexity than grouped-query attention (GQA), low-rank baselines and recently proposed Repeat-all-over/Sequential sharing at comparable parameter budgets. Ablation studies confirm robustness to the dictionary size and the efficacy of shared representations in capturing cross-layer statistical regularities. Extending to Vision Transformers (ViT), MASA matches performance metrics on image classification and detection tasks with 66.7% fewer attention parameters. By combining dictionary learning strategies with transformer efficiency, MASA offers a scalable blueprint for parameter-efficient models without sacrificing performance. Finally, we investigate the possibility of employing MASA on pretrained LLMs to reduce their number of parameters without experiencing any significant drop in their performance.

Method: Replicated 37 data annotation tasks from 21 published studies, tested 13 million labels from 18 different LLMs across 2361 hypotheses, and analyzed 21 mitigation techniques.

Result: Intentional LLM hacking is simple - with just a few prompt paraphrases, virtually anything can be made statistically significant. Accidental LLM hacking affects ~31% of hypotheses for state-of-the-art models and ~50% for smaller models. Human annotations provide crucial protection, and regression corrections can restore valid inference.

Conclusion: LLM hacking poses serious risks to research validity. Higher model capabilities reduce but don’t eliminate risk. Practical recommendations are needed to prevent manipulation and accidental errors, especially near significance thresholds.

Abstract: Large language models are rapidly transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis. However, LLM outputs vary significantly depending on the implementation choices made by researchers (e.g., model selection or prompting strategy). Such variation can introduce systematic biases and random errors, which propagate to downstream analyses and cause Type I (false positive), Type II (false negative), Type S (wrong sign), or Type M (exaggerated effect) errors. We call this phenomenon where configuration choices lead to incorrect conclusions LLM hacking. We find that intentional LLM hacking is strikingly simple. By replicating 37 data annotation tasks from 21 published social science studies, we show that, with just a handful of prompt paraphrases, virtually anything can be presented as statistically significant. Beyond intentional manipulation, our analysis of 13 million labels from 18 different LLMs across 2361 realistic hypotheses shows that there is also a high risk of accidental LLM hacking, even when following standard research practices. We find incorrect conclusions in approximately 31% of hypotheses for state-of-the-art LLMs, and in half the hypotheses for smaller language models. While higher task performance and stronger general model capabilities reduce LLM hacking risk, even highly accurate models remain susceptible. The risk of LLM hacking decreases as effect sizes increase, indicating the need for more rigorous verification of LLM-based findings near significance thresholds. We analyze 21 mitigation techniques and find that human annotations provide crucial protection against false positives. Common regression estimator correction techniques can restore valid inference but trade off Type I vs. Type II errors. We publish a list of practical recommendations to prevent LLM hacking.

Method: Leverages LLMs to generate narrative personas from social media data, applies quality assessment filtering, uses importance sampling for global alignment with psychometric distributions, and includes task-specific adaptation for subpopulations.

Result: Extensive experiments show the method significantly reduces population-level bias and enables accurate, flexible social simulation for various research and policy applications.

Conclusion: The proposed framework provides a robust solution for synthesizing representative persona sets that authentically capture real-world population diversity, advancing LLM-based social simulation capabilities.

Abstract: Recent advances in large language models (LLMs) have enabled human-like social simulations at unprecedented scale and fidelity, offering new opportunities for computational social science. A key challenge, however, is the construction of persona sets that authentically represent the diversity and distribution of real-world populations. Most existing LLM-based social simulation studies focus primarily on designing agentic frameworks and simulation environments, often overlooking the complexities of persona generation and the potential biases introduced by unrepresentative persona sets. In this paper, we propose a systematic framework for synthesizing high-quality, population-aligned persona sets for LLM-driven social simulation. Our approach begins by leveraging LLMs to generate narrative personas from long-term social media data, followed by rigorous quality assessment to filter out low-fidelity profiles. We then apply importance sampling to achieve global alignment with reference psychometric distributions, such as the Big Five personality traits. To address the needs of specific simulation contexts, we further introduce a task-specific module that adapts the globally aligned persona set to targeted subpopulations. Extensive experiments demonstrate that our method significantly reduces population-level bias and enables accurate, flexible social simulation for a wide range of research and policy applications.

Method: Uses fine-tuned large vision language models for contextualized embeddings, employs distributional attention mechanism to weight token contributions, and uses reconstruction objective to align topic representations with document embeddings.

Result: Outperforms unimodal and multimodal baselines on six benchmarks, achieving average LLM score of 2.61, and demonstrates effectiveness in few-shot retrieval and capturing visually grounded semantics in scientific articles.

Conclusion: CEMTM provides an effective solution for multimodal topic modeling that maintains interpretability while handling multiple images efficiently and achieving superior performance across various domains.

Abstract: We introduce CEMTM, a context-enhanced multimodal topic model designed to infer coherent and interpretable topic structures from both short and long documents containing text and images. CEMTM builds on fine-tuned large vision language models (LVLMs) to obtain contextualized embeddings, and employs a distributional attention mechanism to weight token-level contributions to topic inference. A reconstruction objective aligns topic-based representations with the document embedding, encouraging semantic consistency across modalities. Unlike existing approaches, CEMTM can process multiple images per document without repeated encoding and maintains interpretability through explicit word-topic and document-topic distributions. Extensive experiments on six multimodal benchmarks show that CEMTM consistently outperforms unimodal and multimodal baselines, achieving a remarkable average LLM score of 2.61. Further analysis shows its effectiveness in downstream few-shot retrieval and its ability to capture visually grounded semantics in complex domains such as scientific articles.

Method: Dual-layered analytical framework combining Bloom’s Taxonomy (six cognitive levels) and dialogue act categorization (eleven types), applied to 7,077 annotated user utterances from TALKA chatbot interactions.

Result: Generative AI chatbots effectively support language learning by aligning dialogue acts with cognitive intentions, helping learners express themselves confidently and connect with cultural identity through various interaction types including information requests, translations, cultural inquiries, and creative language use.

Conclusion: AI-mediated dialogue facilitates cognitive development, pragmatic negotiation, and socio-cultural affiliation in low-resource language learners, offering empirical insights for technology-supported language preservation and educational practice.

Abstract: With many endangered languages at risk of disappearing, efforts to preserve them now rely more than ever on using technology alongside culturally informed teaching strategies. This study examines user behaviors in TALKA, a generative AI-powered chatbot designed for Hakka language engagement, by employing a dual-layered analytical framework grounded in Bloom’s Taxonomy of cognitive processes and dialogue act categorization. We analyzed 7,077 user utterances, each carefully annotated according to six cognitive levels and eleven dialogue act types. These included a variety of functions, such as asking for information, requesting translations, making cultural inquiries, and using language creatively. Pragmatic classifications further highlight how different types of dialogue acts–such as feedback, control commands, and social greetings–align with specific cognitive intentions. The results suggest that generative AI chatbots can support language learning in meaningful ways–especially when they are designed with an understanding of how users think and communicate. They may also help learners express themselves more confidently and connect with their cultural identity. The TALKA case provides empirical insights into how AI-mediated dialogue facilitates cognitive development in low-resource language learners, as well as pragmatic negotiation and socio-cultural affiliation. By focusing on AI-assisted language learning, this study offers new insights into how technology can support language preservation and educational practice.

Method: Align3 employs Test-Time Deliberation (TTD) with hierarchical reflection and revision to reason over specification boundaries. The method is evaluated using SpecBench, a unified benchmark covering 5 scenarios, 103 specifications, and 1,500 prompts.

Result: Experiments on 15 reasoning and 18 instruct models show that: (i) test-time deliberation enhances specification alignment; (ii) Align3 advances the safety-helpfulness trade-off frontier with minimal overhead; (iii) SpecBench effectively reveals alignment gaps.

Conclusion: Test-time deliberation is an effective strategy for reasoning over real-world specification boundaries, with Align3 demonstrating improved specification alignment while maintaining a good safety-helpfulness balance.

Abstract: Large language models (LLMs) are increasingly applied in diverse real-world scenarios, each governed by bespoke behavioral and safety specifications (spec) custom-tailored by users or organizations. These spec, categorized into safety-spec and behavioral-spec, vary across scenarios and evolve with changing preferences and requirements. We formalize this challenge as specification alignment, focusing on LLMs’ ability to follow dynamic, scenario-specific spec from both behavioral and safety perspectives. To address this challenge, we propose Align3, a lightweight method that employs Test-Time Deliberation (TTD) with hierarchical reflection and revision to reason over the specification boundaries. We further present SpecBench, a unified benchmark for measuring specification alignment, covering 5 scenarios, 103 spec, and 1,500 prompts. Experiments on 15 reasoning and 18 instruct models with several TTD methods, including Self-Refine, TPO, and MoreThink, yield three key findings: (i) test-time deliberation enhances specification alignment; (ii) Align3 advances the safety-helpfulness trade-off frontier with minimal overhead; (iii) SpecBench effectively reveals alignment gaps. These results highlight the potential of test-time deliberation as an effective strategy for reasoning over the real-world specification boundaries.

Method: Simplified human-written texts using LLM while preserving core content, pretrained causal models (28M-500M parameters) from scratch on original vs. simplified data, evaluated in fine-tuning and zero-shot settings.

Result: Smaller models degrade less on simpler texts; text complexity has minimal impact on fine-tuning but affects zero-shot performance: simpler texts help linguistic knowledge tasks, complex texts benefit world knowledge and entity tracking tasks.

Conclusion: Different types of data diversity affect transfer and zero-shot performance differently, suggesting data curation should be tailored to specific goals rather than using a one-size-fits-all approach.

Abstract: Improving pretraining data quality and size is known to boost downstream performance, but the role of text complexity–how hard a text is to read–remains less explored. We reduce surface-level complexity (shorter sentences, simpler words, simpler structure) while keeping core content approximately constant and ask: (i) How does complexity affect language modeling across model sizes? (ii) Can useful representations be learned from simpler text alone? (iii) How does pretraining text complexity influence downstream language understanding? We simplify human-written texts using a large language model, pretrain causal models (28M-500M) from scratch on original vs. simplified data, and evaluate them in fine-tuning and zero-shot setups. We find that perplexity is sensitive to the interaction between model capacity and text complexity–smaller models degrade far less on simpler texts–while text complexity has little impact on fine-tuning evaluations, with zero-shot evaluations indicating that simpler texts benefit performance on linguistic knowledge tasks, whereas more complex texts favor tasks requiring world knowledge and entity tracking. Our findings suggest that different types of data diversity affect transfer and zero-shot performance differently, providing insight into tailoring data curation to specific goals.

Method: Proposes K-DeCore with knowledge decoupling mechanism that disentangles reasoning into task-specific and task-agnostic stages, dual-perspective memory consolidation, and structure-guided pseudo-data synthesis.

Result: Extensive experiments on four benchmark datasets show K-DeCore’s superiority over existing continual learning methods across multiple metrics using various backbone LLMs.

Conclusion: K-DeCore effectively addresses CSKR challenges by decoupling knowledge reasoning and maintaining fixed parameters, demonstrating strong performance across diverse structured knowledge tasks.

Abstract: Continual Structured Knowledge Reasoning (CSKR) focuses on training models to handle sequential tasks, where each task involves translating natural language questions into structured queries grounded in structured knowledge. Existing general continual learning approaches face significant challenges when applied to this task, including poor generalization to heterogeneous structured knowledge and inefficient reasoning due to parameter growth as tasks increase. To address these limitations, we propose a novel CSKR framework, \textsc{K-DeCore}, which operates with a fixed number of tunable parameters. Unlike prior methods, \textsc{K-DeCore} introduces a knowledge decoupling mechanism that disentangles the reasoning process into task-specific and task-agnostic stages, effectively bridging the gaps across diverse tasks. Building on this foundation, \textsc{K-DeCore} integrates a dual-perspective memory consolidation mechanism for distinct stages and introduces a structure-guided pseudo-data synthesis strategy to further enhance the model’s generalization capabilities. Extensive experiments on four benchmark datasets demonstrate the superiority of \textsc{K-DeCore} over existing continual learning methods across multiple metrics, leveraging various backbone large language models.

Method: Proposed BiasDetector framework to quantify LLMs’ ability to identify sparse labels under different Instruction Boundary conditions (sufficient, redundant, insufficient prompt coverage), tested across 8 experimental settings and 5 dataset forms.

Result: Evaluations on 5 mainstream LLMs reveal substantial reasoning biases persist despite high accuracy, directly caused by prompt coverage issues in downstream tasks.

Conclusion: Findings emphasize the importance of addressing sparse labels and the need for developers to recognize and mitigate risks introduced by Instruction Boundary in LLM applications.

Abstract: Nowadays, automatically generated datasets are increasingly used in LLM reasoning tasks; however, large-scale corpora often contain inherent flaws. For example, a single-choice question may include none or multiple correct options, while true-or-false questions may involve vague or unverifiable statements. We refer to these exceptional answer forms as sparse labels. To compare LLMs’ ability to recognize various question forms and produce correct answers, we investigate how different instruction formats can either facilitate or mislead LLM reasoning ability. We introduce the concept of Instruction Boundary, which systematically analyzes how different levels of prompt coverage – sufficient, redundant, or insufficient – can lead to reasoning biases and performance changes in LLMs. To examine this phenomenon, we design eight experimental settings across five dataset forms. We further propose BiasDetector, a unified framework that quantifies LLMs’ ability to identify sparse labels under different kinds of Instruction Boundary conditions. Evaluations on five mainstream LLMs show that, despite their seemingly high accuracy, substantial reasoning biases persist in many downstream tasks as a direct consequence of prompt coverage. We analyze the impact of these biases and outline possible mitigation strategies. Our findings highlight not only the importance of addressing sparse labels, but also the need for developers to recognize and mitigate the risks introduced by Instruction Boundary.

Method: Proposes structured sparse factorization using dense dictionary and column-sparse coefficient matrix, enabling union-of-subspaces representation. Uses calibration data to optimize factorization by minimizing functional reconstruction error rather than weight approximation.

Result: Outperforms state-of-the-art low-rank methods in accuracy and perplexity across Llama and Qwen models at 20-50% compression ratios. Preserves model fidelity without fine-tuning and is compatible with quantization for additional gains.

Conclusion: Structured sparse dictionary learning is a powerful alternative to conventional low-rank approaches for efficient LLM deployment, offering greater expressiveness and better accuracy preservation.

Abstract: Post-training compression of large language models (LLMs) largely relies on low-rank weight approximation, which represents each column of a weight matrix in a shared low-dimensional subspace. While this is a computationally efficient strategy, the imposed structural constraint is rigid and can lead to a noticeable model accuracy drop. In this work, we propose CoSpaDi (Compression via Sparse Dictionary Learning), a novel training-free compression framework that replaces low-rank decomposition with a more flexible structured sparse factorization in which each weight matrix is represented with a dense dictionary and a column-sparse coefficient matrix. This formulation enables a union-of-subspaces representation: different columns of the original weight matrix are approximated in distinct subspaces spanned by adaptively selected dictionary atoms, offering greater expressiveness than a single invariant basis. Crucially, CoSpaDi leverages a small calibration dataset to optimize the factorization such that the output activations of compressed projection layers closely match those of the original ones, thereby minimizing functional reconstruction error rather than mere weight approximation. This data-aware strategy preserves better model fidelity without any fine-tuning under reasonable compression ratios. Moreover, the resulting structured sparsity allows efficient sparse-dense matrix multiplication and is compatible with post-training quantization for further memory and latency gains. We evaluate CoSpaDi across multiple Llama and Qwen models under per-layer and per-group settings at 20-50% compression ratios, demonstrating consistent superiority over state-of-the-art data-aware low-rank methods both in accuracy and perplexity. Our results establish structured sparse dictionary learning as a powerful alternative to conventional low-rank approaches for efficient LLM deployment.

Method: Created ML2B benchmark with 30 Kaggle competitions translated into 13 languages across tabular, text, and image data types. Used AIDE framework for automated end-to-end assessment of data science pipelines.

Result: Substantial 15-45% performance degradation on non-English tasks compared to English, highlighting challenges in multilingual representation learning for code generation.

Conclusion: The benchmark and evaluation framework are made available to facilitate future research in multilingual ML code generation, addressing critical gaps in current evaluation practices.

Abstract: Large language models (LLMs) have recently demonstrated strong capabilities in generating machine learning (ML) code, enabling end-to-end pipeline construction from natural language instructions. However, existing benchmarks for ML code generation are mainly restricted to English, overlooking the global and multilingual nature of ML research and practice. To address this gap, we present ML2B, the first benchmark for evaluating multilingual ML code generation. ML2B consists of 30 Kaggle competitions translated into 13 natural languages, covering tabular, text, and image data types, with structured metadata and validated human-reviewed translations. For evaluation, we employ AIDE, an automated framework for end-to-end assessment of data science pipelines, and provide insights into cross-lingual model performance. Our results reveal substantial 15-45% performance degradation on non-English tasks, highlighting critical challenges in multilingual representation learning for code generation. The benchmark, evaluation framework, and comprehensive results are made available through our GitHub repository to facilitate future research in multilingual ML code generation: https://github.com/enaix/ml2b.

Method: HEART provides feedback on incorrect responses using emotionally charged phrases based on Ekman’s six universal emotions, systematically varying emotional tone across iterations to guide models away from flawed reasoning paths.

Result: When guided by an oracle verifier, HEART unlocks significantly deeper reasoning and achieves substantial accuracy improvements over state-of-the-art baselines on challenging benchmarks. However, it struggles in verifier-free settings.

Conclusion: The next frontier in machine reasoning may lie in understanding and leveraging the ‘HEART’ of models - combining emotional guidance with logical refinement, though practical deployment requires solving the verifier-free challenge.

Abstract: Test-time scaling has shown considerable success in improving the performance of language models on complex reasoning tasks without requiring fine-tuning. However, current strategies such as self-reflection primarily focus on logical or structural refinement. They do not leverage the guiding potential of affective feedback. Inspired by psychological research showing that emotions can modulate cognitive performance, we introduce HEART–a novel framework that uses emotionally-driven prompts for iterative self-correction. HEART provides feedback on a model’s incorrect response using a curated set of concise, emotionally charged phrases based on the six universal emotions categorized by Dr. Paul Ekman. By systematically varying the emotional tone of the feedback across iterations, our method guides the model to escape flawed reasoning paths and explore more promising alternatives. We evaluate our framework on challenging reasoning benchmarks including OlympiadBench, Humanity’s Last Exam, and SimpleQA. Our results reveal a significant new phenomenon: when guided by an oracle verifier, this affective iteration protocol unlocks significantly deeper reasoning, leading to consistent and substantial increases in accuracy over state-of-the-art baselines with the same verifier. However, we also identify a critical bottleneck for practical deployment. In a verifier-free setting, it struggles to harness these gains consistently, highlighting as a key challenge for future work. Our findings suggest that the next frontier in machine reasoning may lie not just in refining logic, but also in understanding and leveraging the `HEART’ of the models.

Method: Novel user simulator architecture that simulates four categories of non-collaborative behaviors: requesting unavailable services, digressing conversations, expressing impatience, and providing incomplete utterances.

Result: Experiments on MultiWOZ and τ-bench show significant performance degradation in state-of-the-art tool agents when facing non-collaborative users, with issues like escalated hallucinations and dialogue breakdowns.

Conclusion: Provides an extensible user simulation framework to help develop more robust tool agents and preemptively diagnose weaknesses under challenging real-world conditions.

Abstract: Tool agents interact with users through multi-turn dialogues to accomplish various tasks. Recent studies have adopted user simulation methods to develop these agents in multi-turn settings. However, existing user simulators tend to be agent-friendly, exhibiting only cooperative behaviors, which fails to train and test agents against non-collaborative users in the real world. To address this, we propose a novel user simulator architecture that simulates four categories of non-collaborative behaviors: requesting unavailable services, digressing into tangential conversations, expressing impatience, and providing incomplete utterances. Our user simulator can simulate challenging and natural non-collaborative behaviors while reliably delivering all intents and information necessary to accomplish the task. Our experiments on MultiWOZ and $\tau$-bench reveal significant performance degradation in state-of-the-art tool agents when encountering non-collaborative users. We provide detailed analyses of agents’ weaknesses under each non-collaborative condition, such as escalated hallucinations and dialogue breakdowns. Ultimately, we contribute an easily extensible user simulation framework to help the research community develop tool agents and preemptively diagnose them under challenging real-world conditions within their own services.

Method: Uses ’last but not late’ interaction with causal attention between query and all candidate documents in same context window, then extracts contextual embeddings from each document’s final token.

Result: Achieves state-of-the-art BEIR performance with 61.94 nDCG@10 while being significantly smaller than comparable models.

Conclusion: The novel ’last but not late’ interaction approach enables superior reranking performance with a much more compact model architecture.

Abstract: jina-reranker-v3 is a 0.6B-parameter multilingual listwise reranker that introduces a novel “last but not late” interaction. Unlike late interaction models like ColBERT that encode documents separately before multi-vector matching, our approach applies causal attention between the query and all candidate documents in the same context window, enabling rich interactions before extracting contextual embeddings from each document’s final token. The new model achieves state-of-the-art BEIR performance with 61.94 nDCG@10 while being significantly smaller than other models with comparable performance.

Method: Developed a latent classifier to distinguish correct vs incorrect latent thinking patterns, then used it as a Latent Reward Model (LRM) in Latent Thinking Optimization (LTO) algorithm to probabilistically optimize latent reasoning processes.

Result: LTO significantly improved latent thinking processes across diverse reasoning tasks. The latent classifier reliably predicted answer correctness, and LRM effectively detected incorrect thinking patterns while generalizing across domains.

Conclusion: Reward modeling and supervised thinking optimization can be performed directly in latent space, offering a general, efficient, domain-agnostic approach to improve LLM thinking processes compared to verbal thinking methods.

Abstract: Large Language Models (LLMs) excel at problem solving by generating chain of thoughts in natural language, but such verbal thinking is computationally costly and prone to overthinking. Recent work instead proposes a latent thinking architecture Huginn-3.5B, which represents intermediate reasoning steps as sequence of latent representations. However, latent thoughts lack interpretability and are difficult to supervise, raising concerns about the correctness and reliability of its latent thinking processes. In this paper, we provide a systematic study of how Huginn-3.5B thinks in the latent space and how external supervision signals can improve its latent thinking processes. We show that latent thoughts leading to correct versus incorrect answers exhibit highly distinguishable patterns, and that a latent classifier can reliably predict answer correctness directly from latent thoughts. Leveraging these insights, we propose Latent Thinking Optimization (LTO), a probabilistic algorithm that employs the latent classifier as a Latent Reward Model (LRM) to optimize the latent thinking processes. Extensive experiments across diverse reasoning tasks demonstrate that LRM is highly effective in detecting incorrect latent thinking patterns, and LTO can significantly improve the latent thinking processes. Furthermore, we show that LRM can generalize across diverse domains, and LTO can be seamlessly applied to general LLMs to improve their thinking processes. In contrast to verbal thinking, our method demonstrates that reward modeling and scaling test-time thinking with supervision can be performed directly in the latent space, highlighting its potential as a general, efficient, and domain-agnostic approach to improving the thinking processes of LLMs.

Method: Fine-tuned different SLMs for domain and skill level alignment using data from college admissions reading/writing tests, compared with embedding-based supervised ML models, and conducted semantic similarity analysis.

Result: SLMs consistently outperformed embedding-based models, especially for fine-grained skill alignment. Including more item text data significantly improved performance beyond sample size increases alone.

Conclusion: Fine-tuned SLMs are effective for automated item alignment, with semantic similarity analysis revealing that misclassifications occur when skills are semantically too close.

Abstract: Aligning test items to content standards is a critical step in test development to collect validity evidence based on content. Item alignment has typically been conducted by human experts. This judgmental process can be subjective and time-consuming. This study investigated the performance of fine-tuned small language models (SLMs) for automated item alignment using data from a large-scale standardized reading and writing test for college admissions. Different SLMs were trained for alignment at both domain and skill levels respectively with 10 skills mapped to 4 content domains. The model performance was evaluated in multiple criteria on two testing datasets. The impact of types and sizes of the input data for training was investigated. Results showed that including more item text data led to substantially better model performance, surpassing the improvements induced by sample size increase alone. For comparison, supervised machine learning models were trained using the embeddings from the multilingual-E5-large-instruct model. The study results showed that fine-tuned SLMs consistently outperformed the embedding-based supervised machine learning models, particularly for the more fine-grained skill alignment. To better understand model misclassifications, multiple semantic similarity analysis including pairwise cosine similarity, Kullback-Leibler divergence of embedding distributions, and two-dimension projections of item embeddings were conducted. These analyses consistently showed that certain skills in SAT and PSAT were semantically too close, providing evidence for the observed misclassification.

Method: The authors formalize typicality bias theoretically, verify it empirically on preference datasets, and introduce Verbalized Sampling (VS) - a training-free prompting strategy that asks models to verbalize probability distributions over multiple responses (e.g., “Generate 5 jokes about coffee and their corresponding probabilities”).

Result: VS significantly improves performance across creative writing (poems, stories, jokes), dialogue simulation, open-ended QA, and synthetic data generation, increasing diversity by 1.6-2.1x in creative writing without sacrificing factual accuracy and safety. More capable models benefit more from VS.

Conclusion: The work provides a new data-centric perspective on mode collapse and a practical inference-time remedy (Verbalized Sampling) that helps unlock pre-trained generative diversity in LLMs.

Abstract: Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: typicality bias in preference data, whereby annotators systematically favor familiar text as a result of well-established findings in cognitive psychology. We formalize this bias theoretically, verify it on preference datasets empirically, and show that it plays a central role in mode collapse. Motivated by this analysis, we introduce Verbalized Sampling, a simple, training-free prompting strategy to circumvent mode collapse. VS prompts the model to verbalize a probability distribution over a set of responses (e.g., “Generate 5 jokes about coffee and their corresponding probabilities”). Comprehensive experiments show that VS significantly improves performance across creative writing (poems, stories, jokes), dialogue simulation, open-ended QA, and synthetic data generation, without sacrificing factual accuracy and safety. For instance, in creative writing, VS increases diversity by 1.6-2.1x over direct prompting. We further observe an emergent trend that more capable models benefit more from VS. In sum, our work provides a new data-centric perspective on mode collapse and a practical inference-time remedy that helps unlock pre-trained generative diversity.

Method: Controlled insertion of padding tokens across three open-source model families (Llama, Gemma, Qwen), evaluating outcomes along four axes: activations, generation quality, bias, and safety.

Result: Even small amounts of padding shift hidden representations, degrade quality in smaller models, alter bias unpredictably, and weaken safety guardrails.

Conclusion: Padding is not a harmless detail but a robustness risk that must be carefully handled in deployment.

Abstract: Padding tokens are widely used in large language models (LLMs) to equalize sequence lengths during batched inference. While they should be fully masked, implementation errors can cause them to influence computation, and the extent of this influence is not well understood. We systematically study this effect across three open-source model families (Llama, Gemma, Qwen), inserting controlled amounts of padding and evaluating outcomes along four axes: activations, generation quality, bias, and safety. Even small amounts of padding shift hidden representations, degrade quality in smaller models, alter bias in unpredictable ways, and weaken safety guardrails. These findings demonstrate that padding is not a harmless detail but a robustness risk that must be carefully handled in deployment.

Method: Introduces AMAS with a dynamic graph designer that autonomously identifies task-specific optimal graph configurations through lightweight LLM adaptation, creating context-sensitive agent pathways.

Result: AMAS systematically exceeds state-of-the-art single-agent and multi-agent approaches across question answering, mathematical deduction, and code generation benchmarks with diverse LLM architectures.

Conclusion: Context-sensitive structural adaptability is a foundational requirement for high-performance LLM multi-agent system deployments.

Abstract: Although large language models (LLMs) have revolutionized natural language processing capabilities, their practical implementation as autonomous multi-agent systems (MAS) for industrial problem-solving encounters persistent barriers. Conventional MAS architectures are fundamentally restricted by inflexible, hand-crafted graph topologies that lack contextual responsiveness, resulting in diminished efficacy across varied academic and commercial workloads. To surmount these constraints, we introduce AMAS, a paradigm-shifting framework that redefines LLM-based MAS through a novel dynamic graph designer. This component autonomously identifies task-specific optimal graph configurations via lightweight LLM adaptation, eliminating the reliance on monolithic, universally applied structural templates. Instead, AMAS exploits the intrinsic properties of individual inputs to intelligently direct query trajectories through task-optimized agent pathways. Rigorous validation across question answering, mathematical deduction, and code generation benchmarks confirms that AMAS systematically exceeds state-of-the-art single-agent and multi-agent approaches across diverse LLM architectures. Our investigation establishes that context-sensitive structural adaptability constitutes a foundational requirement for high-performance LLM MAS deployments.

Method: A data-centric approach that rebalances the turn-count distribution of the training dataset to address the skewed distribution causing Format Inertia.

Result: Experimental results demonstrate that the rebalancing method substantially alleviates Format Inertia in medical pre-consultation dialogues.

Conclusion: Rebalancing turn-count distributions in training data effectively mitigates Format Inertia, improving the quality of LLM-generated medical dialogues by reducing repetitive and uninformative questioning.

Abstract: Recent advances in Large Language Models (LLMs) have brought significant improvements to various service domains, including chatbots and medical pre-consultation applications. In the healthcare domain, the most common approach for adapting LLMs to multi-turn dialogue generation is Supervised Fine-Tuning (SFT). However, datasets for SFT in tasks like medical pre-consultation typically exhibit a skewed turn-count distribution. Training on such data induces a novel failure mechanism we term Format Inertia, where models tend to generate repetitive, format-correct, but diagnostically uninformative questions in long medical dialogues. To mitigate this observed failure mechanism, we adopt a simple, data-centric method that rebalances the turn-count distribution of the training dataset. Experimental results show that our approach substantially alleviates Format Inertia in medical pre-consultation.

Method: Online reinforcement learning framework where LLMs interact with search engines and receive reward signals to shape planning, retrieval, and reasoning behaviors, enabling dynamic interaction that reflects real-world verification scenarios.

Result: Veri-R1 improves joint accuracy by up to 30% and doubles evidence score, often surpassing larger-scale model counterparts. Ablation studies reveal impact of reward components and link between output logits and label accuracy.

Conclusion: Online RL is effective for precise and faithful claim verification, providing important foundation for future research in LLM-empowered verification systems.

Abstract: Claim verification with large language models (LLMs) has recently attracted growing attention, due to their strong reasoning capabilities and transparent verification processes compared to traditional answer-only judgments. However, existing approaches to online claim verification, which requires iterative evidence retrieval and reasoning, still mainly rely on prompt engineering or pre-designed reasoning workflows, without unified training to improve necessary skills. Therefore, we introduce Veri-R1, an online reinforcement learning (RL) framework that enables an LLM to interact with a search engine and to receive reward signals that explicitly shape its planning, retrieval, and reasoning behaviors. This dynamic interaction of LLM with retrieval systems more accurately reflects real-world verification scenarios and fosters comprehensive verification skills. Empirical results show that Veri-R1 improves joint accuracy by up to 30% and doubles the evidence score, often surpassing its larger-scale model counterparts. Ablation studies further reveal the impact of reward components, and the link between output logits and label accuracy. Our results highlight the effectiveness of online RL for precise and faithful claim verification, providing an important foundation for future research. We release our code to support community progress in LLM empowered claim verification.

Method: Created InfoMosaic-Bench covering six domains (medicine, finance, maps, video, web, and multi-domain integration) using InfoMosaic-Flow pipeline that grounds task conditions in verified tool outputs, enforces cross-source dependencies, and filters out trivial cases.

Result: Experiments with 14 state-of-the-art LLM agents show: web information alone is insufficient (38.2% accuracy, 67.5% pass rate); domain tools provide selective but inconsistent benefits; 22.4% of failures come from incorrect tool usage or selection.

Conclusion: Current LLMs still struggle with basic tool handling and effectively integrating multiple information sources, highlighting the need for improved tool-augmented agent capabilities.

Abstract: Information seeking is a fundamental requirement for humans. However, existing LLM agents rely heavily on open-web search, which exposes two fundamental weaknesses: online content is noisy and unreliable, and many real-world tasks require precise, domain-specific knowledge unavailable from the web. The emergence of the Model Context Protocol (MCP) now allows agents to interface with thousands of specialized tools, seemingly resolving this limitation. Yet it remains unclear whether agents can effectively leverage such tools – and more importantly, whether they can integrate them with general-purpose search to solve complex tasks. Therefore, we introduce InfoMosaic-Bench, the first benchmark dedicated to multi-source information seeking in tool-augmented agents. Covering six representative domains (medicine, finance, maps, video, web, and multi-domain integration), InfoMosaic-Bench requires agents to combine general-purpose search with domain-specific tools. Tasks are synthesized with InfoMosaic-Flow, a scalable pipeline that grounds task conditions in verified tool outputs, enforces cross-source dependencies, and filters out shortcut cases solvable by trivial lookup. This design guarantees both reliability and non-triviality. Experiments with 14 state-of-the-art LLM agents reveal three findings: (i) web information alone is insufficient, with GPT-5 achieving only 38.2% accuracy and 67.5% pass rate; (ii) domain tools provide selective but inconsistent benefits, improving some domains while degrading others; and (iii) 22.4% of failures arise from incorrect tool usage or selection, highlighting that current LLMs still struggle with even basic tool handling.

Method: Proposes a task-agnostic ILCL method using guided exploration with a compact TODO forest to prioritize actions and a lightweight plan-act-extract loop to execute them, automatically generating high-precision reusable context documents.

Result: Experiments across TextWorld, ALFWorld, and Crafter show significant improvements: ReAct’s success rate in TextWorld increased from 37% to 95%, and IGE improved from 81% to 95%, with gains in both success and efficiency.

Conclusion: By transforming one-off exploration into persistent, reusable knowledge, ILCL complements existing contexts to enable more reliable and efficient LLM agents, amortizing exploration costs across multiple downstream tasks.

Abstract: Large language model (LLM) agents typically receive two kinds of context: (i) environment-level manuals that define interaction interfaces and global rules, and (ii) task-level guidance or demonstrations tied to specific goals. In this work, we identify a crucial but overlooked third type of context, instance-level context, which consists of verifiable and reusable facts tied to a specific environment instance, such as object locations, crafting recipes, and local rules. We argue that the absence of instance-level context is a common source of failure for LLM agents in complex tasks, as success often depends not only on reasoning over global rules or task prompts but also on making decisions based on precise and persistent facts. Acquiring such context requires more than memorization: the challenge lies in efficiently exploring, validating, and formatting these facts under tight interaction budgets. We formalize this problem as Instance-Level Context Learning (ILCL) and introduce our task-agnostic method to solve it. Our method performs a guided exploration, using a compact TODO forest to intelligently prioritize its next actions and a lightweight plan-act-extract loop to execute them. This process automatically produces a high-precision context document that is reusable across many downstream tasks and agents, thereby amortizing the initial exploration cost. Experiments across TextWorld, ALFWorld, and Crafter demonstrate consistent gains in both success and efficiency: for instance, ReAct’s mean success rate in TextWorld rises from 37% to 95%, while IGE improves from 81% to 95%. By transforming one-off exploration into persistent, reusable knowledge, our method complements existing contexts to enable more reliable and efficient LLM agents.

Method: Memory-augmented architecture with hierarchical parametric memory banks. Small language models fetch context-dependent memory blocks during pretraining and inference. Pretraining learns to store long-tail knowledge in memory while the small model captures common knowledge and reasoning.

Result: A 160M-parameter model with 18M memory from a 4.6B memory bank performs comparably to models with 2x+ parameters. Hierarchical feed-forward memories work robustly across transformers, scaling to over 21B parameters.

Conclusion: Memory-augmented architectures enable efficient knowledge storage and access, making small models competitive with much larger ones while being practical for resource-constrained devices.

Abstract: The impressive performance gains of modern language models currently rely on scaling parameters: larger models store more world knowledge and reason better. Yet compressing all world knowledge into parameters is unnecessary, as only a fraction is used per prompt, and impractical for edge devices with limited inference-time memory and compute. We address this shortcoming by a memory-augmented architecture and a pretraining strategy aligned with existing hardware paradigms. We introduce small language models that access large hierarchical parametric memory banks encoding world knowledge. During pretraining and inference, we fetch a small, context-dependent memory block and add it to the model. Our pretraining learns to store long-tail world knowledge in the memory parameters, while the small language model acts as an anchor capturing common knowledge and general reasoning abilities. Through trillion-token-scale experiments, we show significant gains: a 160M-parameters model augmented with an 18M-parameters memory fetched from a 4.6B memory bank obtains comparable performance to a regular model with more than 2x the parameters. Through extensive experiments, we study the optimal type and size of parametric memories in transformers, scaling them to over 21B parameters. We find that our proposed hierarchical feed-forward memories work robustly across transformer architectures, whether added during pretraining or post-hoc.

Method: Combines Llama 3.1 with Named-Entity-Recognition (NER) using tunable weights to emphasize/de-emphasize specific types of differences, generating both quantitative similarity scores and interpretable feedback.

Result: Achieves 67% exact-match accuracy and 93% accuracy within +/- 1 compared to radiologist-provided ground truth scores, outperforming both LLMs and NER used independently.

Conclusion: The proposed Llama-EntScore method effectively addresses limitations of standalone LLM and NER approaches for radiology report comparison, providing accurate semantic similarity assessment with interpretable feedback for training purposes.

Abstract: Radiology report evaluation is a crucial part of radiologists’ training and plays a key role in ensuring diagnostic accuracy. As part of the standard reporting workflow, a junior radiologist typically prepares a preliminary report, which is then reviewed and edited by a senior radiologist to produce the final report. Identifying semantic differences between preliminary and final reports is essential for junior doctors, both as a training tool and to help uncover gaps in clinical knowledge. While AI in radiology is a rapidly growing field, the application of large language models (LLMs) remains challenging due to the need for specialised domain knowledge. In this paper, we explore the ability of LLMs to provide explainable and accurate comparisons of reports in the radiology domain. We begin by comparing the performance of several LLMs in comparing radiology reports. We then assess a more traditional approach based on Named-Entity-Recognition (NER). However, both approaches exhibit limitations in delivering accurate feedback on semantic similarity. To address this, we propose Llama-EntScore, a semantic similarity scoring method using a combination of Llama 3.1 and NER with tunable weights to emphasise or de-emphasise specific types of differences. Our approach generates a quantitative similarity score for tracking progress and also gives an interpretation of the score that aims to offer valuable guidance in reviewing and refining their reporting. We find our method achieves 67% exact-match accuracy and 93% accuracy within +/- 1 when compared to radiologist-provided ground truth scores - outperforming both LLMs and NER used independently. Code is available at: https://github.com/otmive/llama_reports

Method: Created SurveyBench with: (1) survey topics from 11,343 arXiv papers and 4,947 high-quality surveys; (2) multifaceted metric hierarchy assessing outline quality, content quality, and non-textual richness; (3) dual-mode evaluation protocol with content-based and quiz-based answerability tests aligned with readers’ needs.

Result: SurveyBench effectively challenges existing LLM4Survey approaches, showing they perform on average 21% lower than human-written surveys in content-based evaluation.

Conclusion: The proposed evaluation framework successfully reveals the deficiencies in current automated survey generation methods and provides a more rigorous, reader-aligned benchmark for assessing survey quality.

Abstract: Academic survey writing, which distills vast literature into a coherent and insightful narrative, remains a labor-intensive and intellectually demanding task. While recent approaches, such as general DeepResearch agents and survey-specialized methods, can generate surveys automatically (a.k.a. LLM4Survey), their outputs often fall short of human standards and there lacks a rigorous, reader-aligned benchmark for thoroughly revealing their deficiencies. To fill the gap, we propose a fine-grained, quiz-driven evaluation framework SurveyBench, featuring (1) typical survey topics source from recent 11,343 arXiv papers and corresponding 4,947 high-quality surveys; (2) a multifaceted metric hierarchy that assesses the outline quality (e.g., coverage breadth, logical coherence), content quality (e.g., synthesis granularity, clarity of insights), and non-textual richness; and (3) a dual-mode evaluation protocol that includes content-based and quiz-based answerability tests, explicitly aligned with readers’ informational needs. Results show SurveyBench effectively challenges existing LLM4Survey approaches (e.g., on average 21% lower than human in content-based evaluation).

Method: SoC-DT unifies reaction-diffusion tumor growth models with discrete SoC interventions (surgery, chemotherapy, radiotherapy) and genomic/demographic personalization. Uses IMEX-SoC solver for stability and scalability.

Result: SoC-DT consistently outperforms classical PDE baselines and purely data-driven neural models in predicting tumor dynamics on both synthetic and real glioma data.

Conclusion: SoC-DT bridges mechanistic interpretability with modern differentiable solvers, establishing a principled foundation for patient-specific digital twins in oncology with biologically consistent tumor dynamics estimation.

Abstract: Accurate prediction of tumor trajectories under standard-of-care (SoC) therapies remains a major unmet need in oncology. This capability is essential for optimizing treatment planning and anticipating disease progression. Conventional reaction-diffusion models are limited in scope, as they fail to capture tumor dynamics under heterogeneous therapeutic paradigms. There is hence a critical need for computational frameworks that can realistically simulate SoC interventions while accounting for inter-patient variability in genomics, demographics, and treatment regimens. We introduce Standard-of-Care Digital Twin (SoC-DT), a differentiable framework that unifies reaction-diffusion tumor growth models, discrete SoC interventions (surgery, chemotherapy, radiotherapy) along with genomic and demographic personalization to predict post-treatment tumor structure on imaging. An implicit-explicit exponential time-differencing solver, IMEX-SoC, is also proposed, which ensures stability, positivity, and scalability in SoC treatment situations. Evaluated on both synthetic data and real world glioma data, SoC-DT consistently outperforms classical PDE baselines and purely data-driven neural models in predicting tumor dynamics. By bridging mechanistic interpretability with modern differentiable solvers, SoC-DT establishes a principled foundation for patient-specific digital twins in oncology, enabling biologically consistent tumor dynamics estimation. Code will be made available upon acceptance.

Method: Proposes AgentAug with multiple LLM-driven pipelines simulating four fabrication categories for news video creation, combined with active learning based on uncertainty sampling to select useful augmented samples during training.

Result: Experimental results on two benchmark datasets show that AgentAug consistently improves the performance of short video fake news detectors.

Conclusion: AgentAug effectively addresses the data scarcity and diversity issues in fake news video detection by generating realistic augmented data that captures the complex relationships in real-world fake news creation.

Abstract: The emergence of fake news on short video platforms has become a new significant societal concern, necessitating automatic video-news-specific detection. Current detectors primarily rely on pattern-based features to separate fake news videos from real ones. However, limited and less diversified training data lead to biased patterns and hinder their performance. This weakness stems from the complex many-to-many relationships between video material segments and fabricated news events in real-world scenarios: a single video clip can be utilized in multiple ways to create different fake narratives, while a single fabricated event often combines multiple distinct video segments. However, existing datasets do not adequately reflect such relationships due to the difficulty of collecting and annotating large-scale real-world data, resulting in sparse coverage and non-comprehensive learning of the characteristics of potential fake news video creation. To address this issue, we propose a data augmentation framework, AgentAug, that generates diverse fake news videos by simulating typical creative processes. AgentAug implements multiple LLM-driven pipelines of four fabrication categories for news video creation, combined with an active learning strategy based on uncertainty sampling to select the potentially useful augmented samples during training. Experimental results on two benchmark datasets demonstrate that AgentAug consistently improves the performance of short video fake news detectors.

Method: Combines distributed multi-GPU inference system with optimized ONNX models, heterogeneous batch inference, and high-throughput parallelism for efficient video processing, followed by interactive visualization platform with multiple chart types.

Result: Successfully generates timestamped appearance records and provides comprehensive visualizations including frequency charts, duration analyses, co-appearance matrices, network graphs, and heatmaps for multi-dimensional insights.

Conclusion: The framework enables new possibilities for entertainment analytics, content creation strategies, and audience engagement studies by bridging distributed recognition with structured, visually-driven analytics.

Abstract: In an era dominated by video content, understanding its structure and dynamics has become increasingly important. This paper presents a hybrid framework that combines a distributed multi-GPU inference system with an interactive visualization platform for analyzing celebrity dynamics in video episodes. The inference framework efficiently processes large volumes of video data by leveraging optimized ONNX models, heterogeneous batch inference, and high-throughput parallelism, ensuring scalable generation of timestamped appearance records. These records are then transformed into a comprehensive suite of visualizations, including appearance frequency charts, duration analyses, pie charts, co-appearance matrices, network graphs, stacked area charts, seasonal comparisons, and heatmaps. Together, these visualizations provide multi-dimensional insights into video content, revealing patterns in celebrity prominence, screen-time distribution, temporal dynamics, co-appearance relationships, and intensity across episodes and seasons. The interactive nature of the system allows users to dynamically explore data, identify key moments, and uncover evolving relationships between individuals. By bridging distributed recognition with structured, visually-driven analytics, this work enables new possibilities for entertainment analytics, content creation strategies, and audience engagement studies.

Method: Trained CNNs (MobileNetV2, ResNet-18, EfficientNet-B0) and vision transformers (DeiT-Tiny, ViT-B16) on labeled underwater data, evaluated on cross-domain test set. Also assessed zero-shot models CLIP ViT-L14 and Gemini 2.0 Flash.

Result: MobileNetV2 achieved strongest cross-domain performance (F1 0.97). All fine-tuned models had high Precision (~99%) but varied Recall. Zero-shot CLIP had Recall ~80% but Precision ~56%, while Gemini showed inverse profile (Precision ~99%, Recall ~81%).

Conclusion: Compact CNNs with supervised training generalize effectively for cross-domain underwater detection, while large pretrained vision-language models provide complementary strengths.

Abstract: Marine plastic pollution is a pressing environmental threat, making reliable automation for underwater debris detection essential. However, vision systems trained on one dataset often degrade on new imagery due to domain shift. This study benchmarks models for cross-domain robustness, training convolutional neural networks - CNNs (MobileNetV2, ResNet-18, EfficientNet-B0) and vision transformers (DeiT-Tiny, ViT-B16) on a labeled underwater dataset and then evaluates them on a balanced cross-domain test set built from plastic-positive images drawn from a different source and negatives from the training domain. Two zero-shot models were assessed, CLIP ViT-L14 and Google’s Gemini 2.0 Flash, that leverage pretraining to classify images without fine-tuning. Results show the lightweight MobileNetV2 delivers the strongest cross-domain performance (F1 0.97), surpassing larger models. All fine-tuned models achieved high Precision (around 99%), but differ in Recall, indicating varying sensitivity to plastic instances. Zero-shot CLIP is comparatively sensitive (Recall around 80%) yet prone to false positives (Precision around 56%), whereas Gemini exhibits the inverse profile (Precision around 99%, Recall around 81%). Error analysis highlights recurring confusions with coral textures, suspended particulates, and specular glare. Overall, compact CNNs with supervised training can generalize effectively for cross-domain underwater detection, while large pretrained vision-language models provide complementary strengths.

Method: Ensemble framework combining Swin Transformers and ViTs with texture-based methods, using data-splitting, sequential training, frequency splitting, patch-based attention, and face segmentation techniques.

Result: Achieves state-of-the-art performance on DFWild-Cup dataset (diverse subset of eight deepfake datasets), demonstrating improved accuracy and robustness.

Conclusion: Hybrid models combining transformers and texture-based methods can effectively address deepfake detection challenges, offering robust real-world solutions through complementary feature extraction approaches.

Abstract: Detecting manipulated media has now become a pressing issue with the recent rise of deepfakes. Most existing approaches fail to generalize across diverse datasets and generation techniques. We thus propose a novel ensemble framework, combining the strengths of transformer-based architectures, such as Swin Transformers and ViTs, and texture-based methods, to achieve better detection accuracy and robustness. Our method introduces innovative data-splitting, sequential training, frequency splitting, patch-based attention, and face segmentation techniques to handle dataset imbalances, enhance high-impact regions (e.g., eyes and mouth), and improve generalization. Our model achieves state-of-the-art performance when tested on the DFWild-Cup dataset, a diverse subset of eight deepfake datasets. The ensemble benefits from the complementarity of these approaches, with transformers excelling in global feature extraction and texturebased methods providing interpretability. This work demonstrates that hybrid models can effectively address the evolving challenges of deepfake detection, offering a robust solution for real-world applications.

Method: Two-stage pipeline: 1) Visual label retrieval using three multilingual encoders (mCLIP, AraCLIP, Jina V4) with a hybrid vocabulary from training captions and Visual Genome translations; 2) Caption generation using Qwen-VL and Gemini Pro Vision with retrieved labels as prompts.

Result: Best performance varied by metric: mCLIP + Gemini Pro Vision achieved highest BLEU-1 (5.34%) and cosine similarity (60.01%), while AraCLIP + Qwen-VL obtained best LLM-judge score (36.33%) across six encoder-decoder configurations.

Conclusion: The interpretable pipeline successfully enables culturally coherent and contextually accurate Arabic captions, with different encoder-decoder combinations excelling in different evaluation metrics.

Abstract: We present VLCAP, an Arabic image captioning framework that integrates CLIP-based visual label retrieval with multimodal text generation. Rather than relying solely on end-to-end captioning, VLCAP grounds generation in interpretable Arabic visual concepts extracted with three multilingual encoders, mCLIP, AraCLIP, and Jina V4, each evaluated separately for label retrieval. A hybrid vocabulary is built from training captions and enriched with about 21K general domain labels translated from the Visual Genome dataset, covering objects, attributes, and scenes. The top-k retrieved labels are transformed into fluent Arabic prompts and passed along with the original image to vision-language models. In the second stage, we tested Qwen-VL and Gemini Pro Vision for caption generation, resulting in six encoder-decoder configurations. The results show that mCLIP + Gemini Pro Vision achieved the best BLEU-1 (5.34%) and cosine similarity (60.01%), while AraCLIP + Qwen-VL obtained the highest LLM-judge score (36.33%). This interpretable pipeline enables culturally coherent and contextually accurate Arabic captions.

Method: Proposes ReactDiff framework that incorporates two priors: temporal facial behavioral kinematics and facial action unit dependencies to guide the diffusion process toward realistic human reaction manifolds.

Result: Extensive experiments on REACT2024 dataset show state-of-the-art reaction quality, diversity, and reaction appropriateness compared to existing methods.

Conclusion: ReactDiff successfully addresses the challenge of generating diverse and human-like facial reactions by incorporating anatomical and temporal constraints, achieving superior performance in both quality and diversity.

Abstract: The automatic generation of diverse and human-like facial reactions in dyadic dialogue remains a critical challenge for human-computer interaction systems. Existing methods fail to model the stochasticity and dynamics inherent in real human reactions. To address this, we propose ReactDiff, a novel temporal diffusion framework for generating diverse facial reactions that are appropriate for responding to any given dialogue context. Our key insight is that plausible human reactions demonstrate smoothness, and coherence over time, and conform to constraints imposed by human facial anatomy. To achieve this, ReactDiff incorporates two vital priors (spatio-temporal facial kinematics) into the diffusion process: i) temporal facial behavioral kinematics and ii) facial action unit dependencies. These two constraints guide the model toward realistic human reaction manifolds, avoiding visually unrealistic jitters, unstable transitions, unnatural expressions, and other artifacts. Extensive experiments on the REACT2024 dataset demonstrate that our approach not only achieves state-of-the-art reaction quality but also excels in diversity and reaction appropriateness.

Method: Used SpaceNet dataset with two label-distribution regimes: naturally imbalanced five-class split and balanced-resampled split (700 images per class). Applied matched preprocessing (224x224, ImageNet normalization), lightweight augmentations, and 40-epoch training budget on single NVIDIA P100 GPU.

Result: On imbalanced split: EfficientNet-B0 achieved 93% test accuracy with strong macro-F1 and lower latency; ViT-Base was competitive at 93% but with larger parameter count and runtime. On balanced split: Both models performed strongly, with EfficientNet-B0 reaching 99% accuracy while ViT-Base remained competitive.

Conclusion: Balancing label distributions narrows architecture performance gaps, but CNNs retain efficiency advantages in terms of model size and latency. Both architectures can achieve strong performance when data is properly balanced.

Abstract: We present a controlled comparison of a convolutional neural network (EfficientNet-B0) and a Vision Transformer (ViT-Base) on SpaceNet under two label-distribution regimes: a naturally imbalanced five-class split and a balanced-resampled split with 700 images per class (70:20:10 train/val/test). With matched preprocessing (224x224, ImageNet normalization), lightweight augmentations, and a 40-epoch budget on a single NVIDIA P100, we report accuracy, macro-F1, balanced accuracy, per-class recall, and deployment metrics (model size and latency). On the imbalanced split, EfficientNet-B0 reaches 93% test accuracy with strong macro-F1 and lower latency; ViT-Base is competitive at 93% with a larger parameter count and runtime. On the balanced split, both models are strong; EfficientNet-B0 reaches 99% while ViT-Base remains competitive, indicating that balancing narrows architecture gaps while CNNs retain an efficiency edge. We release manifests, logs, and per-image predictions to support reproducibility.

Method: Developed an end-to-end system that predicts Oriented Bounding Boxes (OBB) for precise logo detection. Created a dataset of 1,103 frames from Swedish elite soccer with 670 unique sponsor logos annotated with OBBs. Integrated language-driven agentic layer for natural language queries and report generation.

Result: Achieved mean Average Precision (mAP@0.5) of 0.859, with precision of 0.96 and recall of 0.87. System provides precise visibility metrics including exposure duration and on-screen coverage. Complete analytics dashboard enables auditable and interpretable sponsor measurement.

Conclusion: ExposureEngine provides a comprehensive, automated solution for accurate sponsor visibility analytics in sports broadcasts using rotation-aware detection, overcoming limitations of traditional methods and enabling scalable, precise measurement.

Abstract: Quantifying sponsor visibility in sports broadcasts is a critical marketing task traditionally hindered by manual, subjective, and unscalable analysis methods. While automated systems offer an alternative, their reliance on axis-aligned Horizontal Bounding Box (HBB) leads to inaccurate exposuremetrics when logos appear rotated or skewed due to dynamic camera angles and perspective distortions. This paper introduces ExposureEngine, an end-to-end system designed for accurate, rotation-aware sponsor visibility analytics in sports broadcasts, demonstrated in a soccer case study. Our approach predicts Oriented Bounding Box (OBB) to provide a geometrically precise fit to each logo regardless of the orientation on-screen. To train and evaluate our detector, we developed a new dataset comprising 1,103 frames from Swedish elite soccer, featuring 670 unique sponsor logos annotated with OBBs. Our model achieves a mean Average Precision (mAP@0.5) of 0.859, with a precision of 0.96 and recall of 0.87, demonstrating robust performance in localizing logos under diverse broadcast conditions. The system integrates these detections into an analytical pipeline that calculates precise visibility metrics, such as exposure duration and on-screen coverage. Furthermore, we incorporate a language-driven agentic layer, enabling users to generate reports, summaries, and media content through natural language queries. The complete system, including the dataset and the analytics dashboard, provides a comprehensive solution for auditable and interpretable sponsor measurement in sports media. An overview of the ExposureEngine is available online: https://youtu.be/tRw6OBISuW4 .

Method: The authors systematically examine four core AI tasks: detection and classification, tracking and reidentification, trajectory prediction, and intent recognition and prediction, with emphasis on developments from the past five years.

Result: The review identifies emerging research trends and provides a foundational reference linking visual AI advances with practical considerations for real-world implementation in intelligent transportation systems.

Conclusion: The survey highlights four major open challenges from data, model, and deployment perspectives to guide future research in developing next-generation sensing systems for enhanced VRU safety.

Abstract: Ensuring the safety of vulnerable road users (VRUs), such as pedestrians and cyclists, remains a critical global challenge, as conventional infrastructure-based measures often prove inadequate in dynamic urban environments. Recent advances in artificial intelligence (AI), particularly in visual perception and reasoning, open new opportunities for proactive and context-aware VRU protection. However, existing surveys on AI applications for VRUs predominantly focus on detection, offering limited coverage of other vision-based tasks that are essential for comprehensive VRU understanding and protection. This paper presents a state-of-the-art review of recent progress in camera-based AI sensing systems for VRU safety, with an emphasis on developments from the past five years and emerging research trends. We systematically examine four core tasks, namely detection and classification, tracking and reidentification, trajectory prediction, and intent recognition and prediction, which together form the backbone of AI-empowered proactive solutions for VRU protection in intelligent transportation systems. To guide future research, we highlight four major open challenges from the perspectives of data, model, and deployment. By linking advances in visual AI with practical considerations for real-world implementation, this survey aims to provide a foundational reference for the development of next-generation sensing systems to enhance VRU safety.

Method: Proposes PaperTalker, a multi-agent framework that integrates slide generation with layout refinement using tree search visual choice, cursor grounding, subtitling, speech synthesis, and talking-head rendering. Uses parallel slide-wise generation for efficiency.

Result: Experiments on Paper2Video benchmark show that the generated presentation videos are more faithful and informative than existing baselines, establishing practical automated academic video generation.

Conclusion: PaperTalker represents a significant step toward automated and ready-to-use academic video generation, with the dataset, agent framework, and code made publicly available.

Abstract: Academic presentation videos have become an essential medium for research communication, yet producing them remains highly labor-intensive, often requiring hours of slide design, recording, and editing for a short 2 to 10 minutes video. Unlike natural video, presentation video generation involves distinctive challenges: inputs from research papers, dense multi-modal information (text, figures, tables), and the need to coordinate multiple aligned channels such as slides, subtitles, speech, and human talker. To address these challenges, we introduce PaperTalker, the first benchmark of 101 research papers paired with author-created presentation videos, slides, and speaker metadata. We further design four tailored evaluation metrics–Meta Similarity, PresentArena, PresentQuiz, and IP Memory–to measure how videos convey the paper’s information to the audience. Building on this foundation, we propose PaperTalker, the first multi-agent framework for academic presentation video generation. It integrates slide generation with effective layout refinement by a novel effective tree search visual choice, cursor grounding, subtitling, speech synthesis, and talking-head rendering, while parallelizing slide-wise generation for efficiency. Experiments on Paper2Video demonstrate that the presentation videos produced by our approach are more faithful and informative than existing baselines, establishing a practical step toward automated and ready-to-use academic video generation. Our dataset, agent, and code are available at https://github.com/showlab/Paper2Video.

Method: Analysis of EOFM representation spaces to demonstrate sensitivity to sensor architecture differences.

Result: EOFMs’ representation spaces are highly sensitive to sensor architecture, revealing significant differences in how models process data from different sensors.

Conclusion: Understanding sensor architecture sensitivity provides crucial insights for improving EOFM design and guiding robust remote-sensing science practices.

Abstract: Earth Observation Foundation Models (EOFMs) have exploded in prevalence as tools for processing the massive volumes of remotely sensed and other earth observation data, and for delivering impact on the many essential earth monitoring tasks. An emerging trend posits using the outputs of pre-trained models as ’embeddings’ which summarize high dimensional data to be used for generic tasks such as similarity search and content-specific queries. However, most EOFM models are trained only on single modalities of data and then applied or benchmarked by matching bands across different modalities. It is not clear from existing work what impact diverse sensor architectures have on the internal representations of the present suite of EOFMs. We show in this work that the representation space of EOFMs is highly sensitive to sensor architecture and that understanding this difference gives a vital perspective on the pitfalls of current EOFM design and signals for how to move forward as model developers, users, and a community guided by robust remote-sensing science.

Method: Uses inpainting-guided perturbation with SAM-refined masks for object removal/replacement and background replacement, applied to YOLOv9 seal detection in drone imagery.

Result: Produces explanations that localize diagnostic structures, avoid deletion artifacts, and provide domain-relevant insights validated by expert review and quantitative metrics.

Conclusion: The approach enables more trustworthy AI deployment in ecology by generating ecologically plausible explanations that support expert validation.

Abstract: Ecological monitoring is increasingly automated by vision models, yet opaque predictions limit trust and field adoption. We present an inpainting-guided, perturbation-based explanation technique that produces photorealistic, mask-localized edits that preserve scene context. Unlike masking or blurring, these edits stay in-distribution and reveal which fine-grained morphological cues drive predictions in tasks such as species recognition and trait attribution. We demonstrate the approach on a YOLOv9 detector fine-tuned for harbor seal detection in Glacier Bay drone imagery, using Segment-Anything-Model-refined masks to support two interventions: (i) object removal/replacement (e.g., replacing seals with plausible ice/water or boats) and (ii) background replacement with original animals composited onto new scenes. Explanations are assessed by re-scoring perturbed images (flip rate, confidence drop) and by expert review for ecological plausibility and interpretability. The resulting explanations localize diagnostic structures, avoid deletion artifacts common to traditional perturbations, and yield domain-relevant insights that support expert validation and more trustworthy deployment of AI in ecology.

Method: Modular framework combining pre-trained state-of-the-art IR networks with generative DPMs, requiring training of only a small task-specific module (0.7M parameters) and employing efficient sampling strategies.

Result: Outperforms existing approaches in perceptual quality, maintains competitive fidelity metrics, and achieves at least 4x reduction in neural function evaluations without performance loss.

Conclusion: DP-IR enables practical adoption of DPMs for image restoration by reducing computational requirements while maintaining high performance across multiple IR tasks.

Abstract: Diffusion Probabilistic Models (DPMs) have been recently utilized to deal with various blind image restoration (IR) tasks, where they have demonstrated outstanding performance in terms of perceptual quality. However, the task-specific nature of existing solutions and the excessive computational costs related to their training, make such models impractical and challenging to use for different IR tasks than those that were initially trained for. This hinders their wider adoption, especially by those who lack access to powerful computational resources and vast amount of training data. In this work we aim to address the above issues and enable the successful adoption of DPMs in practical IR-related applications. Towards this goal, we propose a modular diffusion probabilistic IR framework (DP-IR), which allows us to combine the performance benefits of existing pre-trained state-of-the-art IR networks and generative DPMs, while it requires only the additional training of a relatively small module (0.7M params) related to the particular IR task of interest. Moreover, the architecture of the proposed framework allows for a sampling strategy that leads to at least four times reduction of neural function evaluations without suffering any performance loss, while it can also be combined with existing acceleration techniques such as DDIM. We evaluate our model on four benchmarks for the tasks of burst JDD-SR, dynamic scene deblurring, and super-resolution. Our method outperforms existing approaches in terms of perceptual quality while it retains a competitive performance with respect to fidelity metrics.

Method: Systematic survey methodology covering thresholding, edge detection, region-based segmentation, clustering algorithms, model-based techniques, CNNs, FCNs, U-Net variants, attention mechanisms, semi-supervised learning, GANs, and Transformer-based models.

Result: The survey comprehensively maps the evolution of MIS methodologies and identifies emerging trends including hybrid architectures, cross-modality learning, federated learning, and active learning strategies.

Conclusion: Despite significant progress, critical challenges persist including dataset bias, domain adaptation, model interpretability, and integration into clinical workflows, requiring continued research and development.

Abstract: Medical Image Segmentation (MIS) stands as a cornerstone in medical image analysis, playing a pivotal role in precise diagnostics, treatment planning, and monitoring of various medical conditions. This paper presents a comprehensive and systematic survey of MIS methodologies, bridging the gap between traditional image processing techniques and modern deep learning approaches. The survey encompasses thresholding, edge detection, region-based segmentation, clustering algorithms, and model-based techniques while also delving into state-of-the-art deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), and the widely adopted U-Net and its variants. Moreover, integrating attention mechanisms, semi-supervised learning, generative adversarial networks (GANs), and Transformer-based models is thoroughly explored. In addition to covering established methods, this survey highlights emerging trends, including hybrid architectures, cross-modality learning, federated and distributed learning frameworks, and active learning strategies, which aim to address challenges such as limited labeled datasets, computational complexity, and model generalizability across diverse imaging modalities. Furthermore, a specialized case study on lumbar spine segmentation is presented, offering insights into the challenges and advancements in this relatively underexplored anatomical region. Despite significant progress in the field, critical challenges persist, including dataset bias, domain adaptation, interpretability of deep learning models, and integration into real-world clinical workflows.

Method: The authors integrate texture mapping into 3DGS by augmenting each Gaussian with alpha, RGB, or RGBA texture maps to model spatially varying color and opacity across each Gaussian’s extent.

Result: The method achieves higher image quality than existing approaches while using similar or fewer Gaussians, with alpha-only texture maps significantly improving expressivity and RGB texture maps providing the highest expressivity.

Conclusion: Texture mapping greatly enhances 3DGS expressivity, enabling richer texture patterns and geometric structures while maintaining the efficiency advantages of Gaussian splatting.

Abstract: 3D Gaussian Splatting (3DGS) has recently emerged as a state-of-the-art 3D reconstruction and rendering technique due to its high-quality results and fast training and rendering time. However, pixels covered by the same Gaussian are always shaded in the same color up to a Gaussian falloff scaling factor. Furthermore, the finest geometric detail any individual Gaussian can represent is a simple ellipsoid. These properties of 3DGS greatly limit the expressivity of individual Gaussian primitives. To address these issues, we draw inspiration from texture and alpha mapping in traditional graphics and integrate it with 3DGS. Specifically, we propose a new generalized Gaussian appearance representation that augments each Gaussian with alpha~(A), RGB, or RGBA texture maps to model spatially varying color and opacity across the extent of each Gaussian. As such, each Gaussian can represent a richer set of texture patterns and geometric structures, instead of just a single color and ellipsoid as in naive Gaussian Splatting. Surprisingly, we found that the expressivity of Gaussians can be greatly improved by using alpha-only texture maps, and further augmenting Gaussians with RGB texture maps achieves the highest expressivity. We validate our method on a wide variety of standard benchmark datasets and our own custom captures at both the object and scene levels. We demonstrate image quality improvements over existing methods while using a similar or lower number of Gaussians.

Method: DECOR (Deep Clustering with Orientation Robustness) framework that groups complex defect patterns from wafer maps while explicitly accounting for orientation variations to ensure spatially similar defects are consistently clustered regardless of rotation or alignment.

Result: The method outperforms existing clustering baseline methods on the MixedWM38 dataset and demonstrates ability to discover clusters without manual tuning.

Conclusion: DECOR provides a reliable and scalable solution for automated visual inspection systems in semiconductor manufacturing by handling imperfect data conditions and orientation variations.

Abstract: In semiconductor manufacturing, early detection of wafer defects is critical for product yield optimization. However, raw wafer data from wafer quality tests are often complex, unlabeled, imbalanced and can contain multiple defects on a single wafer, making it crucial to design clustering methods that remain reliable under such imperfect data conditions. We introduce DECOR, a deep clustering with orientation robustness framework that groups complex defect patterns from wafer maps into consistent clusters. We evaluate our method on the open source MixedWM38 dataset, demonstrating its ability to discover clusters without manual tuning. DECOR explicitly accounts for orientation variations in wafer maps, ensuring that spatially similar defects are consistently clustered regardless of its rotation or alignment. Experiments indicate that our method outperforms existing clustering baseline methods, thus providing a reliable and scalable solution in automated visual inspection systems.

Method: Uses an LSTM neural network with attention mechanism focusing on key facial areas. Trained on subsets of 6 emotion classes and performs error correction for the 7th excluded class. Experiments conducted on all possible class subset configurations.

Result: Error correction is possible for all classes with varying success rates - some classes are better restored than others. Test results show improvement in key quality metrics for small classes, indicating effectiveness for rare event detection.

Conclusion: The proposed method is effective for facial expression analysis systems and can be applied to tasks requiring stable classification under imbalanced class distributions, such as anti-fraud systems for detecting rare events.

Abstract: This paper considers the problem of error correction in multi-class classification of face images on unbalanced samples. The study is based on the analysis of a data frame containing images labeled by seven different emotional states of people of different ages. Particular attention is paid to the problem of class imbalance, in which some emotions significantly prevail over others. To solve the classification problem, a neural network model based on LSTM with an attention mechanism focusing on key areas of the face that are informative for emotion recognition is used. As part of the experiments, the model is trained on all possible configurations of subsets of six classes with subsequent error correction for the seventh class, excluded at the training stage. The results show that correction is possible for all classes, although the degree of success varies: some classes are better restored, others are worse. In addition, on the test sample, when correcting some classes, an increase in key quality metrics for small classes was recorded, which indicates the promise of the proposed approach in solving applied problems related to the search for rare events, for example, in anti-fraud systems. Thus, the proposed method can be effectively applied in facial expression analysis systems and in tasks requiring stable classification under skewed class distribution.

Method: Proposes a joint training approach with quality partition: classification-based learning for HQ images and self-supervised image-image contrastive learning for LQ images, using a proxy-updated real-time queue for effective contrastive learning.

Result: Experiments on SCface, Tinyface, IJB-B, and five high-quality datasets demonstrate effectiveness in recognizing face images of different qualities.

Conclusion: The proposed quality-guided joint training approach successfully handles mixed-quality face recognition by applying appropriate learning strategies based on image quality, achieving improved performance across various quality datasets.

Abstract: The quality of a face crop in an image is decided by many factors such as camera resolution, distance, and illumination condition. This makes the discrimination of face images with different qualities a challenging problem in realistic applications. However, most existing approaches are designed specifically for high-quality (HQ) or low-quality (LQ) images, and the performances would degrade for the mixed-quality images. Besides, many methods ask for pre-trained feature extractors or other auxiliary structures to support the training and the evaluation. In this paper, we point out that the key to better understand both the HQ and the LQ images simultaneously is to apply different learning methods according to their qualities. We propose a novel quality-guided joint training approach for mixed-quality face recognition, which could simultaneously learn the images of different qualities with a single encoder. Based on quality partition, classification-based method is employed for HQ data learning. Meanwhile, for the LQ images which lack identity information, we learn them with self-supervised image-image contrastive learning. To effectively catch up the model update and improve the discriminability of contrastive learning in our joint training scenario, we further propose a proxy-updated real-time queue to compose the contrastive pairs with features from the genuine encoder. Experiments on the low-quality datasets SCface and Tinyface, the mixed-quality dataset IJB-B, and five high-quality datasets demonstrate the effectiveness of our proposed approach in recognizing face images of different qualities.

Method: Introduced DCG-Bench benchmark with three task types: Simple Text-to-Chart, Detailed Text-to-Chart, and Video-to-Chart. Built DCG-8K dataset with instruction-code-video triplets and QA pairs. Developed two-stage training recipe using Joint-Code-Visual Reward for group relative policy optimization to create expert MLLM Qwen2.5-VL-DCG-3B.

Result: Benchmarking revealed shortcomings of existing MLLMs in visual-to-chart tasks. The proposed model achieved 8.31% average performance gain over best open-sourced MLLM across three tasks, and showed comparable performance to proprietary models despite having only 3B parameters.

Conclusion: The DCG-Bench benchmark and proposed training methodology effectively address the dynamic chart generation challenge. The results demonstrate the effectiveness of the two-stage training recipe and joint reward optimization for creating capable DCG models with relatively small parameter counts.

Abstract: Dynamic Chart Generation (DCG) involves producing code-rendered animated visualizations as charts. While recent advances in multi-modal large language models (MLLMs) have significantly improved their capability on static chart generation and comprehension, MLLMs’ potential for handling dynamic chart generation and understanding remains underexplored. To bridge this research gap, we introduce DCG-Bench (Dynamic Chart Generation Benchmark), the first benchmark evaluating MLLM’s capability on dynamic chart generation tasks from three dimensions: Simple Text-to-Chart, Detailed Text-to-Chart, and Video-to-Chart tasks. We construct DCG-8K, a high-quality DCG dataset with annotations covering instruction-code-video triplets and QA pairs for both code and video evaluation. Based on DCG-8K, we explored a two-stage training recipe, proposing Joint-Code-Visual Reward for group relative policy optimization to construct expert MLLM Qwen2.5-VL-DCG-3B for the DCG task. Our benchmarking result reveals shortcomings of existing MLLMs in the visual-to-chart task, and our model beats the best open-sourced MLLM with an average 8.31% performance gain across three tasks, and shows on par performance against proprietary models with only 3B parameters, proving the effectiveness of our training recipe. Our code and dataset will be publicly available.

Method: STIV integrates image condition into a Diffusion Transformer (DiT) through frame replacement and incorporates text conditioning via joint image-text conditional classifier-free guidance, enabling simultaneous T2V and TI2V tasks.

Result: An 8.7B model achieves 83.1 on VBench T2V (surpassing CogVideoX-5B, Pika, Kling, Gen-3) and state-of-the-art 90.1 on VBench I2V at 512 resolution. The framework extends to video prediction, frame interpolation, multi-view generation, and long video generation.

Conclusion: STIV provides a transparent and extensible recipe for building cutting-edge video generation models, empowering future research and accelerating progress toward more versatile and reliable video generation solutions.

Abstract: The field of video generation has made remarkable advancements, yet there remains a pressing need for a clear, systematic recipe that can guide the development of robust and scalable models. In this work, we present a comprehensive study that systematically explores the interplay of model architectures, training recipes, and data curation strategies, culminating in a simple and scalable text-image-conditioned video generation method, named STIV. Our framework integrates image condition into a Diffusion Transformer (DiT) through frame replacement, while incorporating text conditioning via a joint image-text conditional classifier-free guidance. This design enables STIV to perform both text-to-video (T2V) and text-image-to-video (TI2V) tasks simultaneously. Additionally, STIV can be easily extended to various applications, such as video prediction, frame interpolation, multi-view generation, and long video generation, etc. With comprehensive ablation studies on T2I, T2V, and TI2V, STIV demonstrate strong performance, despite its simple design. An 8.7B model with 512 resolution achieves 83.1 on VBench T2V, surpassing both leading open and closed-source models like CogVideoX-5B, Pika, Kling, and Gen-3. The same-sized model also achieves a state-of-the-art result of 90.1 on VBench I2V task at 512 resolution. By providing a transparent and extensible recipe for building cutting-edge video generation models, we aim to empower future research and accelerate progress toward more versatile and reliable video generation solutions.

Method: VoT processes monocular frame sequences using feature extraction and global relationship modeling through temporal and spatial attention. It directly predicts camera motion without dense geometry estimation, using only camera poses for supervision.

Result: VoT outperforms traditional methods, runs 3x faster, scales with larger datasets, benefits from stronger pre-trained backbones, and generalizes across diverse camera motions and calibration settings.

Conclusion: Monocular visual odometry can be effectively addressed end-to-end without traditional pipeline components, achieving better performance and generalization while being significantly faster.

Abstract: Modern monocular visual odometry methods typically combine pre-trained deep learning components with optimization modules, resulting in complex pipelines that rely heavily on camera calibration and hyperparameter tuning, and often struggle in unseen real-world scenarios. Recent large-scale 3D models trained on massive amounts of multi-modal data have partially alleviated these challenges, providing generalizable dense reconstruction and camera pose estimation. Still, they remain limited in handling long videos and providing accurate per-frame estimates, which are required for visual odometry. In this work, we demonstrate that monocular visual odometry can be addressed effectively in an end-to-end manner, thereby eliminating the need for handcrafted components such as bundle adjustment, feature matching, camera calibration, or dense 3D reconstruction. We introduce VoT, short for Visual odometry Transformer, which processes sequences of monocular frames by extracting features and modeling global relationships through temporal and spatial attention. Unlike prior methods, VoT directly predicts camera motion without estimating dense geometry and relies solely on camera poses for supervision. The framework is modular and flexible, allowing seamless integration of various pre-trained encoders as feature extractors. Experimental results demonstrate that VoT scales effectively with larger datasets, benefits substantially from stronger pre-trained backbones, generalizes across diverse camera motions and calibration settings, and outperforms traditional methods while running more than 3 times faster. The code will be released.

Method: Proposed an inference-time search algorithm that guides sampling using side information while balancing exploration and exploitation, avoiding gradient-based guidance issues.

Result: Consistently improves qualitative and quantitative performance across various inverse problems including box inpainting, super-resolution, and multiple deblurring tasks, outperforming reward gradient-based guidance baselines.

Conclusion: The approach enables more accurate and reliable reconstructions, can be seamlessly integrated into existing diffusion-based pipelines, and provides an effective alternative to gradient-based guidance methods.

Abstract: Diffusion models have emerged as powerful priors for solving inverse problems. However, existing approaches typically overlook side information that could significantly improve reconstruction quality, especially in severely ill-posed settings. In this work, we propose a novel inference-time search algorithm that guides the sampling process using the side information in a manner that balances exploration and exploitation. This enables more accurate and reliable reconstructions, providing an alternative to the gradient-based guidance that is prone to reward-hacking artifacts. Our approach can be seamlessly integrated into a wide range of existing diffusion-based image reconstruction pipelines. Through extensive experiments on a number of inverse problems, such as box inpainting, super-resolution, and various deblurring tasks including motion, Gaussian, nonlinear, and blind deblurring, we show that our approach consistently improves the qualitative and quantitative performance of diffusion-based image reconstruction algorithms. We also show the superior performance of our approach with respect to other baselines, including reward gradient-based guidance algorithms. The code is available at \href{https://github.com/mhdfb/sideinfo-search-reconstruction}{this repository}.

Method: UCF constructs cost volumes by matching test samples against normal samples, then applies a learnable filtering module with multi-layer attention guidance to reduce matching noise and highlight subtle anomalies.

Result: Comprehensive experiments on 22 benchmarks show UCF consistently improves various UAD methods, achieving new state-of-the-art results in both unimodal (RGB) and multimodal (RGB-3D, RGB-Text) scenarios.

Conclusion: UCF provides a generic and effective framework for refining anomaly detection across diverse settings by addressing the fundamental challenge of matching noise in cost volumes.

Abstract: Unsupervised anomaly detection (UAD) aims to identify image- and pixel-level anomalies using only normal training data, with wide applications such as industrial inspection and medical analysis, where anomalies are scarce due to privacy concerns and cold-start constraints. Existing methods, whether reconstruction-based (restoring normal counterparts) or embedding-based (pretrained representations), fundamentally conduct image- or feature-level matching to generate anomaly maps. Nonetheless, matching noise has been largely overlooked, limiting their detection ability. Beyond earlier focus on unimodal RGB-based UAD, recent advances expand to multimodal scenarios, e.g., RGB–3D and RGB–Text, enabled by point cloud sensing and vision–language models. Despite shared challenges, these lines remain largely isolated, hindering a comprehensive understanding and knowledge transfer. In this paper, we advocate unified UAD for both unimodal and multimodal settings in the matching perspective. Under this insight, we present Unified Cost Filtering (UCF), a generic post-hoc refinement framework for refining anomaly cost volume of any UAD model. The cost volume is constructed by matching a test sample against normal samples from the same or different modalities, followed by a learnable filtering module with multi-layer attention guidance from the test sample, mitigating matching noise and highlighting subtle anomalies. Comprehensive experiments on 22 diverse benchmarks demonstrate the efficacy of UCF in enhancing a variety of UAD methods, consistently achieving new state-of-the-art results in both unimodal (RGB) and multimodal (RGB–3D, RGB–Text) UAD scenarios. Code and models will be released at https://github.com/ZHE-SAPI/CostFilter-AD.

Method: The authors conducted a systematic review of sonar image datasets across multiple modalities (SSS, FLS, SAS, MBES, DIDSON), analyzed applications (classification, detection, segmentation, 3D reconstruction), and synthesized findings into a master table and chronological timeline.

Result: The review mapped publicly accessible datasets, offering clear comparison of characteristics, sizes, and annotation details, while identifying gaps in the current landscape of sonar image resources.

Conclusion: This work serves as a comprehensive base guide for researchers to start or advance in underwater acoustic data analysis, providing a clear roadmap through systematic cataloging and contextualization of existing sonar image datasets.

Abstract: Sonar images are relevant for advancing underwater exploration, autonomous navigation, and ecosystem monitoring. However, the progress depends on data availability. The scarcity of publicly available, well-annotated sonar image datasets creates a significant bottleneck for the development of robust machine learning models. This paper presents a comprehensive and concise review of the current landscape of sonar image datasets, seeking not only to catalog existing resources but also to contextualize them, identify gaps, and provide a clear roadmap, serving as a base guide for researchers of any kind who wish to start or advance in the field of underwater acoustic data analysis. We mapped publicly accessible datasets across various sonar modalities, including Side Scan Sonar (SSS), Forward-Looking Sonar (FLS), Synthetic Aperture Sonar (SAS), Multibeam Echo Sounder (MBES), and Dual-Frequency Identification Sonar (DIDSON). An analysis was conducted on applications such as classification, detection, segmentation, and 3D reconstruction. This work focuses on state-of-the-art advancements, incorporating newly released datasets. The findings are synthesized into a master table and a chronological timeline, offering a clear and accessible comparison of characteristics, sizes, and annotation details datasets.

Method: Employ Visual Language Models (VLMs) to assess object detection results by identifying missing or inconsistent detections using multimodal capabilities.

Result: The framework enables automated quality assessment and improves overall detection performance on complex industrial diagrams.

Conclusion: VLMs provide an effective solution for automated quality evaluation and refinement of object detection in industrial diagram digitalization.

Abstract: Industrial diagrams such as piping and instrumentation diagrams (P&IDs) are essential for the design, operation, and maintenance of industrial plants. Converting these diagrams into digital form is an important step toward building digital twins and enabling intelligent industrial automation. A central challenge in this digitalization process is accurate object detection. Although recent advances have significantly improved object detection algorithms, there remains a lack of methods to automatically evaluate the quality of their outputs. This paper addresses this gap by introducing a framework that employs Visual Language Models (VLMs) to assess object detection results and guide their refinement. The approach exploits the multimodal capabilities of VLMs to identify missing or inconsistent detections, thereby enabling automated quality assessment and improving overall detection performance on complex industrial diagrams.

Method: Co-designed a lensless camera with Implicit Neural Representation algorithm to capture display characteristics from various viewpoints, enabling light field reconstruction from a 46.6° × 37.6° viewing cone.

Result: The pipeline successfully reconstructs light fields emitted from displays across a wide viewing angle without requiring specialized hardware.

Conclusion: This emerging pipeline represents initial progress toward effortless display calibration and characterization, making the process more accessible to users.

Abstract: Calibrating displays is a basic and regular task that content creators must perform to maintain optimal visual experience, yet it remains a troublesome issue. Measuring display characteristics from different viewpoints often requires specialized equipment and a dark room, making it inaccessible to most users. To avoid specialized hardware requirements in display calibrations, our work co-designs a lensless camera and an Implicit Neural Representation based algorithm for capturing display characteristics from various viewpoints. More specifically, our pipeline enables efficient reconstruction of light fields emitted from a display from a viewing cone of 46.6{\deg} X 37.6{\deg}. Our emerging pipeline paves the initial steps towards effortless display calibration and characterization.

Method: Defines attention relative to reference frames from Platonic solid symmetry groups, creating a principled weight-sharing scheme that enables combined equivariance to continuous translations and Platonic symmetries.

Result: Achieves competitive performance across computer vision (CIFAR-10), 3D point clouds (ScanObjectNN), and molecular property prediction (QM9, OMol25) benchmarks while maintaining standard Transformer architecture and computational cost.

Conclusion: The Platonic Transformer resolves the trade-off between geometric equivariance and Transformer efficiency, providing geometric constraints at no additional computational cost.

Abstract: While widespread, Transformers lack inductive biases for geometric symmetries common in science and computer vision. Existing equivariant methods often sacrifice the efficiency and flexibility that make Transformers so effective through complex, computationally intensive designs. We introduce the Platonic Transformer to resolve this trade-off. By defining attention relative to reference frames from the Platonic solid symmetry groups, our method induces a principled weight-sharing scheme. This enables combined equivariance to continuous translations and Platonic symmetries, while preserving the exact architecture and computational cost of a standard Transformer. Furthermore, we show that this attention is formally equivalent to a dynamic group convolution, which reveals that the model learns adaptive geometric filters and enables a highly scalable, linear-time convolutional variant. Across diverse benchmarks in computer vision (CIFAR-10), 3D point clouds (ScanObjectNN), and molecular property prediction (QM9, OMol25), the Platonic Transformer achieves competitive performance by leveraging these geometric constraints at no additional cost.

Method: Joint optimization of primary task and explainability objective where each output is justified by concrete exemplars weighted by relevance in feature space, operating like a learned KNN.

Result: Enables systematic investigation of memorization vs generalization trade-off, verification of training set inclusion, detection of mislabeled data, enhanced resilience to input perturbations, and identification of similar inputs contributing to predictions.

Conclusion: Provenance networks provide complementary explainability approach that improves transparency, robustness, and trustworthiness in neural models, though with computational costs and current scaling limitations to moderately sized datasets.

Abstract: We introduce provenance networks, a novel class of neural models designed to provide end-to-end, training-data-driven explainability. Unlike conventional post-hoc methods, provenance networks learn to link each prediction directly to its supporting training examples as part of the model’s normal operation, embedding interpretability into the architecture itself. Conceptually, the model operates similarly to a learned KNN, where each output is justified by concrete exemplars weighted by relevance in the feature space. This approach facilitates systematic investigations of the trade-off between memorization and generalization, enables verification of whether a given input was included in the training set, aids in the detection of mislabeled or anomalous data points, enhances resilience to input perturbations, and supports the identification of similar inputs contributing to the generation of a new data point. By jointly optimizing the primary task and the explainability objective, provenance networks offer insights into model behavior that traditional deep networks cannot provide. While the model introduces additional computational cost and currently scales to moderately sized datasets, it provides a complementary approach to existing explainability techniques. In particular, it addresses critical challenges in modern deep learning, including model opaqueness, hallucination, and the assignment of credit to data contributors, thereby improving transparency, robustness, and trustworthiness in neural models.

Method: Analyzes DINOv2’s core techniques (multi-crop view augmentation and self-distillation with mean teacher), traces their development in previous work, and compares performance with other SSL and WSL methods across various downstream tasks.

Result: DINOv2 establishes new state-of-the-art performance, surpassing weakly supervised methods like OpenCLIP on most benchmarks, with transformer backbones showing remarkable emergent properties.

Conclusion: Discusses DINOv2’s limitations, its impact on the field, and suggests future research directions for self-supervised learning.

Abstract: Recent advances in self-supervised learning (SSL) have made it possible to learn general-purpose visual features that capture both the high-level semantics and the fine-grained spatial structure of images. Most notably, the recent DINOv2 has established a new state of the art by surpassing weakly supervised methods (WSL) like OpenCLIP on most benchmarks. In this survey, we examine the core ideas behind its approach, multi-crop view augmentation and self-distillation with a mean teacher, and trace their development in previous work. We then compare the performance of DINO and DINOv2 with other SSL and WSL methods across various downstream tasks, and highlight some remarkable emergent properties of their learned features with transformer backbones. We conclude by briefly discussing DINOv2’s limitations, its impact, and future research directions.

Method: Proposed SpatialViLT and MaskedSpatialViLT variants that integrate spatial features (depth maps, 3D coordinates, edge maps) using multi-task learning, with SpatialEnsemble combining both approaches.

Result: Achieved state-of-the-art accuracy on Visual Spatial Reasoning (VSR) dataset, excelling in directional, topological, and proximity relations.

Conclusion: This work significantly advances spatial intelligence in AI systems, which is crucial for advanced multimodal understanding and real-world applications.

Abstract: Vision-language models (VLMs) have advanced multimodal reasoning but still face challenges in spatial reasoning for 3D scenes and complex object configurations. To address this, we introduce SpatialViLT, an enhanced VLM that integrates spatial features like depth maps, 3D coordinates, and edge maps through a multi-task learning framework. This approach enriches multimodal embeddings with spatial understanding. We propose two variants: SpatialViLT and MaskedSpatialViLT, focusing on full and masked object regions, respectively. Additionally, SpatialEnsemble combines both approaches, achieving state-of-the-art accuracy. Our models excel in spatial reasoning categories such as directional, topological, and proximity relations, as demonstrated on the challenging Visual Spatial Reasoning (VSR) dataset. This work represents a significant step in enhancing the spatial intelligence of AI systems, crucial for advanced multimodal understanding and real-world applications.

Method: SPEGNet uses a unified design with multi-scale feature integration via channel calibration and spatial enhancement. Boundaries emerge from context-rich representations with progressive refinement using scale-adaptive edge modulation at intermediate resolutions.

Result: Achieves 0.887 Sα on CAMO, 0.890 on COD10K, and 0.895 on NC4K datasets with real-time inference speed. Excels across scales from tiny intricate objects to large pattern-similar ones while handling occlusion and ambiguous boundaries.

Conclusion: The unified approach strikes a balance between boundary precision and regional consistency, outperforming complex accumulated component methods while maintaining computational efficiency.

Abstract: Camouflaged object detection segments objects with intrinsic similarity and edge disruption. Current detection methods rely on accumulated complex components. Each approach adds components such as boundary modules, attention mechanisms, and multi-scale processors independently. This accumulation creates a computational burden without proportional gains. To manage this complexity, they process at reduced resolutions, eliminating fine details essential for camouflage. We present SPEGNet, addressing fragmentation through a unified design. The architecture integrates multi-scale features via channel calibration and spatial enhancement. Boundaries emerge directly from context-rich representations, maintaining semantic-spatial alignment. Progressive refinement implements scale-adaptive edge modulation with peak influence at intermediate resolutions. This design strikes a balance between boundary precision and regional consistency. SPEGNet achieves 0.887 $S_\alpha$ on CAMO, 0.890 on COD10K, and 0.895 on NC4K, with real-time inference speed. Our approach excels across scales, from tiny, intricate objects to large, pattern-similar ones, while handling occlusion and ambiguous boundaries. Code, model weights, and results are available on \href{https://github.com/Baber-Jan/SPEGNet}{https://github.com/Baber-Jan/SPEGNet}.

Method: Used encoder-decoder networks (asymmetrical denoising autoencoder and U-Net) trained on synthetic pairs created by applying real artifact fields to synthetic images, then evaluated on real OS-SI images.

Result: Both networks improved image clarity, with each network excelling against different types of artifacts. The approach successfully enabled supervised denoising of OS-SI images.

Conclusion: Synthetic training enables effective supervised denoising of OS-SI images, and encoder-decoder networks show potential for streamlining reconstruction workflows in structured illumination microscopy.

Abstract: Structured illumination (SI) enhances image resolution and contrast by projecting patterned light onto a sample. In two-phase optical-sectioning SI (OS-SI), reduced acquisition time introduces residual artifacts that conventional denoising struggles to suppress. Deep learning offers an alternative to traditional methods; however, supervised training is limited by the lack of clean, optically sectioned ground-truth data. We investigate encoder-decoder networks for artifact reduction in two-phase OS-SI, using synthetic training pairs formed by applying real artifact fields to synthetic images. An asymmetrical denoising autoencoder (DAE) and a U-Net are trained on the synthetic data, then evaluated on real OS-SI images. Both networks improve image clarity, with each excelling against different artifact types. These results demonstrate that synthetic training enables supervised denoising of OS-SI images and highlight the potential of encoder-decoder networks to streamline reconstruction workflows.

Method: PEaRL represents transcriptomics through pathway activation scores computed with ssGSEA, encodes pathway signals with a transformer, and aligns them with histology features via contrastive learning.

Result: Across three cancer ST datasets (breast, skin, lymph node), PEaRL outperforms SOTA methods with up to 58.9% and 20.4% increase in Pearson correlation for gene- and pathway-level expression prediction respectively.

Conclusion: Grounding transcriptomic representation in pathways produces more biologically faithful and interpretable multimodal models, advancing computational pathology beyond gene-level embeddings.

Abstract: Integrating histopathology with spatial transcriptomics (ST) provides a powerful opportunity to link tissue morphology with molecular function. Yet most existing multimodal approaches rely on a small set of highly variable genes, which limits predictive scope and overlooks the coordinated biological programs that shape tissue phenotypes. We present PEaRL (Pathway Enhanced Representation Learning), a multimodal framework that represents transcriptomics through pathway activation scores computed with ssGSEA. By encoding biologically coherent pathway signals with a transformer and aligning them with histology features via contrastive learning, PEaRL reduces dimensionality, improves interpretability, and strengthens cross-modal correspondence. Across three cancer ST datasets (breast, skin, and lymph node), PEaRL consistently outperforms SOTA methods, yielding higher accuracy for both gene- and pathway-level expression prediction (up to 58.9 percent and 20.4 percent increase in Pearson correlation coefficient compared to SOTA). These results demonstrate that grounding transcriptomic representation in pathways produces more biologically faithful and interpretable multimodal models, advancing computational pathology beyond gene-level embeddings.

Method: Introduces hierarchical semantic prompts for fine-grained control, dual-prompt mechanism for text-controlled architecture, and parameter-efficient fine-tuning for rapid adaptation to new tasks and modalities.

Result: Outperforms state-of-the-art models on 8/10 datasets across 3 imaging modalities and 30+ organs/tumors, achieves CI=0.69 for prognosis prediction with EHR integration, and enables rapid adaptation to new tasks.

Conclusion: DuPLUS establishes itself as a versatile and clinically relevant solution for medical image analysis with strong generalizability and extensibility capabilities.

Abstract: Deep learning for medical imaging is hampered by task-specific models that lack generalizability and prognostic capabilities, while existing ‘universal’ approaches suffer from simplistic conditioning and poor medical semantic understanding. To address these limitations, we introduce DuPLUS, a deep learning framework for efficient multi-modal medical image analysis. DuPLUS introduces a novel vision-language framework that leverages hierarchical semantic prompts for fine-grained control over the analysis task, a capability absent in prior universal models. To enable extensibility to other medical tasks, it includes a hierarchical, text-controlled architecture driven by a unique dual-prompt mechanism. For segmentation, DuPLUS is able to generalize across three imaging modalities, ten different anatomically various medical datasets, encompassing more than 30 organs and tumor types. It outperforms the state-of-the-art task specific and universal models on 8 out of 10 datasets. We demonstrate extensibility of its text-controlled architecture by seamless integration of electronic health record (EHR) data for prognosis prediction, and on a head and neck cancer dataset, DuPLUS achieved a Concordance Index (CI) of 0.69. Parameter-efficient fine-tuning enables rapid adaptation to new tasks and modalities from varying centers, establishing DuPLUS as a versatile and clinically relevant solution for medical image analysis. The code for this work is made available at: https://anonymous.4open.science/r/DuPLUS-6C52

Method: Two-stage framework integrating Threading Detection Model (TDM) to parallelize YOLOv10-based detection and MobileSAM-based segmentation, executed concurrently for efficient resource use.

Result: Achieved mAP50 of 0.9627, mAP75 of 0.7731, mAP95 of 0.7178, and MobileSAM mIoU of 0.7421 on Houbara Bustard. YOLOv10 operates at 43.7 ms per frame, confirming real-time readiness.

Conclusion: The proposed framework successfully enables real-time animal detection and segmentation with high accuracy, supported by a curated dataset of 40,000 annotated Houbara images, making it suitable for conservation monitoring applications.

Abstract: Real-time animal detection and segmentation in natural environments are vital for wildlife conservation, enabling non-invasive monitoring through remote camera streams. However, these tasks remain challenging due to limited computational resources and the cryptic appearance of many species. We propose a mobile-optimized two-stage deep learning framework that integrates a Threading Detection Model (TDM) to parallelize YOLOv10-based detection and MobileSAM-based segmentation. Unlike prior YOLO+SAM pipelines, our approach improves real-time performance by reducing latency through threading. YOLOv10 handles detection while MobileSAM performs lightweight segmentation, both executed concurrently for efficient resource use. On the cryptic Houbara Bustard, a conservation-priority species, our model achieves mAP50 of 0.9627, mAP75 of 0.7731, mAP95 of 0.7178, and a MobileSAM mIoU of 0.7421. YOLOv10 operates at 43.7 ms per frame, confirming real-time readiness. We introduce a curated Houbara dataset of 40,000 annotated images to support model training and evaluation across diverse conditions. The code and dataset used in this study are publicly available on GitHub at https://github.com/LyesSaadSaoud/mobile-houbara-detseg. For interactive demos and additional resources, visit https://lyessaadsaoud.github.io/LyesSaadSaoud-Threaded-YOLO-SAM-Houbara.

Method: The survey clusters and reviews existing domain generalization approaches for semantic segmentation, with a focus on identifying the emerging trend towards foundation-model-based methods.

Result: The paper provides extensive performance comparisons that demonstrate the significant influence of foundation models on improving domain generalization capabilities in semantic segmentation tasks.

Conclusion: This survey aims to advance domain generalization research and inspire exploration of new research directions, particularly highlighting the transformative impact of foundation models in this rapidly evolving field.

Abstract: The generalization of deep neural networks to unknown domains is a major challenge despite their tremendous progress in recent years. For this reason, the dynamic area of domain generalization (DG) has emerged. In contrast to unsupervised domain adaptation, there is no access to or knowledge about the target domains, and DG methods aim to generalize across multiple different unseen target domains. Domain generalization is particularly relevant for the task semantic segmentation which is used in several areas such as biomedicine or automated driving. This survey provides a comprehensive overview of the rapidly evolving topic of domain generalized semantic segmentation. We cluster and review existing approaches and identify the paradigm shift towards foundation-model-based domain generalization. Finally, we provide an extensive performance comparison of all approaches, which highlights the significant influence of foundation models on domain generalization. This survey seeks to advance domain generalization research and inspire scientists to explore new research directions.

Method: Two-stage training framework: pre-training transformer-based vision encoder and text decoder on image/caption pairs, then fine-tuning on images/report pairs for clinical findings.

Result: The model generates clinically meaningful findings from endoscopic images, streamlining documentation process.

Conclusion: This approach reduces physician workload and improves patient care by automating report generation for endoscopic procedures.

Abstract: Endoscopic procedures such as esophagogastroduodenoscopy (EGD) and colonoscopy play a critical role in diagnosing and managing gastrointestinal (GI) disorders. However, the documentation burden associated with these procedures place significant strain on gastroenterologists, contributing to inefficiencies in clinical workflows and physician burnout. To address this challenge, we propose a novel automated report generation model that leverages a transformer-based vision encoder and text decoder within a two-stage training framework. In the first stage, both components are pre-trained on image/text caption pairs to capture generalized vision-language features, followed by fine-tuning on images/report pairs to generate clinically meaningful findings. Our approach not only streamlines the documentation process but also holds promise for reducing physician workload and improving patient care.

Method: Uses a two-component system: SketchAdapter maps human sketches to 2D paths, and DiffPath (diffusion model) generates 3D trajectories from 2D projections and depth images. Trained on 32k synthetic flight paths with photorealistic 3D Gaussian Splatting scenes, plus 872 human-labeled sketches.

Result: Achieved 100% success in low/medium clutter and 40% in high-clutter real-world environments, outperforming ablations by 20-60% in task completion. Demonstrated effective zero-shot sim-to-real transfer.

Conclusion: SketchPlan successfully bridges the gap between human sketches and 3D drone navigation, with modular design and mixed training data significantly improving interpretation of human intent and 3D path inference capabilities.

Abstract: We propose SketchPlan, a diffusion-based planner that interprets 2D hand-drawn sketches over depth images to generate 3D flight paths for drone navigation. SketchPlan comprises two components: a SketchAdapter that learns to map the human sketches to projected 2D paths, and DiffPath, a diffusion model that infers 3D trajectories from 2D projections and a first person view depth image. Our model achieves zero-shot sim-to-real transfer, generating accurate and safe flight paths in previously unseen real-world environments. To train the model, we build a synthetic dataset of 32k flight paths using a diverse set of photorealistic 3D Gaussian Splatting scenes. We automatically label the data by computing 2D projections of the 3D flight paths onto the camera plane, and use this to train the DiffPath diffusion model. However, since real human 2D sketches differ significantly from ideal 2D projections, we additionally label 872 of the 3D flight paths with real human sketches and use this to train the SketchAdapter to infer the 2D projection from the human sketch. We demonstrate SketchPlan’s effectiveness in both simulated and real-world experiments, and show through ablations that training on a mix of human labeled and auto-labeled data together with a modular design significantly boosts its capabilities to correctly interpret human intent and infer 3D paths. In real-world drone tests, SketchPlan achieved 100% success in low/medium clutter and 40% in unseen high-clutter environments, outperforming key ablations by 20-60% in task completion.

Method: A pose-conditioned, large-margin contrastive encoder that isolates persistent identity cues from pose-expression latents while canceling transient pose and expression information.

Result: The method consistently outperforms existing puppeteering defenses, operates in real-time, and shows strong generalization to out-of-distribution scenarios across multiple talking-head generation models.

Conclusion: The proposed biometric leakage defense effectively detects identity swaps by analyzing pose-expression latents directly, providing real-time protection against puppeteering attacks in talking-head systems.

Abstract: AI-based talking-head videoconferencing systems reduce bandwidth by sending a compact pose-expression latent and re-synthesizing RGB at the receiver, but this latent can be puppeteered, letting an attacker hijack a victim’s likeness in real time. Because every frame is synthetic, deepfake and synthetic video detectors fail outright. To address this security problem, we exploit a key observation: the pose-expression latent inherently contains biometric information of the driving identity. Therefore, we introduce the first biometric leakage defense without ever looking at the reconstructed RGB video: a pose-conditioned, large-margin contrastive encoder that isolates persistent identity cues inside the transmitted latent while cancelling transient pose and expression. A simple cosine test on this disentangled embedding flags illicit identity swaps as the video is rendered. Our experiments on multiple talking-head generation models show that our method consistently outperforms existing puppeteering defenses, operates in real-time, and shows strong generalization to out-of-distribution scenarios.

Method: Proposes DragStream with: 1) adaptive distribution self-rectification using neighboring frames’ statistics to constrain latent embedding drift; 2) spatial-frequency selective optimization to exploit contextual information while mitigating interference.

Result: DragStream can be seamlessly integrated into existing autoregressive video diffusion models and effectively enables streaming drag-oriented interactive video manipulation.

Conclusion: The proposed DragStream approach successfully resolves the REVEL task, providing versatile drag operations for video editing and animation with translation, deformation, and rotation effects.

Abstract: Achieving streaming, fine-grained control over the outputs of autoregressive video diffusion models remains challenging, making it difficult to ensure that they consistently align with user expectations. To bridge this gap, we propose \textbf{stReaming drag-oriEnted interactiVe vidEo manipuLation (REVEL)}, a new task that enables users to modify generated videos \emph{anytime} on \emph{anything} via fine-grained, interactive drag. Beyond DragVideo and SG-I2V, REVEL unifies drag-style video manipulation as editing and animating video frames with both supporting user-specified translation, deformation, and rotation effects, making drag operations versatile. In resolving REVEL, we observe: \emph{i}) drag-induced perturbations accumulate in latent space, causing severe latent distribution drift that halts the drag process; \emph{ii}) streaming drag is easily disturbed by context frames, thereby yielding visually unnatural outcomes. We thus propose a training-free approach, \textbf{DragStream}, comprising: \emph{i}) an adaptive distribution self-rectification strategy that leverages neighboring frames’ statistics to effectively constrain the drift of latent embeddings; \emph{ii}) a spatial-frequency selective optimization mechanism, allowing the model to fully exploit contextual information while mitigating its interference via selectively propagating visual cues along generation. Our method can be seamlessly integrated into existing autoregressive video diffusion models, and extensive experiments firmly demonstrate the effectiveness of our DragStream.

Method: Group-Aggregative Selection Multi-Instance Learning (GAS-MIL) - a flexible ensemble framework that seamlessly integrates features from multiple foundation models while preserving their complementary strengths.

Result: Across three cancer datasets (prostate, ovarian, breast), GAS-MIL consistently achieves superior or on-par performance relative to individual foundation models and established MIL methods.

Conclusion: GAS-MIL enables efficient integration of heterogeneous foundation models, streamlining model deployment for pathology and providing a scalable foundation for future multimodal and precision oncology applications.

Abstract: Foundation models (FMs) have transformed computational pathology by providing powerful, general-purpose feature extractors. However, adapting and benchmarking individual FMs for specific diagnostic tasks is often time-consuming and resource-intensive, especially given their scale and diversity. To address this challenge, we introduce Group-Aggregative Selection Multi-Instance Learning (GAS-MIL), a flexible ensemble framework that seamlessly integrates features from multiple FMs, preserving their complementary strengths without requiring manual feature selection or extensive task-specific fine-tuning. Across classification tasks in three cancer datasets-prostate (PANDA), ovarian (UBC-OCEAN), and breast (TCGA-BrCa)-GAS-MIL consistently achieves superior or on-par performance relative to individual FMs and established MIL methods, demonstrating its robustness and generalizability. By enabling efficient integration of heterogeneous FMs, GAS-MIL streamlines model deployment for pathology and provides a scalable foundation for future multimodal and precision oncology applications.

Method: Created the Drone-Assisted Naloxone Delivery Simulation Dataset (DANDSD) with college students as bystanders, then developed a video-based real-time SA assessment framework using graph embeddings and transformer models that integrate visual perception and comprehension cues.

Result: The framework achieves high-performance SA prediction with strong temporal segmentation, outperforming FINCH baseline by 9% in Mean over Frames (MoF) and 5% in Intersection over Union (IoU).

Conclusion: The work enables development of adaptive drone systems that can effectively guide bystanders during opioid overdose emergencies, potentially improving emergency response outcomes and saving lives.

Abstract: Rapid naloxone delivery via drones offers a promising solution for responding to opioid overdose emergencies (OOEs), by extending lifesaving interventions to medically untrained bystanders before emergency medical services (EMS) arrive. Recognizing the critical role of bystander situational awareness (SA) in human-autonomy teaming (HAT), we address a key research gap in real-time SA assessment by introducing the Drone-Assisted Naloxone Delivery Simulation Dataset (DANDSD). This pioneering dataset captures HAT during simulated OOEs, where college students without medical training act as bystanders tasked with administering intranasal naloxone to a mock overdose victim. Leveraging this dataset, we propose a video-based real-time SA assessment framework that utilizes graph embeddings and transformer models to assess bystander SA in real time. Our approach integrates visual perception and comprehension cues–such as geometric, kinematic, and interaction graph features–and achieves high-performance SA prediction. It also demonstrates strong temporal segmentation accuracy, outperforming the FINCH baseline by 9% in Mean over Frames (MoF) and 5% in Intersection over Union (IoU). This work supports the development of adaptive drone systems capable of guiding bystanders effectively, ultimately improving emergency response outcomes and saving lives.

Method: Used dataset of 231 products (1,628 images) processed by four OCR models, with ground truth subset of 113 images (60 products) for accuracy evaluation using metrics including CER, WER, BLEU, ROUGE-L, F1, coverage, and execution time.

Result: Tesseract achieved lowest CER (0.912) and highest BLEU (0.245). EasyOCR provided good balance between accuracy and multilingual support. PaddleOCR achieved near complete coverage but was slower. TrOCR produced weakest results despite GPU acceleration.

Conclusion: Results provide packaging-specific benchmark, establish baseline, and highlight directions for layout-aware methods and text localization.

Abstract: This study evaluates four open-source Optical Character Recognition (OCR) systems which are Tesseract, EasyOCR, PaddleOCR, and TrOCR on real world food packaging images. The aim is to assess their ability to extract ingredient lists and nutrition facts panels. Accurate OCR for packaging is important for compliance and nutrition monitoring but is challenging due to multilingual text, dense layouts, varied fonts, glare, and curved surfaces. A dataset of 231 products (1,628 images) was processed by all four models to assess speed and coverage, and a ground truth subset of 113 images (60 products) was created for accuracy evaluation. Metrics include Character Error Rate (CER), Word Error Rate (WER), BLEU, ROUGE-L, F1, coverage, and execution time. On the ground truth subset, Tesseract achieved the lowest CER (0.912) and the highest BLEU (0.245). EasyOCR provided a good balance between accuracy and multilingual support. PaddleOCR achieved near complete coverage but was slower because it ran on CPU only due to GPU incompatibility, and TrOCR produced the weakest results despite GPU acceleration. These results provide a packaging-specific benchmark, establish a baseline, and highlight directions for layout-aware methods and text localization.

Method: FrameOracle uses a four-stage curriculum training approach with weak proxy signals initially, then leverages FrameOracle-41K dataset with keyframe annotations for stronger supervision.

Result: Reduces 16-frame inputs to 10.4 frames without accuracy loss, and 64-frame inputs to 13.9 frames with 1.4% accuracy improvement across five VLMs and six benchmarks.

Conclusion: FrameOracle achieves state-of-the-art efficiency-accuracy trade-offs for scalable video understanding by adaptively selecting optimal frames.

Abstract: Vision-language models (VLMs) have advanced video understanding, but their performance is limited by the number of input frames they can process. Existing frame sampling strategies, such as uniform or fixed-budget selection, often fail to adapt to variations in information density or task complexity, resulting in inefficiency and information loss. To address this, we present FrameOracle, a lightweight and plug-and-play module that predicts both (1) which frames are most relevant to a given query and (2) how many frames are needed. FrameOracle is trained using a four-stage curriculum, with the first three stages relying on weak proxy signals such as cross-modal similarity. In the final stage, it leverages stronger supervision from a new dataset we introduce, FrameOracle-41K, the first large-scale VideoQA collection to provide keyframe annotations specifying the minimal set of frames required to answer each question. Extensive experiments across five VLMs and six benchmarks demonstrate that FrameOracle reduces 16-frame inputs to an average of 10.4 frames without any loss in accuracy. When starting from 64-frame candidates, it reduces the input to an average of 13.9 frames while improving accuracy by 1.4%, achieving state-of-the-art efficiency-accuracy trade-offs for scalable video understanding.

Method: Hybrid Co-FineTuning (CFT) that integrates labeled samples from target game and co-domain games with unlabeled data to enhance feature representation learning.

Result: Demonstrates superior performance compared to conventional baselines across multiple gaming environments, maintaining competitive performance with only 50% of labeled data from target game.

Conclusion: CFT method provides enhanced scalability and adaptability for efficient visual bug detection across various game titles while reducing dependency on labeled examples.

Abstract: Manual identification of visual bugs in video games is a resource-intensive and costly process, often demanding specialized domain knowledge. While supervised visual bug detection models offer a promising solution, their reliance on extensive labeled datasets presents a significant challenge due to the infrequent occurrence of such bugs. To overcome this limitation, we propose a hybrid Co-FineTuning (CFT) method that effectively integrates both labeled and unlabeled data. Our approach leverages labeled samples from the target game and diverse co-domain games, additionally incorporating unlabeled data to enhance feature representation learning. This strategy maximizes the utility of all available data, substantially reducing the dependency on labeled examples from the specific target game. The developed framework demonstrates enhanced scalability and adaptability, facilitating efficient visual bug detection across various game titles. Our experimental results show the robustness of the proposed method for game visual bug detection, exhibiting superior performance compared to conventional baselines across multiple gaming environments. Furthermore, CFT maintains competitive performance even when trained with only 50% of the labeled data from the target game.

Method: HRM with two Transformer modules, DEQ-style one-step training, deep supervision, Rotary Position Embeddings, and RMSNorm. Evaluated on MNIST, CIFAR-10, CIFAR-100 with no data augmentation, identical optimizer with one-epoch warmup and cosine-floor decay, and label smoothing.

Result: HRM achieves ~98% on MNIST but performs poorly on natural images: 65.0% on CIFAR-10 vs 77.2% for CNN baseline, and 29.7% on CIFAR-100 vs 45.3% for CNN. HRM trains ~30x slower per epoch and shows significant overfitting.

Conclusion: HRM is not competitive with simple convolutional architectures for small-resolution image classification without augmentation due to insufficient image-specific inductive bias, though modifications could potentially improve it.

Abstract: This paper asks whether the Hierarchical Reasoning Model (HRM) with the two Transformer-style modules $(f_L,f_H)$, one step (DEQ-style) training, deep supervision, Rotary Position Embeddings, and RMSNorm can serve as a practical image classifier. It is evaluated on MNIST, CIFAR-10, and CIFAR-100 under a deliberately raw regime: no data augmentation, identical optimizer family with one-epoch warmup then cosine-floor decay, and label smoothing. HRM optimizes stably and performs well on MNIST ($\approx 98%$ test accuracy), but on small natural images it overfits and generalizes poorly: on CIFAR-10, HRM reaches 65.0% after 25 epochs, whereas a two-stage Conv–BN–ReLU baseline attains 77.2% while training $\sim 30\times$ faster per epoch; on CIFAR-100, HRM achieves only 29.7% test accuracy despite 91.5% train accuracy, while the same CNN reaches 45.3% test with 50.5% train accuracy. Loss traces and error analyses indicate healthy optimization but insufficient image-specific inductive bias for HRM in this regime. It is concluded that, for small-resolution image classification without augmentation, HRM is not competitive with even simple convolutional architectures as the HRM currently exist but this does not exclude possibilities that modifications to the model may allow it to improve greatly.

Method: Proposes Diffusion-Classifier Synergy (DCS) with dynamic multi-faceted reward function at feature level (prototype-anchored maximum mean discrepancy, dimension-wise variance matching) and logits level (confidence recalibration, cross-session confusion-aware mechanisms) to guide diffusion model generation.

Result: Demonstrably achieves state-of-the-art performance on FSCIL benchmarks, significantly enhancing both knowledge retention and new class learning capabilities.

Conclusion: The co-evolutionary process between diffusion model and classifier through reward-aligned learning effectively addresses FSCIL challenges, providing a robust framework for few-shot incremental learning.

Abstract: Few-Shot Class-Incremental Learning (FSCIL) challenges models to sequentially learn new classes from minimal examples without forgetting prior knowledge, a task complicated by the stability-plasticity dilemma and data scarcity. Current FSCIL methods often struggle with generalization due to their reliance on limited datasets. While diffusion models offer a path for data augmentation, their direct application can lead to semantic misalignment or ineffective guidance. This paper introduces Diffusion-Classifier Synergy (DCS), a novel framework that establishes a mutual boosting loop between diffusion model and FSCIL classifier. DCS utilizes a reward-aligned learning strategy, where a dynamic, multi-faceted reward function derived from the classifier’s state directs the diffusion model. This reward system operates at two levels: the feature level ensures semantic coherence and diversity using prototype-anchored maximum mean discrepancy and dimension-wise variance matching, while the logits level promotes exploratory image generation and enhances inter-class discriminability through confidence recalibration and cross-session confusion-aware mechanisms. This co-evolutionary process, where generated images refine the classifier and an improved classifier state yields better reward signals, demonstrably achieves state-of-the-art performance on FSCIL benchmarks, significantly enhancing both knowledge retention and new class learning.

Method: Uses a vision-language framework with three innovations: domain-specific VQA dataset (9,000 samples across 40 regulations), clause filter module for dynamic selection of relevant clauses, and behavior magnifier module that enhances worker regions for fine-grained action recognition.

Result: Significantly outperforms baseline models with 22.01% precision, 34.22% recall, and 28.37% F1 score improvements over 72B unfine-tuned baseline. Clause filter reduces inference latency by 13.56% while maintaining accuracy, and behavior magnifier adds 3.45% precision and 8.62% recall gains.

Conclusion: Demonstrates the potential of multimodal large models to enhance occupational safety monitoring in mining and other high-risk domains, with practical integration through a lightweight web-based interface for automatic violation reporting.

Abstract: Industrial accidents, particularly in high-risk domains such as surface and underground mining, are frequently caused by unsafe worker behaviors. Traditional manual inspection remains labor-intensive, error-prone, and insufficient for large-scale, dynamic environments, highlighting the urgent need for intelligent and automated safety monitoring. In this paper, we present MonitorVLM, a novel vision–language framework designed to detect safety violations directly from surveillance video streams. MonitorVLM introduces three key innovations: (1) a domain-specific violation dataset comprising 9,000 vision–question–answer (VQA) samples across 40 high-frequency mining regulations, enriched with augmentation and auxiliary detection cues; (2) a clause filter (CF) module that dynamically selects the Top-$K$ most relevant clauses, reducing inference latency by 13.56% while maintaining accuracy; and (3) a behavior magnifier (BM) module that enhances worker regions to improve fine-grained action recognition, yielding additional gains of 3.45% in precision and 8.62% in recall. Experimental results demonstrate that MonitorVLM significantly outperforms baseline vision–language models, achieving improvements of 22.01% in precision, 34.22% in recall, and 28.37% in F1 score over the 72B unfine-tuned baseline. A lightweight web-based interface further integrates MonitorVLM into practical workflows, enabling automatic violation reporting with video timestamping. This study highlights the potential of multimodal large models to enhance occupational safety monitoring in mining and beyond.

Method: Hybrid model integrating fine-tuned ExceptionNet architecture outputs as input for diffusion model, using image tensors as conditioning. Features multiple conditional modules with time embeddings and image covariate embeddings to dynamically adapt network behavior during diffusion process. Implemented as cloud-based solution for scalability.

Result: Achieved 97.32% accuracy in image-based accident detection, outperforming baseline models. Comprehensive ablation study investigated diffusion characteristics including timestep schedulers, encoding techniques, timestep count, and architectural design changes.

Conclusion: The proposed diffusion model demonstrates superior performance in accident detection for ITS, providing a robust classification framework that effectively handles complex data distributions through conditional modulation and cloud-based scalability.

Abstract: The integration of Diffusion Models into Intelligent Transportation Systems (ITS) is a substantial improvement in the detection of accidents. We present a novel hybrid model integrating guidance classification with diffusion techniques. By leveraging fine-tuned ExceptionNet architecture outputs as input for our proposed diffusion model and processing image tensors as our conditioning, our approach creates a robust classification framework. Our model consists of multiple conditional modules, which aim to modulate the linear projection of inputs using time embeddings and image covariate embeddings, allowing the network to adapt its behavior dynamically throughout the diffusion process. To address the computationally intensive nature of diffusion models, our implementation is cloud-based, enabling scalable and efficient processing. Our strategy overcomes the shortcomings of conventional classification approaches by leveraging diffusion models inherent capacity to effectively understand complicated data distributions. We investigate important diffusion characteristics, such as timestep schedulers, timestep encoding techniques, timestep count, and architectural design changes, using a thorough ablation study, and have conducted a comprehensive evaluation of the proposed model against the baseline models on a publicly available dataset. The proposed diffusion model performs best in image-based accident detection with an accuracy of 97.32%.

Method: Proposes SAMSOD with unimodal supervision to enhance non-dominant modality learning, gradient deconfliction to reduce conflicting gradients, and two decoupled adapters to separately mask high- and low-activation neurons for better foreground emphasis.

Result: Demonstrates effectiveness through experiments on RGB-T SOD benchmark datasets, scribble supervised RGB-T SOD, fully supervised RGB-D SOD datasets, and RGB-D rail surface defect detection.

Conclusion: The proposed method effectively addresses modality imbalance and gradient conflicts, showing superior performance across multiple SOD tasks and datasets.

Abstract: RGB-T salient object detection (SOD) aims to segment attractive objects by combining RGB and thermal infrared images. To enhance performance, the Segment Anything Model has been fine-tuned for this task. However, the imbalance convergence of two modalities and significant gradient difference between high- and low- activations are ignored, thereby leaving room for further performance enhancement. In this paper, we propose a model called \textit{SAMSOD}, which utilizes unimodal supervision to enhance the learning of non-dominant modality and employs gradient deconfliction to reduce the impact of conflicting gradients on model convergence. The method also leverages two decoupled adapters to separately mask high- and low-activation neurons, emphasizing foreground objects by enhancing background learning. Fundamental experiments on RGB-T SOD benchmark datasets and generalizability experiments on scribble supervised RGB-T SOD, fully supervised RGB-D SOD datasets and full-supervised RGB-D rail surface defect detection all demonstrate the effectiveness of our proposed method.

Method: Proposed progressive-iterative zooming adapter (PIZA) for parameter-efficient fine-tuning that enables models to progressively zoom in and localize small objects. Also created SOREC dataset with 100,000 expression-bounding box pairs for small objects in driving scenarios.

Result: Applied PIZA to GroundingDINO and demonstrated significant improvement in accuracy on the SOREC dataset.

Conclusion: The proposed dataset and method effectively address the challenge of small object localization in referring expression comprehension, with publicly available resources for further research.

Abstract: Referring expression comprehension (REC) aims to localize the target object described by a natural language expression. Recent advances in vision-language learning have led to significant performance improvements in REC tasks. However, localizing extremely small objects remains a considerable challenge despite its importance in real-world applications such as autonomous driving. To address this issue, we introduce a novel dataset and method for REC targeting small objects. First, we present the small object REC (SOREC) dataset, which consists of 100,000 pairs of referring expressions and corresponding bounding boxes for small objects in driving scenarios. Second, we propose the progressive-iterative zooming adapter (PIZA), an adapter module for parameter-efficient fine-tuning that enables models to progressively zoom in and localize small objects. In a series of experiments, we apply PIZA to GroundingDINO and demonstrate a significant improvement in accuracy on the SOREC dataset. Our dataset, codes and pre-trained models are publicly available on the project page.

Method: Incorporates attention mechanism into the WNet deep learning model to create Attention-WNet for artery-vein segmentation.

Result: Outperformed other state-of-the-art models on publicly available HRF and DRIVE datasets.

Conclusion: The proposed Attention-WNet approach demonstrates superior performance for retinal artery-vein segmentation compared to existing methods.

Abstract: Segmenting of clinically important retinal blood vessels into arteries and veins is a prerequisite for retinal vessel analysis. Such analysis can provide potential insights and bio-markers for identifying and diagnosing various retinal eye diseases. Alteration in the regularity and width of the retinal blood vessels can act as an indicator of the health of the vasculature system all over the body. It can help identify patients at high risk of developing vasculature diseases like stroke and myocardial infarction. Over the years, various Deep Learning architectures have been proposed to perform retinal vessel segmentation. Recently, attention mechanisms have been increasingly used in image segmentation tasks. The work proposes a new Deep Learning approach for artery-vein segmentation. The new approach is based on the Attention mechanism that is incorporated into the WNet Deep Learning model, and we call the model as Attention-WNet. The proposed approach has been tested on publicly available datasets such as HRF and DRIVE datasets. The proposed approach has outperformed other state-of-art models available in the literature.

Method: Created person-centric annotations for LAION-400M using validated automatic labeling pipelines combining object detection, multimodal captioning, and finetuned classifiers, including bounding boxes, perceived gender and race/ethnicity labels, and auto-generated captions.

Result: Uncovered demographic imbalances and harmful associations (e.g., men and individuals perceived as Black or Middle Eastern disproportionately linked with crime-related content), and showed 60-70% of gender bias in CLIP and Stable Diffusion can be linearly explained by direct co-occurrences in the data.

Conclusion: The resources establish the first large-scale empirical link between dataset composition and downstream model bias, providing evidence that training data imbalances directly contribute to model biases.

Abstract: Vision-language models trained on large-scale multimodal datasets show strong demographic biases, but the role of training data in producing these biases remains unclear. A major barrier has been the lack of demographic annotations in web-scale datasets such as LAION-400M. We address this gap by creating person-centric annotations for the full dataset, including over 276 million bounding boxes, perceived gender and race/ethnicity labels, and automatically generated captions. These annotations are produced through validated automatic labeling pipelines combining object detection, multimodal captioning, and finetuned classifiers. Using them, we uncover demographic imbalances and harmful associations, such as the disproportionate linking of men and individuals perceived as Black or Middle Eastern with crime-related and negative content. We also show that 60-70% of gender bias in CLIP and Stable Diffusion can be linearly explained by direct co-occurrences in the data. Our resources establish the first large-scale empirical link between dataset composition and downstream model bias.

Method: Compare two types of pretrained neural networks: 1) Generic networks pretrained on large diverse datasets of unspecific images, 2) Specialized network pretrained on satellite imagery.

Result: The paper investigates the trade-off between task specificity (specialized satellite imagery network) and data volume (generic networks with more training images) for urban informal settlement detection.

Conclusion: Research aims to determine which approach - task-specific pretraining or large-scale generic pretraining - provides superior performance for detecting informal settlements like favelas in Rio de Janeiro.

Abstract: While deep learning methods for detecting informal settlements have already been developed, they have not yet fully utilized the potential offered by recent pretrained neural networks. We compare two types of pretrained neural networks for detecting the favelas of Rio de Janeiro: 1. Generic networks pretrained on large diverse datasets of unspecific images, 2. A specialized network pretrained on satellite imagery. While the latter is more specific to the target task, the former has been pretrained on significantly more images. Hence, this research investigates whether task specificity or data volume yields superior performance in urban informal settlement detection.

Method: Proposes Low-Rank Adaptation (LoRA) patching with learnable gating mechanism to prevent gradient explosions, and Multi-Modal Feature Alignment (MMFA) loss for semantic-level feature alignment. Also introduces defensive LoRA patching as a complementary solution.

Result: Successfully defeats multiple proactive defenses with only 1,000 facial examples and a single epoch of fine-tuning, revealing critical weaknesses in current defense paradigms.

Conclusion: Current Deepfake defense strategies have fundamental vulnerabilities and require more robust approaches to address the security risks posed by techniques like LoRA patching.

Abstract: Deepfakes pose significant societal risks, motivating the development of proactive defenses that embed adversarial perturbations in facial images to prevent manipulation. However, in this paper, we show that these preemptive defenses often lack robustness and reliability. We propose a novel approach, Low-Rank Adaptation (LoRA) patching, which injects a plug-and-play LoRA patch into Deepfake generators to bypass state-of-the-art defenses. A learnable gating mechanism adaptively controls the effect of the LoRA patch and prevents gradient explosions during fine-tuning. We also introduce a Multi-Modal Feature Alignment (MMFA) loss, encouraging the features of adversarial outputs to align with those of the desired outputs at the semantic level. Beyond bypassing, we present defensive LoRA patching, embedding visible warnings in the outputs as a complementary solution to mitigate this newly identified security vulnerability. With only 1,000 facial examples and a single epoch of fine-tuning, LoRA patching successfully defeats multiple proactive defenses. These results reveal a critical weakness in current paradigms and underscore the need for more robust Deepfake defense strategies. Our code is available at https://github.com/ZOMIN28/LoRA-Patching.

Method: Proposed Reference-Set-Finetuning (RSF) - fine-tuning VPR models on the test-time reference set (map) before receiving test queries, leveraging the available reference images and poses from the target domain.

Result: RSF boosts SOTA VPR performance by ~2.3% average increase in Recall@1 on challenging datasets, while maintaining generalization and working across diverse test datasets.

Conclusion: Reference-set finetuning is an effective complementary approach that leverages test-time reference data to improve VPR performance on challenging domain-shift scenarios.

Abstract: Given a query image, Visual Place Recognition (VPR) is the task of retrieving an image of the same place from a reference database with robustness to viewpoint and appearance changes. Recent works show that some VPR benchmarks are solved by methods using Vision-Foundation-Model backbones and trained on large-scale and diverse VPR-specific datasets. Several benchmarks remain challenging, particularly when the test environments differ significantly from the usual VPR training datasets. We propose a complementary, unexplored source of information to bridge the train-test domain gap, which can further improve the performance of State-of-the-Art (SOTA) VPR methods on such challenging benchmarks. Concretely, we identify that the test-time reference set, the “map”, contains images and poses of the target domain, and must be available before the test-time query is received in several VPR applications. Therefore, we propose to perform simple Reference-Set-Finetuning (RSF) of VPR models on the map, boosting the SOTA (~2.3% increase on average for Recall@1) on these challenging datasets. Finetuned models retain generalization, and RSF works across diverse test datasets.

Method: ARSAM decomposes SAM’s gradient into SGD gradient and PSF, observes their dynamic evolution during training, and adaptively reuses PSF while timely updating it to maintain performance while reducing computations.

Result: ARSAM achieves comparable accuracies to SAM with about 40% speedup on CIFAR-10/100. It also accelerates various challenging tasks like human pose estimation and model quantization without performance loss.

Conclusion: ARSAM provides an efficient alternative to SAM that maintains generalization performance while significantly reducing computational overhead, demonstrating broad practicality across diverse tasks.

Abstract: Sharpness-Aware Minimization (SAM) improves model generalization but doubles the computational cost of Stochastic Gradient Descent (SGD) by requiring twice the gradient calculations per optimization step. To mitigate this, we propose Adaptively sampling-Reusing-mixing decomposed gradients to significantly accelerate SAM (ARSAM). Concretely, we firstly discover that SAM’s gradient can be decomposed into the SGD gradient and the Projection of the Second-order gradient onto the First-order gradient (PSF). Furthermore, we observe that the SGD gradient and PSF dynamically evolve during training, emphasizing the growing role of the PSF to achieve a flat minima. Therefore, ARSAM is proposed to the reused PSF and the timely updated PSF still maintain the model’s generalization ability. Extensive experiments show that ARSAM achieves state-of-the-art accuracies comparable to SAM across diverse network architectures. On CIFAR-10/100, ARSAM is comparable to SAM while providing a speedup of about 40%. Moreover, ARSAM accelerates optimization for the various challenge tasks (\textit{e.g.}, human pose estimation, and model quantization) without sacrificing performance, demonstrating its broad practicality.% The code is publicly accessible at: https://github.com/ajiaaa/ARSAM.

Method: Uses Concept-aware Embedding Generator to extract concept representations from each encoder layer, and Concept Prompt Tuning to amplify concept-related visual cues through prompt guidance.

Result: Outperforms state-of-the-art methods on three public datasets, effectively capturing and utilizing concept-wise information for improved predictions.

Conclusion: CoPA successfully addresses limitations of existing concept-based methods by leveraging multi-layer features and prompt guidance for enhanced clinical diagnostic transparency.

Abstract: The transparency of deep learning models is essential for clinical diagnostics. Concept Bottleneck Model provides clear decision-making processes for diagnosis by transforming the latent space of black-box models into human-understandable concepts. However, concept-based methods still face challenges in concept capture capabilities. These methods often rely on encode features solely from the final layer, neglecting shallow and multiscale features, and lack effective guidance in concept encoding, hindering fine-grained concept extraction. To address these issues, we introduce Concept Prompting and Aggregating (CoPA), a novel framework designed to capture multilayer concepts under prompt guidance. This framework utilizes the Concept-aware Embedding Generator (CEG) to extract concept representations from each layer of the visual encoder. Simultaneously, these representations serve as prompts for Concept Prompt Tuning (CPT), steering the model towards amplifying critical concept-related visual cues. Visual representations from each layer are aggregated to align with textual concept representations. With the proposed method, valuable concept-wise information in the images is captured and utilized effectively, thus improving the performance of concept and disease prediction. Extensive experimental results demonstrate that CoPA outperforms state-of-the-art methods on three public datasets. Code is available at https://github.com/yihengd/CoPA.

Method: Apply ZFP compression in error tolerance and fixed-rate modes to 3D medical dataset with ground-truth vascular segmentations, comparing segmentation quality on compressed vs uncompressed volumes.

Result: ZFP achieves substantial data reduction (up to 22.89:1 ratio) while maintaining high fidelity with mean Dice coefficient of 0.87656 compared to baseline 0.8774.

Conclusion: ZFP is a viable and powerful tool for enabling more efficient and accessible research on large-scale medical datasets, fostering broader collaboration.

Abstract: The increasing size and complexity of medical imaging datasets, particularly in 3D formats, present significant barriers to collaborative research and transferability. This study investigates whether the ZFP compression technique can mitigate these challenges without compromising the performance of automated cerebrovascular segmentation, a critical first step in intracranial aneurysm detection. We apply ZFP in both its error tolerance and fixed-rate modes to a large scale, and one of the most recent, datasets in the literature, 3D medical dataset containing ground-truth vascular segmentations. The segmentation quality on the compressed volumes is rigorously compared to the uncompressed baseline (Dice approximately equals 0.8774). Our findings reveal that ZFP can achieve substantial data reduction–up to a 22.89:1 ratio in error tolerance mode–while maintaining a high degree of fidelity, with the mean Dice coefficient remaining high at 0.87656. These results demonstrate that ZFP is a viable and powerful tool for enabling more efficient and accessible research on large-scale medical datasets, fostering broader collaboration across the community.

Method: Three-branch encoder integrating CNNs, Transformers, and Mamba-based Attention Fusion (MAF) mechanism, with multi-scale attention-based CNN decoder and co-attention gate for enhanced feature selection and cross-scale communication.

Result: Outperforms state-of-the-art methods in accuracy and generalization on multiple benchmark datasets while maintaining comparable computational complexity.

Conclusion: The architecture effectively balances efficiency and effectiveness, offering a practical and scalable solution for diverse medical imaging tasks.

Abstract: In recent years, deep learning has shown near-expert performance in segmenting complex medical tissues and tumors. However, existing models are often task-specific, with performance varying across modalities and anatomical regions. Balancing model complexity and performance remains challenging, particularly in clinical settings where both accuracy and efficiency are critical. To address these issues, we propose a hybrid segmentation architecture featuring a three-branch encoder that integrates CNNs, Transformers, and a Mamba-based Attention Fusion (MAF) mechanism to capture local, global, and long-range dependencies. A multi-scale attention-based CNN decoder reconstructs fine-grained segmentation maps while preserving contextual consistency. Additionally, a co-attention gate enhances feature selection by emphasizing relevant spatial and semantic information across scales during both encoding and decoding, improving feature interaction and cross-scale communication. Extensive experiments on multiple benchmark datasets show that our approach outperforms state-of-the-art methods in accuracy and generalization, while maintaining comparable computational complexity. By effectively balancing efficiency and effectiveness, our architecture offers a practical and scalable solution for diverse medical imaging tasks. Source code and trained models will be publicly released upon acceptance to support reproducibility and further research.

Method: Uses YOLOv9 algorithm with polygonal annotations (instead of traditional bounding boxes) for precise localization, trained on a novel dataset of 1000+ images from Dhaka, Bangladesh for three classes: Broken, Not Broken, and Manhole.

Result: Achieved 78.1% overall image-level accuracy with strong F1-scores for Broken (86.7%) and Not Broken (89.2%) classes, but poor performance for Manhole detection (18.2% F1-score) due to class imbalance.

Conclusion: The approach provides an efficient and scalable solution for urban infrastructure monitoring in developing countries, though manhole detection needs improvement through better class balancing.

Abstract: Urban safety and infrastructure maintenance are critical components of smart city development. Manual monitoring of road damages is time-consuming, highly costly, and error-prone. This paper presents a deep learning approach for automated road damage and manhole detection using the YOLOv9 algorithm with polygonal annotations. Unlike traditional bounding box annotation, we employ polygonal annotations for more precise localization of road defects. We develop a novel dataset comprising more than one thousand images which are mostly collected from Dhaka, Bangladesh. This dataset is used to train a YOLO-based model for three classes, namely Broken, Not Broken, and Manhole. We achieve 78.1% overall image-level accuracy. The YOLOv9 model demonstrates strong performance for Broken (86.7% F1-score) and Not Broken (89.2% F1-score) classes, with challenges in Manhole detection (18.2% F1-score) due to class imbalance. Our approach offers an efficient and scalable solution for monitoring urban infrastructure in developing countries.

Method: Uses time-dependent contrastive learning with SimCLR, treating an image and its domain-invariant feature as positive pairs. The learned contrastive model then guides inference of a pretrained stochastic differential equation (SDE) for translation.

Result: Achieves comparable results to state-of-the-art on several metrics across three unpaired I2I tasks. Model converges significantly faster and requires no label supervision or classifier training.

Conclusion: Contrastive-SDE provides an efficient alternative for unpaired image-to-image translation, combining the strengths of contrastive learning and diffusion models while eliminating the need for supervision and reducing training time.

Abstract: Unpaired image-to-image translation involves learning mappings between source domain and target domain in the absence of aligned or corresponding samples. Score based diffusion models have demonstrated state-of-the-art performance in generative tasks. Their ability to approximate complex data distributions through stochastic differential equations (SDEs) enables them to generate high-fidelity and diverse outputs, making them particularly well-suited for unpaired I2I settings. In parallel, contrastive learning provides a powerful framework for learning semantic similarities without the need for explicit supervision or paired data. By pulling together representations of semantically similar samples and pushing apart dissimilar ones, contrastive methods are inherently aligned with the objectives of unpaired translation. Its ability to selectively enforce semantic consistency at the feature level makes contrastive learning particularly effective for guiding generation in unpaired scenarios. In this work, we propose a time-dependent contrastive learning approach where a model is trained with SimCLR by considering an image and its domain invarient feature as a positive pair, enabling the preservation of domain-invariant features and the discarding of domain-specific ones. The learned contrastive model then guides the inference of a pretrained SDE for the I2I translation task. We empirically compare Contrastive-SDE with several baselines across three common unpaired I2I tasks, using four metrics for evaluation. Constrastive-SDE achieves comparable results to the state-of-the-art on several metrics. Furthermore, we observe that our model converges significantly faster and requires no label supervision or classifier training, making it a more efficient alternative for this task.

Method: Extended LIBERO benchmark with systematic perturbations across four dimensions: manipulated objects, initial states, task instructions, and environments to test model robustness.

Result: Existing models that achieve over 90% accuracy in standard LIBERO evaluation collapse to 0.0% accuracy under the generalized LIBERO-PRO setting, exposing reliance on rote memorization.

Conclusion: Current evaluation practices are severely flawed, and the community should adopt robust assessments that test genuine model generalization and comprehension rather than memorization capabilities.

Abstract: LIBERO has emerged as a widely adopted benchmark for evaluating Vision-Language-Action (VLA) models; however, its current training and evaluation settings are problematic, often leading to inflated performance estimates and preventing fair model comparison. To address these issues, we introduce LIBERO-PRO, an extended LIBERO benchmark that systematically evaluates model performance under reasonable perturbations across four dimensions: manipulated objects, initial states, task instructions, and environments. Experimental results reveal that, although existing models achieve over 90% accuracy under the standard LIBERO evaluation, their performance collapses to 0.0% under our generalized setting. Crucially, this discrepancy exposes the models’ reliance on rote memorization of action sequences and environment layouts from the training set, rather than genuine task understanding or environmental perception. For instance, models persist in executing grasping actions when the target object is replaced with irrelevant items, and their outputs remain unchanged even when given corrupted instructions or even messy tokens. These findings expose the severe flaws in current evaluation practices, and we call on the community to abandon misleading methodologies in favor of robust assessments of model generalization and comprehension. Our code is available at: https://github.com/Zxy-MLlab/LIBERO-PRO.

Method: Created Mirage dataset with diverse AI-generated images showing visible artifacts. Evaluated Large Vision-Language Models (LVLMs) for explainable AI image detection on both Mirage and existing benchmark datasets.

Result: LVLMs are highly effective at detecting AI-generated images with visible artifacts, but their performance declines when images lack such visual cues.

Conclusion: LVLMs show promise for explainable AI image detection, particularly for images with visible artifacts, but struggle with more sophisticated AI-generated content lacking obvious visual cues.

Abstract: Recent advances in image generation models have led to models that produce synthetic images that are increasingly difficult for standard AI detectors to identify, even though they often remain distinguishable by humans. To identify this discrepancy, we introduce \textbf{Mirage}, a curated dataset comprising a diverse range of AI-generated images exhibiting visible artifacts, where current state-of-the-art detection methods largely fail. Furthermore, we investigate whether Large Vision-Language Models (LVLMs), which are increasingly employed as substitutes for human judgment in various tasks, can be leveraged for explainable AI image detection. Our experiments on both Mirage and existing benchmark datasets demonstrate that while LVLMs are highly effective at detecting AI-generated images with visible artifacts, their performance declines when confronted with images lacking such cues.

Method: Three-stage progressive pruning: (1) Gaussian Sampling to identify critical primitives, (2) Gaussian Pruning to remove redundancies, (3) Gaussian Merging to fuse similar primitives. Also integrates implicit appearance compression and Sub-Vector Quantization for 4D representations.

Result: Reduces model sizes by over 60% while maintaining reconstruction quality, significantly outperforming state-of-the-art methods on standard benchmark datasets.

Conclusion: OMG4 represents a significant advancement in compact 4D scene representation, enabling new applications by dramatically reducing storage requirements without sacrificing quality.

Abstract: 4D Gaussian Splatting has emerged as a new paradigm for dynamic scene representation, enabling real-time rendering of scenes with complex motions. However, it faces a major challenge of storage overhead, as millions of Gaussians are required for high-fidelity reconstruction. While several studies have attempted to alleviate this memory burden, they still face limitations in compression ratio or visual quality. In this work, we present OMG4 (Optimized Minimal 4D Gaussian Splatting), a framework that constructs a compact set of salient Gaussians capable of faithfully representing 4D Gaussian models. Our method progressively prunes Gaussians in three stages: (1) Gaussian Sampling to identify primitives critical to reconstruction fidelity, (2) Gaussian Pruning to remove redundancies, and (3) Gaussian Merging to fuse primitives with similar characteristics. In addition, we integrate implicit appearance compression and generalize Sub-Vector Quantization (SVQ) to 4D representations, further reducing storage while preserving quality. Extensive experiments on standard benchmark datasets demonstrate that OMG4 significantly outperforms recent state-of-the-art methods, reducing model sizes by over 60% while maintaining reconstruction quality. These results position OMG4 as a significant step forward in compact 4D scene representation, opening new possibilities for a wide range of applications. Our source code is available at https://minshirley.github.io/OMG4/.

Method: Uses contrastive image-to-image alignment to enhance similarity between aerial and ground-view embeddings, and multi-instance vocabulary associations to align aerial images with text embeddings from pretrained models.

Result: Achieved improvements of +6.32 mAP on DOTAv2, +4.16 mAP on VisDrone, and +3.46 mAP on HRRSD in zero-shot setting compared to finetuned closed-vocabulary models.

Conclusion: The framework enables flexible and scalable object detection in aerial applications by effectively transferring open-vocabulary representations across domains.

Abstract: Traditional object detection models are typically trained on a fixed set of classes, limiting their flexibility and making it costly to incorporate new categories. Open-vocabulary object detection addresses this limitation by enabling models to identify unseen classes without explicit training. Leveraging pretrained models contrastively trained on abundantly available ground-view image-text classification pairs provides a strong foundation for open-vocabulary object detection in aerial imagery. Domain shifts, viewpoint variations, and extreme scale differences make direct knowledge transfer across domains ineffective, requiring specialized adaptation strategies. In this paper, we propose a novel framework for adapting open-vocabulary representations from ground-view images to solve object detection in aerial imagery through structured domain alignment. The method introduces contrastive image-to-image alignment to enhance the similarity between aerial and ground-view embeddings and employs multi-instance vocabulary associations to align aerial images with text embeddings. Extensive experiments on the xView, DOTAv2, VisDrone, DIOR, and HRRSD datasets are used to validate our approach. Our open-vocabulary model achieves improvements of +6.32 mAP on DOTAv2, +4.16 mAP on VisDrone (Images), and +3.46 mAP on HRRSD in the zero-shot setting when compared to finetuned closed-vocabulary dataset-specific model performance, thus paving the way for more flexible and scalable object detection systems in aerial applications.

Method: The review follows the PRISMA framework methodology and synthesizes recent research on DL approaches for skin cancer diagnosis.

Result: The paper identifies innovative approaches to overcome DL challenges in skin cancer diagnosis, including data augmentation, hybrid models, and feature fusion techniques.

Conclusion: Deep learning has transformative potential for skin disease diagnosis and clinical decision-making, but continued advancements are needed to fully realize this potential in dermatological care.

Abstract: Skin cancer is one of the most prevalent and deadly forms of cancer worldwide, which highlights the critical importance of early detection and diagnosis in improving patient outcomes. Deep learning (DL) has shown significant promise in enhancing the accuracy and efficiency of automated skin disease diagnosis, particularly in detecting and evaluating skin lesions and classification. However, there are still several challenges for DL-based skin cancer diagnosis, including complex features, image noise, intra-class variation, inter-class similarity, and data imbalance. By synthesizing recent research, this review discusses innovative approaches to cope with these challenges, such as data augmentation, hybrid models, and feature fusion, etc. Furthermore, the review highlights the integration of DL models into clinical workflows, offering insights into the potential of deep learning to revolutionize skin disease diagnosis and improve clinical decision-making. This article follows a comprehensive methodology based on the PRISMA framework and emphasizes the need for continued advancements to fully unlock the transformative potential of DL in dermatological care.

Method: Proposes Student Discriminator Assisted Knowledge Distillation (SDAKD) with a three-stage training strategy and adapted feature map distillation in the last two stages.

Result: Experiments on GCFSR and Real-ESRGAN show consistent improvements over baselines and state-of-the-art GAN knowledge distillation methods.

Conclusion: SDAKD effectively addresses the capacity mismatch problem in GAN distillation and enables more efficient deployment of GANs on resource-constrained devices.

Abstract: Generative Adversarial Networks (GANs) achieve excellent performance in generative tasks, such as image super-resolution, but their computational requirements make difficult their deployment on resource-constrained devices. While knowledge distillation is a promising research direction for GAN compression, effectively training a smaller student generator is challenging due to the capacity mismatch between the student generator and the teacher discriminator. In this work, we propose Student Discriminator Assisted Knowledge Distillation (SDAKD), a novel GAN distillation methodology that introduces a student discriminator to mitigate this capacity mismatch. SDAKD follows a three-stage training strategy, and integrates an adapted feature map distillation approach in its last two training stages. We evaluated SDAKD on two well-performing super-resolution GANs, GCFSR and Real-ESRGAN. Our experiments demonstrate consistent improvements over the baselines and SOTA GAN knowledge distillation methods. The SDAKD source code will be made openly available upon acceptance of the paper.

Method: Clinical data extracted from medical literature using LLMs (Claude 3.5 Sonnet) with iterative prompting strategies, then transformed into 3D representations using Neural Head Avatar framework.

Result: Created dataset with 7,920 images from two head textures covering broad ocular conditions, with extraction method achieving 91.92% accuracy.

Conclusion: PoseGaze-AHP is the first publicly available resource for AI-driven ocular-induced AHP diagnosis, supporting development of accurate and privacy-compliant diagnostic tools.

Abstract: Diagnosing ocular-induced abnormal head posture (AHP) requires a comprehensive analysis of both head pose and ocular movements. However, existing datasets focus on these aspects separately, limiting the development of integrated diagnostic approaches and restricting AI-driven advancements in AHP analysis. To address this gap, we introduce PoseGaze-AHP, a novel 3D dataset that synchronously captures head pose and gaze movement information for ocular-induced AHP assessment. Structured clinical data were extracted from medical literature using large language models (LLMs) through an iterative process with the Claude 3.5 Sonnet model, combining stepwise, hierarchical, and complex prompting strategies. The extracted records were systematically imputed and transformed into 3D representations using the Neural Head Avatar (NHA) framework. The dataset includes 7,920 images generated from two head textures, covering a broad spectrum of ocular conditions. The extraction method achieved an overall accuracy of 91.92%, demonstrating its reliability for clinical dataset construction. PoseGaze-AHP is the first publicly available resource tailored for AI-driven ocular-induced AHP diagnosis, supporting the development of accurate and privacy-compliant diagnostic tools.

Method: Proposed DynaMesh-Rater: a large multimodal model (LMM) approach that extracts multi-dimensional features including visual features from projected 2D video, motion features from cropped video clips, and geometry features from 4D human mesh. Uses LoRA-based instruction tuning to teach the LMM to predict quality scores.

Result: Extensive experiments on the DHQA-4D dataset (containing 32 high-quality 4D sequences and 1920 distorted meshes) demonstrate the superiority of DynaMesh-Rater over previous quality assessment methods for both textured and non-textured 4D meshes.

Conclusion: The proposed DHQA-4D dataset and DynaMesh-Rater method effectively address the quality assessment challenge for dynamic 4D digital humans, providing comprehensive multi-dimensional feature analysis and superior performance compared to existing methods.

Abstract: With the rapid development of 3D scanning and reconstruction technologies, dynamic digital human avatars based on 4D meshes have become increasingly popular. A high-precision dynamic digital human avatar can be applied to various fields such as game production, animation generation, and remote immersive communication. However, these 4D human avatar meshes are prone to being degraded by various types of noise during the processes of collection, compression, and transmission, thereby affecting the viewing experience of users. In light of this fact, quality assessment of dynamic 4D digital humans becomes increasingly important. In this paper, we first propose a large-scale dynamic digital human quality assessment dataset, DHQA-4D, which contains 32 high-quality real-scanned 4D human mesh sequences, 1920 distorted textured 4D human meshes degraded by 11 textured distortions, as well as their corresponding textured and non-textured mean opinion scores (MOSs). Equipped with DHQA-4D dataset, we analyze the influence of different types of distortion on human perception for textured dynamic 4D meshes and non-textured dynamic 4D meshes. Additionally, we propose DynaMesh-Rater, a novel large multimodal model (LMM) based approach that is able to assess both textured 4D meshes and non-textured 4D meshes. Concretely, DynaMesh-Rater elaborately extracts multi-dimensional features, including visual features from a projected 2D video, motion features from cropped video clips, and geometry features from the 4D human mesh to provide comprehensive quality-related information. Then we utilize a LMM model to integrate the multi-dimensional features and conduct a LoRA-based instruction tuning technique to teach the LMM model to predict the quality scores. Extensive experimental results on the DHQA-4D dataset demonstrate the superiority of our DynaMesh-Rater method over previous quality assessment methods.

Method: Enhanced ResNet-50 with ASFF using dual-branch design that fuses high-level semantic and mid-level detail features through global average pooling and fully connected layers to generate adaptive weights for weighted fusion.

Result: Achieved 93.18% accuracy on ISIC 2020 dataset (3297 images), outperforming 5 classic CNNs with higher precision, recall, specificity, F1 score, and AUC values of 0.9670 (P-R) and 0.9717 (ROC).

Conclusion: The proposed ASFF-based ResNet-50 provides a more effective and efficient solution for computer-aided skin cancer diagnosis by adaptively focusing on lesion-relevant regions while suppressing background noise.

Abstract: Skin cancer classification remains a challenging problem due to high inter-class similarity, intra-class variability, and image noise in dermoscopic images. To address these issues, we propose an improved ResNet-50 model enhanced with Adaptive Spatial Feature Fusion (ASFF), which adaptively integrates multi-scale semantic and surface features to improve feature representation and reduce overfitting. The ResNet-50 model is enhanced with an adaptive feature fusion mechanism to achieve more effective multi-scale feature extraction and improve overall performance. Specifically, a dual-branch design fuses high-level semantic and mid-level detail features, which are processed through global average pooling and fully connected layers to generate adaptive weights for weighted fusion, thereby strengthening feature learning and reducing the impact of noise on classification. The method is evaluated on a subset of the ISIC 2020 dataset containing 3297 benign and malignant skin lesion images. Experimental results show that the proposed ASFF-based ResNet-50 achieves the best overall performance compared with 5 classic convolutional neural networks (CNNs) models. The proposed model reached an accuracy of 93.18% along with higher precision, recall, specificity, and F1 score. The improved model achieves an AUC value of 0.9670 and 0.9717 in the P-R and ROC curve, respectively. Then, the evaluation based on Grad-CAM further proved that the improved model adaptively focuses on lesion-relevant regions while suppressing irrelevant background information, thereby validating its enhanced feature learning capability from a deep representation perspective. These findings demonstrate that the proposed approach provides a more effective and efficient solution for computer-aided skin cancer diagnosis.

Method: Retrospective study using publicly available datasets across three medical imaging modalities. Each modality trained a DenseNet-121 CNN via transfer learning with augmentation and modality-specific preprocessing. Predictions were fused using validation-weighted ensemble strategy.

Result: High validation accuracy for radiological (100%) and histopathological (95.12%) modalities, with clinical images performing lower (63.10%) due to visual heterogeneity. Ensemble model achieved 84.58% overall accuracy on multimodal validation dataset of 55 samples.

Conclusion: The multimodal ensemble framework provides a non-invasive, AI-assisted triage tool that enhances early identification of high-risk lesions, supports clinical decision-making, and aligns with global oncology guidelines to reduce diagnostic delays and improve patient outcomes.

Abstract: Aims Late diagnosis of Oral Squamous Cell Carcinoma (OSCC) contributes significantly to its high global mortality rate, with over 50% of cases detected at advanced stages and a 5-year survival rate below 50% according to WHO statistics. This study aims to improve early detection of OSCC by developing a multimodal deep learning framework that integrates clinical, radiological, and histopathological images using a weighted ensemble of DenseNet-121 convolutional neural networks (CNNs). Material and Methods A retrospective study was conducted using publicly available datasets representing three distinct medical imaging modalities. Each modality-specific dataset was used to train a DenseNet-121 CNN via transfer learning. Augmentation and modality-specific preprocessing were applied to increase robustness. Predictions were fused using a validation-weighted ensemble strategy. Evaluation was performed using accuracy, precision, recall, F1-score. Results High validation accuracy was achieved for radiological (100%) and histopathological (95.12%) modalities, with clinical images performing lower (63.10%) due to visual heterogeneity. The ensemble model demonstrated improved diagnostic robustness with an overall accuracy of 84.58% on a multimodal validation dataset of 55 samples. Conclusion The multimodal ensemble framework bridges gaps in the current diagnostic workflow by offering a non-invasive, AI-assisted triage tool that enhances early identification of high-risk lesions. It supports clinicians in decision-making, aligning with global oncology guidelines to reduce diagnostic delays and improve patient outcomes.

Method: Proposed IQA-Select, a clustering-based data selection framework with three stages: clustering feature extraction, cluster quota allocation, and cluster sampling strategy. The method systematically selects high-quality subsets from instruction tuning data.

Result: IQA-Select achieved 102.1% and 103.7% performance of full fine-tuning using only 10% selected data in Q-Bench and AesBench respectively, demonstrating superior efficiency and performance.

Conclusion: Data quality is more important than quantity for explainable IQA. The proposed IQA-Select method effectively reduces computational costs while improving performance by selecting high-quality data subsets.

Abstract: In recent years, with the rapid development of powerful multimodal large language models (MLLMs), explainable image quality assessment (IQA) has gradually become popular, aiming at providing quality-related descriptions and answers of images. To achieve this goal, recent methods seek to construct a large-scale instruction tuning dataset to empower the MLLM with quality perception ability following the well-known scaling law. However, a large amount of instruction tuning data may cause substantial computational costs and redundant data, which in turn will cause harm to the performance of the model. To cope with this problem, in this paper, we challenge the scaling law and systematically investigate the role of data quality of the instruction tuning dataset for explainable IQA. Using a powerful pre-trained MLLM, we first investigate the changes in model performance after fine-tuning with different sizes of instruction tuning data. We find that selecting a subset of the data set randomly using an appropriate ratio can even lead to better results than training with the entire instruction tuning dataset, demonstrating the redundancy of current explainable IQA instruction tuning data. Beyond randomly sampling a subset, we propose a clustering-based data selection framework with three stages: clustering feature extraction, cluster quota allocation, and cluster sampling strategy. Then we systematically analyze the choices of each stage and propose a simple but efficient data selection method IQA-Select for explainable IQA. The experimental results demonstrate that IQA-Select can achieve 102.1% and 103.7% performance of full fine-tuning using only 10% selected data in Q-Bench and AesBench respectively, significantly reducing computational costs while achieving better performance.

Method: Uses sparse 3D waypoints generated by VLM as intermediate representation, processed by a generalizable action expert that refines them into dense action sequences using real-time point cloud observations. Implements “Action Pre-training, Pointcloud Fine-tuning” paradigm.

Result: The framework effectively bridges VLM planning capabilities with physical action execution, enabling robust generalization across tasks and environments without requiring fine-tuning on new data.

Conclusion: The approach successfully combines the broad generalization of VLMs with fine-grained action-level generalization, overcoming limitations of previous VLA models and dual-system approaches.

Abstract: Although Vision-Language Models (VLM) have demonstrated impressive planning and reasoning capabilities, translating these abilities into the physical world introduces significant challenges. Conventional Vision-Language-Action (VLA) models, which integrate reasoning and action into a monolithic architecture, generalize poorly because they are constrained by scarce, narrow-domain data. While recent dual-system approaches attempt to decouple “thinking” from “acting”, they are often constrained by semantic ambiguities within the action module. This ambiguity makes large-scale, cross-task training infeasible. Consequently, these systems typically necessitate fine-tuning on newly collected data when deployed to novel environments, and the cooperation mechanism between the two systems remains ill-defined. To address these limitations, we introduce, for the first time, a framework centered around a generalizable action expert. Our approach utilizes sparse 3D trajectories as an intermediate representation, effectively bridging the high-level planning capabilities of the VLM with the low-level physical action module. During the planning phase, the VLM is only required to generate coarse 3D waypoints. These waypoints are then processed by our generalizable action expert, which refines them into dense, executable action sequences by sampling real-time point cloud observations of the environment. To promote training efficiency and robust generalization, we introduce a novel “Action Pre-training, Pointcloud Fine-tuning” paradigm. Our method combines the broad generalization capabilities of VLMs in visual understanding and planning with the fine-grained, action-level generalization of action expert.

Method: Transforms zero-shot fine-grained image classification into a visual question-answering framework, leverages LVLMs’ comprehensive understanding capabilities, and enhances performance through novel attention intervention technique and improved class description benchmarks.

Result: The proposed method consistently outperforms the current state-of-the-art approach across multiple fine-grained image classification benchmarks.

Conclusion: Demonstrates both the effectiveness of the proposed method and the broader potential of LVLMs for zero-shot fine-grained classification tasks.

Abstract: Large Vision-Language Models (LVLMs) have demonstrated impressive performance on vision-language reasoning tasks. However, their potential for zero-shot fine-grained image classification, a challenging task requiring precise differentiation between visually similar categories, remains underexplored. We present a novel method that transforms zero-shot fine-grained image classification into a visual question-answering framework, leveraging LVLMs’ comprehensive understanding capabilities rather than relying on direct class name generation. We enhance model performance through a novel attention intervention technique. We also address a key limitation in existing datasets by developing more comprehensive and precise class description benchmarks. We validate the effectiveness of our method through extensive experimentation across multiple fine-grained image classification benchmarks. Our proposed method consistently outperforms the current state-of-the-art (SOTA) approach, demonstrating both the effectiveness of our method and the broader potential of LVLMs for zero-shot fine-grained classification tasks. Code and Datasets: https://github.com/Atabuzzaman/Fine-grained-classification

Method: Structured empirical study benchmarking: (i) classical filters, (ii) modern defogging networks, (iii) chained variants (filter→model, model→filter), and (iv) prompt-driven visual-language image editing models applied directly to foggy images. Evaluation uses Foggy Cityscapes dataset.

Result: Analysis reveals when defogging helps, when chaining yields synergy or degradation, and how VLM-based editors compare to dedicated approaches. VLM judge qualitative scores show strong correlations with mAP metrics.

Conclusion: Establishes a transparent, task-oriented benchmark for defogging methods and highlights conditions under which preprocessing genuinely improves autonomous perception in adverse weather.

Abstract: Autonomous driving perception systems are particularly vulnerable in foggy conditions, where light scattering reduces contrast and obscures fine details critical for safe operation. While numerous defogging methods exist-from handcrafted filters to learned restoration models-improvements in image fidelity do not consistently translate into better downstream detection and segmentation. Moreover, prior evaluations often rely on synthetic data, leaving questions about real-world transferability. We present a structured empirical study that benchmarks a comprehensive set of pipelines, including (i) classical filters, (ii) modern defogging networks, (iii) chained variants (filter$\rightarrow$model, model$\rightarrow$filter), and (iv) prompt-driven visual–language image editing models (VLM) applied directly to foggy images. Using Foggy Cityscapes, we assess both image quality and downstream performance on object detection (mAP) and segmentation (PQ, RQ, SQ). Our analysis reveals when defogging helps, when chaining yields synergy or degradation, and how VLM-based editors compare to dedicated approaches. In addition, we evaluate qualitative rubric-based scores from a VLM judge and quantify their alignment with task metrics, showing strong correlations with mAP. Together, these results establish a transparent, task-oriented benchmark for defogging methods and highlight the conditions under which preprocessing genuinely improves autonomous perception in adverse weather.

Method: A cascaded framework that analyzes and prepares textual prompts and visual conditions to train VDMs, with a camera-aware conditioning module that automatically selects viewpoints aligned with input text to connect the two stages.

Result: The approach demonstrates effectiveness on both the MovieGen benchmark and a newly introduced benchmark for T2M-VDM combination, showing versatility across diverse use cases.

Conclusion: CAMEO successfully bridges T2M models and conditional VDMs for robust general human motion video generation, mitigating suboptimal factors through carefully designed components including camera-aware conditioning.

Abstract: Human video generation is becoming an increasingly important task with broad applications in graphics, entertainment, and embodied AI. Despite the rapid progress of video diffusion models (VDMs), their use for general-purpose human video generation remains underexplored, with most works constrained to image-to-video setups or narrow domains like dance videos. In this work, we propose CAMEO, a cascaded framework for general human motion video generation. It seamlessly bridges Text-to-Motion (T2M) models and conditional VDMs, mitigating suboptimal factors that may arise in this process across both training and inference through carefully designed components. Specifically, we analyze and prepare both textual prompts and visual conditions to effectively train the VDM, ensuring robust alignment between motion descriptions, conditioning signals, and the generated videos. Furthermore, we introduce a camera-aware conditioning module that connects the two stages, automatically selecting viewpoints aligned with the input text to enhance coherence and reduce manual intervention. We demonstrate the effectiveness of our approach on both the MovieGen benchmark and a newly introduced benchmark tailored to the T2M-VDM combination, while highlighting its versatility across diverse use cases.

Method: Proposes federated image-based localization where independent organizations maintain separate VPS services for their own spaces, with solutions for managing and merging data across maps without sharing private data.

Result: Enables access control of indoor 3D scans, distributed VPS maintenance, and encourages larger coverage while addressing challenges like localization coherency, quality control, and service selection.

Conclusion: OpenFLAME provides a scalable federated approach to VPS that overcomes limitations of centralized systems by allowing distributed maintenance and coverage expansion while preserving privacy.

Abstract: World-scale augmented reality (AR) applications need a ubiquitous 6DoF localization backend to anchor content to the real world consistently across devices. Large organizations such as Google and Niantic are 3D scanning outdoor public spaces in order to build their own Visual Positioning Systems (VPS). These centralized VPS solutions fail to meet the needs of many future AR applications – they do not cover private indoor spaces because of privacy concerns, regulations, and the labor bottleneck of updating and maintaining 3D scans. In this paper, we present OpenFLAME, a federated VPS backend that allows independent organizations to 3D scan and maintain a separate VPS service for their own spaces. This enables access control of indoor 3D scans, distributed maintenance of the VPS backend, and encourages larger coverage. Sharding of VPS services introduces several unique challenges – coherency of localization results across spaces, quality control of VPS services, selection of the right VPS service for a location, and many others. We introduce the concept of federated image-based localization and provide reference solutions for managing and merging data across maps without sharing private data.

Method: Uses CNN-LSTM models to extract biomechanical features (joint angles, limb velocities, kinetic chain patterns) from motion data, then analyzes these features for stroke effectiveness and injury risk, and generates feedback using large language models.

Result: The framework produces technically accurate, biomechanically grounded, and actionable feedback for end-users, evaluated on classification performance and interpretability.

Conclusion: The approach successfully bridges explainable AI and sports biomechanics by connecting deep learning analysis with practical, language-based feedback for tennis stroke improvement.

Abstract: Automated tennis stroke analysis has advanced significantly with the integration of biomechanical motion cues alongside deep learning techniques, enhancing stroke classification accuracy and player performance evaluation. Despite these advancements, existing systems often fail to connect biomechanical insights with actionable language feedback that is both accessible and meaningful to players and coaches. This research project addresses this gap by developing a novel framework that extracts key biomechanical features (such as joint angles, limb velocities, and kinetic chain patterns) from motion data using Convolutional Neural Network Long Short-Term Memory (CNN-LSTM)-based models. These features are analyzed for relationships influencing stroke effectiveness and injury risk, forming the basis for feedback generation using large language models (LLMs). Leveraging the THETIS dataset and feature extraction techniques, our approach aims to produce feedback that is technically accurate, biomechanically grounded, and actionable for end-users. The experimental setup evaluates this framework on classification performance and interpretability, bridging the gap between explainable AI and sports biomechanics.

Method: Proposed TimeWarp - a systematic method to create targeted synthetic temporal datasets for fine-tuning, and introduced a large-scale preference dataset capturing intricate temporal dynamics to ground model responses to visual and temporal information.

Result: Applied to existing models, significantly improved performance on temporal understanding benchmarks with absolute improvement across seven benchmarks.

Conclusion: TimeWarp’s synthetic temporal datasets effectively advance temporal understanding in Video-LLMs, demonstrating the importance of targeted dataset creation for improving fine-grained video understanding capabilities.

Abstract: While Video Large Language Models (Video-LLMs) have demonstrated remarkable performance across general video understanding benchmarks-particularly in video captioning and descriptive tasks-they consistently underperform on tasks that require fine-grained temporal understanding. This limitation arises due to the lack of visual complexity and temporal nuance in current fine-tuning datasets, leading these models to rely heavily on language-based reasoning rather than truly understanding video dynamics. In this work, we propose TimeWarp, a systematic method to create a targeted synthetic temporal dataset to fine-tune the model’s responses to encourage it to focus on the given input video. We introduce a large-scale preference dataset, created using TimeWarp, that captures intricate temporal dynamics often overlooked, grounding the model’s responses to visual and temporal information. We demonstrate that when our method is applied to existing models, it significantly improves performance on temporal understanding benchmarks, highlighting the effectiveness of our proposed datasets in advancing temporal understanding in Video-LLMs, resulting in an absolute improvement in performance across seven benchmarks. Code is available at https://github.com/sameepv21/timewarp.

Method: Extended text encoder context length in VLMs to 512 tokens, pretrained on BIOMEDICA-LongCAP dataset containing 1M image-caption pairs with context-aware descriptions from full-text articles.

Result: BMC-LongCLIP achieved up to +30% absolute gains in Recall@1 for long-caption retrieval, +2% average improvements in classification, faster convergence, and reduced token waste from 55% to 2.2%.

Conclusion: Long-context modeling is a promising direction for advancing biomedical VLMs, as longer context enables additional supervision from long-format captions and correlates with better performance.

Abstract: Embedding vision-language models (VLMs) are typically pretrained with short text windows (<77 tokens), which forces the truncation of long-format captions. Yet, the distribution of biomedical captions from large-scale open source literature reveals that a huge portion of captions far exceed 77 tokens. To this end, we investigate the impact of pretraining on long-format biomedical captions by extending the context length of text encoders in VLMs. We find that longer context (thus, enabling additional supervision provided in long-format captions) correlates with better retrieval and classification performance. Given this finding, we introduce BIOMEDICA-LongCAP, a dataset of 1M image-caption pairs enriched with context-aware descriptions from full-text articles, providing longer and additional textual supervision. Using BIOMEDICA-LongCAP, we train BMC-LongCLIP, a long-context biomedical VLM with a text encoder supporting windows of up to 512 tokens. Our model extends context capacity by 6.6x, reducing token waste from 55% to just 2.2%. On long-caption retrieval benchmarks, BMC-LongCLIP achieves up to +30% absolute gains in Recall@1 and +2% average improvements in classification, while also converging faster than short-context. Our results demonstrate that long-context modeling is a promising direction for advancing biomedical VLMs.

Method: Uses a controllable self-reinforcing optimization cycle: dynamic filtering of pseudo-labels to maintain known distribution, Bayes-optimal classifier with logit adjustment, and class-aware adaptive augmentation for minority classes.

Result: Achieves consistent improvements across benchmark datasets, surpassing state-of-the-art methods by up to 15.97% in accuracy.

Conclusion: CPG effectively handles unknown unlabeled data distributions through its controllable pseudo-label generation framework and theoretically reduces generalization error.

Abstract: Current long-tailed semi-supervised learning methods assume that labeled data exhibit a long-tailed distribution, and unlabeled data adhere to a typical predefined distribution (i.e., long-tailed, uniform, or inverse long-tailed). However, the distribution of the unlabeled data is generally unknown and may follow an arbitrary distribution. To tackle this challenge, we propose a Controllable Pseudo-label Generation (CPG) framework, expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution. Specifically, CPG operates through a controllable self-reinforcing optimization cycle: (i) at each training step, our dynamic controllable filtering mechanism selectively incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset, ensuring that the updated labeled dataset follows a known distribution; (ii) we then construct a Bayes-optimal classifier using logit adjustment based on the updated labeled data distribution; (iii) this improved classifier subsequently helps identify more reliable pseudo-labels in the next training step. We further theoretically prove that this optimization cycle can significantly reduce the generalization error under some conditions. Additionally, we propose a class-aware adaptive augmentation module to further improve the representation of minority classes, and an auxiliary branch to maximize data utilization by leveraging all labeled and unlabeled samples. Comprehensive evaluations on various commonly used benchmark datasets show that CPG achieves consistent improvements, surpassing state-of-the-art methods by up to \textbf{15.97%} in accuracy. The code is available at https://github.com/yaxinhou/CPG.

Method: Fine-tuned PaddleOCRv5’s text recognition module using curated Han-Nom manuscripts, with full training pipeline including preprocessing, LMDB conversion, evaluation, and visualization.

Result: Significant improvement in character recognition accuracy from 37.5% to 50.0%, especially under noisy image conditions. Developed interactive demo for visual comparison.

Conclusion: The fine-tuned model enables better digitization of historical documents and supports downstream applications like Han-Vietnamese semantic alignment, machine translation, and historical linguistics research.

Abstract: Recognizing and processing Classical Chinese (Han-Nom) texts play a vital role in digitizing Vietnamese historical documents and enabling cross-lingual semantic research. However, existing OCR systems struggle with degraded scans, non-standard glyphs, and handwriting variations common in ancient sources. In this work, we propose a fine-tuning approach for PaddleOCRv5 to improve character recognition on Han-Nom texts. We retrain the text recognition module using a curated subset of ancient Vietnamese Chinese manuscripts, supported by a full training pipeline covering preprocessing, LMDB conversion, evaluation, and visualization. Experimental results show a significant improvement over the base model, with exact accuracy increasing from 37.5 percent to 50.0 percent, particularly under noisy image conditions. Furthermore, we develop an interactive demo that visually compares pre- and post-fine-tuning recognition results, facilitating downstream applications such as Han-Vietnamese semantic alignment, machine translation, and historical linguistics research. The demo is available at https://huggingface.co/spaces/MinhDS/Fine-tuned-PaddleOCRv5.

Method: Uses an INR that simultaneously predicts pixel intensities and class labels, with meta-learning to find optimal initial parameters over training data, enabling quick fine-tuning for unseen test images.

Result: Achieved Dice scores comparable to U-Net models on 2D and 3D brain MRI segmentation tasks while using 90% fewer parameters.

Conclusion: MetaSeg provides a scalable alternative to resource-heavy architectures like U-Nets and vision transformers for medical image segmentation.

Abstract: Implicit neural representations (INRs) have achieved remarkable successes in learning expressive yet compact signal representations. However, they are not naturally amenable to predictive tasks such as segmentation, where they must learn semantic structures over a distribution of signals. In this study, we introduce MetaSeg, a meta-learning framework to train INRs for medical image segmentation. MetaSeg uses an underlying INR that simultaneously predicts per pixel intensity values and class labels. It then uses a meta-learning procedure to find optimal initial parameters for this INR over a training dataset of images and segmentation maps, such that the INR can simply be fine-tuned to fit pixels of an unseen test image, and automatically decode its class labels. We evaluated MetaSeg on 2D and 3D brain MRI segmentation tasks and report Dice scores comparable to commonly used U-Net models, but with $90%$ fewer parameters. MetaSeg offers a fresh, scalable alternative to traditional resource-heavy architectures such as U-Nets and vision transformers for medical image segmentation. Our project is available at https://kushalvyas.github.io/metaseg.html .

Method: Two-stage approach: 1) Localize question-relevant intervals using low-fps skim, 2) Answer via span-aware reallocation of visual tokens at higher effective frame rate with interleaved group-relative objective.

Result: Achieves up to 8.6% improvement with 50% less frame input on long-video QA and temporal grounding tasks; span-aware token reallocation consistently outperforms uniform sampling.

Conclusion: ViTL provides an interpretable, compute-efficient solution for scalable long-video QA when combined with the new dataset.

Abstract: We present \emph{Video-in-the-Loop} (ViTL), a two-stage long-video QA framework that preserves a fixed token budget by first \emph{localizing} question-relevant interval(s) with a low-fps skim and then \emph{answering} via span-aware reallocation of visual tokens at higher effective frame rate, emitting an interleaved output with both spans and the final option for direct attribution. We also introduce \dataname{}, which converts description based event graphs into \emph{span-grounded} multiple-choice QA by pairing each question with \emph{ground-truth} time span(s) and related reasoning. ViTL is trained end-to-end with an interleaved group-relative objective that couples temporal IoU for localization with answer correctness, allowing credit to flow from answers back to spans without increasing compute. Under fixed token budgets, ViTL attains up to 8.6% with 50% less frame input on long-video QA and temporal grounding (e.g., Charades-STA, ActivityNet-Captions) and ablations show that span-aware token reallocation consistently surpasses uniform sampling. Together, \dataname{} and ViTL provide an interpretable, compute-efficient recipe for scalable long-video QA.

Method: Conducted comprehensive study of ‘word swap’ method, developed ‘attention re-weight method’ for better adaptability, and proposed ‘CL P2P’ framework to address limitations like cycle inconsistency.

Result: Enhanced understanding of hyperparameter interactions with neural network attention mechanisms, leading to improved composition and quality of generated images in prompt-to-prompt editing.

Conclusion: The research contributes to better understanding and optimization of hyperparameter settings in relation to attention mechanisms, significantly improving the precision and reliability of text-driven image editing frameworks.

Abstract: Recent advances in image editing have shifted from manual pixel manipulation to employing deep learning methods like stable diffusion models, which now leverage cross-attention mechanisms for text-driven control. This transition has simplified the editing process but also introduced variability in results, such as inconsistent hair color changes. Our research aims to enhance the precision and reliability of prompt-to-prompt image editing frameworks by exploring and optimizing hyperparameters. We present a comprehensive study of the “word swap” method, develop an “attention re-weight method” for better adaptability, and propose the “CL P2P” framework to address existing limitations like cycle inconsistency. This work contributes to understanding and improving the interaction between hyperparameter settings and the architectural choices of neural network models, specifically their attention mechanisms, which significantly influence the composition and quality of the generated images.

Method: Train a model for image-grounded reasoning that dynamically invokes multiple specialized tools to iteratively narrow focus to relevant screen regions.

Result: Achieves 52.8% accuracy on ScreenSpot-Pro benchmark with only 18.5K training samples, outperforming V2P-7B (50.6% with 9.6M samples) and GTA-1-7B (50.1% with 1.56M samples).

Conclusion: GUI-Spotlight substantially improves visual grounding accuracy for GUI systems through iterative region focusing, enabling more reliable pointer-level actions with minimal training data.

Abstract: Multimodal large language models (MLLMs) have markedly expanded the competence of graphical user-interface (GUI) systems, propelling them beyond controlled simulations into complex, real-world environments across diverse platforms. However, practical usefulness is still bounded by the reliability of visual grounding, i.e., mapping textual references to exact on-screen elements. This limitation prevents the system from accurately performing pointer-level actions such as clicking or dragging. To address it, we introduce GUI-Spotlight – a model trained for image-grounded reasoning that dynamically invokes multiple specialized tools to iteratively narrow its focus to the relevant region of the screen, thereby substantially improving visual grounding accuracy. On the ScreenSpot-Pro benchmark, GUI-Spotlight trained with only 18.5K training samples achieves 52.8% accuracy, surpassing V2P-7B (50.6% with 9.6M training samples) and GTA-1-7B (50.1% with 1.56M training samples).

Method: Models range estimation as an optimization problem minimizing quantization errors by layer-wise local minima, proves local convexity, and presents an efficient search algorithm applied to transformed weights space.

Result: Outperforms state-of-the-art on top-1 accuracy for ResNet series and Inception-v3 models, with almost no loss in 8-bit/6-bit settings and significant improvement in 4-bit quantization for image classification.

Conclusion: The proposed range estimation method effectively improves quantization performance in post-training quantization with minimal accuracy loss across various bit-widths.

Abstract: Post-training quantization for reducing the storage of deep neural network models has been demonstrated to be an effective way in various tasks. However, low-bit quantization while maintaining model accuracy is a challenging problem. In this paper, we present a range estimation method to improve the quantization performance for post-training quantization. We model the range estimation into an optimization problem of minimizing quantization errors by layer-wise local minima. We prove this problem is locally convex and present an efficient search algorithm to find the optimal solution. We propose the application of the above search algorithm to the transformed weights space to do further improvement in practice. Our experiments demonstrate that our method outperforms state-of-the-art performance generally on top-1 accuracy for image classification tasks on the ResNet series models and Inception-v3 model. The experimental results show that the proposed method has almost no loss of top-1 accuracy in 8-bit and 6-bit settings for image classifications, and the accuracy of 4-bit quantization is also significantly improved. The code is available at https://github.com/codeiscommitting/REQuant.

Method: Uses flexible tri-modal retrieval with text, image, and 3D queries, featuring ESSGNN layout encoder that captures spatial relationships and appearance features, supporting iterative scene construction.

Result: Empirical evaluations show improved spatial and stylistic consistency compared to baseline methods in various retrieval tasks.

Conclusion: MetaFind provides an effective framework for contextually and stylistically coherent 3D asset retrieval in metaverse scene generation.

Abstract: We present MetaFind, a scene-aware tri-modal compositional retrieval framework designed to enhance scene generation in the metaverse by retrieving 3D assets from large-scale repositories. MetaFind addresses two core challenges: (i) inconsistent asset retrieval that overlooks spatial, semantic, and stylistic constraints, and (ii) the absence of a standardized retrieval paradigm specifically tailored for 3D asset retrieval, as existing approaches mainly rely on general-purpose 3D shape representation models. Our key innovation is a flexible retrieval mechanism that supports arbitrary combinations of text, image, and 3D modalities as queries, enhancing spatial reasoning and style consistency by jointly modeling object-level features (including appearance) and scene-level layout structures. Methodologically, MetaFind introduces a plug-and-play equivariant layout encoder ESSGNN that captures spatial relationships and object appearance features, ensuring retrieved 3D assets are contextually and stylistically coherent with the existing scene, regardless of coordinate frame transformations. The framework supports iterative scene construction by continuously adapting retrieval results to current scene updates. Empirical evaluations demonstrate the improved spatial and stylistic consistency of MetaFind in various retrieval tasks compared to baseline methods.

Method: Modified loss function that integrates ordinal information into binary cross-entropy loss, serving as an ordinality-aware regularization method during model optimization.

Result: The approach aims to improve model performance by leveraging ordinal characteristics and penalizing threshold-near misclassifications more heavily.

Conclusion: Incorporating ordinal weighting into loss functions can enhance solar flare prediction models by better handling the inherent ordinal relationships in flare intensity data.

Abstract: The prediction of solar flares is typically formulated as a binary classification task, distinguishing events as either Flare (FL) or No-Flare (NF) according to a specified threshold (for example, greater than or equal to C-class, M-class, or X-class). However, this binary framework neglects the inherent ordinal relationships among the sub-classes contained within each category (FL and NF). Several studies on solar flare prediction have empirically shown that the most frequent misclassifications occur near this prediction threshold. This suggests that the models struggle to differentiate events that are similar in intensity but fall on opposite sides of the binary threshold. To mitigate this limitation, we propose a modified loss function that integrates the ordinal information among the sub-classes of the binarized flare labels into the conventional binary cross-entropy (BCE) loss. This approach serves as an ordinality-aware, data-driven regularization method that penalizes the incorrect predictions of flare events in close proximity to the prediction threshold more heavily than those away from the boundary during model optimization. By incorporating ordinal weighting into the loss function, we aim to enhance the model’s learning process by leveraging the ordinal characteristics of the data, thereby improving its overall performance.

Method: Proposes two key components: (1) Outlier-aware quantizer using sampling-based range estimation to reduce activation outliers and keeping extreme weights in FP16, (2) Frequency-aware calibration strategy that emphasizes low- and mid-frequency components during fine-tuning to mitigate banding artifacts.

Result: QuantDemoire achieves large reductions in parameters and computation while maintaining quality, outperforming existing quantization methods by over 4 dB on W4A4 (4-bit weight and activation quantization).

Conclusion: The proposed QuantDemoire framework effectively addresses quantization challenges for demoiréing models, enabling efficient deployment on edge devices without compromising performance.

Abstract: Demoir'eing aims to remove moir'e artifacts that often occur in images. While recent deep learning-based methods have achieved promising results, they typically require substantial computational resources, limiting their deployment on edge devices. Model quantization offers a compelling solution. However, directly applying existing quantization methods to demoir'eing models introduces severe performance degradation. The main reasons are distribution outliers and weakened representations in smooth regions. To address these issues, we propose QuantDemoire, a post-training quantization framework tailored to demoir'eing. It contains two key components. **First}, we introduce an outlier-aware quantizer to reduce errors from outliers. It uses sampling-based range estimation to reduce activation outliers, and keeps a few extreme weights in FP16 with negligible cost. Second, we design a frequency-aware calibration strategy. It emphasizes low- and mid-frequency components during fine-tuning, which mitigates banding artifacts caused by low-bit quantization. Extensive experiments validate that our QuantDemoire achieves large reductions in parameters and computation while maintaining quality. Meanwhile, it outperforms existing quantization methods by over 4 dB on W4A4. Code is released at: https://github.com/zhengchen1999/QuantDemoire.

Method: Combines diffusion generative prior (NCSN++ with SDE modeling) with multi-regularization constraints including anisotropic TV and nuclear norm (LoRA) in an ADMM framework. Uses 2D slice-based strategy with FFT acceleration and tensor-parallel optimization for efficient inference.

Result: Experiments on AAPM-2016, CTHD, and LIDC datasets with 8, 4, and 2 views show TV-LoRA consistently surpasses benchmarks in SSIM, texture recovery, edge clarity, and artifact suppression, demonstrating strong robustness and generalizability.

Conclusion: TV-LoRA achieves high-fidelity, efficient 3D CT reconstruction with broad clinical applicability in low-dose, sparse-sampling scenarios, with ablation studies confirming the complementary effects of LoRA regularization and diffusion priors.

Abstract: This work presents TV-LoRA, a novel method for low-dose sparse-view CT reconstruction that combines a diffusion generative prior (NCSN++ with SDE modeling) and multi-regularization constraints, including anisotropic TV and nuclear norm (LoRA), within an ADMM framework. To address ill-posedness and texture loss under extremely sparse views, TV-LoRA integrates generative and physical constraints, and utilizes a 2D slice-based strategy with FFT acceleration and tensor-parallel optimization for efficient inference. Experiments on AAPM-2016, CTHD, and LIDC datasets with $N_{\mathrm{view}}=8,4,2$ show that TV-LoRA consistently surpasses benchmarks in SSIM, texture recovery, edge clarity, and artifact suppression, demonstrating strong robustness and generalizability. Ablation studies confirm the complementary effects of LoRA regularization and diffusion priors, while the FFT-PCG module provides a speedup. Overall, Diffusion + TV-LoRA achieves high-fidelity, efficient 3D CT reconstruction and broad clinical applicability in low-dose, sparse-sampling scenarios.

Method: The authors formalize topological consistency as the core property of topological maps and show localization accuracy serves as an efficient surrogate metric. They propose the first quantitative measure of dataset ambiguity and curate a diverse benchmark dataset with calibrated ambiguity levels.

Result: The paper implements and releases deep-learned baseline systems and evaluates them alongside classical methods, yielding new insights into the limitations of current approaches under perceptual aliasing conditions.

Conclusion: All datasets, baselines, and evaluation tools are open-sourced to promote consistent and reproducible research in topological mapping, addressing the field’s standardization challenges.

Abstract: Topological mapping offers a compact and robust representation for navigation, but progress in the field is hindered by the lack of standardized evaluation metrics, datasets, and protocols. Existing systems are assessed using different environments and criteria, preventing fair and reproducible comparisons. Moreover, a key challenge - perceptual aliasing - remains under-quantified, despite its strong influence on system performance. We address these gaps by (1) formalizing topological consistency as the fundamental property of topological maps and showing that localization accuracy provides an efficient and interpretable surrogate metric, and (2) proposing the first quantitative measure of dataset ambiguity to enable fair comparisons across environments. To support this protocol, we curate a diverse benchmark dataset with calibrated ambiguity levels, implement and release deep-learned baseline systems, and evaluate them alongside classical methods. Our experiments and analysis yield new insights into the limitations of current approaches under perceptual aliasing. All datasets, baselines, and evaluation tools are fully open-sourced to foster consistent and reproducible research in topological mapping.

Method: Created HREM dataset (1280x720 resolution), proposed EEMFlow network with encoder-decoder architecture, added CDC module for dense flow, and developed ADM for density adaptation.

Result: EEMFlow achieves 30x faster runtime than SOTA methods. ADM improves EEMFlow and EEMFlow+ performance by 8% and 10% respectively.

Conclusion: The proposed methods successfully address meshflow estimation and data density challenges, demonstrating superior efficiency and performance in event-based motion field prediction.

Abstract: In this paper, we explore the problem of event-based meshflow estimation, a novel task that involves predicting a spatially smooth sparse motion field from event cameras. To start, we review the state-of-the-art in event-based flow estimation, highlighting two key areas for further research: i) the lack of meshflow-specific event datasets and methods, and ii) the underexplored challenge of event data density. First, we generate a large-scale High-Resolution Event Meshflow (HREM) dataset, which showcases its superiority by encompassing the merits of high resolution at 1280x720, handling dynamic objects and complex motion patterns, and offering both optical flow and meshflow labels. These aspects have not been fully explored in previous works. Besides, we propose Efficient Event-based MeshFlow (EEMFlow) network, a lightweight model featuring a specially crafted encoder-decoder architecture to facilitate swift and accurate meshflow estimation. Furthermore, we upgrade EEMFlow network to support dense event optical flow, in which a Confidence-induced Detail Completion (CDC) module is proposed to preserve sharp motion boundaries. We conduct comprehensive experiments to show the exceptional performance and runtime efficiency (30x faster) of our EEMFlow model compared to the recent state-of-the-art flow method. As an extension, we expand HREM into HREM+, a multi-density event dataset contributing to a thorough study of the robustness of existing methods across data with varying densities, and propose an Adaptive Density Module (ADM) to adjust the density of input event data to a more optimal range, enhancing the model’s generalization ability. We empirically demonstrate that ADM helps to significantly improve the performance of EEMFlow and EEMFlow+ by 8% and 10%, respectively. Code and dataset are released at https://github.com/boomluo02/EEMFlowPlus.

Method: Pretrains encoder with direct pose regression head, jointly learns networks via regression and diffusion heads, and introduces sampling guidance with time-dependent score scaling to balance exploration-exploitation trade-off.

Result: Achieves state-of-the-art accuracies on REAL275, HouseCat6D, and ROPE benchmarks with single-pose inference, while being more efficient in both training and inference.

Conclusion: The proposed method is simple yet effective, accelerating training convergence while maintaining multi-modal characteristics for symmetric objects and ensuring high-quality pose generation.

Abstract: Latest diffusion models have shown promising results in category-level 6D object pose estimation by modeling the conditional pose distribution with depth image input. The existing methods, however, suffer from slow convergence during training, learning its encoder with the diffusion denoising network in end-to-end fashion, and require an additional network that evaluates sampled pose hypotheses to filter out low-quality pose candidates. In this paper, we propose a novel pipeline that tackles these limitations by two key components. First, the proposed method pretrains the encoder with the direct pose regression head, and jointly learns the networks via the regression head and the denoising diffusion head, significantly accelerating training convergence while achieving higher accuracy. Second, sampling guidance via time-dependent score scaling is proposed s.t. the exploration-exploitation trade-off is effectively taken, eliminating the need for the additional evaluation network. The sampling guidance maintains multi-modal characteristics of symmetric objects at early denoising steps while ensuring high-quality pose generation at final steps. Extensive experiments on multiple benchmarks including REAL275, HouseCat6D, and ROPE, demonstrate that the proposed method, simple yet effective, achieves state-of-the-art accuracies even with single-pose inference, while being more efficient in both training and inference.

Method: The authors establish a theoretical connection between concept drift and knowledge distillation, framing non-stationary reasoning dynamics as next-token prediction of multi-stream reasoning trajectories. They introduce the “learn, compare, critique” paradigm with autonomous preference optimization (APO), where students learn from teachers, compare multiple teachers, and critically reflect on drifting inferences to perform concept alignment.

Result: Extensive experiments demonstrate superior performance in consistency, robustness, and generalization within knowledge distillation. The authors also contribute CXR-MAX, a large-scale dataset with 170,982 distilled reasoning trajectories derived from publicly accessible MLLMs based on MIMIC-CXR.

Conclusion: The proposed autonomous preference optimization approach effectively addresses concept drift in MLLM distillation, yielding robust, consistent, and generalizable student models through the “learn, compare, critique” paradigm and concept alignment.

Abstract: This paper identifies a critical yet underexplored challenge in distilling from multimodal large language models (MLLMs): the reasoning trajectories generated by multiple drifting teachers exhibit concept drift, whereby their reasoning distributions evolve unpredictably and transmit biases to the student model, ultimately compromising its performance. To tackle this issue, we pioneer a theoretical connection between concept drift and knowledge distillation, casting the non-stationary reasoning dynamics from multiple MLLM teachers as next-token prediction of multi-stream reasoning trajectories.Guided by concept drift, we introduce the “learn, compare, critique” paradigm, culminating in autonomous preference optimization (APO). Under the active guidance of the teachers, the student model first learns and self-distils preferred thinking by comparing multiple teachers. It then engages in critical reflection over the drifting inference from teachers, performing concept alignment through APO, ultimately yielding a robust, consistent, and generalizable model.Extensive experiments demonstrate our superior performance of consistency, robustness and generalization within knowledge distillation. Besides, we also contributed a large-scale dataset, CXR-MAX (Multi-teachers Alignment X-rays), comprising 170,982 distilled reasoning trajectories derived from publicly accessible MLLMs based on MIMIC-CXR. Our code and data are public at: https://anonymous.4open.science/r/Autonomous-Distillation/.

Method: Developed SiteShield, a multi-modal LVLM-based Retrieval-Augmented Generation (RAG) framework that integrates visual and audio inputs for construction safety inspection automation.

Result: SiteShield outperformed unimodal LLMs without RAG with F1 score of 0.82, hamming loss of 0.04, precision of 0.76, and recall of 0.96 using real-world data.

Conclusion: SiteShield offers a novel pathway to enhance information retrieval and efficiency in generating safety reports for construction safety inspections.

Abstract: Conventional construction safety inspection methods are often inefficient as they require navigating through large volume of information. Recent advances in large vision-language models (LVLMs) provide opportunities to automate safety inspections through enhanced visual and linguistic understanding. However, existing applications face limitations including irrelevant or unspecific responses, restricted modal inputs and hallucinations. Utilisation of Large Language Models (LLMs) for this purpose is constrained by availability of training data and frequently lack real-time adaptability. This study introduces SiteShield, a multi-modal LVLM-based Retrieval-Augmented Generation (RAG) framework for automating construction safety inspection reports by integrating visual and audio inputs. Using real-world data, SiteShield outperformed unimodal LLMs without RAG with an F1 score of 0.82, hamming loss of 0.04, precision of 0.76, and recall of 0.96. The findings indicate that SiteShield offers a novel pathway to enhance information retrieval and efficiency in generating safety reports.

Method: BLADE trains a generative model to translate images across bias domains while preserving task-relevant features, then adaptively refines images with synthetic counterparts based on bias susceptibility. It aligns images with bias-translated counterparts sharing task features but differing in bias, while misaligning with same-bias samples.

Result: BLADE significantly outperforms state-of-the-art methods on multiple benchmark datasets, exceeding the closest baseline by ~18% absolute margin on corrupted CIFAR-10 under worst group setting.

Conclusion: BLADE establishes a new benchmark in bias mitigation and demonstrates potential for developing more robust deep learning models without explicit supervision.

Abstract: Neural networks have revolutionized numerous fields, yet they remain vulnerable to a critical flaw: the tendency to learn implicit biases, spurious correlations between certain attributes and target labels in training data. These biases are often more prevalent and easier to learn, causing models to rely on superficial patterns rather than task-relevant features necessary for generalization. Existing methods typically rely on strong assumptions, such as prior knowledge of these biases or access to bias-conflicting samples, i.e., samples that contradict spurious correlations and counterbalance bias-aligned samples, samples that conform to these spurious correlations. However, such assumptions are often impractical in real-world settings. We propose BLADE ({B}ias-{L}inked {A}daptive {DE}biasing), a generative debiasing framework that requires no prior knowledge of bias or bias-conflicting samples. BLADE first trains a generative model to translate images across bias domains while preserving task-relevant features. Then, it adaptively refines each image with its synthetic counterpart based on the image’s susceptibility to bias. To encourage robust representations, BLADE aligns an image with its bias-translated synthetic counterpart that shares task-relevant features but differs in bias, while misaligning it with samples sharing the same bias. We evaluate BLADE on multiple benchmark datasets and show that it significantly outperforms state-of-the-art methods. Notably, it exceeds the closest baseline by an absolute margin of around 18% on the corrupted CIFAR-10 dataset under the worst group setting, establishing a new benchmark in bias mitigation and demonstrating its potential for developing more robust deep learning models without explicit supervision.

Method: Combines concept-guided image segmentation with attention-based multiple instance learning, treating segmented regions as instances and aggregating evidence across them to identify task-relevant concepts.

Result: Achieves robust performance across spurious correlations, input corruptions, and large-scale benchmarks while providing transparent concept-level explanations without concept annotations.

Conclusion: SEG-MIL-CBM enables spatially grounded concept reasoning without manual annotations, improving both interpretability and robustness in deep learning models.

Abstract: Deep neural networks have achieved remarkable success in computer vision; however, their black-box nature in decision-making limits interpretability and trust, particularly in safety-critical applications. Interpretability is crucial in domains where errors have severe consequences. Existing models not only lack transparency but also risk exploiting unreliable or misleading features, which undermines both robustness and the validity of their explanations. Concept Bottleneck Models (CBMs) aim to improve transparency by reasoning through human-interpretable concepts. Still, they require costly concept annotations and lack spatial grounding, often failing to identify which regions support each concept. We propose SEG-MIL-CBM, a novel framework that integrates concept-guided image segmentation into an attention-based multiple instance learning (MIL) framework, where each segmented region is treated as an instance and the model learns to aggregate evidence across them. By reasoning over semantically meaningful regions aligned with high-level concepts, our model highlights task-relevant evidence, down-weights irrelevant cues, and produces spatially grounded, concept-level explanations without requiring annotations of concepts or groups. SEG-MIL-CBM achieves robust performance across settings involving spurious correlations (unintended dependencies between background and label), input corruptions (perturbations that degrade visual quality), and large-scale benchmarks, while providing transparent, concept-level explanations.

Method: Model hidden feature evolution as a mixture of ODEs across dimensions and introduce HyCa, a hybrid ODE solver-inspired caching framework that applies dimension-wise caching strategies.

Result: Achieves near-lossless acceleration: 5.55x speedup on FLUX, 5.56x on HunyuanVideo, 6.24x on Qwen-Image and Qwen-Image-Edit without retraining.

Conclusion: HyCa provides effective training-free acceleration for Diffusion Transformers by modeling feature dynamics with dimension-wise ODEs and hybrid caching strategies.

Abstract: Diffusion Transformers offer state-of-the-art fidelity in image and video synthesis, but their iterative sampling process remains a major bottleneck due to the high cost of transformer forward passes at each timestep. To mitigate this, feature caching has emerged as a training-free acceleration technique that reuses or forecasts hidden representations. However, existing methods often apply a uniform caching strategy across all feature dimensions, ignoring their heterogeneous dynamic behaviors. Therefore, we adopt a new perspective by modeling hidden feature evolution as a mixture of ODEs across dimensions, and introduce HyCa, a Hybrid ODE solver inspired caching framework that applies dimension-wise caching strategies. HyCa achieves near-lossless acceleration across diverse domains and models, including 5.55 times speedup on FLUX, 5.56 times speedup on HunyuanVideo, 6.24 times speedup on Qwen-Image and Qwen-Image-Edit without retraining.

Method: An agent dynamically searches the web to retrieve images for unknown concepts, then performs multimodal prompt optimization to steer generative models toward accurate synthesis.

Result: World-To-Image achieves +8.1% improvement in accuracy-to-prompt on the NICE benchmark, outperforming state-of-the-art methods in both semantic alignment and visual aesthetics with high efficiency in under three iterations.

Conclusion: The framework enables T2I systems to better reflect the ever-changing real world by bridging knowledge gaps through agent-driven web retrieval and multimodal optimization.

Abstract: While text-to-image (T2I) models can synthesize high-quality images, their performance degrades significantly when prompted with novel or out-of-distribution (OOD) entities due to inherent knowledge cutoffs. We introduce World-To-Image, a novel framework that bridges this gap by empowering T2I generation with agent-driven world knowledge. We design an agent that dynamically searches the web to retrieve images for concepts unknown to the base model. This information is then used to perform multimodal prompt optimization, steering powerful generative backbones toward an accurate synthesis. Critically, our evaluation goes beyond traditional metrics, utilizing modern assessments like LLMGrader and ImageReward to measure true semantic fidelity. Our experiments show that World-To-Image substantially outperforms state-of-the-art methods in both semantic alignment and visual aesthetics, achieving +8.1% improvement in accuracy-to-prompt on our curated NICE benchmark. Our framework achieves these results with high efficiency in less than three iterations, paving the way for T2I systems that can better reflect the ever-changing real world. Our demo code is available here\footnote{https://github.com/mhson-kyle/World-To-Image}.

Method: MASC uses geometry-aware distance metrics and density-driven agglomerative construction to build a hierarchical semantic tree that models the underlying manifold of token embeddings, transforming flat prediction into structured hierarchical prediction.

Result: MASC accelerates training by up to 57% and improves generation quality, reducing FID of LlamaGen-XL from 2.87 to 2.58, making AR frameworks competitive with state-of-the-art methods.

Conclusion: Structuring the prediction space through hierarchical semantic organization is as crucial as architectural innovation for scalable generative modeling, with MASC providing a plug-and-play solution that significantly enhances AR image generation.

Abstract: Autoregressive (AR) models have shown great promise in image generation, yet they face a fundamental inefficiency stemming from their core component: a vast, unstructured vocabulary of visual tokens. This conventional approach treats tokens as a flat vocabulary, disregarding the intrinsic structure of the token embedding space where proximity often correlates with semantic similarity. This oversight results in a highly complex prediction task, which hinders training efficiency and limits final generation quality. To resolve this, we propose Manifold-Aligned Semantic Clustering (MASC), a principled framework that constructs a hierarchical semantic tree directly from the codebook’s intrinsic structure. MASC employs a novel geometry-aware distance metric and a density-driven agglomerative construction to model the underlying manifold of the token embeddings. By transforming the flat, high-dimensional prediction task into a structured, hierarchical one, MASC introduces a beneficial inductive bias that significantly simplifies the learning problem for the AR model. MASC is designed as a plug-and-play module, and our extensive experiments validate its effectiveness: it accelerates training by up to 57% and significantly improves generation quality, reducing the FID of LlamaGen-XL from 2.87 to 2.58. MASC elevates existing AR frameworks to be highly competitive with state-of-the-art methods, establishing that structuring the prediction space is as crucial as architectural innovation for scalable generative modeling.

Method: Proposed ZoomIn framework mimics human visual inspection with two stages: scanning images to locate suspicious regions, then performing focused analysis on zoomed-in areas. Created MagniFake dataset of 20,000 real/synthetic images with bounding boxes and explanations using automated VLM-based pipeline.

Result: Achieved 96.39% accuracy with robust generalization. Provides human-understandable explanations grounded in visual evidence.

Conclusion: ZoomIn effectively addresses both detection accuracy and interpretability challenges in AI-generated image forensics through its two-stage approach and comprehensive dataset.

Abstract: The rapid growth of AI-generated imagery has blurred the boundary between real and synthetic content, raising critical concerns for digital integrity. Vision-language models (VLMs) offer interpretability through explanations but often fail to detect subtle artifacts in high-quality synthetic images. We propose ZoomIn, a two-stage forensic framework that improves both accuracy and interpretability. Mimicking human visual inspection, ZoomIn first scans an image to locate suspicious regions and then performs a focused analysis on these zoomed-in areas to deliver a grounded verdict. To support training, we introduce MagniFake, a dataset of 20,000 real and high-quality synthetic images annotated with bounding boxes and forensic explanations, generated through an automated VLM-based pipeline. Our method achieves 96.39% accuracy with robust generalization, while providing human-understandable explanations grounded in visual evidence.

Method: An end-to-end trainable algorithm implemented in a few lines of Python code, demonstrated with stereo vision using 74 images on a 19x15 input window.

Result: The algorithm achieves accurate image registration results with very little training data and training time, excelling in brevity and simplicity.

Conclusion: This simple algorithm serves as an excellent starting point for image registration tasks where training data, training time, or code complexity are constrained.

Abstract: The problem of image registration is finding a transformation that aligns two images, such that the corresponding points are in the same location. This paper introduces a simple, end-to-end trainable algorithm that is implementable in a few lines of Python code. The approach is shown to work with very little training data and training time, while achieving accurate results in some settings. An example application to stereo vision was trained from 74 images on a 19x15 input window. With just a dozen lines of Python code this algorithm excels in brevity and may serve as a good start in related scenarios with limitations to training data, training time or code complexity.

Method: Redesigned and optimized convolutional layers by creating novel ArConv layers, resulting in a new general model architecture with reduced parameter count.

Result: The proposed model with 1.3M parameters achieved 0.9328 accuracy on RfMiD test set, outperforming MobileNetV2 (2.2M parameters) which achieved 0.9266 accuracy under identical conditions.

Conclusion: The ArConv-based model successfully balances computational efficiency and accuracy, making deep learning more accessible for eye disease diagnosis on mobile platforms.

Abstract: Convolutional neural networks are continually evolving, with some efforts aimed at improving accuracy, others at increasing speed, and some at enhancing accessibility. Improving accessibility broadens the application of neural networks across a wider range of tasks, including the detection of eye diseases. Early diagnosis of eye diseases and consulting an ophthalmologist can prevent many vision disorders. Given the importance of this issue, various datasets have been collected from the cornea to facilitate the process of making neural network models. However, most of the methods introduced in the past are computationally complex. In this study, we tried to increase the accessibility of deep neural network models. We did this at the most fundamental level, specifically by redesigning and optimizing the convolutional layers. By doing so, we created a new general model that incorporates our novel convolutional layer named ArConv layers. Thanks to the efficient performance of this new layer, the model has suitable complexity for use in mobile phones and can perform the task of diagnosing the presence of disease with high accuracy. The final model we present contains only 1.3 million parameters. In comparison to the MobileNetV2 model, which has 2.2 million parameters, our model demonstrated better accuracy when trained and evaluated on the RfMiD dataset under identical conditions, achieving an accuracy of 0.9328 versus 0.9266 on the RfMiD test set.

Method: Uses a masked autoregressive framework with decoder-only rectified flow transformer to generate arbitrary 6-DoF views conditioned on reference views, unifying 3D and video modeling.

Result: Sets new state-of-the-art on view synthesis benchmarks, with zero-shot performance outperforming other generative methods in few-view settings and matching per-scene optimization methods in many-view settings.

Conclusion: Kaleido successfully demonstrates that 3D rendering can be effectively treated as a video sequence task, enabling unified modeling and leveraging video data to improve performance while reducing dependency on 3D datasets.

Abstract: We present Kaleido, a family of generative models designed for photorealistic, unified object- and scene-level neural rendering. Kaleido operates on the principle that 3D can be regarded as a specialised sub-domain of video, expressed purely as a sequence-to-sequence image synthesis task. Through a systemic study of scaling sequence-to-sequence generative neural rendering, we introduce key architectural innovations that enable our model to: i) perform generative view synthesis without explicit 3D representations; ii) generate any number of 6-DoF target views conditioned on any number of reference views via a masked autoregressive framework; and iii) seamlessly unify 3D and video modelling within a single decoder-only rectified flow transformer. Within this unified framework, Kaleido leverages large-scale video data for pre-training, which significantly improves spatial consistency and reduces reliance on scarce, camera-labelled 3D datasets – all without any architectural modifications. Kaleido sets a new state-of-the-art on a range of view synthesis benchmarks. Its zero-shot performance substantially outperforms other generative methods in few-view settings, and, for the first time, matches the quality of per-scene optimisation methods in many-view settings.

Method: Built on nnU-Netv2 with semi-supervised mean teacher scheme, domain adaptation module with randomized histogram-based style transfer and contrast-aware network, and continual test-time adaptation strategy.

Result: Outperforms nnU-Netv2 baseline with superior Dice score and Hausdorff Distance, showing strong generalization to unseen domains under low-annotation conditions.

Conclusion: The proposed framework effectively addresses domain generalization challenges in liver MRI segmentation through combined semi-supervised learning, domain adaptation, and test-time adaptation strategies.

Abstract: Accurate liver segmentation from contrast-enhanced MRI is essential for diagnosis, treatment planning, and disease monitoring. However, it remains challenging due to limited annotated data, heterogeneous enhancement protocols, and significant domain shifts across scanners and institutions. Traditional image-to-image translation frameworks have made great progress in domain generalization, but their application is not straightforward. For example, Pix2Pix requires image registration, and cycle-GAN cannot be integrated seamlessly into segmentation pipelines. Meanwhile, these methods are originally used to deal with cross-modality scenarios, and often introduce structural distortions and suffer from unstable training, which may pose drawbacks in our single-modality scenario. To address these challenges, we propose CoSSeg-TTA, a compact segmentation framework for the GED4 (Gd-EOB-DTPA enhanced hepatobiliary phase MRI) modality built upon nnU-Netv2 and enhanced with a semi-supervised mean teacher scheme to exploit large amounts of unlabeled volumes. A domain adaptation module, incorporating a randomized histogram-based style appearance transfer function and a trainable contrast-aware network, enriches domain diversity and mitigates cross-center variability. Furthermore, a continual test-time adaptation strategy is employed to improve robustness during inference. Extensive experiments demonstrate that our framework consistently outperforms the nnU-Netv2 baseline, achieving superior Dice score and Hausdorff Distance while exhibiting strong generalization to unseen domains under low-annotation conditions.

Method: Leverages concept-based explanations to identify and suppress the most influential concept activation vectors, thereby neutralizing patch effects without explicit detection.

Result: Achieves higher robust and clean accuracy than state-of-the-art PatchCleanser on Imagenette with ResNet-50, while maintaining strong performance across varying patch sizes and locations.

Conclusion: Combining interpretability with robustness shows promise, and concept-driven defenses represent a scalable strategy for securing machine learning models against adversarial patch attacks.

Abstract: Adversarial patch attacks pose a practical threat to deep learning models by forcing targeted misclassifications through localized perturbations, often realized in the physical world. Existing defenses typically assume prior knowledge of patch size or location, limiting their applicability. In this work, we propose a patch-agnostic defense that leverages concept-based explanations to identify and suppress the most influential concept activation vectors, thereby neutralizing patch effects without explicit detection. Evaluated on Imagenette with a ResNet-50, our method achieves higher robust and clean accuracy than the state-of-the-art PatchCleanser, while maintaining strong performance across varying patch sizes and locations. Our results highlight the promise of combining interpretability with robustness and suggest concept-driven defenses as a scalable strategy for securing machine learning models against adversarial patch attacks.

Method: Proposed Adapt-STformer with Recurrent Deformable Transformer Encoder (Recurrent-DTE) that uses iterative recurrent mechanism to fuse information from multiple sequential frames.

Result: Achieved up to 17% recall improvement, 36% reduction in sequence extraction time, and 35% lower memory usage compared to second-best baseline on Nordland, Oxford, and NuScenes datasets.

Conclusion: Adapt-STformer successfully addresses the flexibility-efficiency trade-off in Seq-VPR, enabling variable sequence length support with improved performance and reduced computational requirements.

Abstract: Sequential Visual Place Recognition (Seq-VPR) leverages transformers to capture spatio-temporal features effectively; however, existing approaches prioritize performance at the expense of flexibility and efficiency. In practice, a transformer-based Seq-VPR model should be flexible to the number of frames per sequence (seq-length), deliver fast inference, and have low memory usage to meet real-time constraints. To our knowledge, no existing transformer-based Seq-VPR method achieves both flexibility and efficiency. To address this gap, we propose Adapt-STformer, a Seq-VPR method built around our novel Recurrent Deformable Transformer Encoder (Recurrent-DTE), which uses an iterative recurrent mechanism to fuse information from multiple sequential frames. This design naturally supports variable seq-lengths, fast inference, and low memory usage. Experiments on the Nordland, Oxford, and NuScenes datasets show that Adapt-STformer boosts recall by up to 17% while reducing sequence extraction time by 36% and lowering memory usage by 35% compared to the second-best baseline.

Method: Treats input and edited images as first/last video frames, leverages pretrained video models for temporal consistency, introduces temporal reasoning stage with reasoning tokens to imagine plausible editing trajectories, then drops tokens to avoid full video rendering costs.

Result: Outperforms state-of-the-art baselines in both visual fidelity and physical plausibility on the new PBench-Edit benchmark for physically consistent image editing.

Conclusion: ChronoEdit effectively bridges the physical consistency gap in image editing by reformulating it as a video generation problem, with 14B and 2B model variants to be released.

Abstract: Recent advances in large generative models have significantly advanced image editing and in-context image generation, yet a critical gap remains in ensuring physical consistency, where edited objects must remain coherent. This capability is especially vital for world simulation related tasks. In this paper, we present ChronoEdit, a framework that reframes image editing as a video generation problem. First, ChronoEdit treats the input and edited images as the first and last frames of a video, allowing it to leverage large pretrained video generative models that capture not only object appearance but also the implicit physics of motion and interaction through learned temporal consistency. Second, ChronoEdit introduces a temporal reasoning stage that explicitly performs editing at inference time. Under this setting, the target frame is jointly denoised with reasoning tokens to imagine a plausible editing trajectory that constrains the solution space to physically viable transformations. The reasoning tokens are then dropped after a few steps to avoid the high computational cost of rendering a full video. To validate ChronoEdit, we introduce PBench-Edit, a new benchmark of image-prompt pairs for contexts that require physical consistency, and demonstrate that ChronoEdit surpasses state-of-the-art baselines in both visual fidelity and physical plausibility. Code and models for both the 14B and 2B variants of ChronoEdit will be released on the project page: https://research.nvidia.com/labs/toronto-ai/chronoedit

Method: All recordings (RGB video or motion capture) are converted into anonymized SMPL meshes via a harmonized preprocessing pipeline. The dataset supports supervised clinical score prediction (UPDRS gait scores) and unsupervised motion pretext tasks (2D-to-3D keypoint lifting and full-body 3D reconstruction).

Result: Motion encoders consistently outperform handcrafted features. Pretraining on CARE-PD reduces MPJPE from 60.8mm to 7.5mm and boosts PD severity macro-F1 by 17 percentage points.

Conclusion: CARE-PD demonstrates the value of clinically curated, diverse training data for Parkinson’s Disease assessment and is released for non-commercial research.

Abstract: Objective gait assessment in Parkinson’s Disease (PD) is limited by the absence of large, diverse, and clinically annotated motion datasets. We introduce CARE-PD, the largest publicly available archive of 3D mesh gait data for PD, and the first multi-site collection spanning 9 cohorts from 8 clinical centers. All recordings (RGB video or motion capture) are converted into anonymized SMPL meshes via a harmonized preprocessing pipeline. CARE-PD supports two key benchmarks: supervised clinical score prediction (estimating Unified Parkinson’s Disease Rating Scale, UPDRS, gait scores) and unsupervised motion pretext tasks (2D-to-3D keypoint lifting and full-body 3D reconstruction). Clinical prediction is evaluated under four generalization protocols: within-dataset, cross-dataset, leave-one-dataset-out, and multi-dataset in-domain adaptation. To assess clinical relevance, we compare state-of-the-art motion encoders with a traditional gait-feature baseline, finding that encoders consistently outperform handcrafted features. Pretraining on CARE-PD reduces MPJPE (from 60.8mm to 7.5mm) and boosts PD severity macro-F1 by 17 percentage points, underscoring the value of clinically curated, diverse training data. CARE-PD and all benchmark code are released for non-commercial research at https://neurips2025.care-pd.ca/.

Method: GenAR clusters genes into hierarchical groups to capture dependencies, models expression as discrete token generation for raw counts, and uses fused histological and spatial embeddings in a coarse-to-fine decoding process.

Result: Extensive experiments on four Spatial Transcriptomics datasets across different tissue types show GenAR achieves state-of-the-art performance.

Conclusion: GenAR offers a cost-effective alternative for spatial gene expression prediction with potential applications in precision medicine and molecular profiling.

Abstract: Spatial Transcriptomics (ST) offers spatially resolved gene expression but remains costly. Predicting expression directly from widely available Hematoxylin and Eosin (H&E) stained images presents a cost-effective alternative. However, most computational approaches (i) predict each gene independently, overlooking co-expression structure, and (ii) cast the task as continuous regression despite expression being discrete counts. This mismatch can yield biologically implausible outputs and complicate downstream analyses. We introduce GenAR, a multi-scale autoregressive framework that refines predictions from coarse to fine. GenAR clusters genes into hierarchical groups to expose cross-gene dependencies, models expression as codebook-free discrete token generation to directly predict raw counts, and conditions decoding on fused histological and spatial embeddings. From an information-theoretic perspective, the discrete formulation avoids log-induced biases and the coarse-to-fine factorization aligns with a principled conditional decomposition. Extensive experimental results on four Spatial Transcriptomics datasets across different tissue types demonstrate that GenAR achieves state-of-the-art performance, offering potential implications for precision medicine and cost-effective molecular profiling. Code is publicly available at https://github.com/oyjr/genar.

Method: Proposes 3D Rasterization to replace costly rendering with lightweight rasterization of annotated primitives. Enables augmentations like counterfactual recovery maneuvers and cross-agent view synthesis. Introduces Raster-to-Real feature-space alignment to bridge sim-to-real gap.

Result: Achieves state-of-the-art closed-loop robustness and long-tail generalization. Ranks first on four major benchmarks: NAVSIM v1/v2, Waymo Open Dataset Vision-based E2E Driving, and Bench2Drive.

Conclusion: Lightweight rasterization with feature alignment suffices to scale E2E training, offering a practical alternative to photorealistic rendering for end-to-end driving planners.

Abstract: Imitation learning for end-to-end driving trains policies only on expert demonstrations. Once deployed in a closed loop, such policies lack recovery data: small mistakes cannot be corrected and quickly compound into failures. A promising direction is to generate alternative viewpoints and trajectories beyond the logged path. Prior work explores photorealistic digital twins via neural rendering or game engines, but these methods are prohibitively slow and costly, and thus mainly used for evaluation. In this work, we argue that photorealism is unnecessary for training end-to-end planners. What matters is semantic fidelity and scalability: driving depends on geometry and dynamics, not textures or lighting. Motivated by this, we propose 3D Rasterization, which replaces costly rendering with lightweight rasterization of annotated primitives, enabling augmentations such as counterfactual recovery maneuvers and cross-agent view synthesis. To transfer these synthetic views effectively to real-world deployment, we introduce a Raster-to-Real feature-space alignment that bridges the sim-to-real gap. Together, these components form Rasterization Augmented Planning (RAP), a scalable data augmentation pipeline for planning. RAP achieves state-of-the-art closed-loop robustness and long-tail generalization, ranking first on four major benchmarks: NAVSIM v1/v2, Waymo Open Dataset Vision-based E2E Driving, and Bench2Drive. Our results show that lightweight rasterization with feature alignment suffices to scale E2E training, offering a practical alternative to photorealistic rendering. Project page: https://alan-lanfeng.github.io/RAP/.

Method: Two sequentially connected diffusion models: backward diffusion for generating unobserved historical trajectories, and forward diffusion for future trajectory prediction. Includes dual-head parameterization for aleatoric uncertainty estimation and temporally adaptive noise module for dynamic noise scale modulation.

Result: Sets new state-of-the-art performance in momentary trajectory prediction on ETH/UCY and Stanford Drone datasets.

Conclusion: The proposed Diffusion^2 framework effectively addresses momentary trajectory prediction challenges and enhances traffic safety in autonomous driving scenarios.

Abstract: Accurate pedestrian trajectory prediction is crucial for ensuring safety and efficiency in autonomous driving and human-robot interaction scenarios. Earlier studies primarily utilized sufficient observational data to predict future trajectories. However, in real-world scenarios, such as pedestrians suddenly emerging from blind spots, sufficient observational data is often unavailable (i.e. momentary trajectory), making accurate prediction challenging and increasing the risk of traffic accidents. Therefore, advancing research on pedestrian trajectory prediction under extreme scenarios is critical for enhancing traffic safety. In this work, we propose a novel framework termed Diffusion^2, tailored for momentary trajectory prediction. Diffusion^2 consists of two sequentially connected diffusion models: one for backward prediction, which generates unobserved historical trajectories, and the other for forward prediction, which forecasts future trajectories. Given that the generated unobserved historical trajectories may introduce additional noise, we propose a dual-head parameterization mechanism to estimate their aleatoric uncertainty and design a temporally adaptive noise module that dynamically modulates the noise scale in the forward diffusion process. Empirically, Diffusion^2 sets a new state-of-the-art in momentary trajectory prediction on ETH/UCY and Stanford Drone datasets.

Method: Integrates trajectory-guided generation with feature field distillation, allowing edits to be applied interactively without full re-generation. Generates dynamic environments from natural language instructions with object-level controls.

Result: MorphoSim maintains high scene fidelity while enabling controllability and editability. Objects can be directed, recolored, or removed, and scenes can be observed from arbitrary viewpoints.

Conclusion: MorphoSim provides a framework for generating 4D scenes with multi-view consistency and object-level controls, addressing limitations of current text-to-video models and offering practical applications for robotics.

Abstract: World models that support controllable and editable spatiotemporal environments are valuable for robotics, enabling scalable training data, repro ducible evaluation, and flexible task design. While recent text-to-video models generate realistic dynam ics, they are constrained to 2D views and offer limited interaction. We introduce MorphoSim, a language guided framework that generates 4D scenes with multi-view consistency and object-level controls. From natural language instructions, MorphoSim produces dynamic environments where objects can be directed, recolored, or removed, and scenes can be observed from arbitrary viewpoints. The framework integrates trajectory-guided generation with feature field dis tillation, allowing edits to be applied interactively without full re-generation. Experiments show that Mor phoSim maintains high scene fidelity while enabling controllability and editability. The code is available at https://github.com/eric-ai-lab/Morph4D.

Method: Created VLMCountBench - a minimalist benchmark using basic geometric shapes (triangles, circles) and their compositions, with strict variable control to study effects of color, size, and prompt refinement through controlled ablation studies.

Result: VLMs can count reliably when only one shape type is present, but exhibit substantial failures when multiple shape types are combined (compositional counting), showing systematic counting errors in complex scenarios.

Conclusion: Current VLMs have a fundamental empirical limitation in compositional counting, which motivates important future research directions to address this weakness in vision-language understanding.

Abstract: Vision-Language Models (VLMs) have become a central focus of today’s AI community, owing to their impressive abilities gained from training on large-scale vision-language data from the Web. These models have demonstrated strong performance across diverse tasks, including image understanding, video understanding, complex visual reasoning, and embodied AI. Despite these noteworthy successes, a fundamental question remains: Can VLMs count objects correctly? In this paper, we introduce a simple yet effective benchmark, VLMCountBench, designed under a minimalist setting with only basic geometric shapes (e.g., triangles, circles) and their compositions, focusing exclusively on counting tasks without interference from other factors. We adopt strict independent variable control and systematically study the effects of simple properties such as color, size, and prompt refinement in a controlled ablation. Our empirical results reveal that while VLMs can count reliably when only one shape type is present, they exhibit substantial failures when multiple shape types are combined (i.e., compositional counting). This highlights a fundamental empirical limitation of current VLMs and motivates important directions for future research.

Method: Decomposes BFR into three sub-tasks: identity-preserving restoration, high-quality generation, and dynamic fusion. Uses learning-based deformable face registration, texture-guided restoration network, and deep metric learning with informative positive/negative samples.

Result: Extensive experiments on real-world and synthetic datasets show superior performance in both visual fidelity and identity consistency compared to existing methods.

Conclusion: CodeFormer++ effectively maximizes generative priors for high-quality face restoration while preserving identity through its three-component approach.

Abstract: Blind face restoration (BFR) has attracted increasing attention with the rise of generative methods. Most existing approaches integrate generative priors into the restoration pro- cess, aiming to jointly address facial detail generation and identity preservation. However, these methods often suffer from a trade-off between visual quality and identity fidelity, leading to either identity distortion or suboptimal degradation removal. In this paper, we present CodeFormer++, a novel framework that maximizes the utility of generative priors for high-quality face restoration while preserving identity. We decompose BFR into three sub-tasks: (i) identity- preserving face restoration, (ii) high-quality face generation, and (iii) dynamic fusion of identity features with realistic texture details. Our method makes three key contributions: (1) a learning-based deformable face registration module that semantically aligns generated and restored faces; (2) a texture guided restoration network to dynamically extract and transfer the texture of generated face to boost the quality of identity-preserving restored face; and (3) the integration of deep metric learning for BFR with the generation of informative positive and hard negative samples to better fuse identity- preserving and generative features. Extensive experiments on real-world and synthetic datasets demonstrate that, the pro- posed CodeFormer++ achieves superior performance in terms of both visual fidelity and identity consistency.

Method: A.I.R. uses a powerful VLM for deep semantic analysis of complex queries and deploys this analysis in a cost-effective iterative loop that processes only small batches of high-potential frames at a time.

Result: Extensive experiments show A.I.R. outperforms existing frame selection methods, significantly boosts foundation VLM performance, and achieves substantial computational efficiency gains over other VLM-based techniques.

Conclusion: The proposed A.I.R. approach effectively addresses the accuracy-efficiency trade-off in frame selection for VideoQA by combining deep semantic analysis with iterative processing of high-potential frames.

Abstract: Effectively applying Vision-Language Models (VLMs) to Video Question Answering (VideoQA) hinges on selecting a concise yet comprehensive set of frames, as processing entire videos is computationally infeasible. However, current frame selection methods face a critical trade-off: approaches relying on lightweight similarity models, such as CLIP, often fail to capture the nuances of complex queries, resulting in inaccurate similarity scores that cannot reflect the authentic query-frame relevance, which further undermines frame selection. Meanwhile, methods that leverage a VLM for deeper analysis achieve higher accuracy but incur prohibitive computational costs. To address these limitations, we propose A.I.R., a training-free approach for Adaptive, Iterative, and Reasoning-based frame selection. We leverage a powerful VLM to perform deep, semantic analysis on complex queries, and this analysis is deployed within a cost-effective iterative loop that processes only a small batch of the most high-potential frames at a time. Extensive experiments on various VideoQA benchmarks demonstrate that our approach outperforms existing frame selection methods, significantly boosts the performance of the foundation VLM, and achieves substantial gains in computational efficiency over other VLM-based techniques.

Method: Proposes reAR training strategy with token-wise regularization: when predicting next token, the causal transformer also recovers visual embedding of current token and predicts embedding of target token under noisy context. No changes needed to tokenizer, generation order, inference pipeline, or external models.

Result: Substantially improves performance: On ImageNet, reduces gFID from 3.02 to 1.86 and improves IS to 316.9 with standard rasterization tokenizer. With advanced tokenizers, achieves gFID of 1.42 with only 177M parameters, matching performance of larger state-of-the-art diffusion models (675M).

Conclusion: reAR effectively addresses generator-tokenizer inconsistency in visual autoregressive generation through simple regularization, achieving competitive performance with diffusion models while maintaining model simplicity and requiring no architectural changes.

Abstract: Visual autoregressive (AR) generation offers a promising path toward unifying vision and language models, yet its performance remains suboptimal against diffusion models. Prior work often attributes this gap to tokenizer limitations and rasterization ordering. In this work, we identify a core bottleneck from the perspective of generator-tokenizer inconsistency, i.e., the AR-generated tokens may not be well-decoded by the tokenizer. To address this, we propose reAR, a simple training strategy introducing a token-wise regularization objective: when predicting the next token, the causal transformer is also trained to recover the visual embedding of the current token and predict the embedding of the target token under a noisy context. It requires no changes to the tokenizer, generation order, inference pipeline, or external models. Despite its simplicity, reAR substantially improves performance. On ImageNet, it reduces gFID from 3.02 to 1.86 and improves IS to 316.9 using a standard rasterization-based tokenizer. When applied to advanced tokenizers, it achieves a gFID of 1.42 with only 177M parameters, matching the performance with larger state-of-the-art diffusion models (675M).

Method: Automated pipeline converts detection datasets into medical VQA data with CoT reasoning by linking lesion boxes to organ segmentation and structured rationales. Uses Integrated CoT-Curriculum Strategy with Easy (explicit boxes), Medium (implicit localization), and Hard (weakly supervised) stages.

Result: MedCLM attains state-of-the-art performance on several medical VQA benchmarks, demonstrating superior reasoning capabilities.

Conclusion: Provides a scalable framework for developing clinically aligned medical vision-language models that can generate question-answer pairs with step-by-step reasoning.

Abstract: Bridging clinical diagnostic reasoning with AI remains a central challenge in medical imaging. We introduce MedCLM, an automated pipeline that converts detection datasets into large-scale medical visual question answering (VQA) data with Chain-of-Thought (CoT) reasoning by linking lesion boxes to organ segmentation and structured rationales. These contextual signals enable medical vision-language models to generate question-answer pairs with step-by-step reasoning. To utilize this data effectively, we propose an Integrated CoT-Curriculum Strategy composed of an Easy stage with explicit lesion boxes for visual grounding, a Medium stage that encourages implicit localization, and a Hard stage for weakly supervised reasoning. Experimental results demonstrate that MedCLM attains state-of-the-art performance on several medical VQA benchmarks, providing a scalable framework for developing clinically aligned medical vision-language models.

Method: Created VaseVQA-3D dataset with 664 ancient Greek vase 3D models and corresponding QA data, established a complete data construction pipeline, and developed VaseVLM model with domain-adaptive training.

Result: Improved by 12.8% on R@1 metrics and 6.6% on lexical similarity compared to previous state-of-the-art on the VaseVQA-3D dataset, significantly enhancing 3D vase artifact recognition and understanding.

Conclusion: The approach provides new technical pathways for digital heritage preservation research by effectively addressing domain-specific challenges in cultural heritage applications of VLMs.

Abstract: Vision-Language Models (VLMs) have achieved significant progress in multimodal understanding tasks, demonstrating strong capabilities particularly in general tasks such as image captioning and visual reasoning. However, when dealing with specialized cultural heritage domains like 3D vase artifacts, existing models face severe data scarcity issues and insufficient domain knowledge limitations. Due to the lack of targeted training data, current VLMs struggle to effectively handle such culturally significant specialized tasks. To address these challenges, we propose the VaseVQA-3D dataset, which serves as the first 3D visual question answering dataset for ancient Greek pottery analysis, collecting 664 ancient Greek vase 3D models with corresponding question-answer data and establishing a complete data construction pipeline. We further develop the VaseVLM model, enhancing model performance in vase artifact analysis through domain-adaptive training. Experimental results validate the effectiveness of our approach, where we improve by 12.8% on R@1 metrics and by 6.6% on lexical similarity compared with previous state-of-the-art on the VaseVQA-3D dataset, significantly improving the recognition and understanding of 3D vase artifacts, providing new technical pathways for digital heritage preservation research.

Method: Three-part approach: 1) Comprehensive data pipeline for high-quality editing data, 2) Hierarchical model with base model, pattern shifting modules, and consistency enhancement modules, 3) Two-stage training strategy (pattern shifting then consistency enhancement) with separate datasets.

Result: TBStar-Edit outperforms existing general-domain editing models in both objective metrics (VIE Score) and subjective user preference on e-commerce benchmarks.

Conclusion: The proposed specialized approach with data engineering, hierarchical architecture, and staged training effectively addresses e-commerce image editing consistency challenges.

Abstract: Recent advances in image generation and editing technologies have enabled state-of-the-art models to achieve impressive results in general domains. However, when applied to e-commerce scenarios, these general models often encounter consistency limitations. To address this challenge, we introduce TBStar-Edit, an new image editing model tailored for the e-commerce domain. Through rigorous data engineering, model architecture design and training strategy, TBStar-Edit achieves precise and high-fidelity image editing while maintaining the integrity of product appearance and layout. Specifically, for data engineering, we establish a comprehensive data construction pipeline, encompassing data collection, construction, filtering, and augmentation, to acquire high-quality, instruction-following, and strongly consistent editing data to support model training. For model architecture design, we design a hierarchical model framework consisting of a base model, pattern shifting modules, and consistency enhancement modules. For model training, we adopt a two-stage training strategy to enhance the consistency preservation: first stage for editing pattern shifting, and second stage for consistency enhancement. Each stage involves training different modules with separate datasets. Finally, we conduct extensive evaluations of TBStar-Edit on a self-proposed e-commerce benchmark, and the results demonstrate that TBStar-Edit outperforms existing general-domain editing models in both objective metrics (VIE Score) and subjective user preference.

Method: Proposed asynchronous diffusion framework that allocates distinct timesteps to different pixels and dynamically modulates pixel-wise denoising schedules, allowing prompt-related regions to denoise more gradually while leveraging clearer context from less noisy areas.

Result: Extensive experiments show significant improvement in text-to-image alignment across diverse prompts compared to standard synchronous diffusion models.

Conclusion: Asynchronous diffusion models effectively address the alignment limitations of traditional diffusion models by enabling better context utilization through staggered pixel denoising schedules.

Abstract: Diffusion models have achieved impressive results in generating high-quality images. Yet, they often struggle to faithfully align the generated images with the input prompts. This limitation arises from synchronous denoising, where all pixels simultaneously evolve from random noise to clear images. As a result, during generation, the prompt-related regions can only reference the unrelated regions at the same noise level, failing to obtain clear context and ultimately impairing text-to-image alignment. To address this issue, we propose asynchronous diffusion models – a novel framework that allocates distinct timesteps to different pixels and reformulates the pixel-wise denoising process. By dynamically modulating the timestep schedules of individual pixels, prompt-related regions are denoised more gradually than unrelated regions, thereby allowing them to leverage clearer inter-pixel context. Consequently, these prompt-related regions achieve better alignment in the final images. Extensive experiments demonstrate that our asynchronous diffusion models can significantly improve text-to-image alignment across diverse prompts. The code repository for this work is available at https://github.com/hu-zijing/AsynDM.

Method: TAG uses an intermediate sample as projection basis and amplifies tangential components of estimated scores via first-order Taylor expansion to correct sampling trajectory.

Result: The method steers sampling toward higher-probability regions, reducing inconsistencies and enhancing sample quality with minimal computational addition.

Conclusion: TAG provides a plug-and-play, architecture-agnostic guidance approach that improves diffusion sampling fidelity efficiently.

Abstract: Recent diffusion models achieve the state-of-the-art performance in image generation, but often suffer from semantic inconsistencies or hallucinations. While various inference-time guidance methods can enhance generation, they often operate indirectly by relying on external signals or architectural modifications, which introduces additional computational overhead. In this paper, we propose Tangential Amplifying Guidance (TAG), a more efficient and direct guidance method that operates solely on trajectory signals without modifying the underlying diffusion model. TAG leverages an intermediate sample as a projection basis and amplifies the tangential components of the estimated scores with respect to this basis to correct the sampling trajectory. We formalize this guidance process by leveraging a first-order Taylor expansion, which demonstrates that amplifying the tangential component steers the state toward higher-probability regions, thereby reducing inconsistencies and enhancing sample quality. TAG is a plug-and-play, architecture-agnostic module that improves diffusion sampling fidelity with minimal computational addition, offering a new perspective on diffusion guidance.

Method: Uses LLM to generate descriptive texts for user criteria to construct semantic basis, then projects image representations into this conditional feature space using VLM.

Result: Extensive experiments on classification and retrieval tasks demonstrate superiority and generality of CRL over conventional approaches.

Conclusion: CRL effectively extracts task-specific representations without costly fine-tuning, enabling better performance on customized downstream tasks.

Abstract: Conventional representation learning methods learn a universal representation that primarily captures dominant semantics, which may not always align with customized downstream tasks. For instance, in animal habitat analysis, researchers prioritize scene-related features, whereas universal embeddings emphasize categorical semantics, leading to suboptimal results. As a solution, existing approaches resort to supervised fine-tuning, which however incurs high computational and annotation costs. In this paper, we propose Conditional Representation Learning (CRL), aiming to extract representations tailored to arbitrary user-specified criteria. Specifically, we reveal that the semantics of a space are determined by its basis, thereby enabling a set of descriptive words to approximate the basis for a customized feature space. Building upon this insight, given a user-specified criterion, CRL first employs a large language model (LLM) to generate descriptive texts to construct the semantic basis, then projects the image representation into this conditional feature space leveraging a vision-language model (VLM). The conditional representation better captures semantics for the specific criterion, which could be utilized for multiple customized tasks. Extensive experiments on classification and retrieval tasks demonstrate the superiority and generality of the proposed CRL. The code is available at https://github.com/XLearning-SCU/2025-NeurIPS-CRL.

Method: Developed AI Session Recorder to unobtrusively record pathologists’ navigation in standard WSI viewers, converted logs into behavioral commands, created Pathology-CoT dataset through human-in-the-loop review, and built two-stage Pathologist-o3 agent with behavior-guided reasoning.

Result: Achieved 84.5% precision, 100.0% recall, and 75.4% accuracy on gastrointestinal lymph-node metastasis detection, exceeding OpenAI o3 model and generalizing across backbones.

Conclusion: The framework makes agentic pathology practical by converting everyday viewer logs into scalable, expert-validated supervision, establishing a path to human-aligned, upgradeable clinical AI.

Abstract: Diagnosing a whole-slide image is an interactive, multi-stage process involving changes in magnification and movement between fields. Although recent pathology foundation models are strong, practical agentic systems that decide what field to examine next, adjust magnification, and deliver explainable diagnoses are still lacking. The blocker is data: scalable, clinically aligned supervision of expert viewing behavior that is tacit and experience-based, not written in textbooks or online, and therefore absent from large language model training. We introduce the AI Session Recorder, which works with standard WSI viewers to unobtrusively record routine navigation and convert the viewer logs into standardized behavioral commands (inspect or peek at discrete magnifications) and bounding boxes. A lightweight human-in-the-loop review turns AI-drafted rationales into the Pathology-CoT dataset, a form of paired “where to look” and “why it matters” supervision produced at roughly six times lower labeling time. Using this behavioral data, we build Pathologist-o3, a two-stage agent that first proposes regions of interest and then performs behavior-guided reasoning. On gastrointestinal lymph-node metastasis detection, it achieved 84.5% precision, 100.0% recall, and 75.4% accuracy, exceeding the state-of-the-art OpenAI o3 model and generalizing across backbones. To our knowledge, this constitutes one of the first behavior-grounded agentic systems in pathology. Turning everyday viewer logs into scalable, expert-validated supervision, our framework makes agentic pathology practical and establishes a path to human-aligned, upgradeable clinical AI.

Method: Proposes high-frequency sparse enhancement transformer with sparse spatial-spectral attention, two-level spatial-frequency fusion strategy (adaptive frequency channel module and high-frequency resonance mask), and spatial-spectral attention fusion module.

Result: Experiments on four benchmark multimodal datasets with limited labeled data demonstrate superior classification performance, outperforming state-of-the-art methods.

Conclusion: S^2Fin effectively integrates frequency domain learning to model key and sparse detail features, achieving better performance in multimodal remote sensing image classification.

Abstract: Deep learning-based methods have achieved significant success in remote sensing Earth observation data analysis. Numerous feature fusion techniques address multimodal remote sensing image classification by integrating global and local features. However, these techniques often struggle to extract structural and detail features from heterogeneous and redundant multimodal images. With the goal of introducing frequency domain learning to model key and sparse detail features, this paper introduces the spatial-spectral-frequency interaction network (S$^2$Fin), which integrates pairwise fusion modules across the spatial, spectral, and frequency domains. Specifically, we propose a high-frequency sparse enhancement transformer that employs sparse spatial-spectral attention to optimize the parameters of the high-frequency filter. Subsequently, a two-level spatial-frequency fusion strategy is introduced, comprising an adaptive frequency channel module that fuses low-frequency structures with enhanced high-frequency details, and a high-frequency resonance mask that emphasizes sharp edges via phase similarity. In addition, a spatial-spectral attention fusion module further enhances feature extraction at intermediate layers of the network. Experiments on four benchmark multimodal datasets with limited labeled data demonstrate that S$^2$Fin performs superior classification, outperforming state-of-the-art methods. The code is available at https://github.com/HaoLiu-XDU/SSFin.

Method: Compared four top-performing superpixel segmentation methods with SLIC, trained classifiers using PyCaret AutoML library, and applied classifier fusion (ensemble) approaches.

Result: Initial results showed minimal performance variation among segmentation methods with individual classifiers, but ensemble methods demonstrated noticeable improvements in balanced accuracy.

Conclusion: Both segmentation method selection and classifier combination through ensemble approaches are crucial for optimizing deforestation detection performance in remote sensing applications.

Abstract: Image segmentation is a crucial step in various visual applications, including environmental monitoring through remote sensing. In the context of the ForestEyes project, which combines citizen science and machine learning to detect deforestation in tropical forests, image segments are used for labeling by volunteers and subsequent model training. Traditionally, the Simple Linear Iterative Clustering (SLIC) algorithm is adopted as the segmentation method. However, recent studies have indicated that other superpixel-based methods outperform SLIC in remote sensing image segmentation, and might suggest that they are more suitable for the task of detecting deforested areas. In this sense, this study investigated the impact of the four best segmentation methods, together with SLIC, on the training of classifiers for the target application. Initially, the results showed little variation in performance among segmentation methods, even when selecting the top five classifiers using the PyCaret AutoML library. However, by applying a classifier fusion approach (ensemble of classifiers), noticeable improvements in balanced accuracy were observed, highlighting the importance of both the choice of segmentation method and the combination of machine learning-based models for deforestation detection tasks.

Method: Created EduPersona benchmark with 1,308 authentic classroom dialogues expanded to 128k turns through persona stylization. Evaluates three tasks: basic coherence, student realism, and long-term persona consistency. Tests three LLMs and their persona-fine-tuned variants.

Result: Persona-fine-tuned models showed significant improvements: TASK1 +33.6%, TASK2 +30.6%, TASK3 +14.9%, demonstrating dataset effectiveness and revealing heterogeneous difficulty of persona modeling.

Conclusion: EduPersona provides the first classroom benchmark for subjective abilities, establishes a decoupled research paradigm, and will be open-sourced to advance trustworthy AI in education.

Abstract: As large language models are increasingly integrated into education, virtual student agents are becoming vital for classroom simulation and teacher training. Yet their classroom-oriented subjective abilities remain largely unassessed, limiting understanding of model boundaries and hindering trustworthy deployment. We present EduPersona, a large-scale benchmark spanning two languages, three subjects, and ten persona types based on the Big Five theory. The dataset contains 1,308 authentic classroom dialogue rounds, corresponding to 12,814 teacher-student Q&A turns, and is further expanded through persona stylization into roughly 10 times larger scale (128k turns), providing a solid foundation for evaluation. Building on this resource, we decompose hard-to-quantify subjective performance into three progressive tasks: TASK1 basic coherence (whether behavior, emotion, expression, and voice align with classroom context), TASK2 student realism, and TASK3 long-term persona consistency, thereby establishing an evaluation framework grounded in educational theory and research value. We conduct systematic experiments on three representative LLMs, comparing their original versions with ten persona-fine-tuned variants trained on EduPersona. Results show consistent and significant average improvements across all tasks: TASK1 +33.6%, TASK2 +30.6%, and TASK3 +14.9%. These improvements highlight the dataset’s effectiveness and research value, while also revealing the heterogeneous difficulty of persona modeling. In summary, EduPersona delivers the first classroom benchmark centered on subjective abilities, establishes a decoupled and verifiable research paradigm, and we will open-source both the dataset and the framework to support the broader research community in advancing trustworthy and human-like AI for education.

Method: MoME processes walking cycles in four stages of movement complexity using lightweight expert models and task-specific gating modules to adaptively weight experts across traits and stages.

Result: Outperforms state-of-the-art gait analysis models with 37.47% weighted F1 score at run level and 44.6% at subject level. Integration of auxiliary tasks (identity, gender, BMI) further improves psychological trait estimation.

Conclusion: Demonstrates viability of multi-task gait-based learning for psychological trait estimation and provides foundation for movement-informed psychological inference research.

Abstract: Gait encodes rich biometric and behavioural information, yet leveraging the manner of walking to infer psychological traits remains a challenging and underexplored problem. We introduce a hierarchical Multi-Stage Mixture of Movement Experts (MoME) architecture for multi-task prediction of psychological attributes from gait sequences represented as 2D poses. MoME processes the walking cycle in four stages of movement complexity, employing lightweight expert models to extract spatio-temporal features and task-specific gating modules to adaptively weight experts across traits and stages. Evaluated on the PsyMo benchmark covering 17 psychological traits, our method outperforms state-of-the-art gait analysis models, achieving a 37.47% weighted F1 score at the run level and 44.6% at the subject level. Our experiments show that integrating auxiliary tasks such as identity recognition, gender prediction, and BMI estimation further improves psychological trait estimation. Our findings demonstrate the viability of multi-task gait-based learning for psychological trait estimation and provide a foundation for future research on movement-informed psychological inference.

Method: Two key components: Token-wise Value Adaptation (ToVA) - a training method that adapts only the value projection in cross-attention instead of modifying key projection; and Latent Optimization for Disentangled Attention (LODA) - optimizes input latent during inference to alleviate attention entanglement.

Result: ConceptSplit achieves robust multi-concept personalization and mitigates unintended concept interference, as demonstrated through extensive qualitative and quantitative experiments.

Conclusion: The proposed framework effectively addresses concept mixing in multi-concept personalization by focusing on value projection adaptation during training and latent optimization during inference.

Abstract: In recent years, multi-concept personalization for text-to-image (T2I) diffusion models to represent several subjects in an image has gained much more attention. The main challenge of this task is “concept mixing”, where multiple learned concepts interfere or blend undesirably in the output image. To address this issue, in this paper, we present ConceptSplit, a novel framework to split the individual concepts through training and inference. Our framework comprises two key components. First, we introduce Token-wise Value Adaptation (ToVA), a merging-free training method that focuses exclusively on adapting the value projection in cross-attention. Based on our empirical analysis, we found that modifying the key projection, a common approach in existing methods, can disrupt the attention mechanism and lead to concept mixing. Second, we propose Latent Optimization for Disentangled Attention (LODA), which alleviates attention entanglement during inference by optimizing the input latent. Through extensive qualitative and quantitative experiments, we demonstrate that ConceptSplit achieves robust multi-concept personalization, mitigating unintended concept interference. Code is available at https://github.com/KU-VGI/ConceptSplit

Method: Integrates foundation-scale 3D segmentation backbone with fine-tuning, co-training using cross pseudo supervision to leverage unlabeled volumes, and standardized preprocessing pipeline without requiring spatial registration.

Result: The model learns to generalize across MRI phases and vendors, demonstrating robust segmentation performance in both labeled and unlabeled domains.

Conclusion: The approach shows effectiveness for label-efficient liver segmentation in multi-phase, multi-vendor MRI and highlights the potential of combining foundation model adaptation with co-training for real-world clinical imaging tasks.

Abstract: Accurate liver segmentation in multi-phase MRI is vital for liver fibrosis assessment, yet labeled data is often scarce and unevenly distributed across imaging modalities and vendor systems. We propose a label-efficient segmentation approach that promotes cross-modality generalization under real-world conditions, where GED4 hepatobiliary-phase annotations are limited, non-contrast sequences (T1WI, T2WI, DWI) are unlabeled, and spatial misalignment and missing phases are common. Our method integrates a foundation-scale 3D segmentation backbone adapted via fine-tuning, co-training with cross pseudo supervision to leverage unlabeled volumes, and a standardized preprocessing pipeline. Without requiring spatial registration, the model learns to generalize across MRI phases and vendors, demonstrating robust segmentation performance in both labeled and unlabeled domains. Our results exhibit the effectiveness of our proposed label-efficient baseline for liver segmentation in multi-phase, multi-vendor MRI and highlight the potential of combining foundation model adaptation with co-training for real-world clinical imaging tasks.

Method: Built on ID-consistent face foundation model with compositional design featuring expression cross-attention module guided by FLAME blendshape parameters, trained on diverse image/video data with expressive variation, plus pluggable Reference Adapter for real image editing.

Result: Model generalizes beyond basic emotions to subtle micro-expressions and expressive transitions, outperforming existing methods in tailored and identity-consistent expression generation.

Conclusion: The framework successfully achieves faithful reimagining of subjects under any facial expression while maintaining identity consistency and enabling precise control.

Abstract: Human-centric generative models designed for AI-driven storytelling must bring together two core capabilities: identity consistency and precise control over human performance. While recent diffusion-based approaches have made significant progress in maintaining facial identity, achieving fine-grained expression control without compromising identity remains challenging. In this work, we present a diffusion-based framework that faithfully reimagines any subject under any particular facial expression. Building on an ID-consistent face foundation model, we adopt a compositional design featuring an expression cross-attention module guided by FLAME blendshape parameters for explicit control. Trained on a diverse mixture of image and video data rich in expressive variation, our adapter generalizes beyond basic emotions to subtle micro-expressions and expressive transitions, overlooked by prior works. In addition, a pluggable Reference Adapter enables expression editing in real images by transferring the appearance from a reference frame during synthesis. Extensive quantitative and qualitative evaluations show that our model outperforms existing methods in tailored and identity-consistent expression generation. Code and models can be found at https://github.com/foivospar/Arc2Face.

Method: Designs a highly discriminative object feature encoder with contrastive pretraining that decouples object representation learning from scene graph prediction, and effectively combines geometric and semantic features for relationship prediction.

Result: Significantly outperforms previous state-of-the-art methods on the 3DSSG dataset across all evaluation metrics, with substantial performance improvements when plugging the pretrained encoder into existing frameworks.

Conclusion: Object feature quality is critical for 3D scene graph accuracy, and the proposed contrastive pretraining approach with enhanced feature integration effectively addresses previous limitations.

Abstract: 3D Semantic Scene Graph Prediction aims to detect objects and their semantic relationships in 3D scenes, and has emerged as a crucial technology for robotics and AR/VR applications. While previous research has addressed dataset limitations and explored various approaches including Open-Vocabulary settings, they frequently fail to optimize the representational capacity of object and relationship features, showing excessive reliance on Graph Neural Networks despite insufficient discriminative capability. In this work, we demonstrate through extensive analysis that the quality of object features plays a critical role in determining overall scene graph accuracy. To address this challenge, we design a highly discriminative object feature encoder and employ a contrastive pretraining strategy that decouples object representation learning from the scene graph prediction. This design not only enhances object classification accuracy but also yields direct improvements in relationship prediction. Notably, when plugging in our pretrained encoder into existing frameworks, we observe substantial performance improvements across all evaluation metrics. Additionally, whereas existing approaches have not fully exploited the integration of relationship information, we effectively combine both geometric and semantic features to achieve superior relationship prediction. Comprehensive experiments on the 3DSSG dataset demonstrate that our approach significantly outperforms previous state-of-the-art methods. Our code is publicly available at https://github.com/VisualScienceLab-KHU/OCRL-3DSSG-Codes.

Method: Evaluated four state-of-the-art MDE methods (Depth Anything V2, ML Depth Pro, ZoeDepth, Metric3D) and geometric baseline on 93 camera trap images with ground truth distances from calibrated ChARUCO patterns. Compared median vs mean depth extraction approaches.

Result: Depth Anything V2 achieved best performance: 0.454m MAE and 0.962 correlation. ZoeDepth performed worst with 3.087m MAE. Median-based depth extraction consistently outperformed mean-based approaches. ZoeDepth fastest (0.17s/image) but least accurate; Depth Anything V2 balanced accuracy and speed (0.22s/image).

Conclusion: Establishes performance baselines for wildlife applications and provides practical guidance for implementing depth estimation in conservation monitoring systems, with Depth Anything V2 recommended for optimal balance of accuracy and speed.

Abstract: Camera traps are widely used for wildlife monitoring, but extracting accurate distance measurements from monocular images remains challenging due to the lack of depth information. While monocular depth estimation (MDE) methods have advanced significantly, their performance in natural wildlife environments has not been systematically evaluated. This work introduces the first benchmark for monocular metric depth estimation in wildlife monitoring conditions. We evaluate four state-of-the-art MDE methods (Depth Anything V2, ML Depth Pro, ZoeDepth, and Metric3D) alongside a geometric baseline on 93 camera trap images with ground truth distances obtained using calibrated ChARUCO patterns. Our results demonstrate that Depth Anything V2 achieves the best overall performance with a mean absolute error of 0.454m and correlation of 0.962, while methods like ZoeDepth show significant degradation in outdoor natural environments (MAE: 3.087m). We find that median-based depth extraction consistently outperforms mean-based approaches across all deep learning methods. Additionally, we analyze computational efficiency, with ZoeDepth being fastest (0.17s per image) but least accurate, while Depth Anything V2 provides an optimal balance of accuracy and speed (0.22s per image). This benchmark establishes performance baselines for wildlife applications and provides practical guidance for implementing depth estimation in conservation monitoring systems.

Method: Proposed Anomaly-Aware YOLO (AA-YOLO) that integrates a statistical anomaly detection test into the YOLO detection head, treating small targets as unexpected patterns against the background to control false alarms.

Result: Achieves competitive performance on IRSTD benchmarks with remarkable robustness in limited training data, noise, and domain shift scenarios. Successfully applied across various YOLO backbones including lightweight models and instance segmentation YOLO.

Conclusion: AA-YOLO provides a generic, versatile solution for real-world IRSTD deployments with constrained resources, offering low false alarm rates and good performance across different scenarios and model architectures.

Abstract: Infrared Small Target Detection (IRSTD) is a challenging task in defense applications, where complex backgrounds and tiny target sizes often result in numerous false alarms using conventional object detectors. To overcome this limitation, we propose Anomaly-Aware YOLO (AA-YOLO), which integrates a statistical anomaly detection test into its detection head. By treating small targets as unexpected patterns against the background, AA-YOLO effectively controls the false alarm rate. Our approach not only achieves competitive performance on several IRSTD benchmarks, but also demonstrates remarkable robustness in scenarios with limited training data, noise, and domain shifts. Furthermore, since only the detection head is modified, our design is highly generic and has been successfully applied across various YOLO backbones, including lightweight models. It also provides promising results when integrated into an instance segmentation YOLO. This versatility makes AA-YOLO an attractive solution for real-world deployments where resources are constrained. The code will be publicly released.

Method: Two-stream framework using 133 COCO WholeBody keypoints from CANDOR corpus: spatial transformer for body configurations and multi-scale temporal transformer for hierarchical motion modeling. Compared pre-trained vs from-scratch training, velocity features, and feature-level fusion.

Result: Spatial transformer achieved 95.74% accuracy, temporal transformer 93.90%. Feature-level fusion boosted performance to 98.03%. Domain-specific training outperformed transfer learning, and spatial features were more discriminative than temporal dynamics.

Conclusion: Transformers are effective for person identification in natural interactions, with spatial and temporal information being complementary. Provides foundation for future multimodal and cross-cultural studies.

Abstract: This paper investigates the performance of transformer-based architectures for person identification in natural, face-to-face conversation scenario. We implement and evaluate a two-stream framework that separately models spatial configurations and temporal motion patterns of 133 COCO WholeBody keypoints, extracted from a subset of the CANDOR conversational corpus. Our experiments compare pre-trained and from-scratch training, investigate the use of velocity features, and introduce a multi-scale temporal transformer for hierarchical motion modeling. Results demonstrate that domain-specific training significantly outperforms transfer learning, and that spatial configurations carry more discriminative information than temporal dynamics. The spatial transformer achieves 95.74% accuracy, while the multi-scale temporal transformer achieves 93.90%. Feature-level fusion pushes performance to 98.03%, confirming that postural and dynamic information are complementary. These findings highlight the potential of transformer architectures for person identification in natural interactions and provide insights for future multimodal and cross-cultural studies.

Method: Analyzed visual information flow through language models in popular MLMs (LLaVA-OneVision, Qwen2.5-VL, Llama-3-LLaVA-NeXT), studied key-value token processing, tested zero-shot perception tasks, and experimented with text prefix additions to control visual information.

Result: Found that image value tokens contain sufficient information for perception tasks, but language model augmentation reduces visual information compared to original visual encoders. Later layer key tokens contain artifacts that harm perception. Adding text prefixes improves perception, and 33.3% of Art Style questions in BLINK benchmark show unused perception information in language models.

Conclusion: Visual key-value tokens play crucial roles in MLM perception capabilities, with artifacts in later layers reducing performance. Better control of visual information could significantly improve perception, suggesting new training directions for visual encoders and language model components.

Abstract: Despite interpretability work analyzing VIT encoders and transformer activations, we don’t yet understand why Multimodal Language Models (MLMs) struggle on perception-heavy tasks. We offer an under-studied perspective by examining how popular MLMs (LLaVA-OneVision, Qwen2.5-VL, and Llama-3-LLaVA-NeXT) process their visual key-value tokens. We first study the flow of visual information through the language model, finding that image value tokens encode sufficient information to perform several perception-heavy tasks zero-shot: segmentation, semantic correspondence, temporal correspondence, and referring expression detection. We find that while the language model does augment the visual information received from the projection of input visual encodings-which we reveal correlates with overall MLM perception capability-it contains less visual information on several tasks than the equivalent visual encoder (SigLIP) that has not undergone MLM finetuning. Further, we find that the visual information corresponding to input-agnostic image key tokens in later layers of language models contains artifacts which reduce perception capability of the overall MLM. Next, we discuss controlling visual information in the language model, showing that adding a text prefix to the image input improves perception capabilities of visual representations. Finally, we reveal that if language models were able to better control their visual information, their perception would significantly improve; e.g., in 33.3% of Art Style questions in the BLINK benchmark, perception information present in the language model is not surfaced to the output! Our findings reveal insights into the role of key-value tokens in multimodal systems, paving the way for deeper mechanistic interpretability of MLMs and suggesting new directions for training their visual encoder and language model components.

Method: Uses progressive online densification to gradually enhance 3D Gaussian representation, and anisotropy-aware sampling with spatio-temporal fusion to adaptively assign receptive fields.

Result: Achieves state-of-the-art performance with 14.3% mIoU improvement over previous best method.

Conclusion: PG-Occ effectively balances computational efficiency and fine-grained scene detail capture for open-vocabulary 3D occupancy prediction.

Abstract: The 3D occupancy prediction task has witnessed remarkable progress in recent years, playing a crucial role in vision-based autonomous driving systems. While traditional methods are limited to fixed semantic categories, recent approaches have moved towards predicting text-aligned features to enable open-vocabulary text queries in real-world scenes. However, there exists a trade-off in text-aligned scene modeling: sparse Gaussian representation struggles to capture small objects in the scene, while dense representation incurs significant computational overhead. To address these limitations, we present PG-Occ, an innovative Progressive Gaussian Transformer Framework that enables open-vocabulary 3D occupancy prediction. Our framework employs progressive online densification, a feed-forward strategy that gradually enhances the 3D Gaussian representation to capture fine-grained scene details. By iteratively enhancing the representation, the framework achieves increasingly precise and detailed scene understanding. Another key contribution is the introduction of an anisotropy-aware sampling strategy with spatio-temporal fusion, which adaptively assigns receptive fields to Gaussians at different scales and stages, enabling more effective feature aggregation and richer scene information capture. Through extensive evaluations, we demonstrate that PG-Occ achieves state-of-the-art performance with a relative 14.3% mIoU improvement over the previous best performing method. Code and pretrained models will be released upon publication on our project page: https://yanchi-3dv.github.io/PG-Occ

Method: The method consists of: 1) A class-domain-wise data generation pipeline that generates unseen-class data guided by hierarchical semantic trees and domain information from seen-class data, 2) A distribution alignment algorithm that estimates and maximizes posterior probability using the generated data.

Result: Extensive experiments on 11 datasets show the method outperforms baseline approaches by up to 14%, demonstrating its effectiveness and superiority in open-vocabulary learning.

Conclusion: The paper theoretically and empirically shows that generating unseen-class data enables effective distribution estimation in open environments, with the proposed method providing significant performance improvements over existing approaches.

Abstract: Open-vocabulary learning requires modeling the data distribution in open environments, which consists of both seen-class and unseen-class data. Existing methods estimate the distribution in open environments using seen-class data, where the absence of unseen classes makes the estimation error inherently unidentifiable. Intuitively, learning beyond the seen classes is crucial for distribution estimation to bound the estimation error. We theoretically demonstrate that the distribution can be effectively estimated by generating unseen-class data, through which the estimation error is upper-bounded. Building on this theoretical insight, we propose a novel open-vocabulary learning method, which generates unseen-class data for estimating the distribution in open environments. The method consists of a class-domain-wise data generation pipeline and a distribution alignment algorithm. The data generation pipeline generates unseen-class data under the guidance of a hierarchical semantic tree and domain information inferred from the seen-class data, facilitating accurate distribution estimation. With the generated data, the distribution alignment algorithm estimates and maximizes the posterior probability to enhance generalization in open-vocabulary learning. Extensive experiments on $11$ datasets demonstrate that our method outperforms baseline approaches by up to $14%$, highlighting its effectiveness and superiority.

Method: Participants used foundation models with linear probing, metric learning with triplet loss, and FL aggregation schemes (FedAvg, FedMedian, FedSAM) to classify inflammation stages in appendicitis videos. Performance was evaluated using F1-score and Expected Cost.

Result: Generalization to unseen centers was limited, but all teams improved after fine-tuning in the adaptation task. ViViT-based submission achieved strongest performance. Challenges included sensitivity to class imbalance and hyperparameter tuning difficulties.

Conclusion: Establishes first FL benchmark for surgical video classification, highlighting trade-offs between local personalization and global robustness, and emphasizing importance of architecture choice, preprocessing, and loss design for future clinical AI development.

Abstract: Purpose: The FedSurg challenge was designed to benchmark the state of the art in federated learning for surgical video classification. Its goal was to assess how well current methods generalize to unseen clinical centers and adapt through local fine-tuning while enabling collaborative model development without sharing patient data. Methods: Participants developed strategies to classify inflammation stages in appendicitis using a preliminary version of the multi-center Appendix300 video dataset. The challenge evaluated two tasks: generalization to an unseen center and center-specific adaptation after fine-tuning. Submitted approaches included foundation models with linear probing, metric learning with triplet loss, and various FL aggregation schemes (FedAvg, FedMedian, FedSAM). Performance was assessed using F1-score and Expected Cost, with ranking robustness evaluated via bootstrapping and statistical testing. Results: In the generalization task, performance across centers was limited. In the adaptation task, all teams improved after fine-tuning, though ranking stability was low. The ViViT-based submission achieved the strongest overall performance. The challenge highlighted limitations in generalization, sensitivity to class imbalance, and difficulties in hyperparameter tuning in decentralized training, while spatiotemporal modeling and context-aware preprocessing emerged as promising strategies. Conclusion: The FedSurg Challenge establishes the first benchmark for evaluating FL strategies in surgical video classification. Findings highlight the trade-off between local personalization and global robustness, and underscore the importance of architecture choice, preprocessing, and loss design. This benchmarking offers a reference point for future development of imbalance-aware, adaptive, and robust FL methods in clinical surgical AI.

Method: Two-robot system combining scanner-equipped robot and tray-handling robot with coordinated motion planning, parameterized scanning space, optimized trajectory planning, and waypoint distribution for comprehensive coverage.

Result: Achieves significantly lower Chamfer Distance and higher F-score compared to baseline methods, offering superior geometric accuracy and improved digitization efficiency.

Conclusion: The automated system provides high-fidelity 3D scanning with reduced reliance on expert operators, making cultural heritage digitization more efficient and accessible.

Abstract: High-fidelity 3D scanning is essential for preserving cultural heritage artefacts, supporting documentation, analysis, and long-term conservation. However, conventional methods typically require specialized expertise and manual intervention to maintain optimal scanning conditions and coverage. We present an automated two-robot scanning system that eliminates the need for handheld or semi-automatic workflows by combining coordinated robotic manipulation with high-resolution 3D scanning. Our system parameterizes the scanning space into distinct regions, enabling coordinated motion planning between a scanner-equipped robot and a tray-handling robot. Optimized trajectory planning and waypoint distribution ensure comprehensive surface coverage, minimize occlusions, and balance reconstruction accuracy with system efficiency. Experimental results show that our approach achieves significantly lower Chamfer Distance and higher F-score compared to baseline methods, offering superior geometric accuracy, improved digitization efficiency, and reduced reliance on expert operators.

Method: Systematic comparison of large-scale CNNs (ResNet, EfficientNet, CLIP-ResNet) with ViT-based foundation models (CLIP-ViT variants and DINO) across different data size settings including few-shot scenarios. Analysis of model refinement performance in both large and small downstream-data scenarios.

Result: ViTs outperform CNNs during refinement in large downstream-data scenarios, similar to training from scratch. In small data scenarios, CNNs match ViT performance due to their inductive bias and smaller capacity. ViTs exhibit stronger generalization in cross-domain evaluation.

Conclusion: Careful selection of model architectures for refinement is crucial. Future research should focus on developing hybrid architectures that balance local and global representations for geometric estimation tasks.

Abstract: Vision-transformers (ViTs) and large-scale convolution-neural-networks (CNNs) have reshaped computer vision through pretrained feature representations that enable strong transfer learning for diverse tasks. However, their efficiency as backbone architectures for geometric estimation tasks involving image deformations in low-data regimes remains an open question. This work considers two such tasks: 1) estimating 2D rigid transformations between pairs of images and 2) predicting the fundamental matrix for stereo image pairs, an important problem in various applications, such as autonomous mobility, robotics, and 3D scene reconstruction. Addressing this intriguing question, this work systematically compares large-scale CNNs (ResNet, EfficientNet, CLIP-ResNet) with ViT-based foundation models (CLIP-ViT variants and DINO) in various data size settings, including few-shot scenarios. These pretrained models are optimized for classification or contrastive learning, encouraging them to focus mostly on high-level semantics. The considered tasks require balancing local and global features differently, challenging the straightforward adoption of these models as the backbone. Empirical comparative analysis shows that, similar to training from scratch, ViTs outperform CNNs during refinement in large downstream-data scenarios. However, in small data scenarios, the inductive bias and smaller capacity of CNNs improve their performance, allowing them to match that of a ViT. Moreover, ViTs exhibit stronger generalization in cross-domain evaluation where the data distribution changes. These results emphasize the importance of carefully selecting model architectures for refinement, motivating future research towards hybrid architectures that balance local and global representations.

Method: Leverages Diffusion Transformer (DiT) adapted for image-conditioned VTO, exploring multiple configurations (in-context token concatenation, channel concatenation, ControlNet integration). Trained on expanded dataset with varied backgrounds, unstructured references, and non-garment categories.

Result: Surpasses state-of-the-art methods on VITON-HD with superior detail preservation and robustness. Outperforms models with VTA and image editing capabilities across thousands of product categories without additional condition encoders.

Conclusion: DiT-VTON successfully redefines VTO as Virtual Try-All, offering versatile capabilities for diverse product categories and advanced image editing functionalities while demonstrating the benefits of data scaling for adaptability.

Abstract: The rapid growth of e-commerce has intensified the demand for Virtual Try-On (VTO) technologies, enabling customers to realistically visualize products overlaid on their own images. Despite recent advances, existing VTO models face challenges with fine-grained detail preservation, robustness to real-world imagery, efficient sampling, image editing capabilities, and generalization across diverse product categories. In this paper, we present DiT-VTON, a novel VTO framework that leverages a Diffusion Transformer (DiT), renowned for its performance on text-conditioned image generation, adapted here for the image-conditioned VTO task. We systematically explore multiple DiT configurations, including in-context token concatenation, channel concatenation, and ControlNet integration, to determine the best setup for VTO image conditioning. To enhance robustness, we train the model on an expanded dataset encompassing varied backgrounds, unstructured references, and non-garment categories, demonstrating the benefits of data scaling for VTO adaptability. DiT-VTON also redefines the VTO task beyond garment try-on, offering a versatile Virtual Try-All (VTA) solution capable of handling a wide range of product categories and supporting advanced image editing functionalities such as pose preservation, localized editing, texture transfer, and object-level customization. Experimental results show that our model surpasses state-of-the-art methods on VITON-HD, achieving superior detail preservation and robustness without reliance on additional condition encoders. It also outperforms models with VTA and image editing capabilities on a diverse dataset spanning thousands of product categories.

Method: Couples geometry-driven neural rendering with diffusion-based view enhancement to synthesize arbitrary egocentric viewpoints from wall-mounted fixed-camera video.

Result: Successfully reconstructs person-specific visual fields and arbitrary viewpoints with high visual quality and fidelity across multi-site surgical cases and controlled studies.

Conclusion: Transforms existing OR camera infrastructure into navigable dynamic 3D records, establishing a foundation for immersive surgical data science that enables surgical practice to be visualized and analyzed from every angle.

Abstract: Observing surgical practice has historically relied on fixed vantage points or recollections, leaving the egocentric visual perspectives that guide clinical decisions undocumented. Fixed-camera video can capture surgical workflows at the room-scale, but cannot reconstruct what each team member actually saw. Thus, these videos only provide limited insights into how decisions that affect surgical safety, training, and workflow optimization are made. Here we introduce EgoSurg, the first framework to reconstruct the dynamic, egocentric replays for any operating room (OR) staff directly from wall-mounted fixed-camera video, and thus, without intervention to clinical workflow. EgoSurg couples geometry-driven neural rendering with diffusion-based view enhancement, enabling high-visual fidelity synthesis of arbitrary and egocentric viewpoints at any moment. In evaluation across multi-site surgical cases and controlled studies, EgoSurg reconstructs person-specific visual fields and arbitrary viewpoints with high visual quality and fidelity. By transforming existing OR camera infrastructure into a navigable dynamic 3D record, EgoSurg establishes a new foundation for immersive surgical data science, enabling surgical practice to be visualized, experienced, and analyzed from every angle.

Method: Uses two key modules: (1) Reciprocal Flow Rectifier for optical-flow correction to stabilize avatar fitting and ensure temporal coherence, and (2) Non-Linear Deformer that decomposes Gaussian maps into view-pose-invariant and view-pose-specific components for adaptive garment deformations.

Result: Extensive experiments show AvatarVTON achieves high fidelity, diversity, and dynamic garment realism. The framework establishes a benchmark for 4D virtual try-on with fair qualitative and quantitative comparisons.

Conclusion: AvatarVTON is well-suited for AR/VR, gaming, and digital-human applications, demonstrating superior performance in generating realistic 4D virtual try-on experiences from single garment images.

Abstract: We propose AvatarVTON, the first 4D virtual try-on framework that generates realistic try-on results from a single in-shop garment image, enabling free pose control, novel-view rendering, and diverse garment choices. Unlike existing methods, AvatarVTON supports dynamic garment interactions under single-view supervision, without relying on multi-view garment captures or physics priors. The framework consists of two key modules: (1) a Reciprocal Flow Rectifier, a prior-free optical-flow correction strategy that stabilizes avatar fitting and ensures temporal coherence; and (2) a Non-Linear Deformer, which decomposes Gaussian maps into view-pose-invariant and view-pose-specific components, enabling adaptive, non-linear garment deformations. To establish a benchmark for 4D virtual try-on, we extend existing baselines with unified modules for fair qualitative and quantitative comparisons. Extensive experiments show that AvatarVTON achieves high fidelity, diversity, and dynamic garment realism, making it well-suited for AR/VR, gaming, and digital-human applications.

Method: A fully 3D Flow Matching framework that transforms Gaussian noise into synthetic CT images by integrating a learned velocity field, conditioned on features from input MRI or CBCT using a lightweight 3D encoder. Separate models were trained for MRI→sCT and CBCT→sCT across abdomen, head and neck, and thorax regions.

Result: The method accurately reconstructs global anatomical structures but has limited preservation of fine details due to low training resolution constraints from memory and runtime limitations.

Conclusion: Future work will explore patch-based training and latent-space flow models to improve resolution and local structural fidelity for better synthetic CT generation.

Abstract: Generating synthetic CT (sCT) from MRI or CBCT plays a crucial role in enabling MRI-only and CBCT-based adaptive radiotherapy, improving treatment precision while reducing patient radiation exposure. To address this task, we adopt a fully 3D Flow Matching (FM) framework, motivated by recent work demonstrating FM’s efficiency in producing high-quality images. In our approach, a Gaussian noise volume is transformed into an sCT image by integrating a learned FM velocity field, conditioned on features extracted from the input MRI or CBCT using a lightweight 3D encoder. We evaluated the method on the SynthRAD2025 Challenge benchmark, training separate models for MRI $\rightarrow$ sCT and CBCT $\rightarrow$ sCT across three anatomical regions: abdomen, head and neck, and thorax. Validation and testing were performed through the challenge submission system. The results indicate that the method accurately reconstructs global anatomical structures; however, preservation of fine details was limited, primarily due to the relatively low training resolution imposed by memory and runtime constraints. Future work will explore patch-based training and latent-space flow models to improve resolution and local structural fidelity.

Method: Proposes Automatic Truncated Backpropagation Through Time (AT-BPTT) with three key components: probabilistic stage-aware timestep selection, adaptive window sizing based on gradient variation, and low-rank Hessian approximation for computational efficiency.

Result: AT-BPTT achieves state-of-the-art performance, improving accuracy by an average of 6.16% over baseline methods, accelerates inner-loop optimization by 3.9x, and saves 63% memory cost on datasets including CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K.

Conclusion: AT-BPTT effectively addresses the limitations of random truncation in dataset distillation by dynamically adapting to neural network learning dynamics, achieving superior performance and efficiency.

Abstract: The growing demand for efficient deep learning has positioned dataset distillation as a pivotal technique for compressing training dataset while preserving model performance. However, existing inner-loop optimization methods for dataset distillation typically rely on random truncation strategies, which lack flexibility and often yield suboptimal results. In this work, we observe that neural networks exhibit distinct learning dynamics across different training stages-early, middle, and late-making random truncation ineffective. To address this limitation, we propose Automatic Truncated Backpropagation Through Time (AT-BPTT), a novel framework that dynamically adapts both truncation positions and window sizes according to intrinsic gradient behavior. AT-BPTT introduces three key components: (1) a probabilistic mechanism for stage-aware timestep selection, (2) an adaptive window sizing strategy based on gradient variation, and (3) a low-rank Hessian approximation to reduce computational overhead. Extensive experiments on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K show that AT-BPTT achieves state-of-the-art performance, improving accuracy by an average of 6.16% over baseline methods. Moreover, our approach accelerates inner-loop optimization by 3.9x while saving 63% memory cost.

Method: Uses visual segmentation of PV modules in aerial overview images, infers structural information to assign modules to benches/rows/columns, identifies visual keypoints for layout, and merges detections from multiple images while maintaining structural integrity.

Result: Successfully tested on two different power plants, resulting in compact georeferenced 3D models with semantic structures suitable for maintenance applications.

Conclusion: The approach enables automated PV power plant mapping with detailed structural modeling down to individual module level, eliminating reliance on third-party data sources.

Abstract: An accurate and up-to-date model of a photovoltaic (PV) power plant is essential for its optimal operation and maintenance. However, such a model may not be easily available. This work introduces a novel approach for PV power plant mapping based on aerial overview images. It enables the automation of the mapping process while removing the reliance on third-party data. The presented mapping method takes advantage of the structural layout of the power plants to achieve detailed modeling down to the level of individual PV modules. The approach relies on visual segmentation of PV modules in overview images and the inference of structural information in each image, assigning modules to individual benches, rows, and columns. We identify visual keypoints related to the layout and use these to merge detections from multiple images while maintaining their structural integrity. The presented method was experimentally verified and evaluated on two different power plants. The final fusion of 3D positions and semantic structures results in a compact georeferenced model suitable for power plant maintenance.

Method: Combines spatial-temporal graph convolutional network (ST-GCN) with convolutional neural network (CNN) using transfer learning on 3D skeleton joint data to infer kinesics (communicative functions of human activity) without manual mappings.

Result: Demonstrated on Dyadic User Engagement (DUET) dataset, the method enables scalable, accurate, and human-centered modeling of behavior while preserving user anonymity.

Conclusion: Provides a new pathway for enhancing RL-driven simulations of human-environment interaction by uncovering latent structures in bodily movements that reflect cognitive and emotional states.

Abstract: Understanding the dynamic relationship between humans and the built environment is a key challenge in disciplines ranging from environmental psychology to reinforcement learning (RL). A central obstacle in modeling these interactions is the inability to capture human psychological states in a way that is both generalizable and privacy preserving. Traditional methods rely on theoretical models or questionnaires, which are limited in scope, static, and labor intensive. We present a kinesics recognition framework that infers the communicative functions of human activity – known as kinesics – directly from 3D skeleton joint data. Combining a spatial-temporal graph convolutional network (ST-GCN) with a convolutional neural network (CNN), the framework leverages transfer learning to bypass the need for manually defined mappings between physical actions and psychological categories. The approach preserves user anonymity while uncovering latent structures in bodily movements that reflect cognitive and emotional states. Our results on the Dyadic User EngagemenT (DUET) dataset demonstrate that this method enables scalable, accurate, and human-centered modeling of behavior, offering a new pathway for enhancing RL-driven simulations of human-environment interaction.

Method: The study compares five skeleton-based interaction recognition algorithms on a dataset of 12 dyadic interactions. Depth sensors are used instead of RGB cameras to address privacy concerns, analyzing skeletal movements to recognize interactions categorized into communication types like emblems and affect displays.

Result: The research provides a comparative analysis of five skeleton-based algorithms for dyadic interaction recognition, using a dataset specifically designed for understanding cultural and emotional aspects of human interactions through privacy-preserving depth sensing.

Conclusion: Skeleton-based interaction recognition using depth sensors offers a viable privacy-preserving approach for understanding human interactions, laying the foundation for cyber-physical-social infrastructure systems that can better address social objectives while respecting privacy concerns.

Abstract: Cyber-physical systems (CPS) integrate sensing, computing, and control to improve infrastructure performance, focusing on economic goals like performance and safety. However, they often neglect potential human-centered (or ‘‘social’’) benefits. Cyber-physical-social infrastructure systems (CPSIS) aim to address this by aligning CPS with social objectives. This involves defining social benefits, understanding human interactions with each other and infrastructure, developing privacy-preserving measurement methods, modeling these interactions for prediction, linking them to social benefits, and actuating the physical environment to foster positive social outcomes. This paper delves into recognizing dyadic human interactions using real-world data, which is the backbone to measuring social behavior. This lays a foundation to address the need to enhance understanding of the deeper meanings and mutual responses inherent in human interactions. While RGB cameras are informative for interaction recognition, privacy concerns arise. Depth sensors offer a privacy-conscious alternative by analyzing skeletal movements. This study compares five skeleton-based interaction recognition algorithms on a dataset of 12 dyadic interactions. Unlike single-person datasets, these interactions, categorized into communication types like emblems and affect displays, offer insights into the cultural and emotional aspects of human interactions.

Method: Integrates early exits and knowledge distillation, training a reduced student early-exit model from a complex teacher early-exit model with a new entropy-based loss for images where teacher classification was incorrect.

Result: Achieves significant reductions in computational complexity without compromising classification performance on CIFAR10, CIFAR100 and SVHN datasets.

Conclusion: The approach effectively optimizes the accuracy-efficiency trade-off and opens new research perspectives for Knowledge Distillation in other contexts.

Abstract: Although deep neural networks and in particular Convolutional Neural Networks have demonstrated state-of-the-art performance in image classification with relatively high efficiency, they still exhibit high computational costs, often rendering them impractical for real-time and edge applications. Therefore, a multitude of compression techniques have been developed to reduce these costs while maintaining accuracy. In addition, dynamic architectures have been introduced to modulate the level of compression at execution time, which is a desirable property in many resource-limited application scenarios. The proposed method effectively integrates two well-established optimization techniques: early exits and knowledge distillation, where a reduced student early-exit model is trained from a more complex teacher early-exit model. The primary contribution of this research lies in the approach for training the student early-exit model. In comparison to the conventional Knowledge Distillation loss, our approach incorporates a new entropy-based loss for images where the teacher’s classification was incorrect. The proposed method optimizes the trade-off between accuracy and efficiency, thereby achieving significant reductions in computational complexity without compromising classification performance. The validity of this approach is substantiated by experimental results on image classification datasets CIFAR10, CIFAR100 and SVHN, which further opens new research perspectives for Knowledge Distillation in other contexts.

Method: The approach retrains a deep convolutional neural network architecture designed for natural image IQA on optical microscopy data to predict both individual quality metrics and global quality scores.

Result: μDeepIQA provides fast and stable quality predictions that generalize well even outside standard method ranges, with the ability to visualize spatially varying quality through patch-wise predictions.

Conclusion: Deep learning models like μDeepIQA benefit optical microscopy studies through stable outlier performance, small patch assessment capability, and rapid predictions, enhancing generalizability over traditional methods.

Abstract: Optical microscopy is one of the most widely used techniques in research studies for life sciences and biomedicine. These applications require reliable experimental pipelines to extract valuable knowledge from the measured samples and must be supported by image quality assessment (IQA) to ensure correct processing and analysis of the image data. IQA methods are implemented with variable complexity. However, while most quality metrics have a straightforward implementation, they might be time consuming and computationally expensive when evaluating a large dataset. In addition, quality metrics are often designed for well-defined image features and may be unstable for images out of the ideal domain. To overcome these limitations, recent works have proposed deep learning-based IQA methods, which can provide superior performance, increased generalizability and fast prediction. Our method, named $\mathrm{\mu}$DeepIQA, is inspired by previous studies and applies a deep convolutional neural network designed for IQA on natural images to optical microscopy measurements. We retrained the same architecture to predict individual quality metrics and global quality scores for optical microscopy data. The resulting models provide fast and stable predictions of image quality by generalizing quality estimation even outside the ideal range of standard methods. In addition, $\mathrm{\mu}$DeepIQA provides patch-wise prediction of image quality and can be used to visualize spatially varying quality in a single image. Our study demonstrates that optical microscopy-based studies can benefit from the generalizability of deep learning models due to their stable performance in the presence of outliers, the ability to assess small image patches, and rapid predictions.

Method: End-to-end system with two modules: high-performance model for grape bunch detection and weight estimation, and Light-Invariant Spectral Autoencoder (LISA) - a domain-adversarial framework that learns illumination-invariant features from uncalibrated hyperspectral data across different lighting conditions.

Result: Complete pipeline achieves 0.82 recall for bunch detection and R² of 0.76 for weight prediction. LISA module improves quality prediction generalization by over 20% compared to baselines, validated across three illumination domains (lab, morning sunlight, afternoon sunlight).

Conclusion: The system successfully generates high-resolution, georeferenced data of both grape yield and quality, providing actionable insights for precision viticulture by overcoming illumination challenges through robust domain-invariant learning.

Abstract: This paper presents an end-to-end, IoT-enabled robotic system for the non-destructive, real-time, and spatially-resolved mapping of grape yield and quality (Brix, Acidity) in vineyards. The system features a comprehensive analytical pipeline that integrates two key modules: a high-performance model for grape bunch detection and weight estimation, and a novel deep learning framework for quality assessment from hyperspectral (HSI) data. A critical barrier to in-field HSI is the ``domain shift" caused by variable illumination. To overcome this, our quality assessment is powered by the Light-Invariant Spectral Autoencoder (LISA), a domain-adversarial framework that learns illumination-invariant features from uncalibrated data. We validated the system’s robustness on a purpose-built HSI dataset spanning three distinct illumination domains: controlled artificial lighting (lab), and variable natural sunlight captured in the morning and afternoon. Results show the complete pipeline achieves a recall (0.82) for bunch detection and a $R^2$ (0.76) for weight prediction, while the LISA module improves quality prediction generalization by over 20% compared to the baselines. By combining these robust modules, the system successfully generates high-resolution, georeferenced data of both grape yield and quality, providing actionable, data-driven insights for precision viticulture.

Method: Collection of multi-modal data including side-scan sonar tiles, bathymetric maps, and optical images from AUV surveys, with spatial association between optical images and SSS tiles for cross-modal representation learning.

Result: Created a standardized dataset with ~1M SSS tiles, bathymetric maps, optical images, and 36K annotated segmentation masks, plus open-source preprocessing and annotation tools.

Conclusion: This resource establishes a benchmark for underwater habitat mapping and promotes advancements in autonomous seafloor classification and multi-sensor integration.

Abstract: Benthic habitat mapping is fundamental for understanding marine ecosystems, guiding conservation efforts, and supporting sustainable resource management. Yet, the scarcity of large, annotated datasets limits the development and benchmarking of machine learning models in this domain. This paper introduces a thorough multi-modal dataset, comprising about a million side-scan sonar (SSS) tiles collected along the coast of Catalonia (Spain), complemented by bathymetric maps and a set of co-registered optical images from targeted surveys using an autonomous underwater vehicle (AUV). Approximately \num{36000} of the SSS tiles have been manually annotated with segmentation masks to enable supervised fine-tuning of classification models. All the raw sensor data, together with mosaics, are also released to support further exploration and algorithm development. To address challenges in multi-sensor data fusion for AUVs, we spatially associate optical images with corresponding SSS tiles, facilitating self-supervised, cross-modal representation learning. Accompanying open-source preprocessing and annotation tools are provided to enhance accessibility and encourage research. This resource aims to establish a standardized benchmark for underwater habitat mapping, promoting advancements in autonomous seafloor classification and multi-sensor integration.

Method: Used four object detection models implemented in PyTorch with transfer learning on a custom dataset of 198 images collected in Kigali. Evaluated models based on accuracy, localization, and inference speed.

Result: The study identified implementation challenges including dataset limitations and model complexities. Performance comparisons were made across the four models for real-time navigation suitability.

Conclusion: Recommended simplified architectures for future work to enhance accessibility of autonomous systems in developing countries like Rwanda, addressing resource constraints and implementation challenges.

Abstract: In Kigali, Rwanda, motorcycle taxis are a primary mode of transportation, often navigating unpredictably and disregarding traffic rules, posing significant challenges for autonomous driving systems. This study compares four object detection models–YOLOv5, Faster R-CNN, SSD, and RetinaNet–for motorbike detection using a custom dataset of 198 images collected in Kigali. Implemented in PyTorch with transfer learning, the models were evaluated for accuracy, localization, and inference speed to assess their suitability for real-time navigation in resource-constrained settings. We identify implementation challenges, including dataset limitations and model complexities, and recommend simplified architectures for future work to enhance accessibility for autonomous systems in developing countries like Rwanda.

Method: Integrates hierarchy-specific classification heads with trainable hierarchy matrices for self-supervised hierarchical learning, plus hierarchical consensus mechanism for probability distribution consistency across levels.

Result: Effective in guiding network learning and robust for hierarchical classification tasks across three benchmark datasets with different hierarchical complexity and backbone architectures.

Conclusion: SAHC method successfully leverages hierarchical structure in remote sensing image classification, demonstrating effectiveness and adaptability across various datasets and architectures.

Abstract: Deep learning has become increasingly important in remote sensing image classification due to its ability to extract semantic information from complex data. Classification tasks often include predefined label hierarchies that represent the semantic relationships among classes. However, these hierarchies are frequently overlooked, and most approaches focus only on fine-grained classification schemes. In this paper, we present a novel Semantics-Aware Hierarchical Consensus (SAHC) method for learning hierarchical features and relationships by integrating hierarchy-specific classification heads within a deep network architecture, each specialized in different degrees of class granularity. The proposed approach employs trainable hierarchy matrices, which guide the network through the learning of the hierarchical structure in a self-supervised manner. Furthermore, we introduce a hierarchical consensus mechanism to ensure consistent probability distributions across different hierarchical levels. This mechanism acts as a weighted ensemble being able to effectively leverage the inherent structure of the hierarchical classification task. The proposed SAHC method is evaluated on three benchmark datasets with different degrees of hierarchical complexity on different tasks, using distinct backbone architectures to effectively emphasize its adaptability. Experimental results show both the effectiveness of the proposed approach in guiding network learning and the robustness of the hierarchical consensus for remote sensing image classification tasks.

Method: Leverages anatomical priors to train seven specialized experts for distinct lung lobes and bilateral lung combinations. Uses multi-modal gating mechanisms that integrate radiomics biomarkers and deep learning features (CNN, ViT, Mamba) to optimally weight expert contributions.

Result: Achieved average AUC of 0.8646 ± 0.0467, a 12.5% improvement over SwinUNETR baseline (AUC 0.7685, p=0.031). Region-specific experts showed lower-lobe models achieving AUCs of 0.88-0.90, surpassing DL counterparts (CNN: 0.76-0.79).

Conclusion: REN demonstrates strong generalizability and clinical interpretability, presenting a scalable, anatomically-guided approach extensible to other structured medical imaging applications.

Abstract: Mixture-of-Experts (MoE) architectures have significantly contributed to scalable machine learning by enabling specialized subnetworks to tackle complex tasks efficiently. However, traditional MoE systems lack domain-specific constraints essential for medical imaging, where anatomical structure and regional disease heterogeneity strongly influence pathological patterns. Here, we introduce Regional Expert Networks (REN), the first anatomically-informed MoE framework tailored specifically for medical image classification. REN leverages anatomical priors to train seven specialized experts, each dedicated to distinct lung lobes and bilateral lung combinations, enabling precise modeling of region-specific pathological variations. Multi-modal gating mechanisms dynamically integrate radiomics biomarkers and deep learning (DL) features (CNN, ViT, Mamba) to weight expert contributions optimally. Applied to interstitial lung disease (ILD) classification, REN achieves consistently superior performance: the radiomics-guided ensemble reached an average AUC of 0.8646 +/- 0.0467, a +12.5 percent improvement over the SwinUNETR baseline (AUC 0.7685, p = 0.031). Region-specific experts further revealed that lower-lobe models achieved AUCs of 0.88-0.90, surpassing DL counterparts (CNN: 0.76-0.79) and aligning with known disease progression patterns. Through rigorous patient-level cross-validation, REN demonstrates strong generalizability and clinical interpretability, presenting a scalable, anatomically-guided approach readily extensible to other structured medical imaging applications.

Method: Proposes Natural Feature Progressive Framework (NFPF) with Specific Feature Learning Machine (SFLM) to quantify sample contribution to model performance and uses Reconstruction Difference metric for sample selection.

Result: NFPF significantly outperforms all established UAL methods and achieves performance on par with supervised AL methods on vision datasets, with superior robustness and better data distribution coverage.

Conclusion: NFPF revolutionizes sample importance measurement in UAL, providing a more effective framework that reduces annotation burden while maintaining high performance comparable to supervised approaches.

Abstract: The effectiveness of modern deep learning models is predicated on the availability of large-scale, human-annotated datasets, a process that is notoriously expensive and time-consuming. While Active Learning (AL) offers a strategic solution by labeling only the most informative and representative data, its iterative nature still necessitates significant human involvement. Unsupervised Active Learning (UAL) presents an alternative by shifting the annotation burden to a single, post-selection step. Unfortunately, prevailing UAL methods struggle to achieve state-of-the-art performance. These approaches typically rely on local, gradient-based scoring for sample importance estimation, which not only makes them vulnerable to ambiguous and noisy data but also hinders their capacity to select samples that adequately represent the full data distribution. Moreover, their use of shallow, one-shot linear selection falls short of a true UAL paradigm. In this paper, we propose the Natural Feature Progressive Framework (NFPF), a UAL method that revolutionizes how sample importance is measured. At its core, NFPF employs a Specific Feature Learning Machine (SFLM) to effectively quantify each sample’s contribution to model performance. We further utilize the SFLM to define a powerful Reconstruction Difference metric for initial sample selection. Our comprehensive experiments show that NFPF significantly outperforms all established UAL methods and achieves performance on par with supervised AL methods on vision datasets. Detailed ablation studies and qualitative visualizations provide compelling evidence for NFPF’s superior performance, enhanced robustness, and improved data distribution coverage.

Method: Proposes CA3D-Diff framework with two key components: 1) Column-aware cross-attention mechanism that leverages geometric properties where anatomically corresponding regions lie in similar column positions across views, using Gaussian-decayed bias to emphasize local correlations; 2) Implicit 3D structure reconstruction module that back-projects noisy 2D latents into coarse 3D feature volume based on breast-view projection geometry, then refines and injects it into the denoising UNet.

Result: Extensive experiments show CA3D-Diff achieves superior performance in bidirectional view translation tasks, outperforming state-of-the-art methods in visual fidelity and structural consistency. The synthesized views effectively improve single-view malignancy classification in screening settings.

Conclusion: CA3D-Diff demonstrates practical value in real-world diagnostics by enabling effective view translation when one mammography view is unavailable, improving both image quality and downstream diagnostic tasks like malignancy classification.

Abstract: Dual-view mammography, including craniocaudal (CC) and mediolateral oblique (MLO) projections, offers complementary anatomical views crucial for breast cancer diagnosis. However, in real-world clinical workflows, one view may be missing, corrupted, or degraded due to acquisition errors or compression artifacts, limiting the effectiveness of downstream analysis. View-to-view translation can help recover missing views and improve lesion alignment. Unlike natural images, this task in mammography is highly challenging due to large non-rigid deformations and severe tissue overlap in X-ray projections, which obscure pixel-level correspondences. In this paper, we propose Column-Aware and Implicit 3D Diffusion (CA3D-Diff), a novel bidirectional mammogram view translation framework based on conditional diffusion model. To address cross-view structural misalignment, we first design a column-aware cross-attention mechanism that leverages the geometric property that anatomically corresponding regions tend to lie in similar column positions across views. A Gaussian-decayed bias is applied to emphasize local column-wise correlations while suppressing distant mismatches. Furthermore, we introduce an implicit 3D structure reconstruction module that back-projects noisy 2D latents into a coarse 3D feature volume based on breast-view projection geometry. The reconstructed 3D structure is refined and injected into the denoising UNet to guide cross-view generation with enhanced anatomical awareness. Extensive experiments demonstrate that CA3D-Diff achieves superior performance in bidirectional tasks, outperforming state-of-the-art methods in visual fidelity and structural consistency. Furthermore, the synthesized views effectively improve single-view malignancy classification in screening settings, demonstrating the practical value of our method in real-world diagnostics.

Method: Introduces a pixel diffusion decoder with transformer components and GAN-free training, then uses distillation to create an efficient single-step decoder (SSDD).

Result: SSDD improves reconstruction FID from 0.87 to 0.50 with 1.4× higher throughput, and preserves DiT generation quality with 3.8× faster sampling.

Conclusion: SSDD can serve as a drop-in replacement for KL-VAE, enabling higher-quality and faster generative models without adversarial training.

Abstract: Tokenizers are a key component of state-of-the-art generative image models, extracting the most important features from the signal while reducing data dimension and redundancy. Most current tokenizers are based on KL-regularized variational autoencoders (KL-VAE), trained with reconstruction, perceptual and adversarial losses. Diffusion decoders have been proposed as a more principled alternative to model the distribution over images conditioned on the latent. However, matching the performance of KL-VAE still requires adversarial losses, as well as a higher decoding time due to iterative sampling. To address these limitations, we introduce a new pixel diffusion decoder architecture for improved scaling and training stability, benefiting from transformer components and GAN-free training. We use distillation to replicate the performance of the diffusion decoder in an efficient single-step decoder. This makes SSDD the first diffusion decoder optimized for single-step reconstruction trained without adversarial losses, reaching higher reconstruction quality and faster sampling than KL-VAE. In particular, SSDD improves reconstruction FID from $0.87$ to $0.50$ with $1.4\times$ higher throughput and preserve generation quality of DiTs with $3.8\times$ faster sampling. As such, SSDD can be used as a drop-in replacement for KL-VAE, and for building higher-quality and faster generative models.

Method: Fine-tune expressive layers of VFM with encoder-decoder network to embed digital watermarks into internal representations of hold-out input images.

Result: Watermarks remain detectable in functional copies after fine-tuning, with low false detection and misdetection probabilities demonstrated theoretically and experimentally.

Conclusion: The proposed method provides effective ownership verification for VFMs, protecting against unauthorized redistribution while maintaining model functionality.

Abstract: Being trained on large and vast datasets, visual foundation models (VFMs) can be fine-tuned for diverse downstream tasks, achieving remarkable performance and efficiency in various computer vision applications. The high computation cost of data collection and training motivates the owners of some VFMs to distribute them alongside the license to protect their intellectual property rights. However, a dishonest user of the protected model’s copy may illegally redistribute it, for example, to make a profit. As a consequence, the development of reliable ownership verification tools is of great importance today, since such methods can be used to differentiate between a redistributed copy of the protected model and an independent model. In this paper, we propose an approach to ownership verification of visual foundation models by fine-tuning a small set of expressive layers of a VFM along with a small encoder-decoder network to embed digital watermarks into an internal representation of a hold-out set of input images. Importantly, the watermarks embedded remain detectable in the functional copies of the protected model, obtained, for example, by fine-tuning the VFM for a particular downstream task. Theoretically and experimentally, we demonstrate that the proposed method yields a low probability of false detection of a non-watermarked model and a low probability of false misdetection of a watermarked model.

Method: Extends pre-trained DNNs with transformation layers to produce multiple latent representations for uncertainty estimation. Compares LUR and RLUR against eight probabilistic deep learning methods across four driver action and intention recognition datasets.

Result: LUR and RLUR achieve comparable in-distribution classification performance to other approaches. For uncertainty-based OOD detection, LUR matches top-performing methods while being more efficient to train and easier to tune than approaches requiring Markov-Chain Monte Carlo sampling or repulsive training.

Conclusion: The proposed LUR and RLUR methods provide effective uncertainty estimation for safety-critical applications with better efficiency and ease of tuning compared to existing probabilistic deep learning approaches.

Abstract: Deep neural networks (DNNs) are increasingly applied to safety-critical tasks in resource-constrained environments, such as video-based driver action and intention recognition. While last layer probabilistic deep learning (LL-PDL) methods can detect out-of-distribution (OOD) instances, their performance varies. As an alternative to last layer approaches, we propose extending pre-trained DNNs with transformation layers to produce multiple latent representations to estimate the uncertainty. We evaluate our latent uncertainty representation (LUR) and repulsively trained LUR (RLUR) approaches against eight PDL methods across four video-based driver action and intention recognition datasets, comparing classification performance, calibration, and uncertainty-based OOD detection. We also contribute 28,000 frame-level action labels and 1,194 video-level intention labels for the NuScenes dataset. Our results show that LUR and RLUR achieve comparable in-distribution classification performance to other LL-PDL approaches. For uncertainty-based OOD detection, LUR matches top-performing PDL methods while being more efficient to train and easier to tune than approaches that require Markov-Chain Monte Carlo sampling or repulsive training procedures.

Method: Three-phase architecture: pre-trained CNNs, custom convolutional layers with attention mechanisms, and ensemble hard voting. Dataset augmentation for spiral and wave images.

Result: Spiral images: 90% precision, recall, F1-score. Wave images: 96.67% metrics. Combined ensemble voting: 93.3% overall accuracy.

Conclusion: Machine learning with hand-drawn images shows strong potential for early PD diagnosis, providing a non-invasive and cost-effective solution.

Abstract: Parkinson’s disease (PD) is a progressive neurodegenerative condition characterized by the death of dopaminergic neurons, leading to various movement disorder symptoms. Early diagnosis of PD is crucial to prevent adverse effects, yet traditional diagnostic methods are often cumbersome and costly. In this study, a machine learning-based approach is proposed using hand-drawn spiral and wave images as potential biomarkers for PD detection. Our methodology leverages convolutional neural networks (CNNs), transfer learning, and attention mechanisms to improve model performance and resilience against overfitting. To enhance the diversity and richness of both spiral and wave categories, the training dataset undergoes augmentation to increase the number of images. The proposed architecture comprises three phases: utilizing pre-trained CNNs, incorporating custom convolutional layers, and ensemble voting. Employing hard voting further enhances performance by aggregating predictions from multiple models. Experimental results show promising accuracy rates. For spiral images, weighted average precision, recall, and F1-score are 90%, and for wave images, they are 96.67%. After combining the predictions through ensemble hard voting, the overall accuracy is 93.3%. These findings underscore the potential of machine learning in early PD diagnosis, offering a non-invasive and cost-effective solution to improve patient outcomes.

Method: Systematic analysis of three fundamental post-training pillars: supervised fine-tuning with chain-of-thought, reinforcement learning from verifiable objectives, and test-time scaling through enhanced inference computation.

Result: Presents a structured taxonomy clarifying roles, interconnections, and video-specific adaptations of these techniques, addressing challenges like temporal localization and spatiotemporal grounding.

Conclusion: Provides researchers with a unified framework for advancing Video-LMM capabilities, including curated benchmarks and metrics for rigorous assessment of post-training effectiveness.

Abstract: Video understanding represents the most challenging frontier in computer vision, requiring models to reason about complex spatiotemporal relationships, long-term dependencies, and multimodal evidence. The recent emergence of Video-Large Multimodal Models (Video-LMMs), which integrate visual encoders with powerful decoder-based language models, has demonstrated remarkable capabilities in video understanding tasks. However, the critical phase that transforms these models from basic perception systems into sophisticated reasoning engines, post-training, remains fragmented across the literature. This survey provides the first comprehensive examination of post-training methodologies for Video-LMMs, encompassing three fundamental pillars: supervised fine-tuning (SFT) with chain-of-thought, reinforcement learning (RL) from verifiable objectives, and test-time scaling (TTS) through enhanced inference computation. We present a structured taxonomy that clarifies the roles, interconnections, and video-specific adaptations of these techniques, addressing unique challenges such as temporal localization, spatiotemporal grounding, long video efficiency, and multimodal evidence integration. Through systematic analysis of representative methods, we synthesize key design principles, insights, and evaluation protocols while identifying critical open challenges in reward design, scalability, and cost-performance optimization. We further curate essential benchmarks, datasets, and metrics to facilitate rigorous assessment of post-training effectiveness. This survey aims to provide researchers and practitioners with a unified framework for advancing Video-LMM capabilities. Additional resources and updates are maintained at: https://github.com/yunlong10/Awesome-Video-LMM-Post-Training

Method: Proposed architecture using inductive bias from 3D foundation models to match segments across image pairs with up to 180-degree viewpoint changes.

Result: Outperforms state-of-the-art methods (SAM2 video propagator and local feature matching) by up to 30% on AUPRC metric on ScanNet++ and Replica datasets, with demonstrated benefits for 3D instance segmentation and image-goal navigation.

Conclusion: The approach successfully leverages 3D foundation models to achieve superior segment matching performance in extreme viewpoint scenarios, enabling better downstream applications.

Abstract: Segment matching is an important intermediate task in computer vision that establishes correspondences between semantically or geometrically coherent regions across images. Unlike keypoint matching, which focuses on localized features, segment matching captures structured regions, offering greater robustness to occlusions, lighting variations, and viewpoint changes. In this paper, we leverage the spatial understanding of 3D foundation models to tackle wide-baseline segment matching, a challenging setting involving extreme viewpoint shifts. We propose an architecture that uses the inductive bias of these 3D foundation models to match segments across image pairs with up to 180 degree view-point change. Extensive experiments show that our approach outperforms state-of-the-art methods, including the SAM2 video propagator and local feature matching methods, by upto 30% on the AUPRC metric, on ScanNet++ and Replica datasets. We further demonstrate benefits of the proposed model on relevant downstream tasks, including 3D instance segmentation and image-goal navigation. Project Page: https://segmast3r.github.io/

Method: Uses contrast enhancement algorithms to generate pseudo-reference images, trains a classification network to select the best algorithm based on image content and distortion, then performs full-reference assessment between pseudo-reference and degraded images.

Result: The method shows promising performance when evaluated on three databases containing contrast distortions (CCID2014, TID2013, and CSIQ).

Conclusion: The proposed approach effectively addresses contrast distortion assessment by transforming no-reference problems into more accurate full-reference evaluations through intelligent pseudo-reference generation.

Abstract: Contrast change is an important factor that affects the quality of images. During image capturing, unfavorable lighting conditions can cause contrast change and visual quality loss. While various methods have been proposed to assess the quality of images under different distortions such as blur and noise, contrast distortion has been largely overlooked as its visual impact and properties are different from other conventional types of distortions. In this paper, we propose a no-reference image quality assessment (NR-IQA) metric for contrast-distorted images. Using a set of contrast enhancement algorithms, we aim to generate pseudo-reference images that are visually close to the actual reference image, such that the NR problem is transformed to a Full-reference (FR) assessment with higher accuracy. To this end, a large dataset of contrast-enhanced images is produced to train a classification network that can select the most suitable contrast enhancement algorithm based on image content and distortion for pseudo-reference image generation. Finally, the evaluation is performed in the FR manner to assess the quality difference between the contrast-enhanced (pseudoreference) and degraded images. Performance evaluation of the proposed method on three databases containing contrast distortions (CCID2014, TID2013, and CSIQ), indicates the promising performance of the proposed method.

Method: Hybrid architecture combining ResNet-50 for localized features and Vision Transformer for global context, with FAISS-based similarity retrieval and neuroplastic modular design featuring expandable blocks that grow during training when performance plateaus.

Result: Outperforms traditional static models in both accuracy and adaptability, validated on waste classification and KolektorSDD2 industrial defect dataset.

Conclusion: The Neuroplastic Modular Classifier provides a scalable, high-performance solution for real-world image classification with strong applicability in environmental and industrial domains.

Abstract: Efficient and accurate classification of waste and industrial surface defects is essential for ensuring sustainable waste management and maintaining high standards in quality control. This paper introduces the Neuroplastic Modular Classifier, a novel hybrid architecture designed for robust and adaptive image classification in dynamic environments. The model combines a ResNet-50 backbone for localized feature extraction with a Vision Transformer (ViT) to capture global semantic context. Additionally, FAISS-based similarity retrieval is incorporated to provide a memory-like reference to previously encountered data, enriching the model’s feature space. A key innovation of our architecture is the neuroplastic modular design composed of expandable, learnable blocks that dynamically grow during training when performance plateaus. Inspired by biological learning systems, this mechanism allows the model to adapt to data complexity over time, improving generalization. Beyond garbage classification, we validate the model on the Kolektor Surface Defect Dataset 2 (KolektorSDD2), which involves industrial defect detection on metal surfaces. Experimental results across domains show that the proposed architecture outperforms traditional static models in both accuracy and adaptability. The Neuroplastic Modular Classifier offers a scalable, high-performance solution for real-world image classification, with strong applicability in both environmental and industrial domains.

Method: Constructed 1.3M structured image pairs from executable programs, trained unified VLM-FLUX.1 model with lightweight connector using three-stage curriculum, and developed StructBench benchmark with StructScore metric.

Result: Evaluation of 15 models shows even leading closed-source systems perform poorly, while their model achieves strong editing performance with consistent gains from inference-time reasoning across architectures.

Conclusion: The released dataset, model, and benchmark advance unified multimodal foundations for structured visual generation and editing, addressing a critical gap in current AI capabilities.

Abstract: While modern visual generation models excel at creating aesthetically pleasing natural images, they struggle with producing or editing structured visuals like charts, diagrams, and mathematical figures, which demand composition planning, text rendering, and multimodal reasoning for factual fidelity. To address this, we present the first comprehensive, systematic investigation of this domain, encompassing data construction, model training, and an evaluation benchmark. First, we construct a large-scale dataset of 1.3 million high-quality structured image pairs derived from executable drawing programs and augmented with chain-of-thought reasoning annotations. Building on it, we train a unified model that integrates a VLM with FLUX.1 Kontext via a lightweight connector for enhanced multimodal understanding. A three-stage training curriculum enables progressive feature alignment, knowledge infusion, and reasoning-augmented generation, further boosted by an external reasoner at inference time. Finally, we introduce StructBench, a novel benchmark for generation and editing with over 1,700 challenging instances, and an accompanying evaluation metric, StructScore, which employs a multi-round Q&A protocol to assess fine-grained factual accuracy. Evaluations of 15 models reveal that even leading closed-source systems remain far from satisfactory. Our model attains strong editing performance, and inference-time reasoning yields consistent gains across diverse architectures. By releasing the dataset, model, and benchmark, we aim to advance unified multimodal foundations for structured visuals.

Method: Uses Cross-Character Embedding (CCE) to learn identity and behavioral logic across multimodal sources, and Cross-Character Augmentation (CCA) to enrich training with synthetic co-existence and mixed-style data.

Result: Experiments on a curated benchmark with 10 characters from cartoons and live-action series show clear improvements in identity preservation, interaction quality, and robustness to style delusion.

Conclusion: The framework enables new forms of generative storytelling by allowing natural interactions between previously uncoexistent characters while maintaining stylistic fidelity.

Abstract: Imagine Mr. Bean stepping into Tom and Jerry–can we generate videos where characters interact naturally across different worlds? We study inter-character interaction in text-to-video generation, where the key challenge is to preserve each character’s identity and behaviors while enabling coherent cross-context interaction. This is difficult because characters may never have coexisted and because mixing styles often causes style delusion, where realistic characters appear cartoonish or vice versa. We introduce a framework that tackles these issues with Cross-Character Embedding (CCE), which learns identity and behavioral logic across multimodal sources, and Cross-Character Augmentation (CCA), which enriches training with synthetic co-existence and mixed-style data. Together, these techniques allow natural interactions between previously uncoexistent characters without losing stylistic fidelity. Experiments on a curated benchmark of cartoons and live-action series with 10 characters show clear improvements in identity preservation, interaction quality, and robustness to style delusion, enabling new forms of generative storytelling.Additional results and videos are available on our project page: https://tingtingliao.github.io/mimix/.

Method: VChain leverages large multimodal models to generate sparse critical keyframes as snapshots, then uses these to guide sparse inference-time tuning of pre-trained video generators only at key moments.

Result: Extensive experiments on complex, multi-step scenarios show that VChain significantly enhances the quality of generated videos.

Conclusion: The approach is tuning-efficient, introduces minimal overhead, avoids dense supervision, and effectively bridges multimodal reasoning with video generation.

Abstract: Recent video generation models can produce smooth and visually appealing clips, but they often struggle to synthesize complex dynamics with a coherent chain of consequences. Accurately modeling visual outcomes and state transitions over time remains a core challenge. In contrast, large language and multimodal models (e.g., GPT-4o) exhibit strong visual state reasoning and future prediction capabilities. To bridge these strengths, we introduce VChain, a novel inference-time chain-of-visual-thought framework that injects visual reasoning signals from multimodal models into video generation. Specifically, VChain contains a dedicated pipeline that leverages large multimodal models to generate a sparse set of critical keyframes as snapshots, which are then used to guide the sparse inference-time tuning of a pre-trained video generator only at these key moments. Our approach is tuning-efficient, introduces minimal overhead and avoids dense supervision. Extensive experiments on complex, multi-step scenarios show that VChain significantly enhances the quality of generated videos.

Method: Implemented foveated retinotopic transformation in the input layer of standard ResNet models and retrained them for image classification tasks.

Result: Maintained comparable classification accuracy while enhancing robustness to scale and rotational perturbations. Variations in classification probabilities across gaze positions served as effective indicators for object localization.

Conclusion: Foveated retinotopic mapping encodes implicit knowledge about visual object geometry and offers an efficient solution to visual search problems, similar to biological vision systems.

Abstract: From a falcon detecting prey to humans recognizing faces, many species exhibit extraordinary abilities in rapid visual localization and classification. These are made possible by a specialized retinal region called the fovea, which provides high acuity at the center of vision while maintaining lower resolution in the periphery. This distinctive spatial organization, preserved along the early visual pathway through retinotopic mapping, is fundamental to biological vision, yet remains largely unexplored in machine learning. Our study investigates how incorporating foveated retinotopy may benefit deep convolutional neural networks (CNNs) in image classification tasks. By implementing a foveated retinotopic transformation in the input layer of standard ResNet models and re-training them, we maintain comparable classification accuracy while enhancing the network’s robustness to scale and rotational perturbations. Although this architectural modification introduces increased sensitivity to fixation point shifts, we demonstrate how this apparent limitation becomes advantageous: variations in classification probabilities across different gaze positions serve as effective indicators for object localization. Our findings suggest that foveated retinotopic mapping encodes implicit knowledge about visual object geometry, offering an efficient solution to the visual search problem - a capability crucial for many living species.

Method: Two-fold framework: 1) Latte++ uses multi-window aggregation for better geometric correspondences and local prediction consistency evaluation, 2) ITTA employs interactive segmentation with point clicks and bounding boxes, using a lightweight promptable branch with momentum gradient module to capture human feedback.

Result: Extensive experiments across five MM-TTA benchmarks show consistent and notable improvements with robust performance gains for target classes in challenging imbalanced scenarios, with Latte++ providing complementary benefits for temporal stability.

Conclusion: The proposed framework effectively addresses both temporal inconsistency and consistently incorrect predictions in MM-TTA, achieving robust performance through geometric correspondences and human-in-the-loop feedback.

Abstract: Multi-modal test-time adaptation (MM-TTA) adapts models to an unlabeled target domain by leveraging the complementary multi-modal inputs in an online manner. While previous MM-TTA methods for 3D segmentation offer a promising solution by leveraging self-refinement per frame, they suffer from two major limitations: 1) unstable frame-wise predictions caused by temporal inconsistency, and 2) consistently incorrect predictions that violate the assumption of reliable modality guidance. To address these limitations, this work introduces a comprehensive two-fold framework. Firstly, building upon our previous work ReLiable Spatial-temporal Voxels (Latte), we propose Latte++ that better suppresses the unstable frame-wise predictions with more informative geometric correspondences. Instead of utilizing a universal sliding window, Latte++ employs multi-window aggregation to capture more reliable correspondences to better evaluate the local prediction consistency of different semantic categories. Secondly, to tackle the consistently incorrect predictions, we propose Interactive Test-Time Adaptation (ITTA), a flexible add-on to empower effortless human feedback with existing MM-TTA methods. ITTA introduces a novel human-in-the-loop approach that efficiently integrates minimal human feedback through interactive segmentation, requiring only simple point clicks and bounding box annotations. Instead of using independent interactive networks, ITTA employs a lightweight promptable branch with a momentum gradient module to capture and reuse knowledge from scarce human feedback during online inference. Extensive experiments across five MM-TTA benchmarks demonstrate that ITTA achieves consistent and notable improvements with robust performance gains for target classes of interest in challenging imbalanced scenarios, while Latte++ provides complementary benefits for temporal stability.

Method: Maximum likelihood estimation applied to a binomial decision model for a pure probabilistic approach to evaluate distance models using 2AFC psychophysical data.

Result: The method demonstrates superior simplicity, interpretability, flexibility, and computational efficiency compared to existing approaches.

Conclusion: The probabilistic maximum likelihood approach provides a robust and efficient alternative for evaluating perceptual distance models with 2AFC data.

Abstract: The Two Alternative Forced Choice (2AFC) paradigm offers advantages over the Mean Opinion Score (MOS) paradigm in psychophysics (PF), such as simplicity and robustness. However, when evaluating perceptual distance models, MOS enables direct correlation between model predictions and PF data. In contrast, 2AFC only allows pairwise comparisons to be converted into a quality ranking similar to MOS when comparisons include shared images. In large datasets, like BAPPS, where image patches and distortions are combined randomly, deriving rankings from 2AFC PF data becomes infeasible, as distorted images included in each comparisons are independent. To address this, instead of relying on MOS correlation, researchers have trained ad-hoc neural networks to reproduce 2AFC PF data based on pairs of model distances - a black-box approach with conceptual and operational limitations. This paper introduces a more robust distance-model evaluation method using a pure probabilistic approach, applying maximum likelihood estimation to a binomial decision model. Our method demonstrates superior simplicity, interpretability, flexibility, and computational efficiency, as shown through evaluations of various visual distance models on two 2AFC PF datasets.

Method: Derives density fields from meshes using distance fields, encodes radiance field in compact tri-planes, and introduces mesh-tailored adaptations to volume rendering for better convergence.

Result: Achieves superior reconstruction quality compared to previous approaches.

Conclusion: Validates the feasibility of integrating mesh and neural volume rendering for industrial 3D face asset creation.

Abstract: Industrial 3D face assets creation typically reconstructs topology-consistent face meshes from multi-view images for downstream production. However, high-quality reconstruction usually requires manual processing or specific capture settings. Recently NeRF has shown great advantages in 3D reconstruction, by representing scenes as density and radiance fields and utilizing neural volume rendering for novel view synthesis. Inspired by this, we introduce a novel method which combines explicit mesh with neural volume rendering to optimize geometry of an artist-made template face mesh from multi-view images while keeping the topology unchanged. Our method derives density fields from meshes using distance fields as an intermediary and encodes radiance field in compact tri-planes. To improve convergence, several adaptions tailored for meshes are introduced to the volume rendering. Experiments demonstrate that our method achieves superior reconstruction quality compared to previous approaches, validating the feasibility of integrating mesh and neural volume rendering.

Method: Proposes CapsIE architecture using Capsule Networks to capture equivariance with respect to novel viewpoints. Introduces a new objective function based on entropy minimization to accommodate CapsNet architectural changes.

Result: Achieves state-of-the-art performance on equivariant rotation tasks in 3DIEBench dataset compared to prior equivariant SSL methods. Performs competitively against supervised counterparts with higher efficiency and fewer network parameters.

Conclusion: CapsNets can learn complex and generalized representations for large-scale, multi-task datasets, demonstrating their effectiveness in equivariant self-supervised learning compared to previous benchmarks.

Abstract: Learning invariant representations has been the long-standing approach to self-supervised learning. However, recently progress has been made in preserving equivariant properties in representations, yet do so with highly prescribed architectures. In this work, we propose an invariant-equivariant self-supervised architecture that employs Capsule Networks (CapsNets), which have been shown to capture equivariance with respect to novel viewpoints. We demonstrate that the use of CapsNets in equivariant self-supervised architectures achieves improved downstream performance on equivariant tasks with higher efficiency and fewer network parameters. To accommodate the architectural changes of CapsNets, we introduce a new objective function based on entropy minimisation. This approach, which we name CapsIE (Capsule Invariant Equivariant Network), achieves state-of-the-art performance on the equivariant rotation tasks on the 3DIEBench dataset compared to prior equivariant SSL methods, while performing competitively against supervised counterparts. Our results demonstrate the ability of CapsNets to learn complex and generalised representations for large-scale, multi-task datasets compared to previous CapsNet benchmarks. Code is available at https://github.com/AberdeenML/CapsIE.

Method: Proposes a projection module that maps new model embeddings to old model embeddings, pretrained with text data only, and uses parameter-efficient training strategies to preserve the new model’s knowledge without modifications.

Result: Experimental results on cross-modal retrieval datasets demonstrate XBT’s effectiveness in achieving backward compatibility between VLP models like CLIP.

Conclusion: XBT enables backfill-free upgrades when new VLP models emerge, providing an efficient solution for cross-modal retrieval system upgrades.

Abstract: Modern retrieval systems often struggle with upgrading to new and more powerful models due to the incompatibility of embeddings between the old and new models. This necessitates a costly process known as backfilling, which involves re-computing the embeddings for a large number of data samples. In vision, Backward-compatible Training (BT) has been proposed to ensure that the new model aligns with the old model’s embeddings. This paper extends the concept of vision-only BT to the field of cross-modal retrieval, marking the first attempt to address Cross-modal BT (XBT). Our goal is to achieve backward-compatibility between Vision-Language Pretraining (VLP) models, such as CLIP, for the cross-modal retrieval task. To address XBT challenges, we propose an efficient solution: a projection module that maps the new model’s embeddings to those of the old model. This module, pretrained solely with text data, significantly reduces the number of image-text pairs required for XBT learning, and, once it is pretrained, it avoids using the old model during training. Furthermore, we utilize parameter-efficient training strategies that improve efficiency and preserve the off-the-shelf new model’s knowledge by avoiding any modifications. Experimental results on cross-modal retrieval datasets demonstrate the effectiveness of XBT and its potential to enable backfill-free upgrades when a new VLP model emerges.

Method: Uses a 3D convolutional network trained on monocular 3D cues (motion, depth, appearance) to distinguish real from synthetic videos, avoiding unreliable 3D reconstruction. Provides confidence scores and gradient-based visualizations.

Result: L3DE shows strong alignment with 3D reconstruction quality and human judgments. Evaluations reveal persistent simulation gaps and subtle inconsistencies in leading models like Kling, Sora, and MiniMax.

Conclusion: L3DE provides an objective, interpretable framework for assessing 3D visual coherence in generated videos, with applications in model benchmarking, deepfake detection, and video synthesis improvement.

Abstract: Recent advancements in video diffusion models enable the generation of photorealistic videos with impressive 3D consistency and temporal coherence. However, the extent to which these AI-generated videos simulate the 3D visual world remains underexplored. In this paper, we introduce Learned 3D Evaluation (L3DE), an objective, quantifiable, and interpretable method for assessing AI-generated videos’ ability to simulate the real world in terms of 3D visual qualities and consistencies, without requiring manually labeled defects or quality annotations. Instead of relying on 3D reconstruction, which is prone to failure with in-the-wild videos, L3DE employs a 3D convolutional network, trained on monocular 3D cues of motion, depth, and appearance, to distinguish real from synthetic videos. Confidence scores from L3DE quantify the gap between real and synthetic videos in terms of 3D visual coherence, while a gradient-based visualization pinpoints unrealistic regions, improving interpretability. We validate L3DE through extensive experiments, demonstrating strong alignment with 3D reconstruction quality and human judgments. Our evaluations on leading generative models (e.g., Kling, Sora, and MiniMax) reveal persistent simulation gaps and subtle inconsistencies. Beyond generative video assessment, L3DE extends to broader applications: benchmarking video generation models, serving as a deepfake detector, and enhancing video synthesis by inpainting flagged inconsistencies. Project page: https://justin-crchang.github.io/l3de-project-page/

Method: Uses two 360° cameras on a portable monopod to simultaneously record standard exposure and fast shutter speed panoramic videos, processed by a two-stage NeRF-based algorithm with fine alignment of fast- and well-exposed frames.

Result: PanDORA generates non-saturated HDR radiance maps and achieves superior visual fidelity compared to existing methods on real indoor scenes with HDR ground truth lighting.

Conclusion: The system provides a scalable solution for capturing real environments in HDR with casual, high-quality acquisition.

Abstract: Most novel view synthesis methods-including Neural Radiance Fields (NeRF)-struggle to capture the true high dynamic range (HDR) radiance of scenes. This is primarily due to their dependence on low dynamic range (LDR) images from conventional cameras. Exposure bracketing techniques aim to address this challenge, but they introduce a considerable time burden during the acquisition process. In this work, we introduce PanDORA: PANoramic Dual-Observer Radiance Acquisition, a system designed for the casual, high quality HDR capture of indoor environments. Our approach uses two 360{\deg} cameras mounted on a portable monopod to simultaneously record two panoramic 360{\deg} videos: one with standard exposure and another at fast shutter speed. The resulting video data is processed by a proposed two-stage NeRF-based algorithm, including an algorithm for the fine alignment of the fast- and well-exposed frames, generating non-saturated HDR radiance maps. Compared to existing methods on a novel dataset of real indoor scenes captured with our apparatus and including HDR ground truth lighting, PanDORA achieves superior visual fidelity and provides a scalable solution for capturing real environments in HDR.

Method: The paper provides a comprehensive review organizing defense strategies into detection, disruption, and authentication categories, proposing a multidimensional taxonomy and analyzing evaluation datasets, criteria, and metrics.

Result: The review establishes a unified framework for understanding defense mechanisms against AI-generated visual media and identifies mainstream defense-related tasks and their characteristics.

Conclusion: The analysis reveals current research challenges and suggests potential future directions for improving defenses against malicious use of AI-generated visual content.

Abstract: Deep generative models have demonstrated impressive performance in various computer vision applications, including image synthesis, video generation, and medical analysis. Despite their significant advancements, these models may be used for malicious purposes, such as misinformation, deception, and copyright violation. In this paper, we provide a systematic and timely review of research efforts on defenses against AI-generated visual media, covering detection, disruption, and authentication. We review existing methods and summarize the mainstream defense-related tasks within a unified passive and proactive framework. Moreover, we survey the derivative tasks concerning the trustworthiness of defenses, such as their robustness and fairness. For each defense strategy, we formulate its general pipeline and propose a multidimensional taxonomy applicable across defense tasks, based on methodological strategies. Additionally, we summarize the commonly used evaluation datasets, criteria, and metrics. Finally, by analyzing the reviewed studies, we provide insights into current research challenges and suggest possible directions for future research.

Method: Uses iterative estimation based on ICP algorithm with a novel mesh-based active shape model that maintains vertex connectivity. Does not require learning from pose-annotated data.

Result: ShapeICP surpasses many data-driven approaches that rely on pose data for training, despite not using pose-annotated data.

Conclusion: Opens up a new solution space for category-level pose and shape estimation by demonstrating that non-data-driven iterative methods can outperform data-driven approaches.

Abstract: Category-level object pose and shape estimation from a single depth image has recently drawn research attention due to its potential utility for tasks such as robotics manipulation. The task is particularly challenging because the three unknowns, object pose, object shape, and model-to-measurement correspondences, are compounded together, but only a single view of depth measurements is provided. Most of the prior work heavily relies on data-driven approaches to obtain solutions to at least one of the unknowns, and typically two, risking generalization failures if not designed and trained carefully. The shape representations used in the prior work also mainly focus on point clouds and signed distance fields (SDFs). In stark contrast to the prior work, we approach the problem using an iterative estimation method that does not require learning from pose-annotated data. Moreover, we construct and adopt a novel mesh-based object active shape model (ASM), which additionally maintains vertex connectivity compared to the commonly used point-based object ASM. Our algorithm, ShapeICP, is based on the iterative closest point (ICP) algorithm but is equipped with additional features for the category-level pose and shape estimation task. Although not using pose-annotated data, ShapeICP surpasses many data-driven approaches that rely on pose data for training, opening up a new solution space for researchers to consider.

Method: Quantify cross-modal alignment and correspondence using AC score, conduct experiments with 13 vision representation settings across 8 benchmarks.

Result: AC score shows linear correlation with model performance, enabling identification of optimal vision representation without language model finetuning.

Conclusion: The proposed approach achieves 99.7% reduction in computational cost while maintaining performance through optimal vision representation selection.

Abstract: We present the “Law of Vision Representation” in multimodal large language models (MLLMs). It reveals a strong correlation between the combination of cross-modal alignment, correspondence in vision representation, and MLLM performance. We quantify the two factors using the cross-modal Alignment and Correspondence score (AC score). Through extensive experiments involving thirteen different vision representation settings and evaluations across eight benchmarks, we find that the AC score is linearly correlated to model performance. By leveraging this relationship, we are able to identify and train the optimal vision representation only, which does not require finetuning the language model every time, resulting in a 99.7% reduction in computational cost.

Method: SteerDiff acts as an intermediary between user input and diffusion model, identifying and manipulating inappropriate concepts within the text embedding space to guide the model away from harmful outputs.

Result: Extensive experiments across concept unlearning tasks show SteerDiff effectively prevents inappropriate content generation. It demonstrates robustness against multiple red-teaming strategies and versatility in concept forgetting tasks.

Conclusion: SteerDiff provides an effective and flexible safety mechanism for T2I diffusion models that maintains usability while preventing harmful content generation through text embedding manipulation.

Abstract: Text-to-image (T2I) diffusion models have drawn attention for their ability to generate high-quality images with precise text alignment. However, these models can also be misused to produce inappropriate content. Existing safety measures, which typically rely on text classifiers or ControlNet-like approaches, are often insufficient. Traditional text classifiers rely on large-scale labeled datasets and can be easily bypassed by rephrasing. As diffusion models continue to scale, fine-tuning these safeguards becomes increasingly challenging and lacks flexibility. Recent red-teaming attack researches further underscore the need for a new paradigm to prevent the generation of inappropriate content. In this paper, we introduce SteerDiff, a lightweight adaptor module designed to act as an intermediary between user input and the diffusion model, ensuring that generated images adhere to ethical and safety standards with little to no impact on usability. SteerDiff identifies and manipulates inappropriate concepts within the text embedding space to guide the model away from harmful outputs. We conduct extensive experiments across various concept unlearning tasks to evaluate the effectiveness of our approach. Furthermore, we benchmark SteerDiff against multiple red-teaming strategies to assess its robustness. Finally, we explore the potential of SteerDiff for concept forgetting tasks, demonstrating its versatility in text-conditioned image generation.

Method: Uses an approximation to Newton’s optimization method in denoising diffusion models to speed up inference and avoid expensive backpropagation operations.

Result: Produces results that rival or surpass state-of-the-art training-free inference methods while requiring significantly less time. Effective for both linear (inpainting, super-resolution) and non-linear (style-guided generation) constraints.

Conclusion: The proposed algorithm enables fast, high-quality constrained sampling in diffusion models, making it practical for various image generation tasks with constraints.

Abstract: Large denoising diffusion models, such as Stable Diffusion, have been trained on billions of image-caption pairs to perform text-conditioned image generation. As a byproduct of this training, these models have acquired general knowledge about image statistics, which can be useful for other inference tasks. However, when confronted with sampling an image under new constraints, e.g. generating the missing parts of an image, using large pre-trained text-to-image diffusion models is inefficient and often unreliable. Previous approaches either utilized backpropagation through the denoiser network, making them significantly slower and more memory-demanding than simple text-to-image generation, or only enforced the constraint locally, failing to capture critical long-range correlations in the sampled image. In this work, we propose an algorithm that enables fast, high-quality generation under arbitrary constraints. We show that in denoising diffusion models, we can employ an approximation to Newton’s optimization method that allows us to speed up inference and avoid the expensive backpropagation operations. Our approach produces results that rival or surpass the state-of-the-art training-free inference methods while requiring a fraction of the time. We demonstrate the effectiveness of our algorithm under both linear (inpainting, super-resolution) and non-linear (style-guided generation) constraints. An implementation is provided at https://github.com/cvlab-stonybrook/fast-constrained-sampling.

Method: Uses probabilistic objectives with an “uncertainty token” for efficient uncertainty estimation, and introduces a novel inclusion loss that enforces distributional inclusion relationships between image-text pairs and between original/masked inputs.

Result: Achieves 74.6% ImageNet zero-shot accuracy with ViT-B/16, and uncertainty estimates align with intuitive notions (shorter texts more uncertain, general inputs include specific ones). Text uncertainties further improve ImageNet accuracy to 75.8% in few-shot setting.

Conclusion: ProLIP demonstrates practical advantages of probabilistic approaches for VLMs, enabling better uncertainty estimation and improved downstream task performance.

Abstract: Vision-language models (VLMs) embed aligned image-text pairs into a joint space but often rely on deterministic embeddings, assuming a one-to-one correspondence between images and texts. This oversimplifies real-world relationships, which are inherently many-to-many, with multiple captions describing a single image and vice versa. We introduce Probabilistic Language-Image Pre-training (ProLIP), the first probabilistic VLM pre-trained on a billion-scale image-text dataset using only probabilistic objectives, achieving a strong zero-shot capability (e.g., 74.6% ImageNet zero-shot accuracy with ViT-B/16). ProLIP efficiently estimates uncertainty by an “uncertainty token” without extra parameters. We also introduce a novel inclusion loss that enforces distributional inclusion relationships between image-text pairs and between original and masked inputs. Experiments demonstrate that, by leveraging uncertainty estimates, ProLIP benefits downstream tasks and aligns with intuitive notions of uncertainty, e.g., shorter texts being more uncertain and more general inputs including specific ones. Utilizing text uncertainties, we further improve ImageNet accuracy from 74.6% to 75.8% (under a few-shot setting), supporting the practical advantages of our probabilistic approach. The code is available at https://github.com/naver-ai/prolip

Method: Leverages advanced machine learning models specifically tailored for coral reef segmentation to automatically generate dense segmentation masks with minimal manual effort.

Result: Extensive evaluations show CoralSCOP-LAT surpasses existing tools in time efficiency, accuracy, precision, and flexibility, significantly accelerating the coral reef annotation process.

Conclusion: CoralSCOP-LAT provides an efficient solution for obtaining high-quality coral reef segmentation and analysis outcomes, addressing key challenges in coral reef monitoring.

Abstract: Coral reef imagery offers critical data for monitoring ecosystem health, in particular as the ease of image datasets continues to rapidly expand. Whilst semi-automated analytical platforms for reef imagery are becoming more available, the dominant approaches face fundamental limitations. To address these challenges, we propose CoralSCOP-LAT, a coral reef image analysis and labeling tool that automatically segments and analyzes coral regions. By leveraging advanced machine learning models tailored for coral reef segmentation, CoralSCOP-LAT enables users to generate dense segmentation masks with minimal manual effort, significantly enhancing both the labeling efficiency and precision of coral reef analysis. Our extensive evaluations demonstrate that CoralSCOP-LAT surpasses existing coral reef analysis tools in terms of time efficiency, accuracy, precision, and flexibility. CoralSCOP-LAT, therefore, not only accelerates the coral reef annotation process but also assists users in obtaining high-quality coral reef segmentation and analysis outcomes. Github Page: https://github.com/ykwongaq/CoralSCOP-LAT.

Method: Proposes a self-entropy regularization mechanism integrated with reinforcement learning from human feedback to encourage broader exploration and improve training robustness.

Result: The regularization technique effectively mitigates reward hacking, improves training stability, enhances image quality across latent space, and boosts image diversity and specificity.

Conclusion: Integrating human feedback with self-entropy regularization achieves state-of-the-art results on key image generation metrics by stabilizing DPO training for diffusion models.

Abstract: Direct Preference Optimization (DPO) has been successfully used to align large language models (LLMs) according to human preferences, and more recently it has also been applied to improving the quality of text-to-image diffusion models. However, DPO-based methods such as SPO, Diffusion-DPO, and D3PO are highly susceptible to overfitting and reward hacking, especially when the generative model is optimized to fit out-of-distribution during prolonged training. To overcome these challenges and stabilize the training of diffusion models, we introduce a self-entropy regularization mechanism in reinforcement learning from human feedback. This enhancement improves DPO training by encouraging broader exploration and greater robustness. Our regularization technique effectively mitigates reward hacking, leading to improved stability and enhanced image quality across the latent space. Extensive experiments demonstrate that integrating human feedback with self-entropy regularization can significantly boost image diversity and specificity, achieving state-of-the-art results on key image generation metrics.

Method: TimeFormer uses a Cross-Temporal Transformer Encoder to learn temporal relationships of deformable 3D Gaussians, combined with a two-stream optimization strategy that transfers motion knowledge during training but removes TimeFormer during inference.

Result: Extensive experiments in multi-view and monocular dynamic scenes show qualitative and quantitative improvements in reconstruction quality.

Conclusion: TimeFormer effectively addresses limitations in current dynamic scene reconstruction by learning implicit motion patterns while preserving rendering efficiency during inference.

Abstract: Dynamic scene reconstruction is a long-term challenge in 3D vision. Recent methods extend 3D Gaussian Splatting to dynamic scenes via additional deformation fields and apply explicit constraints like motion flow to guide the deformation. However, they learn motion changes from individual timestamps independently, making it challenging to reconstruct complex scenes, particularly when dealing with violent movement, extreme-shaped geometries, or reflective surfaces. To address the above issue, we design a plug-and-play module called TimeFormer to enable existing deformable 3D Gaussians reconstruction methods with the ability to implicitly model motion patterns from a learning perspective. Specifically, TimeFormer includes a Cross-Temporal Transformer Encoder, which adaptively learns the temporal relationships of deformable 3D Gaussians. Furthermore, we propose a two-stream optimization strategy that transfers the motion knowledge learned from TimeFormer to the base stream during the training phase. This allows us to remove TimeFormer during inference, thereby preserving the original rendering speed. Extensive experiments in the multi-view and monocular dynamic scenes validate qualitative and quantitative improvement brought by TimeFormer. Project Page: https://patrickddj.github.io/TimeFormer/

Method: Proposed grounding-IQA paradigm with two subtasks: GIQA-DES (detailed descriptions with bounding boxes) and GIQA-VQA (quality QA for local regions). Created GIQA-160K dataset through automated annotation pipeline and developed GIQA-Bench benchmark for evaluation.

Result: Experiments demonstrate that the proposed method facilitates more fine-grained IQA applications by enabling detailed quality assessments with precise location information.

Conclusion: Grounding-IQA extends existing IQA capabilities by integrating multimodal referring and grounding, providing a more comprehensive framework for fine-grained image quality assessment through location-aware descriptions and region-specific quality evaluation.

Abstract: The development of multimodal large language models (MLLMs) enables the evaluation of image quality through natural language descriptions. This advancement allows for more detailed assessments. However, these MLLM-based IQA methods primarily rely on general contextual descriptions, sometimes limiting fine-grained quality assessment. To address this limitation, we introduce a new image quality assessment (IQA) task paradigm, grounding-IQA. This paradigm integrates multimodal referring and grounding with IQA to realize more fine-grained quality perception, thereby extending existing IQA. Specifically, grounding-IQA comprises two subtasks: grounding-IQA-description (GIQA-DES) and visual question answering (GIQA-VQA). GIQA-DES involves detailed descriptions with precise locations (e.g., bounding boxes), while GIQA-VQA focuses on quality QA for local regions. To realize grounding-IQA, we construct a corresponding dataset, GIQA-160K, through our proposed automated annotation pipeline. Furthermore, we develop a well-designed benchmark, GIQA-Bench. The benchmark evaluates the grounding-IQA performance from three perspectives: description quality, VQA accuracy, and grounding precision. Experiments demonstrate that our proposed method facilitates the more fine-grained IQA application. Code: https://github.com/zhengchen1999/Grounding-IQA.

Method: Autonomous imagination approach where MLLMs iteratively modify visual inputs (isolating objects, rearranging puzzle pieces) into intermediate visual states, decomposing visual-to-textual conversion into closed-loop visual modification steps.

Result: Without retraining, MLLMs can now solve tasks initially beyond their perceptual capability, demonstrating that closed-loop visual modification effectively decomposes visual reasoning into solvable substeps.

Conclusion: Closed-loop visual modification through autonomous imagination is an effective approach for decomposing complex visual reasoning tasks that MLLMs previously couldn’t handle due to perceptual limitations.

Abstract: Under pure textual modality, Large Language Models (LLMs) have demonstrated remarkable success in complex reasoning tasks by decomposing them into simpler sub-problems. However, Multimodal Large Language Models (MLLMs) still struggle with some seemingly straightforward visual tasks, such as counting and solving jigsaw puzzles. We argue that these tasks challenge the ability of visual-to-textual conversion, where MLLMs convert visual information perceived from the input scene, to textual information for further reasoning and generating the answer. If the complexity of the visual input is beyond the perceptual capability of the MLLMs, without decomposing this conversion process, simply scaling inference-time reasoning cannot solve the task because it repeatedly encounters the same perceptual bottleneck. We propose an approach, autonomous imagination, to enable MLLMs to iteratively modify visual inputs (e.g. isolating objects, rearranging puzzle pieces) into intermediate visual states, decomposing visual-to-textual conversion into closed-loop visual modification steps. We show that, without any retraining, MLLMs can now solve tasks initially beyond their perceptual capability, highlighting that closed-loop visual modification can be an effective way of decomposing the visual reasoning task into solvable substeps. Our code and data are released at https://future-item.github.io/autoimagine-site/.

Method: Uses lightweight backbone with hybrid encoder, PolySampler module, multi-scale deformable attention, and PolyOptLoss energy-based loss function for geometric alignment optimization.

Result: Achieved F1 score up to 0.74, lane position deviation as low as 0.405, and real-time inference latency of 6.7ms (optimized to 3.5ms with INT8 quantization).

Conclusion: RowDetr demonstrates the efficiency of polynomial parameterization for crop row detection, making it suitable for deployment on edge computing devices in agricultural robotics.

Abstract: Crop row detection enables autonomous robots to navigate in gps denied environments. Vision based strategies often struggle in the environments due to gaps, curved crop rows and require post-processing steps. Furthermore, labeling crop rows in under the canopy environments accurately is very difficult due to occlusions. This study introduces RowDetr, an efficient end-to-end transformer-based neural network for crop row detection in precision agriculture. RowDetr leverages a lightweight backbone and a hybrid encoder to model straight, curved, or occluded crop rows with high precision. Central to the architecture is a novel polynomial representation that enables direct parameterization of crop rows, eliminating computationally expensive post-processing. Key innovations include a PolySampler module and multi-scale deformable attention, which work together with PolyOptLoss, an energy-based loss function designed to optimize geometric alignment between predicted and the annotated crop rows, while also enhancing robustness against labeling noise. RowDetr was evaluated against other state-of-the-art end-to-end crop row detection methods like AgroNav and RolColAttention on a diverse dataset of 6,962 high-resolution images, used for training, validation, and testing across multiple crop types with annotated crop rows. The system demonstrated superior performance, achieved an F1 score up to 0.74 and a lane position deviation as low as 0.405. Furthermore, RowDetr achieves a real-time inference latency of 6.7ms, which was optimized to 3.5ms with INT8 quantization on an NVIDIA Jetson Orin AGX. This work highlighted the critical efficiency of polynomial parameterization, making RowDetr particularly suitable for deployment on edge computing devices in agricultural robotics and autonomous farming equipment. Index terms > Crop Row Detection, Under Canopy Navigation, Transformers, RT-DETR, RT-DETRv2

Method: Proposes Offset Shifting Denoiser with seamless rotating Window for efficient synchronous denoising, and Global Motion Guidance mechanism to ensure local detail fidelity and global motion continuity.

Result: Achieves superior content and motion quality in panoramic scene-level video generation with constant VRAM consumption regardless of output video resolution.

Conclusion: DynamicScaler provides a training-free, efficient, and scalable solution for immersive dynamic scene creation that preserves coherence across panoramic scenes of arbitrary size.

Abstract: The increasing demand for immersive AR/VR applications and spatial intelligence has heightened the need to generate high-quality scene-level and 360${\deg}$ panoramic video. However, most video diffusion models are constrained by limited resolution and aspect ratio, which restricts their applicability to scene-level dynamic content synthesis. In this work, we propose $\textbf{DynamicScaler}$, addressing these challenges by enabling spatially scalable and panoramic dynamic scene synthesis that preserves coherence across panoramic scenes of arbitrary size. Specifically, we introduce a Offset Shifting Denoiser, facilitating efficient, synchronous, and coherent denoising panoramic dynamic scenes via a diffusion model with fixed resolution through a seamless rotating Window, which ensures seamless boundary transitions and consistency across the entire panoramic space, accommodating varying resolutions and aspect ratios. Additionally, we employ a Global Motion Guidance mechanism to ensure both local detail fidelity and global motion continuity. Extensive experiments demonstrate our method achieves superior content and motion quality in panoramic scene-level video generation, offering a training-free, efficient, and scalable solution for immersive dynamic scene creation with constant VRAM consumption regardless of the output video resolution. Project page is available at $\href{https://dynamic-scaler.pages.dev/new}{https://dynamic-scaler.pages.dev/new}$.

Method: Cut target image regions where detector is confident, augment them with pseudo-labels, join into challenging target images, and use teacher-student learning to prevent model collapse.

Result: Achieved state-of-the-art performance on two out of three traffic scene adaptation benchmarks.

Conclusion: The proposed data augmentation approach effectively adapts object detectors to target domains without source data access, demonstrating strong performance on traffic scene benchmarks.

Abstract: Source-free domain-adaptive object detection is an interesting but scarcely addressed topic. It aims at adapting a source-pretrained detector to a distinct target domain without resorting to source data during adaptation. So far, there is no data augmentation scheme tailored to source-free domain-adaptive object detection. To this end, this paper presents a novel data augmentation approach that cuts out target image regions where the detector is confident, augments them along with their respective pseudo-labels, and joins them into a challenging target image to adapt the detector. As the source data is out of reach during adaptation, we implement our approach within a teacher-student learning paradigm to ensure that the model does not collapse during the adaptation procedure. We evaluated our approach on three adaptation benchmarks of traffic scenes, scoring new state-of-the-art on two of them.

Method: Three-phase approach: 1) Use Adam optimization to derive reference shimming weights from multi-channel B1+ fields, 2) Train ResNet to map B1+ fields directly to RF shimming outputs with confidence parameter in loss function, 3) Optional NFD post-processing to identify extreme non-uniform outcomes.

Result: Achieves 5000x speed-up compared to traditional MLS optimization while maintaining or improving predictive accuracy. Comparative evaluations show significant gains in both processing speed and accuracy.

Conclusion: Fast-RF-Shimming provides a promising solution for persistent B1+ inhomogeneity challenges in ultrahigh field MRI, offering substantial speed improvements without compromising performance.

Abstract: Ultrahigh field (UHF) Magnetic Resonance Imaging (MRI) offers an elevated signal-to-noise ratio (SNR), enabling exceptionally high spatial resolution that benefits both clinical diagnostics and advanced research. However, the jump to higher fields introduces complications, particularly transmit radiofrequency (RF) field ($B_{1}^{+}$) inhomogeneities, manifesting as uneven flip angles and image intensity irregularities. These artifacts can degrade image quality and impede broader clinical adoption. Traditional RF shimming methods, such as Magnitude Least Squares (MLS) optimization, effectively mitigate $B_{1}^{+}$ inhomogeneity, but remain time-consuming. Recent machine learning approaches, including RF Shim Prediction by Iteratively Projected Ridge Regression and other deep learning architectures, suggest alternative pathways. Although these approaches show promise, challenges such as extensive training periods, limited network complexity, and practical data requirements persist. In this paper, we introduce a holistic learning-based framework called Fast-RF-Shimming, which achieves a 5000x speed-up compared to the traditional MLS method. In the initial phase, we employ random-initialized Adaptive Moment Estimation (Adam) to derive the desired reference shimming weights from multi-channel $B_{1}^{+}$ fields. Next, we train a Residual Network (ResNet) to map $B_{1}^{+}$ fields directly to the ultimate RF shimming outputs, incorporating the confidence parameter into its loss function. Finally, we design Non-uniformity Field Detector (NFD), an optional post-processing step, to ensure the extreme non-uniform outcomes are identified. Comparative evaluations with standard MLS optimization underscore notable gains in both processing speed and predictive accuracy, which indicates that our technique shows a promising solution for addressing persistent inhomogeneity challenges.

Method: Two-stage approach: 1) Prompt LVLM to detect predefined concepts in images, 2) Use concept predictions to classify images, with retrieval-based in-context learning for both stages.

Result: Outperforms Concept Bottleneck Models and task-specific supervised methods across four medical datasets using twelve LVLMs, without training and with minimal annotations.

Conclusion: CBVLM provides an effective solution for interpretable medical image analysis by combining concept-based reasoning with LVLMs’ few-shot capabilities, eliminating training requirements and reducing annotation burden.

Abstract: The main challenges limiting the adoption of deep learning-based solutions in medical workflows are the availability of annotated data and the lack of interpretability of such systems. Concept Bottleneck Models (CBMs) tackle the latter by constraining the model output on a set of predefined and human-interpretable concepts. However, the increased interpretability achieved through these concept-based explanations implies a higher annotation burden. Moreover, if a new concept needs to be added, the whole system needs to be retrained. Inspired by the remarkable performance shown by Large Vision-Language Models (LVLMs) in few-shot settings, we propose a simple, yet effective, methodology, CBVLM, which tackles both of the aforementioned challenges. First, for each concept, we prompt the LVLM to answer if the concept is present in the input image. Then, we ask the LVLM to classify the image based on the previous concept predictions. Moreover, in both stages, we incorporate a retrieval module responsible for selecting the best examples for in-context learning. By grounding the final diagnosis on the predicted concepts, we ensure explainability, and by leveraging the few-shot capabilities of LVLMs, we drastically lower the annotation cost. We validate our approach with extensive experiments across four medical datasets and twelve LVLMs (both generic and medical) and show that CBVLM consistently outperforms CBMs and task-specific supervised methods without requiring any training and using just a few annotated examples. More information on our project page: https://cristianopatricio.github.io/CBVLM/.

Method: Proposes UniUIR with three key components: Mamba Mixture-of-Experts module to decouple degradation-specific issues, spatial-frequency prior generator to extract degradation priors in both domains, and depth information integration from pre-trained models for region-dependent distortion handling.

Result: Extensive experiments show UniUIR produces more attractive results in qualitative and quantitative comparisons, with strong generalization compared to state-of-the-art methods.

Conclusion: UniUIR effectively addresses complex underwater mixed distortions through its all-in-one approach, demonstrating superior performance and generalization capabilities.

Abstract: Existing underwater image restoration (UIR) methods generally only handle color distortion or jointly address color and haze issues, but they often overlook the more complex degradations that can occur in underwater scenes. To address this limitation, we propose a Universal Underwater Image Restoration method, termed as UniUIR, considering the complex scenario of real-world underwater mixed distortions as an all-in-one manner. To decouple degradation-specific issues and explore the inter-correlations among various degradations in UIR task, we designed the Mamba Mixture-of-Experts module. This module enables each expert to identify distinct types of degradation and collaboratively extract task-specific priors while maintaining global feature representation based on linear complexity. Building upon this foundation, to enhance degradation representation and address the task conflicts that arise when handling multiple types of degradation, we introduce the spatial-frequency prior generator. This module extracts degradation prior information in both spatial and frequency domains, and adaptively selects the most appropriate task-specific prompts based on image content, thereby improving the accuracy of image restoration. Finally, to more effectively address complex, region-dependent distortions in UIR task, we incorporate depth information derived from a large-scale pre-trained depth prediction model, thereby enabling the network to perceive and leverage depth variations across different image regions to handle localized degradation. Extensive experiments demonstrate that UniUIR can produce more attractive results across qualitative and quantitative comparisons, and shows strong generalization than state-of-the-art methods.

Method: Leverages 2D generative diffusion model priors with Dual-Space Optimization (applying Score Distillation Sampling in both canonical and observation spaces), View Selection strategy, and Pose Feature Injection to ensure visual consistency and realism.

Result: Achieves state-of-the-art performance in producing photorealistic renderings from monocular videos, particularly for challenging unseen body parts.

Conclusion: WonderHuman effectively solves the problem of reconstructing dynamic human avatars from limited viewpoint videos by combining diffusion priors with novel optimization techniques.

Abstract: In this paper, we present WonderHuman to reconstruct dynamic human avatars from a monocular video for high-fidelity novel view synthesis. Previous dynamic human avatar reconstruction methods typically require the input video to have full coverage of the observed human body. However, in daily practice, one typically has access to limited viewpoints, such as monocular front-view videos, making it a cumbersome task for previous methods to reconstruct the unseen parts of the human avatar. To tackle the issue, we present WonderHuman, which leverages 2D generative diffusion model priors to achieve high-quality, photorealistic reconstructions of dynamic human avatars from monocular videos, including accurate rendering of unseen body parts. Our approach introduces a Dual-Space Optimization technique, applying Score Distillation Sampling (SDS) in both canonical and observation spaces to ensure visual consistency and enhance realism in dynamic human reconstruction. Additionally, we present a View Selection strategy and Pose Feature Injection to enforce the consistency between SDS predictions and observed data, ensuring pose-dependent effects and higher fidelity in the reconstructed avatar. In the experiments, our method achieves SOTA performance in producing photorealistic renderings from the given monocular video, particularly for those challenging unseen parts. The project page and source code can be found at https://wyiguanw.github.io/WonderHuman/.

Method: The method disentangles static and dynamic latent variables using the minimal change principle. It achieves component-wise identifiability of dynamic variables through sufficient change property, implements a Temporal Transition Module to disentangle latent dynamics, and minimizes latent dynamic variable dimensionality while imposing temporal conditional independence.

Result: Extensive experiments on various video generation benchmarks show that CoVoGAN significantly improves generation quality and controllability across diverse real-world scenarios, both qualitatively and quantitatively.

Conclusion: CoVoGAN successfully enables efficient and independent control over individual video concepts through latent variable disentanglement, providing a theoretically grounded and practical solution for controllable video generation that outperforms existing methods.

Abstract: Controllable video generation remains a significant challenge, despite recent advances in generating high-quality and consistent videos. Most existing methods for controlling video generation treat the video as a whole, neglecting intricate fine-grained spatiotemporal relationships, which limits both control precision and efficiency. In this paper, we propose Controllable Video Generative Adversarial Networks (CoVoGAN) to disentangle the video concepts, thus facilitating efficient and independent control over individual concepts. Specifically, following the minimal change principle, we first disentangle static and dynamic latent variables. We then leverage the sufficient change property to achieve component-wise identifiability of dynamic latent variables, enabling disentangled control of video generation. To establish the theoretical foundation, we provide a rigorous analysis demonstrating the identifiability of our approach. Building on these theoretical insights, we design a Temporal Transition Module to disentangle latent dynamics. To enforce the minimal change principle and sufficient change property, we minimize the dimensionality of latent dynamic variables and impose temporal conditional independence. To validate our approach, we integrate this module as a plug-in for GANs. Extensive qualitative and quantitative experiments on various video generation benchmarks demonstrate that our method significantly improves generation quality and controllability across diverse real-world scenarios.

Method: Incorporates KAN layers into transformers, uses efficient KAN blocks, enhanced Dilated Neighborhood Attention (DiNA) for global context, and hierarchical hybrid strategy for local-global feature balance.

Result: Achieved SOTA in 27/29 experiments on 17 medical image datasets and 12 corrupted datasets, with 44% computational efficiency improvement and accuracy gains of 4.6-13.4% on benchmarks.

Conclusion: MedViTV2 effectively addresses medical image corruption challenges with improved efficiency and performance, demonstrating strong generalization across diverse medical imaging tasks.

Abstract: Convolutional networks, transformers, hybrid models, and Mamba-based architectures have demonstrated strong performance across various medical image classification tasks. However, these methods were primarily designed to classify clean images using labeled data. In contrast, real-world clinical data often involve image corruptions that are unique to multi-center studies and stem from variations in imaging equipment across manufacturers. In this paper, we introduce the Medical Vision Transformer (MedViTV2), a novel architecture incorporating Kolmogorov-Arnold Network (KAN) layers into the transformer architecture for the first time, aiming for generalized medical image classification. We have developed an efficient KAN block to reduce computational load while enhancing the accuracy of the original MedViT. Additionally, to counteract the fragility of our MedViT when scaled up, we propose an enhanced Dilated Neighborhood Attention (DiNA), an adaptation of the efficient fused dot-product attention kernel capable of capturing global context and expanding receptive fields to scale the model effectively and addressing feature collapse issues. Moreover, a hierarchical hybrid strategy is introduced to stack our Local Feature Perception and Global Feature Perception blocks in an efficient manner, which balances local and global feature perceptions to boost performance. Extensive experiments on 17 medical image classification datasets and 12 corrupted medical image datasets demonstrate that MedViTV2 achieved state-of-the-art results in 27 out of 29 experiments with reduced computational complexity. MedViTV2 is 44% more computationally efficient than the previous version and significantly enhances accuracy, achieving improvements of 4.6% on MedMNIST, 5.8% on NonMNIST, and 13.4% on the MedMNIST-C benchmark.

Method: Two key techniques: encoding distributions as 2D feature maps instead of network parameters, and using SMPL models as domain with refined flow velocity field. Two-stage training paradigm with diffusion flow model for compression and another flow model on latent space.

Result: Outperforms state-of-the-art methods with 57% improvement in geometry quality on pose-conditioned random avatar generation and avatar-consistent novel pose synthesis tasks.

Conclusion: The proposed geometry distribution model effectively addresses the challenges of realistic human geometry generation and demonstrates superior performance compared to existing methods.

Abstract: Realistic human geometry generation is an important yet challenging task, requiring both the preservation of fine clothing details and the accurate modeling of clothing-body interactions. To tackle this challenge, we build upon Geometry distributions, a recently proposed representation that can model a single human geometry with high fidelity using a flow matching model. However, extending a single-geometry distribution to a dataset is non-trivial and inefficient for large-scale learning. To address this, we propose a new geometry distribution model by two key techniques: (1) encoding distributions as 2D feature maps rather than network parameters, and (2) using SMPL models as the domain instead of Gaussian and refining the associated flow velocity field. We then design a generative framework adopting a two staged training paradigm analogous to state-of-the-art image and 3D generative models. In the first stage, we compress geometry distributions into a latent space using a diffusion flow model; the second stage trains another flow model on this latent space. We validate our approach on two key tasks: pose-conditioned random avatar generation and avatar-consistent novel pose synthesis. Experimental results demonstrate that our method outperforms existing state-of-the-art methods, achieving a 57% improvement in geometry quality.

Method: Uses iterative point-wise alignment in a common coordinate system based on state space models (SSMs). RGB and Points SSMs capture long-range dependencies and spatial information from single views with linear complexity.

Result: Achieves on-par performance with CAD-based and dense reference view-based methods across six popular datasets and real-world robotic scenes, despite the more challenging single reference setting.

Conclusion: SinRef-6D provides a scalable solution for novel object 6D pose estimation that works with only a single reference view, eliminating the need for CAD models or dense reference views while maintaining competitive performance.

Abstract: Existing novel object 6D pose estimation methods typically rely on CAD models or dense reference views, which are both difficult to acquire. Using only a single reference view is more scalable, but challenging due to large pose discrepancies and limited geometric and spatial information. To address these issues, we propose a Single-Reference-based novel object 6D (SinRef-6D) pose estimation method. Our key idea is to iteratively establish point-wise alignment in a common coordinate system based on state space models (SSMs). Specifically, iterative object-space point-wise alignment can effectively handle large pose discrepancies, while our proposed RGB and Points SSMs can capture long-range dependencies and spatial information from a single view, offering linear complexity and superior spatial modeling capability. Once pre-trained on synthetic data, SinRef-6D can estimate the 6D pose of a novel object using only a single reference view, without requiring retraining or a CAD model. Extensive experiments on six popular datasets and real-world robotic scenes demonstrate that we achieve on-par performance with CAD-based and dense reference view-based methods, despite operating in the more challenging single reference setting. Code will be released at https://github.com/CNJianLiu/SinRef-6D.

Method: Proposes PRO-VPT framework with iterative prompt relocation strategy: pruning idle prompts from saturated blocks and allocating them to prompt-needed blocks, formulated as nested optimization between ADO and VPT.

Result: Significantly outperforms advanced VPT methods, achieving 1.6 pp and 2.0 pp average accuracy improvements over VPT on VTAB-1k and FGVC benchmarks, leading to state-of-the-art performance.

Conclusion: Adaptive prompt distribution optimization through iterative relocation unlocks the full potential of VPT, demonstrating the importance of task-specific prompt allocation strategies.

Abstract: Visual prompt tuning (VPT), i.e., fine-tuning some lightweight prompt tokens, provides an efficient and effective approach for adapting pre-trained models to various downstream tasks. However, most prior art indiscriminately uses a fixed prompt distribution across different tasks, neglecting the importance of each block varying depending on the task. In this paper, we introduce adaptive distribution optimization (ADO) by tackling two key questions: (1) How to appropriately and formally define ADO, and (2) How to design an adaptive distribution strategy guided by this definition? Through empirical analysis, we first confirm that properly adjusting the distribution significantly improves VPT performance, and further uncover a key insight that a nested relationship exists between ADO and VPT. Based on these findings, we propose a new VPT framework, termed PRO-VPT (iterative Prompt RelOcation-based VPT), which adaptively adjusts the distribution built upon a nested optimization formulation. Specifically, we develop a prompt relocation strategy derived from this formulation, comprising two steps: pruning idle prompts from prompt-saturated blocks, followed by allocating these prompts to the most prompt-needed blocks. By iteratively performing prompt relocation and VPT, our proposal can adaptively learn the optimal prompt distribution in a nested optimization-based manner, thereby unlocking the full potential of VPT. Extensive experiments demonstrate that our proposal significantly outperforms advanced VPT methods, e.g., PRO-VPT surpasses VPT by 1.6 pp and 2.0 pp average accuracy, leading prompt-based methods to state-of-the-art performance on VTAB-1k and FGVC benchmarks. The code is available at https://github.com/ckshang/PRO-VPT.

Method: Proposed FEB-Cache, a joint caching strategy that separates Attention and MLP caching based on frequency-guided cache table to better align with non-exposed bias diffusion process.

Result: Empirical results show FEB-Cache optimizes model performance while facilitating acceleration, reducing exposure bias in cached diffusion processes.

Conclusion: FEB-Cache provides a new perspective on leveraging caching to accelerate diffusion processes by understanding and addressing the frequency response mismatch in Attention and MLP components.

Abstract: Diffusion Transformer (DiT) has exhibited impressive generation capabilities but faces great challenges due to its high computational complexity. To address this issue, various methods, notably feature caching, have been introduced. However, these approaches focus on aligning non-cache diffusion without analyzing why caching damage the generation processes. In this paper, we first confirm that the cache greatly amplifies the exposure bias, resulting in a decline in the generation quality. However, directly applying noise scaling is challenging for this issue due to the non-smoothness of exposure bias. We found that this phenomenon stems from the mismatch between its frequency response characteristics and the simple cache of Attention and MLP. Since these two components exhibit unique preferences for frequency signals, which provides us with a caching strategy to separate Attention and MLP to achieve an enhanced fit of exposure bias and reduce it. Based on this, we introduced FEB-Cache, a joint caching strategy that aligns with the non-exposed bias diffusion process (which gives us a higher performance cap) of caching Attention and MLP based on the frequency-guided cache table. Our approach combines a comprehensive understanding of the caching mechanism and offers a new perspective on leveraging caching to accelerate the diffusion process. Empirical results indicate that FEB-Cache optimizes model performance while concurrently facilitating acceleration. Code is available at https://github.com/aSleepyTree/EB-Cache.

Method: Two-stage approach: 1) Erase malicious semantics in textual prompts by optimizing cross-attention layers to map malicious prompts to contextually similar safe concepts using [EoT] embedding; 2) Use contrastive learning to steer early denoising predictions toward safe directions and away from unsafe ones, leveraging the deterministic properties of diffusion model sampling trajectories.

Result: Comprehensive evaluations on multiple malicious concept erasure benchmarks demonstrate TRCE’s effectiveness in erasing malicious concepts while better preserving the model’s original generation ability compared to existing methods.

Conclusion: TRCE provides an effective solution for achieving reliable concept erasure in text-to-image diffusion models, successfully balancing the removal of malicious content with the preservation of normal generation capabilities through its two-stage strategy.

Abstract: Recent advances in text-to-image diffusion models enable photorealistic image generation, but they also risk producing malicious content, such as NSFW images. To mitigate risk, concept erasure methods are studied to facilitate the model to unlearn specific concepts. However, current studies struggle to fully erase malicious concepts implicitly embedded in prompts (e.g., metaphorical expressions or adversarial prompts) while preserving the model’s normal generation capability. To address this challenge, our study proposes TRCE, using a two-stage concept erasure strategy to achieve an effective trade-off between reliable erasure and knowledge preservation. Firstly, TRCE starts by erasing the malicious semantics implicitly embedded in textual prompts. By identifying a critical mapping objective(i.e., the [EoT] embedding), we optimize the cross-attention layers to map malicious prompts to contextually similar prompts but with safe concepts. This step prevents the model from being overly influenced by malicious semantics during the denoising process. Following this, considering the deterministic properties of the sampling trajectory of the diffusion model, TRCE further steers the early denoising prediction toward the safe direction and away from the unsafe one through contrastive learning, thus further avoiding the generation of malicious content. Finally, we conduct comprehensive evaluations of TRCE on multiple malicious concept erasure benchmarks, and the results demonstrate its effectiveness in erasing malicious concepts while better preserving the model’s original generation ability. The code is available at: http://github.com/ddgoodgood/TRCE. CAUTION: This paper includes model-generated content that may contain offensive material.

Method: Representation Invariance FineTuning (RIFT) - a regularization that leverages orthogonal invariance of manifolds to maximize representation similarity between pretrained and finetuned models in a computationally efficient way.

Result: RIFT is compatible with mainstream finetuning methods, offering competitive or enhanced performance while better preserving model generalizability.

Conclusion: Downstream tasks can be effectively adapted without sacrificing the benefits of pretrained representations through RIFT regularization.

Abstract: Foundation models pretrained on large-scale natural images are widely adapted to various cross-domain low-resource downstream tasks, benefiting from generalizable and transferable patterns captured by their representations. However, these representations are later found to gradually vanish during finetuning, accompanied by a degradation of model’s original generalizability. In this paper, we argue that such tasks can be effectively adapted without sacrificing the benefits of pretrained representations. We approach this by introducing \textit{Representation Invariance FineTuning (RIFT)}, a regularization that maximizes the representation similarity between pretrained and finetuned models by leveraging orthogonal invariance of manifolds in a computationally efficient way. Experiments demonstrate that our method is compatible with mainstream finetuning methods, offering competitive or even enhanced performance and better preservation of the generalizability.

Method: The paper analyzes existing metrics, discusses their pitfalls, conducts expert preference studies, proposes systematic unit tests to verify desirable properties, provides context-aware metric recommendations, and develops a learned metric distilled from human expert judgments.

Result: The research provides a detailed framework for evaluating 3D reconstruction metrics, identifies limitations of current approaches, and introduces a new learned metric that better aligns with human expert preferences.

Conclusion: The work establishes comprehensive guidelines for metric selection in 3D reconstruction evaluation and proposes a human-aligned learned metric as a superior alternative to traditional approaches, with open-source implementation available.

Abstract: “What cannot be measured cannot be improved” while likely never uttered by Lord Kelvin, summarizes effectively the driving force behind this work. This paper presents a detailed discussion of automated metrics for evaluating structured 3D reconstructions. Pitfalls of each metric are discussed, and an analysis through the lens of expert 3D modelers’ preferences is presented. A set of systematic “unit tests” are proposed to empirically verify desirable properties, and context aware recommendations regarding which metric to use depending on application are provided. Finally, a learned metric distilled from human expert judgments is proposed and analyzed. The source code is available at https://github.com/s23dr/wireframe-metrics-iccv2025

Method: Uses motion graphs with dual quaternion skinning and learnable weight painting functions to propagate motion to individual Gaussians, jointly optimized via differentiable rendering.

Result: Achieves state-of-the-art performance on iPhone dataset and competitive results on HyperNeRF, enabling novel pose animation, robot demonstration synthesis, and visual planning.

Conclusion: MBGS provides explicit motion control for Gaussian splatting, expanding applications in robotics through motion graph representation and differentiable optimization.

Abstract: Gaussian splatting has emerged as a powerful tool for high-fidelity reconstruction of dynamic scenes. However, existing methods primarily rely on implicit motion representations, such as encoding motions into neural networks or per-Gaussian parameters, which makes it difficult to further manipulate the reconstructed motions. This lack of explicit controllability limits existing methods to replaying recorded motions only, which hinders a wider application in robotics. To address this, we propose Motion Blender Gaussian Splatting (MBGS), a novel framework that uses motion graphs as an explicit and sparse motion representation. The motion of a graph’s links is propagated to individual Gaussians via dual quaternion skinning, with learnable weight painting functions that determine the influence of each link. The motion graphs and 3D Gaussians are jointly optimized from input videos via differentiable rendering. Experiments show that MBGS achieves state-of-the-art performance on the highly challenging iPhone dataset while being competitive on HyperNeRF. We demonstrate the application potential of our method in animating novel object poses, synthesizing real robot demonstrations, and predicting robot actions through visual planning. The source code, models, video demonstrations can be found at http://mlzxy.github.io/motion-blender-gs.

Method: A staged optimization strategy that functions as an implicit curriculum - first establishing structural scaffolds by prioritizing inter-object relationships, then shifting to high-fidelity refinement of individual components.

Result: Outperforms state-of-the-art methods in fidelity, disentanglement, and spatial coherence on diverse compositional prompts in both qualitative and quantitative evaluations.

Conclusion: Temporal decoupling of competing objectives through staged optimization provides a robust solution to gradient conflict in compositional text-to-3D generation.

Abstract: Current text-to-3D methods excel at generating single objects but falter on compositional prompts. We argue this failure is fundamental to their optimization schedules, as simultaneous or iterative heuristics predictably collapse under a combinatorial explosion of conflicting gradients, leading to entangled geometry or catastrophic divergence. In this paper, we reframe the core challenge of compositional generation as one of optimization scheduling. We introduce DecompDreamer, a framework built on a novel staged optimization strategy that functions as an implicit curriculum. Our method first establishes a coherent structural scaffold by prioritizing inter-object relationships before shifting to the high-fidelity refinement of individual components. This temporal decoupling of competing objectives provides a robust solution to gradient conflict. Qualitative and quantitative evaluations on diverse compositional prompts demonstrate that DecompDreamer outperforms state-of-the-art methods in fidelity, disentanglement, and spatial coherence.

Method: Proposes LIAM model with CLIP backbone for encoding language and image inputs, designed two pre-training tasks to fine-tune weights and pre-align latent spaces, incorporates semantic maps.

Result: Evaluated on ALFRED dataset, demonstrates importance of pre-aligning embedding spaces from different modalities and efficacy of incorporating semantic maps.

Conclusion: The approach enables flexible domestic task execution through multimodal inputs and proper modality alignment, showing promising results for service robotics.

Abstract: The availability of large language models and open-vocabulary object perception methods enables more flexibility for domestic service robots. The large variability of domestic tasks can be addressed without implementing each task individually by providing the robot with a task description along with appropriate environment information. In this work, we propose LIAM - an end-to-end model that predicts action transcripts based on language, image, action, and map inputs. Language and image inputs are encoded with a CLIP backbone, for which we designed two pre-training tasks to fine-tune its weights and pre-align the latent spaces. We evaluate our method on the ALFRED dataset, a simulator-generated benchmark for domestic tasks. Our results demonstrate the importance of pre-aligning embedding spaces from different modalities and the efficacy of incorporating semantic maps.

Method: Systematically converts three driving QA datasets (DriveLM, NuScenes-QA, LingoQA) into multiple-choice questions with distractors grounded in five realistic error categories: Driving Domain Misconceptions, Logical Inconsistencies, Misinterpreted Sensor Inputs, Computational Oversights, and Question Ambiguity.

Result: Fine-tuned LLaVA-1.5-7B improved accuracy by ~6 percentage points across tasks, GPT-4V achieved strongest zero-shot performance (up to 69.8% accuracy), and Qwen2-VL performed competitively, especially in multi-view settings. Traditional metrics like BLEU and CIDEr failed to distinguish model performance.

Conclusion: AutoDrive-QA provides an objective, domain-grounded evaluation protocol for more transparent benchmarking of urban AI systems, supporting development of safer autonomous driving technologies.

Abstract: Evaluating vision-language models (VLMs) in urban driving contexts remains challenging, as existing benchmarks rely on open-ended responses that are ambiguous, annotation-intensive, and inconsistent to score. This lack of standardized evaluation slows progress toward safe and reliable AI for urban mobility. We introduce AutoDrive-QA, the first benchmark that systematically converts open-ended driving QA datasets (DriveLM, NuScenes-QA, LingoQA) into structured multiple-choice questions (MCQs) with distractors grounded in five realistic error categories: Driving Domain Misconceptions, Logical Inconsistencies, Misinterpreted Sensor Inputs, Computational Oversights, and Question Ambiguity. This framework enables reproducible and interpretable evaluation of VLMs across perception, prediction, and planning tasks in complex urban scenes. Experiments show that fine-tuning LLaVA-1.5-7B improves accuracy by about six percentage points across tasks, GPT-4V achieves the strongest zero-shot performance with up to 69.8% accuracy, and Qwen2-VL models also perform competitively, particularly in multi-view settings. Moreover, traditional metrics such as BLEU and CIDEr fail to distinguish strong from weak models. By providing an objective, domain-grounded evaluation protocol, AutoDrive-QA contributes to more transparent benchmarking of urban AI systems, supporting the development of safer and more trustworthy autonomous driving technologies for smart cities.

Method: Proposes a hybrid image reconstruction surrogate task that preserves SD’s detail priors without additional supervision, and introduces Scale-Shift GRU to bridge distribution gaps between SD’s scale-shift invariant estimation and self-supervised scale-invariant depth estimation.

Result: Jasmine achieves state-of-the-art performance on the KITTI benchmark and demonstrates superior zero-shot generalization across multiple datasets.

Conclusion: The framework successfully harnesses Stable Diffusion’s visual priors for self-supervised depth estimation while overcoming inherent challenges through hybrid reconstruction and scale-shift alignment techniques.

Abstract: In this paper, we propose Jasmine, the first Stable Diffusion (SD)-based self-supervised framework for monocular depth estimation, which effectively harnesses SD’s visual priors to enhance the sharpness and generalization of unsupervised prediction. Previous SD-based methods are all supervised since adapting diffusion models for dense prediction requires high-precision supervision. In contrast, self-supervised reprojection suffers from inherent challenges (e.g., occlusions, texture-less regions, illumination variance), and the predictions exhibit blurs and artifacts that severely compromise SD’s latent priors. To resolve this, we construct a novel surrogate task of hybrid image reconstruction. Without any additional supervision, it preserves the detail priors of SD models by reconstructing the images themselves while preventing depth estimation from degradation. Furthermore, to address the inherent misalignment between SD’s scale and shift invariant estimation and self-supervised scale-invariant depth estimation, we build the Scale-Shift GRU. It not only bridges this distribution gap but also isolates the fine-grained texture of SD output against the interference of reprojection loss. Extensive experiments demonstrate that Jasmine achieves SoTA performance on the KITTI benchmark and exhibits superior zero-shot generalization across multiple datasets.

Method: Uses multi-turn RL-based optimization with physics-aware constraints and rendered image evaluations, featuring adaptive iterative reasoning where VLMs refine spatial arrangements over multiple turns by analyzing rendered outputs.

Result: Significantly enhances spatial consistency and formatting stability of various scale models, with more realistic, aligned, and functionally coherent object placements.

Conclusion: Validates RL’s effectiveness for 3D spatial reasoning in metaverse, AR/VR, digital twins, and game development applications, with publicly available code, data, and training pipeline.

Abstract: We present MetaSpatial, the first reinforcement learning (RL)-based framework designed to enhance 3D spatial reasoning in vision-language models (VLMs), enabling real-time 3D scene generation without the need for hard-coded optimizations. MetaSpatial addresses two core challenges: (i) the lack of internalized 3D spatial reasoning in VLMs, which limits their ability to generate realistic layouts, and (ii) the inefficiency of traditional supervised fine-tuning (SFT) for layout generation tasks, as perfect ground truth annotations are unavailable. Our key innovation is a multi-turn RL-based optimization mechanism that integrates physics-aware constraints and rendered image evaluations, ensuring generated 3D layouts are coherent, physically plausible, and aesthetically consistent. Methodologically, MetaSpatial introduces an adaptive, iterative reasoning process, where the VLM refines spatial arrangements over multiple turns by analyzing rendered outputs, improving scene coherence progressively. Empirical evaluations demonstrate that MetaSpatial significantly enhances the spatial consistency and formatting stability of various scale models. Post-training, object placements are more realistic, aligned, and functionally coherent, validating the effectiveness of RL for 3D spatial reasoning in metaverse, AR/VR, digital twins, and game development applications. Our code, data, and training pipeline are publicly available at https://github.com/PzySeere/MetaSpatial.

Method: Two-stage framework: 1) SFT with curated CoT data to activate reasoning potential, 2) reinforcement learning using Group Relative Policy Optimization (GRPO) to generate multiple reasoning-response pairs for domain shift adaptability.

Result: Achieves SOTA results, outperforms open-source and proprietary models, maintains robust performance under domain shifts, and excels in few-shot learning scenarios surpassing full-dataset SFT baselines.

Conclusion: Reason-RFT introduces a novel training paradigm for visual reasoning and represents significant progress in multimodal research, addressing limitations of existing CoT fine-tuning approaches.

Abstract: Visual reasoning abilities play a crucial role in understanding complex multimodal data, advancing both domain-specific applications and artificial general intelligence (AGI). Existing methods enhance Vision-Language Models (VLMs) through Chain-of-Thought (CoT) supervised fine-tuning using meticulously annotated data. However, this approach may lead to overfitting and cognitive rigidity, limiting the model’s generalization ability under domain shifts and reducing real-world applicability. To overcome these limitations, we propose Reason-RFT, a two-stage reinforcement fine-tuning framework for visual reasoning. First, Supervised Fine-Tuning (SFT) with curated CoT data activates the reasoning potential of VLMs. This is followed by reinforcement learning based on Group Relative Policy Optimization (GRPO), which generates multiple reasoning-response pairs to enhance adaptability to domain shifts. To evaluate Reason-RFT, we reconstructed a comprehensive dataset covering visual counting, structural perception, and spatial transformation, serving as a benchmark for systematic assessment across three key dimensions. Experimental results highlight three advantages: (1) performance enhancement, with Reason-RFT achieving state-of-the-art results and outperforming both open-source and proprietary models; (2) generalization superiority, maintaining robust performance under domain shifts across various tasks; and (3) data efficiency, excelling in few-shot learning scenarios and surpassing full-dataset SFT baselines. Reason-RFT introduces a novel training paradigm for visual reasoning and marks a significant step forward in multimodal research. Project website: https://tanhuajie.github.io/ReasonRFT

Method: Proposed FVQ-Rater method extracts multi-dimensional features (spatial, temporal, portrait features, and face embeddings) and uses LoRA-based instruction tuning for quality-specific fine-tuning of large multimodal models.

Result: The method shows superior performance on both FVQ-20K and CFVQA datasets, demonstrating significant potential for FVQA development.

Conclusion: The FVQ-20K dataset and FVQ-Rater method significantly advance face video quality assessment research and show promising results for future development in this domain.

Abstract: Face video quality assessment (FVQA) deserves to be explored in addition to general video quality assessment (VQA), as face videos are the primary content on social media platforms and human visual system (HVS) is particularly sensitive to human faces. However, FVQA is rarely explored due to the lack of large-scale FVQA datasets. To fill this gap, we present the first large-scale in-the-wild FVQA dataset, FVQ-20K, which contains 20,000 in-the-wild face videos together with corresponding mean opinion score (MOS) annotations. Along with the FVQ-20K dataset, we further propose a specialized FVQA method named FVQ-Rater to achieve human-like rating and scoring for face video, which is the first attempt to explore the potential of large multimodal models (LMMs) for the FVQA task. Concretely, we elaborately extract multi-dimensional features including spatial features, temporal features, and face-specific features (i.e., portrait features and face embeddings) to provide comprehensive visual information, and take advantage of the LoRA-based instruction tuning technique to achieve quality-specific fine-tuning, which shows superior performance on both FVQ-20K and CFVQA datasets. Extensive experiments and comprehensive analysis demonstrate the significant potential of the FVQ-20K dataset and FVQ-Rater method in promoting the development of FVQA.

Method: Teacher-student framework where teacher learns from gaze points enhanced by VLM-generated lesion descriptions. Uses multi-scale feature alignment, confidence-weighted consistency constraints, and adaptive masking to guide student model.

Result: Achieved Dice scores of 80.78% (Kvasir-SEG), 80.53% (NCI-ISBI), and 84.22% (ISIC) - improving 3-5% over gaze baselines while maintaining clinical interpretability.

Conclusion: Integrating human visual attention with AI-generated semantic context effectively overcomes limitations of individual weak supervision signals, advancing annotation-efficient medical AI systems.

Abstract: Medical image segmentation remains challenging due to the high cost of pixel-level annotations for training. In the context of weak supervision, clinician gaze data captures regions of diagnostic interest; however, its sparsity limits its use for segmentation. In contrast, vision-language models (VLMs) provide semantic context through textual descriptions but lack the explanation precision required. Recognizing that neither source alone suffices, we propose a teacher-student framework that integrates both gaze and language supervision, leveraging their complementary strengths. Our key insight is that gaze data indicates where clinicians focus during diagnosis, while VLMs explain why those regions are significant. To implement this, the teacher model first learns from gaze points enhanced by VLM-generated descriptions of lesion morphology, establishing a foundation for guiding the student model. The teacher then directs the student through three strategies: (1) Multi-scale feature alignment to fuse visual cues with textual semantics; (2) Confidence-weighted consistency constraints to focus on reliable predictions; (3) Adaptive masking to limit error propagation in uncertain areas. Experiments on the Kvasir-SEG, NCI-ISBI, and ISIC datasets show that our method achieves Dice scores of 80.78%, 80.53%, and 84.22%, respectively-improving 3-5% over gaze baselines without increasing the annotation burden. By preserving correlations among predictions, gaze data, and lesion descriptions, our framework also maintains clinical interpretability. This work illustrates how integrating human visual attention with AI-generated semantic context can effectively overcome the limitations of individual weak supervision signals, thereby advancing the development of deployable, annotation-efficient medical AI systems. Code is available at: https://github.com/jingkunchen/FGI.

Method: Proposes MambaMoE with Mixture of Mamba Expert Block (MoMEB) for adaptive spectral-spatial feature modeling via sparse expert activation, and uncertainty-guided corrective learning (UGCL) strategy that samples supervision from high-uncertainty regions.

Result: Extensive experiments on multiple public HSI benchmark datasets show MambaMoE achieves state-of-the-art performance in both classification accuracy and computational efficiency compared to existing advanced methods.

Conclusion: MambaMoE represents the first MoE-based approach in HSI classification domain and demonstrates superior performance through adaptive expert modeling and uncertainty-guided learning.

Abstract: Mamba-based models have recently demonstrated significant potential in hyperspectral image (HSI) classification, primarily due to their ability to perform contextual modeling with linear computational complexity. However, existing Mamba-based approaches often overlook the directional modeling heterogeneity across different land-cover types, leading to limited classification performance. To address these limitations, we propose MambaMoE, a novel spectral-spatial Mixture-of-Experts (MoE) framework, which represents the first MoE-based approach in the HSI classification domain. Specifically, we design a Mixture of Mamba Expert Block (MoMEB) that performs adaptive spectral-spatial feature modeling via a sparse expert activation mechanism. Additionally, we introduce an uncertainty-guided corrective learning (UGCL) strategy that encourages the model to focus on complex regions prone to prediction ambiguity. This strategy dynamically samples supervision signals from regions with high predictive uncertainty, guiding the model to adaptively refine feature representations and thereby enhancing its focus on challenging areas. Extensive experiments conducted on multiple public HSI benchmark datasets show that MambaMoE achieves state-of-the-art performance in both classification accuracy and computational efficiency compared to existing advanced methods, particularly Mamba-based ones. The code will be available online at https://github.com/YichuXu/MambaMoE.

Method: Uses MLLM-based parser to extract structured semantics from text, leverages foundation models to disentangle object generation from spatial placement, and employs hierarchical refinement with spatial reasoning for pose initialization and visual enhancement.

Result: Achieves semantically coherent, spatially precise, and visually realistic 3D insertions without spatial priors, providing a user-friendly editing experience.

Conclusion: FreeInsert offers a flexible and scalable solution for text-driven 3D object insertion by leveraging foundation models to overcome limitations of existing spatial-prior-dependent methods.

Abstract: Text-driven object insertion in 3D scenes is an emerging task that enables intuitive scene editing through natural language. However, existing 2D editing-based methods often rely on spatial priors such as 2D masks or 3D bounding boxes, and they struggle to ensure consistency of the inserted object. These limitations hinder flexibility and scalability in real-world applications. In this paper, we propose FreeInsert, a novel framework that leverages foundation models including MLLMs, LGMs, and diffusion models to disentangle object generation from spatial placement. This enables unsupervised and flexible object insertion in 3D scenes without spatial priors. FreeInsert starts with an MLLM-based parser that extracts structured semantics, including object types, spatial relationships, and attachment regions, from user instructions. These semantics guide both the reconstruction of the inserted object for 3D consistency and the learning of its degrees of freedom. We leverage the spatial reasoning capabilities of MLLMs to initialize object pose and scale. A hierarchical, spatially aware refinement stage further integrates spatial semantics and MLLM-inferred priors to enhance placement. Finally, the appearance of the object is improved using the inserted-object image to enhance visual fidelity. Experimental results demonstrate that FreeInsert achieves semantically coherent, spatially precise, and visually realistic 3D insertions without relying on spatial priors, offering a user-friendly and flexible editing experience.

Method: Created MMLongBench with 13,331 examples spanning 5 task categories (Visual RAG, Many-Shot ICL, etc.), covering various natural and synthetic images, delivered at 5 standardized input lengths using cross-modal tokenization.

Result: Benchmarked 46 LCVLMs revealing that: single task performance poorly predicts overall capability; both closed-source and open-source models struggle with long-context tasks; models with stronger reasoning ability perform better.

Conclusion: MMLongBench provides the essential foundation for diagnosing and advancing next-generation LCVLMs through comprehensive task coverage, diverse image types, and rigorous length control.

Abstract: The rapid extension of context windows in large vision-language models has given rise to long-context vision-language models (LCVLMs), which are capable of handling hundreds of images with interleaved text tokens in a single forward pass. In this work, we introduce MMLongBench, the first benchmark covering a diverse set of long-context vision-language tasks, to evaluate LCVLMs effectively and thoroughly. MMLongBench is composed of 13,331 examples spanning five different categories of downstream tasks, such as Visual RAG and Many-Shot ICL. It also provides broad coverage of image types, including various natural and synthetic images. To assess the robustness of the models to different input lengths, all examples are delivered at five standardized input lengths (8K-128K tokens) via a cross-modal tokenization scheme that combines vision patches and text tokens. Through a thorough benchmarking of 46 closed-source and open-source LCVLMs, we provide a comprehensive analysis of the current models’ vision-language long-context ability. Our results show that: i) performance on a single task is a weak proxy for overall long-context capability; ii) both closed-source and open-source models face challenges in long-context vision-language tasks, indicating substantial room for future improvement; iii) models with stronger reasoning ability tend to exhibit better long-context performance. By offering wide task coverage, various image types, and rigorous length control, MMLongBench provides the missing foundation for diagnosing and advancing the next generation of LCVLMs.

Method: Developed a theoretical framework linking attention block conditioning to embedded token conditioning, then introduced conditioned embedded tokens - a method that systematically modifies embedded tokens to improve attention mechanism conditioning.

Result: The approach significantly mitigates ill-conditioning, leading to more stable and efficient training. Validated across various transformer architectures with consistent improvements in image classification, object detection, instance segmentation, and NLP tasks.

Conclusion: Conditioned embedded tokens provide an effective solution to transformer attention ill-conditioning, demonstrating broad applicability and effectiveness across multiple domains and architectures.

Abstract: Transformers have transformed modern machine learning, driving breakthroughs in computer vision, natural language processing, and robotics. At the core of their success lies the attention mechanism, which enables the modeling of global dependencies among input tokens. However, we reveal that the attention block in transformers suffers from inherent ill-conditioning, which hampers gradient-based optimization and leads to inefficient training. To address this, we develop a theoretical framework that establishes a direct relationship between the conditioning of the attention block and that of the embedded tokenized data. Building on this insight, we introduce conditioned embedded tokens, a method that systematically modifies the embedded tokens to improve the conditioning of the attention mechanism. Our analysis demonstrates that this approach significantly mitigates ill-conditioning, leading to more stable and efficient training. We validate our methodology across various transformer architectures, achieving consistent improvements in image classification, object detection, instance segmentation, and natural language processing, highlighting its broad applicability and effectiveness.

Method: The authors leverage existing polarization datasets to create a benchmark and evaluate diverse state-of-the-art deep learning models, including both restoration-oriented and generative architectures, through extensive quantitative and qualitative analysis.

Result: The benchmark establishes the current performance ceiling for RGB-to-polarization estimation and systematically reveals the strengths and limitations of different model families (direct reconstruction vs generative synthesis, task-specific training vs large-scale pre-training).

Conclusion: This benchmark serves as a foundational resource to facilitate future research on polarization estimation from RGB inputs, with the authors providing potential directions for future work in this area.

Abstract: Polarization images provide rich physical information that is fundamentally absent from standard RGB images, benefiting a wide range of computer vision applications such as reflection separation and material classification. However, the acquisition of polarization images typically requires additional optical components, which increases both the cost and the complexity of the applications. To bridge this gap, we introduce a new task: RGB-to-polarization image estimation, which aims to infer polarization information directly from RGB images. In this work, we establish the first comprehensive benchmark for this task by leveraging existing polarization datasets and evaluating a diverse set of state-of-the-art deep learning models, including both restoration-oriented and generative architectures. Through extensive quantitative and qualitative analysis, our benchmark not only establishes the current performance ceiling of RGB-to-polarization estimation, but also systematically reveals the respective strengths and limitations of different model families – such as direct reconstruction versus generative synthesis, and task-specific training versus large-scale pre-training. In addition, we provide some potential directions for future research on polarization estimation. This benchmark is intended to serve as a foundational resource to facilitate the design and evaluation of future methods for polarization estimation from standard RGB inputs.

Method: Proposes GeoRanker framework with large vision-language models to jointly encode query-candidate interactions, introduces multi-order distance loss for ranking both absolute and relative distances, and creates GeoRanking dataset for geographic ranking tasks.

Result: Achieves state-of-the-art results on IM2GPS3K and YFCC4K benchmarks, significantly outperforming current best methods.

Conclusion: GeoRanker effectively models spatial relationships in image geolocalization through distance-aware ranking and structured reasoning, demonstrating superior performance over existing approaches.

Abstract: Worldwide image geolocalization-the task of predicting GPS coordinates from images taken anywhere on Earth-poses a fundamental challenge due to the vast diversity in visual content across regions. While recent approaches adopt a two-stage pipeline of retrieving candidates and selecting the best match, they typically rely on simplistic similarity heuristics and point-wise supervision, failing to model spatial relationships among candidates. In this paper, we propose GeoRanker, a distance-aware ranking framework that leverages large vision-language models to jointly encode query-candidate interactions and predict geographic proximity. In addition, we introduce a multi-order distance loss that ranks both absolute and relative distances, enabling the model to reason over structured spatial relationships. To support this, we curate GeoRanking, the first dataset explicitly designed for geographic ranking tasks with multimodal candidate information. GeoRanker achieves state-of-the-art results on two well-established benchmarks (IM2GPS3K and YFCC4K), significantly outperforming current best methods.

Method: Decomposes input image into overlapping regions, generates each using a pretrained 3D object generator, then applies masked rectified flow inpainting to fill missing geometry while maintaining structural continuity. Uses region-based generation for better alignment and spatial-aware 3D inpainting for coherence.

Result: Outperforms state-of-the-art baselines (Trellis, Hunyuan3D-2, TripoSG, LGM) in geometry quality, spatial coherence, and texture fidelity across diverse scenes. Achieves high-quality coherent 3D scene generation without 3D supervision or fine-tuning.

Conclusion: High-quality coherent 3D scene-level asset generation is achievable from single top-down images using a principled, training-free pipeline that overcomes resolution limitations and preserves spatial structure.

Abstract: Acquiring detailed 3D scenes typically demands costly equipment, multi-view data, or labor-intensive modeling. Therefore, a lightweight alternative, generating complex 3D scenes from a single top-down image, plays an essential role in real-world applications. While recent 3D generative models have achieved remarkable results at the object level, their extension to full-scene generation often leads to inconsistent geometry, layout hallucinations, and low-quality meshes. In this work, we introduce SceneFuse-3D, a training-free framework designed to synthesize coherent 3D scenes from a single top-down view. Our method is grounded in two principles: region-based generation to improve image-to-3D alignment and resolution, and spatial-aware 3D inpainting to ensure global scene coherence and high-quality geometry generation. Specifically, we decompose the input image into overlapping regions and generate each using a pretrained 3D object generator, followed by a masked rectified flow inpainting process that fills in missing geometry while maintaining structural continuity. This modular design allows us to overcome resolution bottlenecks and preserve spatial structure without requiring 3D supervision or fine-tuning. Extensive experiments across diverse scenes show that SceneFuse-3D outperforms state-of-the-art baselines, including Trellis, Hunyuan3D-2, TripoSG, and LGM, in terms of geometry quality, spatial coherence, and texture fidelity. Our results demonstrate that high-quality coherent 3D scene-level asset generation is achievable from a single top-down image using a principled, training-free pipeline.

Method: Propose Per-Token Distance metric to quantify positional encoding independence, then introduce Circle-RoPE which projects image token indices onto a ring orthogonal to the linear text token axis, forming a cone-like structure. Also use staggered strategy applying different RoPE variants across layers.

Result: Extensive experiments show the method effectively preserves spatial information from images while reducing relative positional bias, providing a more robust and flexible positional encoding framework for VLMs.

Conclusion: Circle-RoPE offers an effective solution to eliminate spurious cross-modal biases in VLMs while maintaining intra-image spatial information, creating a more balanced positional encoding framework for multimodal learning.

Abstract: Rotary Position Embedding (RoPE) is a widely adopted technique for encoding relative positional information in large language models (LLMs). However, when extended to vision-language models (VLMs), RoPE and its variants enforce relative positional dependencies separately within text and image tokens, introducing unintended cross-modal positional biases. For example, image tokens depicting semantically consistent content are assigned distinct positional encodings solely due to spatial location variations. As a result, such tokens exhibit entirely different relative positional relationships with their corresponding text tokens, ultimately leading to misaligned cross-modal representations. To address this, we propose Per-Token Distance, a simple yet effective metric for quantifying the independence of positional encodings across modalities. Informed by this analysis, we introduce Circle-RoPE, a novel encoding scheme designed to eliminate spurious cross-modal biases. Our key idea is to project image token indices onto a \emph{ring} that is orthogonal to the linear axis of text token indices, thereby forming a cone-like structure in the positional encoding space. In this configuration, each text token (point on the linear text axis) becomes the apex of a cone and maintains an equal distance to all image tokens (points on the circular image \emph{ring}), reducing artificial cross-modal biases while preserving intra-image spatial information. To further enhance performance, we propose a staggered strategy that applies different RoPE variants across layers. Extensive experiments demonstrate that our method effectively preserves spatial information from images while reducing relative positional bias, offering a more robust and flexible positional encoding framework for VLMs. The code is available at https://github.com/lose4578/CircleRoPE.

Method: Proposes ViP²-CLIP with Visual-Perception Prompting (ViP-Prompt) mechanism that fuses global and multi-scale local visual context to adaptively generate fine-grained textual prompts, eliminating manual templates and class-name dependencies.

Result: Extensive experiments on 15 industrial and medical benchmarks demonstrate state-of-the-art performance and robust cross-domain generalization.

Conclusion: ViP²-CLIP effectively addresses limitations of existing CLIP-based zero-shot anomaly detection methods by enabling adaptive prompt generation that focuses on precise abnormal regions, making it valuable for scenarios with ambiguous category labels or privacy constraints.

Abstract: Zero-shot anomaly detection (ZSAD) aims to detect anomalies without any target domain training samples, relying solely on external auxiliary data. Existing CLIP-based methods attempt to activate the model’s ZSAD potential via handcrafted or static learnable prompts. The former incur high engineering costs and limited semantic coverage, whereas the latter apply identical descriptions across diverse anomaly types, thus fail to adapt to complex variations. Furthermore, since CLIP is originally pretrained on large-scale classification tasks, its anomaly segmentation quality is highly sensitive to the exact wording of class names, severely constraining prompting strategies that depend on class labels. To address these challenges, we introduce ViP$^{2}$-CLIP. The key insight of ViP$^{2}$-CLIP is a Visual-Perception Prompting (ViP-Prompt) mechanism, which fuses global and multi-scale local visual context to adaptively generate fine-grained textual prompts, eliminating manual templates and class-name priors. This design enables our model to focus on precise abnormal regions, making it particularly valuable when category labels are ambiguous or privacy-constrained. Extensive experiments on 15 industrial and medical benchmarks demonstrate that ViP$^{2}$-CLIP achieves state-of-the-art performance and robust cross-domain generalization.

Method: Uses grounding model for initial bounding box, visual prompt boosting module to enhance prompts, and promptable segmentation model. Introduces test-time adaptation with learnable adaptors optimized via Bayesian Optimization guided by proxy validation model.

Result: Achieves 71.81 Dice Score (69% relative improvement from 42.53) across seven medical imaging datasets, performing competitively with weakly-prompted interactive foundation models.

Conclusion: The pipeline provides an annotation-efficient and scalable solution for zero-shot medical image segmentation across diverse tasks by proper decomposition and test-time adaptation.

Abstract: Medical image segmentation is vital for clinical diagnosis, yet current deep learning methods often demand extensive expert effort, i.e., either through annotating large training datasets or providing prompts at inference time for each new case. This paper introduces a zero-shot and automatic segmentation pipeline that combines off-the-shelf vision-language and segmentation foundation models. Given a medical image and a task definition (e.g., “segment the optic disc in an eye fundus image”), our method uses a grounding model to generate an initial bounding box, followed by a visual prompt boosting module that enhance the prompts, which are then processed by a promptable segmentation model to produce the final mask. To address the challenges of domain gap and result verification, we introduce a test-time adaptation framework featuring a set of learnable adaptors that align the medical inputs with foundation model representations. Its hyperparameters are optimized via Bayesian Optimization, guided by a proxy validation model without requiring ground-truth labels. Our pipeline offers an annotation-efficient and scalable solution for zero-shot medical image segmentation across diverse tasks. Our pipeline is evaluated on seven diverse medical imaging datasets and shows promising results. By proper decomposition and test-time adaptation, our fully automatic pipeline not only substantially surpasses the previously best-performing method, yielding a 69% relative improvement in accuracy (Dice Score from 42.53 to 71.81), but also performs competitively with weakly-prompted interactive foundation models.

Method: Augment only the portion of data that is not learned early in training with faithful images containing same features but different noise, rather than augmenting the entire dataset.

Result: Method boosts generalization by up to 2.8% across ResNet, ViT, ConvNeXt, and Swin Transformer on CIFAR-10/100 and TinyImageNet, with various optimizers. Applied with SGD, it outperforms SOTA optimizer SAM on CIFAR-100 and TinyImageNet.

Conclusion: Selective augmentation of poorly learned data promotes homogeneity in feature learning speed without amplifying noise, providing more effective generalization than full dataset augmentation.

Abstract: Synthetically augmenting training datasets with diffusion models has been an effective strategy for improving generalization of image classifiers. However, existing techniques struggle to ensure the diversity of generation and increase the size of the data by up to 10-30x to improve the in-distribution performance. In this work, we show that synthetically augmenting part of the data that is not learned early in training with faithful images-containing same features but different noise-outperforms augmenting the entire dataset. By analyzing a two-layer CNN, we prove that this strategy improves generalization by promoting homogeneity in feature learning speed without amplifying noise. Our extensive experiments show that by augmenting only 30%-40% of the data, our method boosts generalization by up to 2.8% in a variety of scenarios, including training ResNet, ViT, ConvNeXt, and Swin Transformer on CIFAR-10/100, and TinyImageNet, with various optimizers including SGD and SAM. Notably, our method applied with SGD outperforms the SOTA optimizer, SAM, on CIFAR-100 and TinyImageNet.

Method: CryoCCD unifies versatile biophysical modeling with a conditional cycle-consistent diffusion model, enhanced with mask-guided contrastive learning to ensure realistic noise while preserving structural fidelity.

Result: Extensive experiments show CryoCCD generates structurally faithful micrographs, improves particle picking and pose estimation, and outperforms state-of-the-art baselines while generalizing to held-out protein families.

Conclusion: CryoCCD provides an effective solution for synthetic cryo-EM data generation that captures authentic biological organization and realistic noise, enabling better development of cryo-EM processing tools.

Abstract: Single-particle cryo-electron microscopy (cryo-EM) has become a cornerstone of structural biology, enabling near-atomic resolution analysis of macromolecules through advanced computational methods. However, the development of cryo-EM processing tools is constrained by the scarcity of high-quality annotated datasets. Synthetic data generation offers a promising alternative, but existing approaches lack thorough biophysical modeling of heterogeneity and fail to reproduce the complex noise observed in real imaging. To address these limitations, we present CryoCCD, a synthesis framework that unifies versatile biophysical modeling with the first conditional cycle-consistent diffusion model tailored for cryo-EM. The biophysical engine provides multi-functional generation capabilities to capture authentic biological organization, and the diffusion model is enhanced with cycle consistency and mask-guided contrastive learning to ensure realistic noise while preserving structural fidelity. Extensive experiments demonstrate that CryoCCD generates structurally faithful micrographs, enhances particle picking and pose estimation, as well as achieves superior performance over state-of-the-art baselines, while also generalizing effectively to held-out protein families.

Method: A dual-scale coarse-to-fine framework with two stages: first generates low-resolution try-on results for robust structural alignment, then uses a blend-refine diffusion process to reconstruct high-resolution outputs by refining scale residuals through noise-image blending.

Result: DS-VTON achieves state-of-the-art performance and significantly surpasses prior methods in both structural alignment and texture fidelity across multiple standard virtual try-on benchmarks.

Conclusion: The proposed DS-VTON framework effectively addresses the core challenges of virtual try-on through its dual-scale approach and mask-free generation strategy, demonstrating superior performance in both structural alignment and texture preservation.

Abstract: Despite recent progress, most existing virtual try-on methods still struggle to simultaneously address two core challenges: accurately aligning the garment image with the target human body, and preserving fine-grained garment textures and patterns. These two requirements map directly onto a coarse-to-fine generation paradigm, where the coarse stage handles structural alignment and the fine stage recovers rich garment details. Motivated by this observation, we propose DS-VTON, an enhanced dual-scale coarse-to-fine framework that tackles the try-on problem more effectively. DS-VTON consists of two stages: the first stage generates a low-resolution try-on result to capture the semantic correspondence between garment and body, where reduced detail facilitates robust structural alignment. In the second stage, a blend-refine diffusion process reconstructs high-resolution outputs by refining the residual between scales through noise-image blending, emphasizing texture fidelity and effectively correcting fine-detail errors from the low-resolution stage. In addition, our method adopts a fully mask-free generation strategy, eliminating reliance on human parsing maps or segmentation masks. Extensive experiments show that DS-VTON not only achieves state-of-the-art performance but consistently and significantly surpasses prior methods in both structural alignment and texture fidelity across multiple standard virtual try-on benchmarks.

Method: Propose UTAMoE framework that decouples internal AR modules via Task-Aware MoE Layer to create task-specific optimization subpaths, combined with a novel Two-Stage Training Strategy to enhance task differentiation while maintaining coordination.

Result: Extensive experiments show UTAMoE mitigates task objective conflicts and achieves state-of-the-art performance across various multimodal benchmarks, with visualizations and ablation studies validating effectiveness.

Conclusion: The proposed UTAMoE framework successfully resolves task objective conflicts in multimodal LLMs through internal AR module decoupling and task-aware optimization, demonstrating superior performance over existing approaches.

Abstract: Unified multimodal large language models (MLLMs) based on end-to-end autoregressive (AR) transformers effectively integrate both understanding and generation tasks within a single framework. However, intrinsic Task Objective Conflicts between high-level semantic abstraction in understanding and fine-grained detail preservation in generation pose significant challenges, often leading to suboptimal trade-offs and task interference. Existing solutions, such as decoupling shared visual encoders, fall short of fundamentally resolving these conflicts due to inherent AR architecture. In this paper, we propose a novel approach that decouples internal components of AR to resolve task objective conflicts. Specifically, we design UTAMoE, a Unified Task-Aware Mixture-of-Experts (MoE) framework that decouples internal AR modules via a Task-Aware MoE Layer to create task-specific optimization subpaths. To enhance task differentiation while maintaining overall coordination, we introduce a novel Two-Stage Training Strategy. Extensive experiments on multimodal benchmarks demonstrate that UTAMoE mitigates task objective conflicts, achieving state-of-the-art performance across various tasks. Visualizations and ablation studies further validate the effectiveness of our approach.

Method: Two-stage neuro-symbolic system: front-end LLM generates structured reasoning plans, back-end executes them using neural models and symbolic functions, with SS-parser for error detection/repair and execution verifier for stepwise validation.

Result: Outperforms a dozen state-of-the-art visual reasoning models on six visual benchmarks.

Conclusion: VLAgent demonstrates that neuro-symbolic approaches with structured planning and verification mechanisms can significantly improve compositional visual reasoning capabilities.

Abstract: The advancement in large language models (LLMs) and large vision models has fueled the rapid progress in multi-modal vision-language reasoning capabilities. However, existing vision-language models (VLMs) remain challenged by compositional visual reasoning. This paper presents VLAgent, a neuro-symbolic approach to developing a Vision-Language Agent system for efficient compositional visual reasoning with three novel features. First, VLAgent develops an interpretable visualization-enhanced two-stage neuro-symbolic reasoning system. The first stage is managed by a front-end engine that generates a structured visual reasoning plan (symbolic program script) for each compositional visual reasoning task by utilizing a pre-trained LLM powered with few-shot chain-of-thought in-context learning. The second stage is managed by a high-performance back-end engine. It transforms the planning script into executable code based on visual input (image or video) and the combination of neural models and symbolic functions and then performs a sequence of actions for the compositional visual reason task. Second, to ensure and enhance the quality of mapping the logic plan to a sequence of executable instructions, VLAgent introduces the SS-parser, which examines the syntax and semantic correctness of the planning script, detects and repairs the logic errors found in the LLM-generated logic plan before generating the executable program. Third, VLAgent introduces the execution verifier in critical reasoning steps to validate and refine its compositional reasoning results in a stepwise manner, for example, ensemble methods for critical visual reasoning and caption analysis for low-confidence compositional reasoning. Extensive experiments on six visual benchmarks compared to a dozen SoTA visual reasoning models show that VLAgent outperforms existing representative approaches to compositional visual reasoning.

Method: Proposes a novel video editing pipeline that integrates GANs and diffusion models. First segments robotic arms from backgrounds, trains an unpaired GAN to translate one robotic arm to another, blends the translated arm with original background, and refines with diffusion model for coherence and motion realism.

Result: Outperforms state-of-the-art video and image editing models on three benchmarks in terms of structural coherence and motion consistency, providing a robust solution for generating reliable cross-embodiment data.

Conclusion: RoboSwap offers an effective framework for cross-embodiment learning by leveraging unpaired data and combining the advantages of both GANs and diffusion models, reducing data collection needs while maintaining high quality video synthesis.

Abstract: Recent advancements in generative models have revolutionized video synthesis and editing. However, the scarcity of diverse, high-quality datasets continues to hinder video-conditioned robotic learning, limiting cross-platform generalization. In this work, we address the challenge of swapping a robotic arm in one video with another: a key step for crossembodiment learning. Unlike previous methods that depend on paired video demonstrations in the same environmental settings, our proposed framework, RoboSwap, operates on unpaired data from diverse environments, alleviating the data collection needs. RoboSwap introduces a novel video editing pipeline integrating both GANs and diffusion models, combining their isolated advantages. Specifically, we segment robotic arms from their backgrounds and train an unpaired GAN model to translate one robotic arm to another. The translated arm is blended with the original video background and refined with a diffusion model to enhance coherence, motion realism and object interaction. The GAN and diffusion stages are trained independently. Our experiments demonstrate that RoboSwap outperforms state-of-the-art video and image editing models on three benchmarks in terms of both structural coherence and motion consistency, thereby offering a robust solution for generating reliable, cross-embodiment data in robotic learning.

Method: Created WetCat dataset with high-resolution recordings of cataract surgeries on artificial eyes, including comprehensive phase annotations and semantic segmentations of key anatomical structures during capsulorhexis and phacoemulsification phases.

Result: Developed a publicly available dataset that enables interpretable AI-driven evaluation tools for surgical skill assessment, supporting standardized clinical metrics and workflow analysis.

Conclusion: WetCat establishes a foundation for objective, scalable surgical education and sets a new benchmark for automated skill assessment in ophthalmology training.

Abstract: To meet the growing demand for systematic surgical training, wet-lab environments have become indispensable platforms for hands-on practice in ophthalmology. Yet, traditional wet-lab training depends heavily on manual performance evaluations, which are labor-intensive, time-consuming, and often subject to variability. Recent advances in computer vision offer promising avenues for automated skill assessment, enhancing both the efficiency and objectivity of surgical education. Despite notable progress in ophthalmic surgical datasets, existing resources predominantly focus on real surgeries or isolated tasks, falling short of supporting comprehensive skill evaluation in controlled wet-lab settings. To address these limitations, we introduce WetCat, the first dataset of wet-lab cataract surgery videos specifically curated for automated skill assessment. WetCat comprises high-resolution recordings of surgeries performed by trainees on artificial eyes, featuring comprehensive phase annotations and semantic segmentations of key anatomical structures. These annotations are meticulously designed to facilitate skill assessment during the critical capsulorhexis and phacoemulsification phases, adhering to standardized surgical skill assessment frameworks. By focusing on these essential phases, WetCat enables the development of interpretable, AI-driven evaluation tools aligned with established clinical metrics. This dataset lays a strong foundation for advancing objective, scalable surgical education and sets a new benchmark for automated workflow analysis and skill assessment in ophthalmology training. The dataset and annotations are publicly available in Synapse.

Method: Proposes Efficient Probing (EP) - a simple multi-query cross-attention mechanism that eliminates redundant projections and reduces trainable parameters while selectively aggregating patch-level features.

Result: EP outperforms linear probing and prior attentive probing approaches across seven benchmarks, generalizes well to diverse pre-training paradigms, and delivers strong low-shot and layer-wise gains.

Conclusion: Efficient Probing provides an effective alternative to linear probing, uncovering emerging properties like complementary attention maps that open new directions for leveraging probing beyond protocol design.

Abstract: As fine-tuning becomes increasingly impractical at scale, probing is emerging as the preferred evaluation protocol. Yet, the standard linear probing fails to adequately reflect the potential of models whose pre-training optimizes representations of patch tokens rather than an explicit global representation. This motivates the need for attentive probing, an alternative that uses attention to selectively aggregate patch-level features. Despite its growing adoption, attentive probing remains under-explored, with existing methods suffering from excessive parameterization and poor computational efficiency. In this work, we revisit attentive probing through the lens of the accuracy vs. parameter efficiency trade-off. We present the first comprehensive study of existing methods, analyzing their design choices and benchmarking their performance. Building on this, we propose efficient probing (EP), a simple yet effective multi-query cross-attention mechanism that eliminates redundant projections and reduces the number of trainable parameters. Despite its simplicity, EP outperforms linear probing and prior attentive probing approaches across seven benchmarks, generalizes well to diverse pre-training paradigms, and delivers strong low-shot and layer-wise gains. Beyond evaluation, our analysis uncovers emerging properties of EP, such as complementary attention maps, which open new directions for leveraging probing beyond protocol design. Code available at https://github.com/billpsomas/efficient-probing.

Method: Developed an Automatic Scene Creation Engine using collaborative LLM agents to translate abstract mathematical problems into faithful 3D scenes, creating 9,000 video-question-answer triplets embedded in realistic 3D environments.

Result: State-of-the-art VLMs struggle significantly on SIRI-Bench, highlighting the challenge of structural spatial reasoning and the gap in current VLM capabilities.

Conclusion: The study aims to bring attention to spatially grounded reasoning and advance VLMs in visual problem-solving by providing a comprehensive benchmark for evaluating structural spatial intelligence.

Abstract: Large Language Models (LLMs) have undergone rapid progress, largely attributed to reinforcement learning on complex reasoning tasks. In contrast, while spatial intelligence is fundamental for Vision-Language Models (VLMs) in real-world interaction, the systematic study of their complex spatial reasoning remains underexplored. To bridge this gap, we introduce SIRI-Bench, a benchmark designed to evaluate VLMs’ structural spatial intelligence through spatial-grounded reasoning tasks. SIRI-Bench comprises 9,000 video-question-answer triplets, where each problem is embedded in a realistic 3D scene. The benchmark is carefully designed so that solving each problem requires both spatial comprehension and structural reasoning. To facilitate large-scale data synthesis, we develop an Automatic Scene Creation Engine that employs collaborative LLM agents to translate abstract mathematical problems into faithful 3D scenes. Experimental results reveal that state-of-the-art VLMs struggle significantly on SIRI-Bench, underscoring the challenge of structural spatial reasoning. We hope that our study will bring researchers’ attention to spatially grounded reasoning and advance VLMs in visual problem-solving.

Method: Uses mask-based visual grounding to identify action-relevant objects and classify them for targeted modifications. Includes automated pipeline to extract high-quality initial, action, and final state frames from videos.

Result: Shows improvements over state-of-the-art methods in quantitative and qualitative evaluations on three egocentric video datasets.

Conclusion: VisualChef effectively generates contextual cooking visuals that depict both action execution and resulting appearance while maintaining environmental consistency through simplified alignment approach.

Abstract: Cooking requires not only following instructions but also understanding, executing, and monitoring each step - a process that can be challenging without visual guidance. Although recipe images and videos offer helpful cues, they often lack consistency in focus, tools, and setup. To better support the cooking process, we introduce VisualChef, a method for generating contextual visual aids tailored to cooking scenarios. Given an initial frame and a specified action, VisualChef generates images depicting both the action’s execution and the resulting appearance of the object, while preserving the initial frame’s environment. Previous work aims to integrate knowledge extracted from large language models by generating detailed textual descriptions to guide image generation, which requires fine-grained visual-textual alignment and involves additional annotations. In contrast, VisualChef simplifies alignment through mask-based visual grounding. Our key insight is identifying action-relevant objects and classifying them to enable targeted modifications that reflect the intended action and outcome while maintaining a consistent environment. In addition, we propose an automated pipeline to extract high-quality initial, action, and final state frames. We evaluate VisualChef quantitatively and qualitatively on three egocentric video datasets and show its improvements over state-of-the-art methods.

Method: Light Register Tokens with alignment supervision, Interleaved Attention Blocks with global cross-image attention, Wavelet-based Dual-branch Architecture, and Normal-gradient Perception Loss. Also introduces PS-Verse dataset and curriculum training.

Result: Achieves state-of-the-art results on DiLiGenT and Luces benchmarks, shows stronger generalization to real materials, and improved efficiency. Ablations confirm effectiveness of proposed components.

Conclusion: LINO UniPS successfully addresses illumination-normal decoupling and detail preservation challenges in universal photometric stereo through novel architectural designs and training strategies.

Abstract: Universal photometric stereo (PS) is defined by two factors: it must (i) operate under arbitrary, unknown lighting conditions and (ii) avoid reliance on specific illumination models. Despite progress (e.g., SDM UniPS), two challenges remain. First, current encoders cannot guarantee that illumination and normal information are decoupled. To enforce decoupling, we introduce LINO UniPS with two key components: (i) Light Register Tokens with light alignment supervision to aggregate point, direction, and environment lights; (ii) Interleaved Attention Block featuring global cross-image attention that takes all lighting conditions together so the encoder can factor out lighting while retaining normal-related evidence. Second, high-frequency geometric details are easily lost. We address this with (i) a Wavelet-based Dual-branch Architecture and (ii) a Normal-gradient Perception Loss. These techniques yield a unified feature space in which lighting is explicitly represented by register tokens, while normal details are preserved via wavelet branch. We further introduce PS-Verse, a large-scale synthetic dataset graded by geometric complexity and lighting diversity, and adopt curriculum training from simple to complex scenes. Extensive experiments show new state-of-the-art results on public benchmarks (e.g., DiLiGenT, Luces), stronger generalization to real materials, and improved efficiency; ablations confirm that Light Register Tokens + Interleaved Attention Block drive better feature decoupling, while Wavelet-based Dual-branch Architecture + Normal-gradient Perception Loss recover finer details.

Method: Created ImplicitQA benchmark with 1K QA pairs from 1K creative video clips across 15 genres and 7 decades. Questions systematically cover 9 reasoning dimensions including spatial reasoning, motion, causal reasoning, social interactions, and more. Annotations were crafted by authors, validated by multiple annotators, and benchmarked against human performance.

Result: Evaluation of 11 leading VideoQA models showed consistent and significant performance degradation on ImplicitQA, demonstrating their reliance on surface-level visual cues and inability to perform complex implicit reasoning.

Conclusion: ImplicitQA exposes critical limitations in current VideoQA models’ reasoning capabilities and highlights the need for models that can perform human-like implicit reasoning across discontinuous visual contexts in creative content.

Abstract: Video Question Answering (VideoQA) has made significant strides by leveraging multimodal learning to align visual and textual modalities. However, current benchmarks overwhelmingly focus on questions answerable through explicit visual content - actions, objects, and events directly observable within individual frames or short clips. In contrast, creative and cinematic videos - such as movies, TV shows, and narrative-driven content - employ storytelling techniques that deliberately omit certain depictions, requiring viewers to infer motives, relationships across discontinuous frames with disjoint visual contexts. Humans naturally excel at such implicit reasoning, seamlessly integrating information across time and context to construct coherent narratives. Yet current benchmarks fail to capture this essential dimension of human-like understanding. To bridge this gap, we present ImplicitQA, a novel benchmark specifically designed to test VideoQA models on human-like implicit reasoning. ImplicitQA comprises 1K meticulously annotated QA pairs drawn from 1K high-quality creative video clips covering 15 genres across 7 decades of content. Questions are systematically categorized into nine key reasoning dimensions: lateral and vertical spatial reasoning, depth and proximity, viewpoint and visibility, motion and trajectory, causal and motivational reasoning, social interactions, physical context, and inferred counting. These annotations are deliberately challenging, crafted by authors, validated through multiple annotators, and benchmarked against human performance to ensure high quality. Our extensive evaluations on 11 leading VideoQA models reveals consistent and significant performance degradation, underscoring their reliance on surface-level visual cues and highlighting the difficulty of implicit reasoning. https://huggingface.co/datasets/ucf-crcv/ImplicitQA.

Method: Proposes VSRM framework with Spatial-to-Temporal and Temporal-to-Spatial Mamba blocks for long-range spatio-temporal feature extraction. Introduces Deformable Cross-Mamba Alignment for dynamic frame alignment and Frequency Charbonnier-like loss to minimize frequency domain gaps.

Result: VSRM achieves state-of-the-art results on diverse benchmarks, demonstrating superior performance in video super-resolution tasks compared to existing methods.

Conclusion: VSRM establishes itself as a solid foundation for future video super-resolution research by effectively leveraging Mamba’s capabilities for efficient long-sequence modeling and high-quality reconstruction.

Abstract: Video super-resolution remains a major challenge in low-level vision tasks. To date, CNN- and Transformer-based methods have delivered impressive results. However, CNNs are limited by local receptive fields, while Transformers struggle with quadratic complexity, posing challenges for processing long sequences in VSR. Recently, Mamba has drawn attention for its long-sequence modeling, linear complexity, and large receptive fields. In this work, we propose VSRM, a novel \textbf{V}ideo \textbf{S}uper-\textbf{R}esolution framework that leverages the power of \textbf{M}amba. VSRM introduces Spatial-to-Temporal Mamba and Temporal-to-Spatial Mamba blocks to extract long-range spatio-temporal features and enhance receptive fields efficiently. To better align adjacent frames, we propose Deformable Cross-Mamba Alignment module. This module utilizes a deformable cross-mamba mechanism to make the compensation stage more dynamic and flexible, preventing feature distortions. Finally, we minimize the frequency domain gaps between reconstructed and ground-truth frames by proposing a simple yet effective Frequency Charbonnier-like loss that better preserves high-frequency content and enhances visual quality. Through extensive experiments, VSRM achieves state-of-the-art results on diverse benchmarks, establishing itself as a solid foundation for future research.

Method: Evaluated eight LMMs (four open-weight, four closed-weight) using contextual metadata from GPS coordinates (location/venue type), timestamps (meal/day type), and food items. Tested various reasoning modifiers including Chain-of-Thought, Multimodal Chain-of-Thought, Scale Hint, Few-Shot, and Expert Persona.

Result: Integrating contextual metadata significantly reduces Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) in predicted nutritional values compared to straightforward prompting with images alone.

Conclusion: Context-aware LMMs have significant potential for improved nutrition analysis, and intelligent metadata integration enhances the efficacy of reasoning modifiers.

Abstract: Large Multimodal Models (LMMs) are increasingly applied to meal images for nutrition analysis. However, existing work primarily evaluates proprietary models, such as GPT-4. This leaves the broad range of LLMs underexplored. Additionally, the influence of integrating contextual metadata and its interaction with various reasoning modifiers remains largely uncharted. This work investigates how interpreting contextual metadata derived from GPS coordinates (converted to location/venue type), timestamps (transformed into meal/day type), and the food items present can enhance LMM performance in estimating key nutritional values. These values include calories, macronutrients (protein, carbohydrates, fat), and portion sizes. We also introduce \textbf{ACETADA}, a new food-image dataset slated for public release. This open dataset provides nutrition information verified by the dietitian and serves as the foundation for our analysis. Our evaluation across eight LMMs (four open-weight and four closed-weight) first establishes the benefit of contextual metadata integration over straightforward prompting with images alone. We then demonstrate how this incorporation of contextual information enhances the efficacy of reasoning modifiers, such as Chain-of-Thought, Multimodal Chain-of-Thought, Scale Hint, Few-Shot, and Expert Persona. Empirical results show that integrating metadata intelligently, when applied through straightforward prompting strategies, can significantly reduce the Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) in predicted nutritional values. This work highlights the potential of context-aware LMMs for improved nutrition analysis.

Method: Divergence Minimization Preference Optimization (DMPO) - a method that aligns diffusion models by minimizing reverse KL divergence, which asymptotically matches the optimization direction of original reinforcement learning.

Result: DMPO consistently outperforms all baseline models across different base models and test sets, achieving the best PickScore in every case, demonstrating superior alignment with desired outputs.

Conclusion: DMPO provides a robust and elegant pathway for preference alignment in diffusion models, effectively bridging principled theory with practical performance improvements.

Abstract: Diffusion models have achieved remarkable success in generating realistic and versatile images from text prompts. Inspired by the recent advancements of language models, there is an increasing interest in further improving the models by aligning with human preferences. However, we investigate alignment from a divergence minimization perspective and reveal that existing preference optimization methods are typically trapped in suboptimal mean-seeking optimization. In this paper, we introduce Divergence Minimization Preference Optimization (DMPO), a novel and principled method for aligning diffusion models by minimizing reverse KL divergence, which asymptotically enjoys the same optimization direction as original RL. We provide rigorous analysis to justify the effectiveness of DMPO and conduct comprehensive experiments to validate its empirical strength across both human evaluations and automatic metrics. Our extensive results show that diffusion models fine-tuned with DMPO can consistently outperform or match existing techniques, specifically consistently outperforming all baseline models across different base models and test sets, achieving the best PickScore in every case, demonstrating the method’s superiority in aligning generative behavior with desired outputs. Overall, DMPO unlocks a robust and elegant pathway for preference alignment, bridging principled theory with practical performance in diffusion models.

Method: Modified StarDist-3D architecture using convolutional neural networks to predict instances as closed surfaces composed of bicubic Bézier triangles, decoupling parameterization from resolution.

Result: SurfDist outperforms StarDist-3D on both synthetic and real-world datasets with blob-shaped instances, achieving better performance with more compact parameterizations.

Conclusion: Interpretable instance surface models can be effectively learned alongside instance membership, providing high-resolution segmentation without voxelization artifacts.

Abstract: We present SurfDist, a convolutional neural network architecture for three-dimensional volumetric instance segmentation. SurfDist enables prediction of instances represented as closed surfaces composed of smooth parametric surface patches, specifically bicubic B'ezier triangles. SurfDist is a modification of the popular model architecture StarDist-3D which breaks StarDist-3D’s coupling of instance parameterization dimension and instance voxel resolution, and it produces predictions which may be upsampled to arbitrarily high resolutions without introduction of voxelization artifacts. For datasets with blob-shaped instances, common in biomedical imaging, SurfDist can outperform StarDist-3D with more compact instance parameterizations. We detail SurfDist’s technical implementation and show one synthetic and one real-world dataset for which it outperforms StarDist-3D. These results demonstrate that interpretable instance surface models can be learned effectively alongside instance membership.

Method: Created UGC-VideoCap benchmark with 1000 TikTok videos annotated through three-stage human-in-the-loop pipeline (audio-only, visual-only, joint audio-visual). Proposed UGC-VideoCaptioner(3B) model distilled from Gemini 2.5 Flash using two-stage training: supervised fine-tuning followed by Group Relative Policy Optimization (GRPO).

Result: Developed a comprehensive benchmark with 4000 carefully crafted QA pairs probing unimodal and cross-modal understanding. The model enables efficient adaptation from limited data while maintaining competitive performance.

Conclusion: UGC-VideoCap provides a high-quality foundation and data-efficient solution for advancing omnimodal video captioning in unconstrained real-world user-generated content settings, addressing the audio-visual imbalance in current approaches.

Abstract: Real-world user-generated videos, especially on platforms like TikTok, often feature rich and intertwined audio visual content. However, existing video captioning benchmarks and models remain predominantly visual centric, overlooking the crucial role of audio in conveying scene dynamics, speaker intent, and narrative context. This lack of omni datasets and lightweight, capable models hampers progress in fine grained, multimodal video understanding. To address these challenges, we introduce UGC-VideoCap, a new benchmark and model framework specifically designed for detailed omnimodal captioning of short form user-generated videos. Unlike prior datasets, UGC-VideoCap emphasizes balanced integration of audio and visual modalities, featuring 1000 TikTok videos annotated through a structured three stage human-in-the-loop pipeline covering audio only, visual only, and joint audio visual semantics. The benchmark also includes 4000 carefully crafted QA pairs probing both unimodal and cross modal understanding. Alongside the dataset, we propose UGC-VideoCaptioner(3B), a 3B parameter captioning model distilled from Gemini 2.5 Flash. Using a novel two-stage training strategy supervised fine tuning followed by Group Relative Policy Optimization (GRPO), our approach enables efficient adaptation from limited data while maintaining competitive performance. Together, our benchmark and model offer a high-quality foundation and a data-efficient solution for advancing omnimodal video captioning in unconstrained real-world UGC settings.

Method: DiViD uses a sequence encoder to extract global static tokens from the first frame and per-frame dynamic tokens, with a conditional DDPM decoder incorporating shared-noise schedules, time-varying KL-based bottlenecks, cross-attention mechanisms, and orthogonality regularization to prevent static-dynamic leakage.

Result: DiViD achieves the highest swap-based joint accuracy, preserves static fidelity while improving dynamic transfer, and reduces average cross-leakage compared to state-of-the-art sequential disentanglement methods on real-world benchmarks.

Conclusion: DiViD successfully demonstrates effective static-dynamic factorization in videos through its novel diffusion-based framework with explicit inductive biases and regularization techniques.

Abstract: Unsupervised disentanglement of static appearance and dynamic motion in video remains a fundamental challenge, often hindered by information leakage and blurry reconstructions in existing VAE- and GAN-based approaches. We introduce DiViD, the first end-to-end video diffusion framework for explicit static-dynamic factorization. DiViD’s sequence encoder extracts a global static token from the first frame and per-frame dynamic tokens, explicitly removing static content from the motion code. Its conditional DDPM decoder incorporates three key inductive biases: a shared-noise schedule for temporal consistency, a time-varying KL-based bottleneck that tightens at early timesteps (compressing static information) and relaxes later (enriching dynamics), and cross-attention that routes the global static token to all frames while keeping dynamic tokens frame-specific. An orthogonality regularizer further prevents residual static-dynamic leakage. We evaluate DiViD on real-world benchmarks using swap-based accuracy and cross-leakage metrics. DiViD outperforms state-of-the-art sequential disentanglement methods: it achieves the highest swap-based joint accuracy, preserves static fidelity while improving dynamic transfer, and reduces average cross-leakage.

Method: Three-stage process: (1) visual abstraction via captioning, (2) intent inference through few-shot chain-of-thought prompting, (3) intent-conditioned response generation.

Result: Extensive experiments on safety benchmarks (SIUO, MM-SafetyBench, HoliSafe) show SIA consistently improves safety and outperforms prior training-free methods.

Conclusion: SIA effectively mitigates harmful outputs in vision-language models by dynamically adapting to implicit intent without requiring extensive retraining.

Abstract: With the growing deployment of Vision-Language Models (VLMs) in real-world applications, previously overlooked safety risks are becoming increasingly evident. In particular, seemingly innocuous multimodal inputs can combine to reveal harmful intent, leading to unsafe model outputs. While multimodal safety has received increasing attention, existing approaches often fail to address such latent risks, especially when harmfulness arises only from the interaction between modalities. We propose SIA (Safety via Intent Awareness), a training-free, intent-aware safety framework that proactively detects harmful intent in multimodal inputs and uses it to guide the generation of safe responses. SIA follows a three-stage process: (1) visual abstraction via captioning; (2) intent inference through few-shot chain-of-thought (CoT) prompting; and (3) intent-conditioned response generation. By dynamically adapting to the implicit intent inferred from an image-text pair, SIA mitigates harmful outputs without extensive retraining. Extensive experiments on safety benchmarks, including SIUO, MM-SafetyBench, and HoliSafe, show that SIA consistently improves safety and outperforms prior training-free methods.

Method: Constructed SAR-TEXT dataset using SAR-Narrator framework (multi-stage strategy for generating textual descriptions), then built three models: SAR-RS-CLIP for retrieval, SAR-RS-CoCa for captioning, and SAR-GPT for VQA.

Result: SAR-RS-CLIP improved average recall by 12.97% and 10.0% on test sets; SAR-RS-CoCa achieved significant improvements in captioning metrics; SAR-GPT outperformed baselines on VQA tasks with stronger semantic understanding.

Conclusion: SAR-TEXT dataset effectively advances SAR semantic understanding, and SAR-Narrator provides a flexible tool for the community to construct larger-scale datasets.

Abstract: Vision Language Models (VLMs) have achieved remarkable breakthroughs in the field of remote sensing in recent years. Synthetic Aperture Radar (SAR) imagery, with its all-weather capability, is essential in remote sensing, yet the lack of large-scale, high-quality SAR image-text datasets hinders its semantic understanding. In this paper, we construct SAR-TEXT, a large-scale and high-quality dataset consisting of over 130,000 SAR image-text pairs. To construct the SAR-TEXT dataset, we design the SAR-Narrator framework, which generates textual descriptions for SAR images through a multi-stage strategy. To verify the effectiveness of the SAR-TEXT dataset, we conduct experiments on three typical vision-language tasks: image-text retrieval, image captioning, and visual question answering (VQA). Specifically, we construct three representative models on SAR-TEXT: SAR-RS-CLIP, SAR-RS-CoCa, and SAR-GPT. SAR-RS-CLIP achieves notable improvements in retrieval performance, boosting average recall by 12.97% and 10.0% on the OSdataset_512 and HRSID test sets, respectively. In the captioning task, SAR-RS-CoCa achieves significant improvements over the original CoCa models in terms of BLEU-4, SPICE, and CIDEr scores. In the VQA task, SAR-GPT outperforms baseline and single-stage models on multiple SAR-VQA datasets, demonstrating stronger semantic understanding and reasoning ability, as further confirmed by qualitative results. It is worth noting that, as a flexible captioning tool, SAR-Narrator can be readily adopted by the community to construct larger-scale SAR image-text datasets. All code, pretrained models, and the SAR-Text dataset are publicly available at: https://github.com/YiguoHe/SAR-TEXT.

Method: Proposed framework integrates 3D-adaptive SFM Decision Mechanism and Personalized Gait Control Motor, enabling parallel agent movement with dynamic crowd awareness and part-level force visualization.

Result: Framework supports dynamic trajectory planning and personalized behavior for each agent, compatible with uneven terrain, generating more realistic and plausible evacuation results.

Conclusion: The method provides enhanced insights for crowd simulation with more realistic evacuation behaviors across various scenarios, with code publicly available.

Abstract: Crowd evacuation simulation is critical for enhancing public safety, and demanded for realistic virtual environments. Current mainstream evacuation models overlook the complex human behaviors that occur during evacuation, such as pedestrian collisions, interpersonal interactions, and variations in behavior influenced by terrain types or individual body shapes. This results in the failure to accurately simulate the escape of people in the real world. In this paper, aligned with the sensory-decision-motor (SDM) flow of the human brain, we propose a real-time 3D crowd evacuation simulation framework that integrates a 3D-adaptive SFM (Social Force Model) Decision Mechanism and a Personalized Gait Control Motor. This framework allows multiple agents to move in parallel and is suitable for various scenarios, with dynamic crowd awareness. Additionally, we introduce Part-level Force Visualization to assist in evacuation analysis. Experimental results demonstrate that our framework supports dynamic trajectory planning and personalized behavior for each agent throughout the evacuation process, and is compatible with uneven terrain. Visually, our method generates evacuation results that are more realistic and plausible, providing enhanced insights for crowd simulation. The code is available at http://cic.tju.edu.cn/faculty/likun/projects/RESCUE.

Method: ReMoMask integrates three innovations: 1) Bidirectional Momentum Text-Motion Model with momentum queues, 2) Semantic Spatio-temporal Attention for biomechanical constraints, 3) RAG-Classier-Free Guidance with minor unconditional generation.

Result: ReMoMask achieves state-of-the-art performance with 3.88% and 10.97% improvement in FID scores on HumanML3D and KIT-ML benchmarks respectively compared to previous SOTA method RAG-T2M.

Conclusion: ReMoMask effectively addresses key limitations in text-to-motion generation by unifying retrieval and generative approaches, producing temporally coherent motions with improved physical plausibility and semantic alignment.

Abstract: Text-to-Motion (T2M) generation aims to synthesize realistic and semantically aligned human motion sequences from natural language descriptions. However, current approaches face dual challenges: Generative models (e.g., diffusion models) suffer from limited diversity, error accumulation, and physical implausibility, while Retrieval-Augmented Generation (RAG) methods exhibit diffusion inertia, partial-mode collapse, and asynchronous artifacts. To address these limitations, we propose ReMoMask, a unified framework integrating three key innovations: 1) A Bidirectional Momentum Text-Motion Model decouples negative sample scale from batch size via momentum queues, substantially improving cross-modal retrieval precision; 2) A Semantic Spatio-temporal Attention mechanism enforces biomechanical constraints during part-level fusion to eliminate asynchronous artifacts; 3) RAG-Classier-Free Guidance incorporates minor unconditional generation to enhance generalization. Built upon MoMask’s RVQ-VAE, ReMoMask efficiently generates temporally coherent motions in minimal steps. Extensive experiments on standard benchmarks demonstrate the state-of-the-art performance of ReMoMask, achieving a 3.88% and 10.97% improvement in FID scores on HumanML3D and KIT-ML, respectively, compared to the previous SOTA method RAG-T2M. Code: https://github.com/AIGeeksGroup/ReMoMask. Website: https://aigeeksgroup.github.io/ReMoMask.

Method: Uses Trajectory Divergence Maps to locate editable regions and Scheduled KV Injection for stable editing, without requiring training or masks.

Result: Achieves superior editability and visual fidelity in shape replacement tasks, outperforming existing methods.

Conclusion: The proposed framework enables precise shape editing while preserving non-target content, validated by the new ReShapeBench benchmark.

Abstract: While recent flow-based image editing models demonstrate general-purpose capabilities across diverse tasks, they often struggle to specialize in challenging scenarios – particularly those involving large-scale shape transformations. When performing such structural edits, these methods either fail to achieve the intended shape change or inadvertently alter non-target regions, resulting in degraded background quality. We propose Follow-Your-Shape, a training-free and mask-free framework that supports precise and controllable editing of object shapes while strictly preserving non-target content. Motivated by the divergence between inversion and editing trajectories, we compute a Trajectory Divergence Map (TDM) by comparing token-wise velocity differences between the inversion and denoising paths. The TDM enables precise localization of editable regions and guides a Scheduled KV Injection mechanism that ensures stable and faithful editing. To facilitate a rigorous evaluation, we introduce ReShapeBench, a new benchmark comprising 120 new images and enriched prompt pairs specifically curated for shape-aware editing. Experiments demonstrate that our method achieves superior editability and visual fidelity, particularly in tasks requiring large-scale shape replacement.

Method: The DSER framework unifies deep spectral feature learning with epipolar-domain regularization, exploiting frequency-domain correlations across epipolar plane images to enforce global structural coherence and mitigate artifacts.

Result: Comprehensive experiments on the 4D Light Field Benchmark and real-world datasets show DSER achieves superior performance in precision, structural consistency, and computational efficiency compared to state-of-the-art methods.

Conclusion: DSER demonstrates the potential of integrating spectral priors with epipolar geometry for scalable and noise-resilient dense light field depth estimation, establishing it as a promising direction for next-generation high-dimensional vision systems.

Abstract: Accurate and efficient dense depth reconstruction from light field imagery remains a central challenge in computer vision, underpinning applications such as augmented reality, biomedical imaging, and 3D scene reconstruction. Existing deep convolutional approaches, while effective, often incur high computational overhead and are sensitive to noise and disparity inconsistencies in real-world scenarios. This paper introduces a novel Deep Spectral Epipolar Representation (DSER) framework for dense light field reconstruction, which unifies deep spectral feature learning with epipolar-domain regularization. The proposed approach exploits frequency-domain correlations across epipolar plane images to enforce global structural coherence, thereby mitigating artifacts and enhancing depth accuracy. Unlike conventional supervised models, DSER operates efficiently with limited training data while maintaining high reconstruction fidelity. Comprehensive experiments on the 4D Light Field Benchmark and a diverse set of real-world datasets demonstrate that DSER achieves superior performance in terms of precision, structural consistency, and computational efficiency compared to state-of-the-art methods. These results highlight the potential of integrating spectral priors with epipolar geometry for scalable and noise-resilient dense light field depth estimation, establishing DSER as a promising direction for next-generation high-dimensional vision systems.

Method: Developed a novel spiking neural network for robust fitting on Intel Loihi 2 neuromorphic hardware, with event-driven formulations of model estimation and algorithmic strategies to overcome hardware limitations in precision and instruction sets.

Result: The neuromorphic robust fitting implementation consumes only 15% of the energy required by established robust fitting algorithms on standard CPUs while achieving equivalent accuracy.

Conclusion: Neuromorphic computing offers a promising pathway for energy-efficient robust fitting in computer vision, demonstrating significant energy savings without compromising accuracy.

Abstract: Robust fitting of geometric models is a fundamental task in many computer vision pipelines. Numerous innovations have been produced on the topic, from improving the efficiency and accuracy of random sampling heuristics to generating novel theoretical insights that underpin new approaches with mathematical guarantees. However, one aspect of robust fitting that has received little attention is energy efficiency. This performance metric has become critical as high energy consumption is a growing concern for AI adoption. In this paper, we explore energy-efficient robust fitting via the neuromorphic computing paradigm. Specifically, we designed a novel spiking neural network for robust fitting on real neuromorphic hardware, the Intel Loihi 2. Enabling this are novel event-driven formulations of model estimation that allow robust fitting to be implemented in the unique architecture of Loihi 2, and algorithmic strategies to alleviate the current limited precision and instruction set of the hardware. Results show that our neuromorphic robust fitting consumes only a fraction (15%) of the energy required to run the established robust fitting algorithm on a standard CPU to equivalent accuracy.

Method: Three-tier control mechanism (width multiplier, classifier depth, classifier kernel) for fine-grained model adaptation, combined with Bayesian Optimization for efficient hyperparameter search under computational constraints.

Result: Enables broad model scaling, targeted layer refinement, and scenario-specific optimization, leading to improved resource allocation and performance through Task-Specific Learning Adaptation (TSLA).

Conclusion: The approach successfully customizes semantic segmentation networks for autonomous driving hardware, maximizing computational capacity and model accuracy while optimizing hardware utilization across diverse self-driving tasks.

Abstract: Autonomous driving platforms encounter diverse driving scenarios, each with varying hardware resources and precision requirements. Given the computational limitations of embedded devices, it is crucial to consider computing costs when deploying on target platforms like the NVIDIA\textsuperscript{\textregistered} DRIVE PX 2. Our objective is to customize the semantic segmentation network according to the computing power and specific scenarios of autonomous driving hardware. We implement dynamic adaptability through a three-tier control mechanism – width multiplier, classifier depth, and classifier kernel – allowing fine-grained control over model components based on hardware constraints and task requirements. This adaptability facilitates broad model scaling, targeted refinement of the final layers, and scenario-specific optimization of kernel sizes, leading to improved resource allocation and performance. Additionally, we leverage Bayesian Optimization with surrogate modeling to efficiently explore hyperparameter spaces under tight computational budgets. Our approach addresses scenario-specific and task-specific requirements through automatic parameter search, accommodating the unique computational complexity and accuracy needs of autonomous driving. It scales its Multiply-Accumulate Operations (MACs) for Task-Specific Learning Adaptation (TSLA), resulting in alternative configurations tailored to diverse self-driving tasks. These TSLA customizations maximize computational capacity and model accuracy, optimizing hardware utilization.

Method: Proposed Semantic Drift Protocol (SDP) - a cyclic evaluation protocol that alternates I2T and T2I over multiple generations. Introduces two metrics: Mean Cumulative Drift (MCD) for embedding-based semantic drift measurement, and Multi-Generation GenEval (MGG) for object-level compliance.

Result: Evaluation on seven models using Nocaps+Docci400 benchmark revealed substantial variation in cross-modal stability. Some models like BAGEL maintained semantic meaning over many alternations, while others like VILA-U drifted quickly despite strong single-pass scores.

Conclusion: SDP serves as a necessary complement to standard I2T and T2I evaluations, revealing cross-modal consistency issues that isolated single-pass metrics cannot detect.

Abstract: Employing a single, unified model (UM) for both visual understanding (image-to-text: I2T) and visual generation (text-to-image: T2I) has opened a new direction in Visual Language Model (VLM) research. While UMs can also support broader unimodal tasks (e.g., text-to-text, image-to-image), we focus on the core cross-modal pair T2I and I2T. Existing evaluation benchmarks consider these capabilities in isolation: FID and GenEval for T2I, and benchmarks such as MME, MMBench for I2T. These isolated single-pass metrics do not reveal cross-consistency: whether a model that “understands” a concept can also “render” it, nor whether semantic meaning is preserved when cycling between image and text modalities. To address this, we introduce the Semantic Drift Protocol (SDP) for Unified Models, a cyclic evaluation protocol that alternates I2T and T2I over multiple generations to quantify semantic drift. We propose two metrics: (i) Mean Cumulative Drift (MCD), an embedding-based measure of overall semantic drift; and (ii) Multi-Generation GenEval (MGG), an object-level compliance score extending GenEval. To assess generalization beyond COCO dataset, which is widely used in training; we create a new benchmark Nocaps+Docci400, sampled from NoCaps and DOCCI and evaluated on seven recent models. SDP reveals substantial variation in cross-modal stability: some models like BAGEL maintain semantic meaning over many alternations, whereas others like VILA-U drift quickly despite strong single-pass scores. Our results highlight SDP as a necessary complement to standard I2T and T2I evaluations. Code is available at https://github.com/mollahsabbir/Semantic-Drift-in-Unified-Models

Method: Systematically perturb non-gender features across four benchmarks (COCO-gender, FACET, MIAP, PHASE) and various VLMs to quantify impact on bias evaluation.

Result: Minimal perturbations (10% object masking, weak background blurring) can dramatically alter bias scores by up to 175% in generative VLMs and 43% in CLIP variants.

Conclusion: Current bias evaluations reflect model responses to spurious features rather than true gender bias. Recommendations include reporting bias metrics alongside feature-sensitivity measurements for more reliable assessment.

Abstract: Gender bias in vision-language foundation models (VLMs) raises concerns about their safe deployment and is typically evaluated using benchmarks with gender annotations on real-world images. However, as these benchmarks often contain spurious correlations between gender and non-gender features, such as objects and backgrounds, we identify a critical oversight in gender bias evaluation: Do spurious features distort gender bias evaluation? To address this question, we systematically perturb non-gender features across four widely used benchmarks (COCO-gender, FACET, MIAP, and PHASE) and various VLMs to quantify their impact on bias evaluation. Our findings reveal that even minimal perturbations, such as masking just 10% of objects or weakly blurring backgrounds, can dramatically alter bias scores, shifting metrics by up to 175% in generative VLMs and 43% in CLIP variants. This suggests that current bias evaluations often reflect model responses to spurious features rather than gender bias, undermining their reliability. Since creating spurious feature-free benchmarks is fundamentally challenging, we recommend reporting bias metrics alongside feature-sensitivity measurements to enable a more reliable bias assessment.

Method: Initialize directed graph adjacency matrix A using a known interpolator Θ, then learn perturbation matrices P and P(2) from data. Implement restoration effects via Douglas-Rachford iterations, which are unrolled into an interpretable neural network.

Result: Experimental results show state-of-the-art image interpolation performance while drastically reducing the number of network parameters compared to conventional approaches.

Conclusion: The proposed method successfully addresses the limitations of random initialization in DNNs by leveraging graph-based initialization and data-driven perturbations, achieving superior interpolation results with improved parameter efficiency.

Abstract: Conventional deep neural nets (DNNs) initialize network parameters at random and then optimize each one via stochastic gradient descent (SGD), resulting in substantial risk of poor-performing local minima.Focusing on the image interpolation problem and leveraging a recent theorem that maps a (pseudo-)linear interpolator {\Theta} to a directed graph filter that is a solution to a MAP problem regularized with a graph shift variation (GSV) prior, we first initialize a directed graph adjacency matrix A based on a known interpolator {\Theta}, establishing a baseline performance.Then, towards further gain, we learn perturbation matrices P and P(2) from data to augment A, whose restoration effects are implemented via Douglas-Rachford (DR) iterations, which we unroll into a lightweight interpretable neural net.Experimental results demonstrate state-of-the-art image interpolation results, while drastically reducing network parameters.

Method: Used 100 Montreal street images (50 real, 50 synthetic) with 230 human annotations across 30 dimensions. Evaluated 7 VLMs in zero-shot setup with structured prompts and deterministic parser. Measured accuracy for single-choice items and Jaccard overlap for multi-label items.

Result: Models performed better on visible, objective properties than subjective appraisals. Claude-sonnet achieved macro 0.31 and mean Jaccard 0.48 on multi-label items. Higher human agreement correlated with better model scores. Synthetic images slightly lowered performance.

Conclusion: The benchmark enables reproducible, uncertainty-aware evaluation of VLMs for participatory urban analysis, showing current limitations in capturing subjective urban perceptions compared to objective properties.

Abstract: Understanding how people read city scenes can inform design and planning. We introduce a small benchmark for testing vision-language models (VLMs) on urban perception using 100 Montreal street images, evenly split between photographs and photorealistic synthetic scenes. Twelve participants from seven community groups supplied 230 annotation forms across 30 dimensions mixing physical attributes and subjective impressions. French responses were normalized to English. We evaluated seven VLMs in a zero-shot setup with a structured prompt and deterministic parser. We use accuracy for single-choice items and Jaccard overlap for multi-label items; human agreement uses Krippendorff’s alpha and pairwise Jaccard. Results suggest stronger model alignment on visible, objective properties than subjective appraisals. The top system (claude-sonnet) reaches macro 0.31 and mean Jaccard 0.48 on multi-label items. Higher human agreement coincides with better model scores. Synthetic images slightly lower scores. We release the benchmark, prompts, and harness for reproducible, uncertainty-aware evaluation in participatory urban analysis.

Method: Projects images into visual tokens in joint semantic space, trains language model to generate latent states that reconstruct key visual tokens, and adapts GRPO algorithm for reinforcement learning on latent reasoning.

Result: Achieves 71.67% on MMVP benchmark compared to 66.67% with Qwen2.5-VL, showing substantial gains in fine-grained visual understanding.

Conclusion: LVR paradigm enables direct visual reasoning in embedding space, significantly enhancing perception capabilities of multimodal models.

Abstract: Multimodal Large Language Models (MLLMs) have achieved notable gains in various tasks by incorporating Chain-of-Thought (CoT) reasoning in language spaces. Recent work extends this direction by leveraging external tools for visual editing, thereby enhancing the visual signal along the reasoning trajectories. Nevertheless, these approaches remain fundamentally constrained: reasoning is still confined to the language space, with visual information treated as static preconditions. We introduce Latent Visual Reasoning (LVR), a new paradigm that enables autoregressive reasoning directly in the visual embedding space. A visual encoder first projects images into visual tokens within a joint semantic space shared with the language model. The language model is then trained to generate latent states that reconstruct key visual tokens critical for answering the query, constituting the process of latent visual reasoning. By interleaving LVR with standard text generation, our model achieves substantial gains on perception-intensive visual question answering tasks. In addition, we adapt the GRPO algorithm to conduct reinforcement learning on latent reasoning, further balancing LVR and textual generation. We show that LVR substantially improves fine-grained visual understanding and perception, achieving 71.67% on MMVP compared to 66.67% with Qwen2.5-VL. Code base and model weights will be released later.

Method: Systematically examined three training approaches (natural training, adversarial training, and pruning) on traffic sign classifiers, evaluating their impact on post-hoc explanation quality using saliency maps through extensive empirical evaluation.

Result: Pruning significantly enhances comprehensibility and faithfulness of explanations. It enforces sparsity in learned representation, leading to more interpretable and reliable decisions while improving model efficiency.

Conclusion: Pruning is a promising strategy for developing transparent deep learning models, especially beneficial for resource-constrained vehicular AI systems where both efficiency and interpretability are crucial.

Abstract: Connected and autonomous vehicles continue to heavily rely on AI systems, where transparency and security are critical for trust and operational safety. Post-hoc explanations provide transparency to these black-box like AI models but the quality and reliability of these explanations is often questioned due to inconsistencies and lack of faithfulness in representing model decisions. This paper systematically examines the impact of three widely used training approaches, namely natural training, adversarial training, and pruning, affect the quality of post-hoc explanations for traffic sign classifiers. Through extensive empirical evaluation, we demonstrate that pruning significantly enhances the comprehensibility and faithfulness of explanations (using saliency maps). Our findings reveal that pruning not only improves model efficiency but also enforces sparsity in learned representation, leading to more interpretable and reliable decisions. Additionally, these insights suggest that pruning is a promising strategy for developing transparent deep learning models, especially in resource-constrained vehicular AI systems.

Method: Evaluated 36 model variants on a curated dataset of 661 canopy images with 85,879 labeled blueberry instances. Fine-tuned models using Unbiased Mean Teacher-based semi-supervised learning on 1,035 unlabeled images.

Result: YOLOv12m achieved 93.3% mAP@50, RT-DETRv2-X achieved 93.6% mAP@50. SSL fine-tuning improved accuracy by up to 2.9%, with RT-DETR-v2-X reaching 94.8% mAP@50. Mid-sized models offered best accuracy-speed balance.

Conclusion: RT-DETR-v2-X achieved the highest accuracy after SSL fine-tuning. Semi-supervised learning shows promise but needs more research for cross-domain data. Dataset and software are publicly available for further research.

Abstract: Blueberry detection in natural environments remains challenging due to variable lighting, occlusions, and motion blur due to environmental factors and imaging devices. Deep learning-based object detectors promise to address these challenges, but they demand a large-scale, diverse dataset that captures the real-world complexities. Moreover, deploying these models in practical scenarios often requires the right accuracy/speed/memory trade-off in model selection. This study presents a novel comparative benchmark analysis of advanced real-time object detectors, including YOLO (You Only Look Once) (v8-v12) and RT-DETR (Real-Time Detection Transformers) (v1-v2) families, consisting of 36 model variants, evaluated on a newly curated dataset for blueberry detection. This dataset comprises 661 canopy images collected with smartphones during the 2022-2023 seasons, consisting of 85,879 labelled instances (including 36,256 ripe and 49,623 unripe blueberries) across a wide range of lighting conditions, occlusions, and fruit maturity stages. Among the YOLO models, YOLOv12m achieved the best accuracy with a mAP@50 of 93.3%, while RT-DETRv2-X obtained a mAP@50 of 93.6%, the highest among all the RT-DETR variants. The inference time varied with the model scale and complexity, and the mid-sized models appeared to offer a good accuracy-speed balance. To further enhance detection performance, all the models were fine-tuned using Unbiased Mean Teacher-based semi-supervised learning (SSL) on a separate set of 1,035 unlabeled images acquired by a ground-based machine vision platform in 2024. This resulted in accuracy gains ranging from -1.4% to 2.9%, with RT-DETR-v2-X achieving the best mAP@50 of 94.8%. More in-depth research into SSL is needed to better leverage cross-domain unlabeled data. Both the dataset and software programs of this study are made publicly available to support further research.

Method: Lightweight ROI augmentation: probabilistically replace full images with random ROI crops from precomputed bounding-box bank during training.

Result: Modest ROC-AUC gains with best parameters (p_roi=0.10, alpha=0.10), variable performance across folds, flat/slightly lower PR-AUC.

Conclusion: Simple ROI strategies can enhance mammography classification in constrained settings without requiring additional labels or architectural changes.

Abstract: Breast cancer screening with mammography remains central to early detection and mortality reduction. Deep learning has shown strong potential for automating mammogram interpretation, yet limited-resolution datasets and small sample sizes continue to restrict performance. We revisit the Mini-DDSM dataset (9,684 images; 2,414 patients) and introduce a lightweight region-of-interest (ROI) augmentation strategy. During training, full images are probabilistically replaced with random ROI crops sampled from a precomputed, label-free bounding-box bank, with optional jitter to increase variability. We evaluate under strict patient-level cross-validation and report ROC-AUC, PR-AUC, and training-time efficiency metrics (throughput and GPU memory). Because ROI augmentation is training-only, inference-time cost remains unchanged. On Mini-DDSM, ROI augmentation (best: p_roi = 0.10, alpha = 0.10) yields modest average ROC-AUC gains, with performance varying across folds; PR-AUC is flat to slightly lower. These results demonstrate that simple, data-centric ROI strategies can enhance mammography classification in constrained settings without requiring additional labels or architectural modifications.

Method: Used ResNet-18 on ImageNette, applied magnitude-based pruning followed by fine-tuning, evaluated with Vanilla Gradients and Integrated Gradients for saliency maps, and CRAFT-based concept extraction for semantic coherence analysis.

Result: Light-to-moderate pruning improved saliency-map focus and faithfulness while maintaining distinct concepts. Aggressive pruning merged features, reducing sparsity and concept coherence despite preserved accuracy.

Conclusion: Pruning can shape representations toward human-aligned attention patterns, but excessive pruning undermines interpretability, suggesting a trade-off between model compression and interpretability.

Abstract: Prior works have shown that neural networks can be heavily pruned while preserving performance, but the impact of pruning on model interpretability remains unclear. In this work, we investigate how magnitude-based pruning followed by fine-tuning affects both low-level saliency maps and high-level concept representations. Using a ResNet-18 trained on ImageNette, we compare post-hoc explanations from Vanilla Gradients (VG) and Integrated Gradients (IG) across pruning levels, evaluating sparsity and faithfulness. We further apply CRAFT-based concept extraction to track changes in semantic coherence of learned concepts. Our results show that light-to-moderate pruning improves saliency-map focus and faithfulness while retaining distinct, semantically meaningful concepts. In contrast, aggressive pruning merges heterogeneous features, reducing saliency map sparsity and concept coherence despite maintaining accuracy. These findings suggest that while pruning can shape internal representations toward more human-aligned attention patterns, excessive pruning undermines interpretability.

Method: Three pretraining tasks: Caption (enhance video detail perception), VQA (deepen understanding of issue definitions), and Chain-of-Thought (improve reasoning capability).

Result: Significantly improves MLLM performance in both zero-shot and supervised fine-tuning settings, with strong generalization to emergent, unseen issues.

Conclusion: The proposed pretraining paradigm effectively addresses distribution gaps and complex issue definitions, enabling unified inappropriate content detection with improved generalization.

Abstract: Short video platforms are evolving rapidly, making the identification of inappropriate content increasingly critical. Existing approaches typically train separate and small classification models for each type of issue, which requires extensive human-labeled data and lacks cross-issue generalization. We propose a reasoning-enhanced multimodal large language model (MLLM) pretraining paradigm for unified inappropriate content detection. To address the distribution gap between short video content and the original pretraining data of MLLMs, as well as the complex issue definitions, we introduce three targeted pretraining tasks: (1) \textit{Caption}, to enhance the MLLM’s perception of video details; (2) \textit{Visual Question Answering (VQA)}, to deepen the MLLM’s understanding of issue definitions and annotation guidelines; (3) \textit{Chain-of-Thought (CoT)}, to enhance the MLLM’s reasoning capability. Experimental results show that our pretraining approach significantly improves the MLLM’s performance in both zero-shot and supervised fine-tuning (SFT) settings. In addition, our pretrained model demonstrates strong generalization capabilities to emergent, previously unseen issues.

Method: Dual Experts framework with frozen source model (Conv-Adapter) and pretrained vision-language model (trainable text prompt), plus three-stage RAIN pipeline: collaborative retrieval, separate fine-tuning, and learning object consistency enforcement.

Result: Extensive experiments on four benchmark datasets demonstrate state-of-the-art matching performance.

Conclusion: EXCL effectively adapts models to target domains without source data by leveraging dual experts and retrieval-augmented interaction, achieving competitive performance while addressing privacy and cost constraints.

Abstract: Source-Free Unsupervised Domain Adaptation (SFUDA) addresses the realistic challenge of adapting a source-trained model to a target domain without access to the source data, driven by concerns over privacy and cost. Existing SFUDA methods either exploit only the source model’s predictions or fine-tune large multimodal models, yet both neglect complementary insights and the latent structure of target data. In this paper, we propose the Experts Cooperative Learning (EXCL). EXCL contains the Dual Experts framework and Retrieval-Augmentation-Interaction optimization pipeline. The Dual Experts framework places a frozen source-domain model (augmented with Conv-Adapter) and a pretrained vision-language model (with a trainable text prompt) on equal footing to mine consensus knowledge from unlabeled target samples. To effectively train these plug-in modules under purely unsupervised conditions, we introduce Retrieval-Augmented-Interaction(RAIN), a three-stage pipeline that (1) collaboratively retrieves pseudo-source and complex target samples, (2) separately fine-tunes each expert on its respective sample set, and (3) enforces learning object consistency via a shared learning result. Extensive experiments on four benchmark datasets demonstrate that our approach matches state-of-the-art performance.

Method: Formalize copyright infringement from Differential Privacy perspective, introduce conditional sensitivity metric, simulate inclusion/exclusion via fine-tuning (learning/unlearning), compute confidence scores over orthogonal prompt distributions using statistical metrics.

Result: DPM reliably detects infringement content without requiring access to original training data or text prompts, demonstrating effectiveness across diverse categories in the Copyright Infringement Detection Dataset (CIDD).

Conclusion: DPM offers an interpretable and practical solution for safeguarding intellectual property in generative AI, providing robust copyright infringement detection with theoretical underpinnings.

Abstract: The widespread deployment of large vision models such as Stable Diffusion raises significant legal and ethical concerns, as these models can memorize and reproduce copyrighted content without authorization. Existing detection approaches often lack robustness and fail to provide rigorous theoretical underpinnings. To address these gaps, we formalize the concept of copyright infringement and its detection from the perspective of Differential Privacy (DP), and introduce the conditional sensitivity metric, a concept analogous to sensitivity in DP, that quantifies the deviation in a diffusion model’s output caused by the inclusion or exclusion of a specific training data point. To operationalize this metric, we propose D-Plus-Minus (DPM), a novel post-hoc detection framework that identifies copyright infringement in text-to-image diffusion models. Specifically, DPM simulates inclusion and exclusion processes by fine-tuning models in two opposing directions: learning or unlearning. Besides, to disentangle concept-specific influence from the global parameter shifts induced by fine-tuning, DPM computes confidence scores over orthogonal prompt distributions using statistical metrics. Moreover, to facilitate standardized benchmarking, we also construct the Copyright Infringement Detection Dataset (CIDD), a comprehensive resource for evaluating detection across diverse categories. Our results demonstrate that DPM reliably detects infringement content without requiring access to the original training dataset or text prompts, offering an interpretable and practical solution for safeguarding intellectual property in the era of generative AI.

Method: Leverages vision-language models for prompt-based region localization to apply differentiated masking that emphasizes diagnostically relevant regions while reducing redundancy in background areas, enabling controllable text-guided masking.

Result: Outperforms existing MIM methods across multiple medical imaging modalities (brain MRI, chest CT, lung X-ray) with gains of up to +3.1 percentage points in classification accuracy, +1.3 in BoxAP, and +1.1 in MaskAP, while using substantially lower masking ratios (40% vs 70%).

Conclusion: Controllable, text-driven masking enables semantically aligned self-supervised learning and advances robust vision models for medical image analysis.

Abstract: The scarcity of annotated data in specialized domains such as medical imaging presents significant challenges to training robust vision models. While self-supervised masked image modeling (MIM) offers a promising solution, existing approaches largely rely on random high-ratio masking, leading to inefficiency and poor semantic alignment. Moreover, region-aware variants typically depend on reconstruction heuristics or supervised signals, limiting their adaptability across tasks and modalities. We propose Mask What Matters, a controllable text-guided masking framework for self-supervised medical image analysis. By leveraging vision-language models for prompt-based region localization, our method flexibly applies differentiated masking to emphasize diagnostically relevant regions while reducing redundancy in background areas. This controllable design enables better semantic alignment, improved representation learning, and stronger cross-task generalizability. Comprehensive evaluation across multiple medical imaging modalities, including brain MRI, chest CT, and lung X-ray, shows that Mask What Matters consistently outperforms existing MIM methods (e.g., SparK), achieving gains of up to +3.1 percentage points in classification accuracy, +1.3 in box average precision (BoxAP), and +1.1 in mask average precision (MaskAP) for detection. Notably, it achieves these improvements with substantially lower overall masking ratios (e.g., 40% vs. 70%). This work demonstrates that controllable, text-driven masking can enable semantically aligned self-supervised learning, advancing the development of robust vision models for medical image analysis.

Method: Uses a propose-and-validate framework: first synthesizes novel views with a 3D-aware diffusion model to propose complete scenes, then validates 3D uncertainties using MVS model attention maps to guide Gaussian Splatting optimization with uncertainty-weighted loss.

Result: Outperforms state-of-the-art methods on Mip-NeRF 360 and NeRF Synthetic datasets by filtering hallucinatory artifacts while preserving plausible completions in under-constrained regions.

Conclusion: OracleGS successfully reconciles generative completeness with regressive fidelity by conditioning generative priors on multi-view geometric evidence, achieving superior sparse-view novel view synthesis.

Abstract: Sparse-view novel view synthesis is fundamentally ill-posed due to severe geometric ambiguity. Current methods are caught in a trade-off: regressive models are geometrically faithful but incomplete, whereas generative models can complete scenes but often introduce structural inconsistencies. We propose OracleGS, a novel framework that reconciles generative completeness with regressive fidelity for sparse view Gaussian Splatting. Instead of using generative models to patch incomplete reconstructions, our “propose-and-validate” framework first leverages a pre-trained 3D-aware diffusion model to synthesize novel views to propose a complete scene. We then repurpose a multi-view stereo (MVS) model as a 3D-aware oracle to validate the 3D uncertainties of generated views, using its attention maps to reveal regions where the generated views are well-supported by multi-view evidence versus where they fall into regions of high uncertainty due to occlusion, lack of texture, or direct inconsistency. This uncertainty signal directly guides the optimization of a 3D Gaussian Splatting model via an uncertainty-weighted loss. Our approach conditions the powerful generative prior on multi-view geometric evidence, filtering hallucinatory artifacts while preserving plausible completions in under-constrained regions, outperforming state-of-the-art methods on datasets including Mip-NeRF 360 and NeRF Synthetic.

Method: Proposed EfficientMIL with Adaptive Patch Selector (APS) module that replaces quadratic-complexity self-attention mechanisms in Transformer-based MIL methods with efficient linear-complexity sequence models including RNN-based GRU, LSTM, and State Space Model Mamba.

Result: Achieved AUC of 0.976 and accuracy of 0.933 on TCGA-Lung dataset with EfficientMIL-Mamba, and AUC of 0.990 and accuracy of 0.975 on CAMELYON16 dataset with EfficientMIL-GRU, surpassing previous state-of-the-art methods. APS proved more effective for patches selection than conventional strategies.

Conclusion: EfficientMIL provides a computationally efficient solution for whole slide image classification that outperforms existing methods while significantly reducing computational complexity from quadratic to linear, making it suitable for large-scale pathology applications.

Abstract: Whole slide images (WSIs) classification represents a fundamental challenge in computational pathology, where multiple instance learning (MIL) has emerged as the dominant paradigm. Current state-of-the-art (SOTA) MIL methods rely on attention mechanisms, achieving good performance but requiring substantial computational resources due to quadratic complexity when processing hundreds of thousands of patches. To address this computational bottleneck, we introduce EfficientMIL, a novel linear-complexity MIL approach for WSIs classification with the patches selection module Adaptive Patch Selector (APS) that we designed, replacing the quadratic-complexity self-attention mechanisms in Transformer-based MIL methods with efficient sequence models including RNN-based GRU, LSTM, and State Space Model (SSM) Mamba. EfficientMIL achieves significant computational efficiency improvements while outperforming other MIL methods across multiple histopathology datasets. On TCGA-Lung dataset, EfficientMIL-Mamba achieved AUC of 0.976 and accuracy of 0.933, while on CAMELYON16 dataset, EfficientMIL-GRU achieved AUC of 0.990 and accuracy of 0.975, surpassing previous state-of-the-art methods. Extensive experiments demonstrate that APS is also more effective for patches selection than conventional selection strategies.

Method: AutoPrune quantifies mutual information between visual and textual tokens, then projects this signal to budget-constrained logistic retention curves that adapt to different task complexities while maintaining computational constraints.

Result: Applied to LLaVA-1.5-7B, AutoPrune prunes 89% of visual tokens, reduces inference FLOPs by 76.8%, and retains 96.7% of original accuracy across tasks - a 9.1% improvement over PDrop.

Conclusion: Complexity-adaptive pruning effectively reduces computational demands while maintaining performance, demonstrating that adaptive strategies outperform fixed pruning schedules for vision-language models.

Abstract: The established redundancy in visual tokens within large vision-language models allows pruning to effectively reduce their substantial computational demands. Previous methods typically employ heuristic layer-specific pruning strategies where, although the number of tokens removed may differ across decoder layers, the overall pruning schedule is fixed and applied uniformly to all input samples and tasks, failing to align token elimination with the model’s holistic reasoning trajectory. Cognitive science indicates that human visual processing often begins with broad exploration to accumulate evidence before narrowing focus as the target becomes distinct. Our experiments reveal an analogous pattern in these models. This observation suggests that neither a fixed pruning schedule nor a heuristic layer-wise strategy can optimally accommodate the diverse complexities inherent in different inputs. To overcome this limitation, we introduce Complexity-Adaptive Pruning (AutoPrune), a training-free, plug-and-play framework that tailors pruning policies to varying sample and task complexities. Specifically, AutoPrune quantifies the mutual information between visual and textual tokens, then projects this signal to a budget-constrained logistic retention curve. Each such logistic curve, defined by its unique shape, corresponds to the specific complexity of different tasks and can guarantee adherence to predefined computational constraints. We evaluate AutoPrune on standard vision-language tasks and on Vision-Language-Action models for autonomous driving. Notably, when applied to LLaVA-1.5-7B, our method prunes 89% of visual tokens and reduces inference FLOPs by 76.8% while retaining 96.7% of the original accuracy averaged over all tasks. This corresponds to a 9.1% improvement over the recent work PDrop, demonstrating the effectiveness. Code is available at https://github.com/AutoLab-SAI-SJTU/AutoPrune.

Method: FrameMind uses Frame-Interleaved Chain-of-Thought (FiCOT) for multi-turn reasoning alternating between textual reasoning and active visual perception, with Dynamic Resolution Frame Sampling (DRFS) and DRFS-GRPO policy optimization algorithm trained without frame-level annotations.

Result: Extensive experiments on MLVU and VideoMME benchmarks show FrameMind significantly outperforms existing models and advances state-of-the-art in flexible and efficient video understanding.

Conclusion: The proposed dynamic sampling approach enables adaptive visual evidence gathering, overcoming limitations of static frame sampling in video understanding tasks.

Abstract: Current video understanding models rely on fixed frame sampling strategies, processing predetermined visual inputs regardless of the specific reasoning requirements of each question. This static approach limits their ability to adaptively gather visual evidence, leading to suboptimal performance on tasks that require either broad temporal coverage or fine-grained spatial detail. In this paper, we introduce FrameMind, an end-to-end framework trained with reinforcement learning that enables models to dynamically request visual information during reasoning through Frame-Interleaved Chain-of-Thought (FiCOT). Unlike traditional approaches, FrameMind operates in multiple turns where the model alternates between textual reasoning and active visual perception, using tools to extract targeted frames or video clips based on identified knowledge gaps. To train effective dynamic sampling policies, we propose Dynamic Resolution Frame Sampling (DRFS), which exposes models to diverse temporal-spatial trade-offs during learning, and DRFS-GRPO, a group-relative policy optimization algorithm that learns from outcome-based rewards without requiring frame-level annotations. Extensive experiments on challenging benchmarks like MLVU and VideoMME demonstrate that our method significantly outperforms existing models, advancing the state of the art in flexible and efficient video understanding.

Method: Used representation analysis to study embedding space alignment and conducted causal interventions to verify that Grounding IDs mediate binding between objects and symbolic cues.

Result: Found that Grounding IDs emerge as robust within-partition alignment, reduce modality gap between image and text, strengthen attention between related components, improve cross-modal grounding, and reduce hallucinations.

Conclusion: Grounding IDs are a key symbolic mechanism that explains how external cues enhance multimodal binding, offering both interpretability and practical improvements in robustness.

Abstract: Large vision-language models (LVLMs) show strong performance across multimodal benchmarks but remain limited in structured reasoning and precise grounding. Recent work has demonstrated that adding simple visual structures, such as partitions and annotations, improves accuracy, yet the internal mechanisms underlying these gains remain unclear. We investigate this phenomenon and propose the concept of Grounding IDs, latent identifiers induced by external cues that bind objects to their designated partitions across modalities. Through representation analysis, we find that these identifiers emerge as robust within-partition alignment in embedding space and reduce the modality gap between image and text. Causal interventions further confirm that these identifiers mediate binding between objects and symbolic cues. We show that Grounding IDs strengthen attention between related components, which in turn improves cross-modal grounding and reduces hallucinations. Taken together, our results identify Grounding IDs as a key symbolic mechanism explaining how external cues enhance multimodal binding, offering both interpretability and practical improvements in robustness.

Method: Three key components: 1) CryoEngine - large-scale synthetic data generator producing 904k subtomograms from 452 particle classes; 2) APT-ViT - Adaptive Phase Tokenization-enhanced Vision Transformer with equivariance-enhancing module; 3) NRCL - Noise-Resilient Contrastive Learning strategy for stable representation learning under noise.

Result: State-of-the-art performance across 24 synthetic and real datasets on all three major subtomogram tasks (classification, alignment, averaging) with strong generalization to unseen datasets.

Conclusion: The proposed foundation model advances scalable and robust subtomogram analysis in cryo-ET by addressing key challenges through synthetic data generation, adaptive architecture design, and noise-resilient learning strategies.

Abstract: Cryo-electron tomography (cryo-ET) enables in situ visualization of macromolecular structures, where subtomogram analysis tasks such as classification, alignment, and averaging are critical for structural determination. However, effective analysis is hindered by scarce annotations, severe noise, and poor generalization. To address these challenges, we take the first step towards foundation models for cryo-ET subtomograms. First, we introduce CryoEngine, a large-scale synthetic data generator that produces over 904k subtomograms from 452 particle classes for pretraining. Second, we design an Adaptive Phase Tokenization-enhanced Vision Transformer (APT-ViT), which incorporates adaptive phase tokenization as an equivariance-enhancing module that improves robustness to both geometric and semantic variations. Third, we introduce a Noise-Resilient Contrastive Learning (NRCL) strategy to stabilize representation learning under severe noise conditions. Evaluations across 24 synthetic and real datasets demonstrate state-of-the-art (SOTA) performance on all three major subtomogram tasks and strong generalization to unseen datasets, advancing scalable and robust subtomogram analysis in cryo-ET.

Method: ExGS unifies Universal Gaussian Compression (UGC) for re-optimization-free pruning to reduce Gaussian primitives, and GaussPainter which uses diffusion priors with mask-guided refinement to restore high-quality renderings from heavily pruned scenes.

Result: The framework achieves over 100x compression (reducing 354.77 MB to 3.31 MB) while preserving fidelity and significantly improving image quality under challenging conditions. It enables real-time restoration with lightweight VAE and one-step diffusion design.

Conclusion: Diffusion priors play a central role in bridging the gap between extreme compression and high-quality neural rendering, making ExGS a practical solution for resource-constrained environments.

Abstract: Neural scene representations, such as 3D Gaussian Splatting (3DGS), have enabled high-quality neural rendering; however, their large storage and transmission costs hinder deployment in resource-constrained environments. Existing compression methods either rely on costly optimization, which is slow and scene-specific, or adopt training-free pruning and quantization, which degrade rendering quality under high compression ratios. In contrast, recent data-driven approaches provide a promising direction to overcome this trade-off, enabling efficient compression while preserving high rendering quality.We introduce ExGS, a novel feed-forward framework that unifies Universal Gaussian Compression (UGC) with GaussPainter for Extreme 3DGS compression. UGC performs re-optimization-free pruning to aggressively reduce Gaussian primitives while retaining only essential information, whereas GaussPainter leverages powerful diffusion priors with mask-guided refinement to restore high-quality renderings from heavily pruned Gaussian scenes. Unlike conventional inpainting, GaussPainter not only fills in missing regions but also enhances visible pixels, yielding substantial improvements in degraded renderings.To ensure practicality, it adopts a lightweight VAE and a one-step diffusion design, enabling real-time restoration. Our framework can even achieve over 100X compression (reducing a typical 354.77 MB model to about 3.31 MB) while preserving fidelity and significantly improving image quality under challenging conditions. These results highlight the central role of diffusion priors in bridging the gap between extreme compression and high-quality neural rendering.Our code repository will be released at: https://github.com/chenttt2001/ExGS

Method: EvoQuality adapts self-consistency to ranking-based IQA by generating pseudo-labels through pairwise majority voting on the VLM’s outputs. These pseudo-rankings create a fidelity reward that guides iterative evolution using group relative policy optimization (GRPO).

Result: EvoQuality boosts the base VLM’s zero-shot performance by 31.8% on PLCC across diverse IQA benchmarks. It achieves competitive or superior performance to state-of-the-art supervised VLM-based IQA models, outperforming them on 5 out of 7 benchmarks.

Conclusion: EvoQuality demonstrates that vision-language models can autonomously refine their perceptual capabilities through self-supervised learning, achieving remarkable performance without ground-truth labels and competing with supervised methods.

Abstract: Improving vision-language models (VLMs) in the post-training stage typically relies on supervised fine-tuning or reinforcement learning, methods that necessitate costly, human-annotated data. While self-supervised techniques such as self-consistency have proven effective for enhancing reasoning capabilities, their application to perceptual domains such as image quality assessment (IQA) remains largely unexplored. In this work, we introduce EvoQuality, a novel framework that enables a VLM to autonomously refine its quality perception capabilities without any ground-truth labels. EvoQuality adapts the principle of self-consistency to the ranking-based nature of IQA. It generates pseudo-labels by performing pairwise majority voting on the VLM’s own outputs to establish a consensus on relative quality. These pseudo-rankings are then formulated into a fidelity reward that guides the model’s iterative evolution through group relative policy optimization (GRPO). By iteratively leveraging its own predictions, EvoQuality progressively refines the VLM’s perceptual capability. Extensive experiments show that EvoQuality boosts the base VLM’s zero-shot performance by 31.8% on PLCC across diverse IQA benchmarks. Remarkably, despite being entirely self-supervised, EvoQuality achieves performance that is competitive with, or even surpasses, state-of-the-art supervised VLM-based IQA models, outperforming these models on 5 out of 7 IQA benchmarks.

Method: PAL-UI combines dual-level summarization (capturing observation-level cues and action-level outcomes) with a dedicated retrieval tool that allows agents to recall specific historical screenshots during planning. Models are trained on 8.6K mobile GUI navigation samples based on Qwen2.5-VL.

Result: PAL-UI significantly outperforms baseline models and prior methods in mobile GUI navigation tasks, even under data-efficient settings, and exhibits strong cross-domain generalization with notable improvements in web navigation without additional training.

Conclusion: The work demonstrates the potential of active memory retrieval for enhancing long-horizon planning capabilities of vision-based GUI agents.

Abstract: Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) promise human-like interaction with software applications, yet long-horizon tasks remain challenging due to memory limitations. Existing approaches either truncate history or rely on simple textual summaries, which risk losing critical information when past visual details become necessary for future decisions. In this paper, we propose \textbf{PAL-UI} (\textbf{P}lanning with \textbf{A}ctive \textbf{L}ook-back), a novel framework that enables GUI agents to adaptively retrieve past observations when required. PAL-UI combines a dual-level summarization agent, capturing both observation-level cues and action-level outcomes, with a dedicated retrieval tool that allows the agent to recall specific historical screenshots during planning. We curate a step-level instruction dataset of 8.6K samples from mobile GUI navigation trajectories and train \textbf{PAL-UI-3B} and \textbf{PAL-UI-7B} models based on Qwen2.5-VL. Extensive experiments demonstrate that PAL-UI significantly outperforms baseline models and prior methods in mobile GUI navigation tasks, even under data-efficient settings. Moreover, PAL-UI exhibits strong cross-domain generalization, achieving notable improvements in web navigation without additional training. Our work highlights the potential of active memory retrieval for long-horizon planning capabilities of vision-based GUI agents.

Method: The approach uses reverse-attention optimization to reactivate suppressed signals while stabilizing attention, reinforced by velocity-guided dynamics for robust concept reactivation and consistency-preserving objectives to maintain global layout.

Result: Extensive experiments demonstrate the method’s effectiveness and efficiency in attacking concept erasure in Flux models, establishing a reliable benchmark for evaluating erasure robustness.

Conclusion: ReFlux provides the first specialized attack method for assessing concept erasure robustness in rectified flow transformers, revealing vulnerabilities in current erasure techniques and setting a benchmark for future safety evaluations.

Abstract: Recent advances in text-to-image (T2I) diffusion models have enabled impressive generative capabilities, but they also raise significant safety concerns due to the potential to produce harmful or undesirable content. While concept erasure has been explored as a mitigation strategy, most existing approaches and corresponding attack evaluations are tailored to Stable Diffusion (SD) and exhibit limited effectiveness when transferred to next-generation rectified flow transformers such as Flux. In this work, we present ReFlux, the first concept attack method specifically designed to assess the robustness of concept erasure in the latest rectified flow-based T2I framework. Our approach is motivated by the observation that existing concept erasure techniques, when applied to Flux, fundamentally rely on a phenomenon known as attention localization. Building on this insight, we propose a simple yet effective attack strategy that specifically targets this property. At its core, a reverse-attention optimization strategy is introduced to effectively reactivate suppressed signals while stabilizing attention. This is further reinforced by a velocity-guided dynamic that enhances the robustness of concept reactivation by steering the flow matching process, and a consistency-preserving objective that maintains the global layout and preserves unrelated content. Extensive experiments consistently demonstrate the effectiveness and efficiency of the proposed attack method, establishing a reliable benchmark for evaluating the robustness of concept erasure strategies in rectified flow transformers.

Method: VirDA prepends domain-specific visual reprogramming layers to the backbone that produce visual prompts acting as textural bias to adapt input images to target domains, using multiple objective functions to optimize intra- and inter-domain distribution differences without modifying backbone parameters.

Result: On Office-31, VirDA achieves 92.8% mean accuracy with only 1.5M trainable parameters, surpassing PDA by +1.6% accuracy using 46% of its parameters, and outperforms full-backbone fine-tuning methods like CDTrans and FixBi while requiring only 1.7% and 2.8% of their parameters respectively.

Conclusion: VirDA enables efficient domain adaptation by reusing backbone parameters across domains through visual reprogramming, achieving competitive performance with dramatically reduced parameter requirements compared to existing methods.

Abstract: Existing UDA pipelines fine-tune already well-trained backbone parameters for every new source-and-target pair, resulting in the number of training parameters and storage memory growing linearly with each new pair, and also preventing the reuse of these well-trained backbone parameters. Inspired by recent implications that existing backbones have textural biases, we propose making use of domain-specific textural bias for domain adaptation via visual reprogramming, namely VirDA. Instead of fine-tuning the full backbone, VirDA prepends a domain-specific visual reprogramming layer to the backbone. This layer produces visual prompts that act as an added textural bias to the input image, adapting its “style” to a target domain. To optimize these visual reprogramming layers, we use multiple objective functions that optimize the intra- and inter-domain distribution differences when domain-adapting visual prompts are applied. This process does not require modifying the backbone parameters, allowing the same backbone to be reused across different domains. We evaluate VirDA on Office-31 and obtain 92.8% mean accuracy with only 1.5M trainable parameters. VirDA surpasses PDA, the state-of-the-art parameter-efficient UDA baseline, by +1.6% accuracy while using just 46% of its parameters. Compared with full-backbone fine-tuning, VirDA outperforms CDTrans and FixBi by +0.2% and +1.4%, respectively, while requiring only 1.7% and 2.8% of their trainable parameters. Relative to the strongest current methods (PMTrans and TVT), VirDA uses ~1.7% of their parameters and trades off only 2.2% and 1.1% accuracy, respectively.

Method: Uses digital camera optics and deep learning-based model with ConSinGAN for dataset generation. Investigates four YOLO models (v3, v4, v7, v9) with and without ConSinGAN augmentation. Develops SCADA system with sensor architecture.

Result: YOLOv7 with ConSinGAN achieves superior performance with 95.50% accuracy and 285 ms detection time, outperforming other YOLO versions and threshold-based approaches.

Conclusion: The proposed automated defect detection system can be easily established for various defect types or when defect data is insufficient, providing an efficient solution for industrial quality inspection.

Abstract: Since the defect detection of conventional industry components is time-consuming and labor-intensive, it leads to a significant burden on quality inspection personnel and makes it difficult to manage product quality. In this paper, we propose an automated defect detection system for the dual in-line package (DIP) that is widely used in industry, using digital camera optics and a deep learning (DL)-based model. The two most common defect categories of DIP are examined: (1) surface defects, and (2) pin-leg defects. However, the lack of defective component images leads to a challenge for detection tasks. To solve this problem, the ConSinGAN is used to generate a suitable-sized dataset for training and testing. Four varieties of the YOLO model are investigated (v3, v4, v7, and v9), both in isolation and with the ConSinGAN augmentation. The proposed YOLOv7 with ConSinGAN is superior to the other YOLO versions in accuracy of 95.50%, detection time of 285 ms, and is far superior to threshold-based approaches. In addition, the supervisory control and data acquisition (SCADA) system is developed, and the associated sensor architecture is described. The proposed automated defect detection can be easily established with numerous types of defects or insufficient defect data.

Method: Implemented a 10-step SLSO framework including image analysis, structured data generation, tooth number extraction and consistency checking, iterative regeneration when inconsistencies detected, and finding generation with restructuring and verification. Compared against conventional Chain-of-Thought method.

Result: SLSO improved output accuracy with 66.9%, 33.3%, and 28.6% improvement rates for tooth number, tooth movement, and root resorption respectively. Successful cases achieved consistent structured output after up to five regenerations. Framework enforced negative finding descriptions and suppressed hallucinations.

Conclusion: SLSO framework shows promise for improving automated finding generation but requires further refinement for extensive lesions spanning multiple teeth and overall performance enhancement for practical clinical use.

Abstract: In this study, we utilized the multimodal capabilities of OpenAI GPT-4o to automatically generate jaw cyst findings on dental panoramic radiographs. To improve accuracy, we constructed a Self-correction Loop with Structured Output (SLSO) framework and verified its effectiveness. A 10-step process was implemented for 22 cases of jaw cysts, including image input and analysis, structured data generation, tooth number extraction and consistency checking, iterative regeneration when inconsistencies were detected, and finding generation with subsequent restructuring and consistency verification. A comparative experiment was conducted using the conventional Chain-of-Thought (CoT) method across seven evaluation items: transparency, internal structure, borders, root resorption, tooth movement, relationships with other structures, and tooth number. The results showed that the proposed SLSO framework improved output accuracy for many items, with 66.9%, 33.3%, and 28.6% improvement rates for tooth number, tooth movement, and root resorption, respectively. In the successful cases, a consistently structured output was achieved after up to five regenerations. Although statistical significance was not reached because of the small size of the dataset, the overall SLSO framework enforced negative finding descriptions, suppressed hallucinations, and improved tooth number identification accuracy. However, the accurate identification of extensive lesions spanning multiple teeth is limited. Nevertheless, further refinement is required to enhance overall performance and move toward a practical finding generation system.

Method: Fine-tunes Qwen-VL using group relative policy optimization (GRPO) with two specialized reward models targeting temporal artifacts and generation complexity. Uses a curated dataset of 140k real and AI-generated videos designed to maximize discrimination difficulty.

Result: Achieves state-of-the-art zero-shot performance on existing benchmarks, with additional training pushing accuracy above 95%. Produces precise and interpretable rationales behind predictions.

Conclusion: VidGuard-R1 successfully addresses the need for both accurate detection and interpretable explanations in AI-generated video detection, demonstrating superior performance through multi-modal fine-tuning with specialized reward models.

Abstract: With the rapid advancement of AI-generated videos, there is an urgent need for effective detection tools to mitigate societal risks such as misinformation and reputational harm. In addition to accurate classification, it is essential that detection models provide interpretable explanations to ensure transparency for regulators and end users. To address these challenges, we introduce VidGuard-R1, the first video authenticity detector that fine-tunes a multi-modal large language model (MLLM) using group relative policy optimization (GRPO). Our model delivers both highly accurate judgments and insightful reasoning. We curate a challenging dataset of 140k real and AI-generated videos produced by state-of-the-art generation models, carefully designing the generation process to maximize discrimination difficulty. We then fine-tune Qwen-VL using GRPO with two specialized reward models that target temporal artifacts and generation complexity. Extensive experiments demonstrate that VidGuard-R1 achieves state-of-the-art zero-shot performance on existing benchmarks, with additional training pushing accuracy above 95%. Case studies further show that VidGuard-R1 produces precise and interpretable rationales behind its predictions. The code is publicly available at https://VidGuard-R1.github.io.

Method: Treat individual patches as atomic captioning units and aggregate them to describe arbitrary regions (single patches, non-contiguous areas, entire images) using dense visual features from backbones like DINO.

Result: Achieves state-of-the-art performance on zero-shot dense, region-set, and trace captioning tasks, outperforming other baselines and competitors.

Conclusion: Patch-wise semantic representations are effective for scalable caption generation, with dense visual features from backbones like DINO being key to performance.

Abstract: Zero-shot captioners are recently proposed models that utilize common-space vision-language representations to caption images without relying on paired image-text data. To caption an image, they proceed by textually decoding a text-aligned image feature, but they limit their scope to global representations and whole-image captions. We present Patch-ioner, a unified framework for zero-shot captioning that shifts from an image-centric to a patch-centric paradigm, enabling the captioning of arbitrary regions without the need of region-level supervision. Instead of relying on global image representations, we treat individual patches as atomic captioning units and aggregate them to describe arbitrary regions, from single patches to non-contiguous areas and entire images. We analyze the key ingredients that enable current latent captioners to work in our novel proposed framework. Experiments demonstrate that backbones producing meaningful, dense visual features, such as DINO, are key to achieving state-of-the-art performance in multiple region-based captioning tasks. Compared to other baselines and state-of-the-art competitors, our models achieve better performance on zero-shot dense, region-set, and a newly introduced trace captioning task, highlighting the effectiveness of patch-wise semantic representations for scalable caption generation. Project page at https://paciosoft.com/Patch-ioner/ .

Method: Conducted controlled experiments to study design choices, then proposed relaxing MaskGIT’s discrete loss with an auxiliary continuous JEPA-based objective to improve compositional performance.

Result: Identified two key factors influencing compositional generalization: (i) whether training objective operates on discrete vs continuous distribution, and (ii) extent of conditioning information about constituent concepts during training.

Conclusion: Auxiliary continuous objectives can improve compositional performance in discrete models like MaskGIT, bridging the gap between discrete and continuous approaches for better compositional generalization.

Abstract: Compositional generalization, the ability to generate novel combinations of known concepts, is a key ingredient for visual generative models. Yet, not all mechanisms that enable or inhibit it are fully understood. In this work, we conduct a systematic study of how various design choices influence compositional generalization in image and video generation in a positive or negative way. Through controlled experiments, we identify two key factors: (i) whether the training objective operates on a discrete or continuous distribution, and (ii) to what extent conditioning provides information about the constituent concepts during training. Building on these insights, we show that relaxing the MaskGIT discrete loss with an auxiliary continuous JEPA-based objective can improve compositional performance in discrete models like MaskGIT.

Method: WAREX extends three popular benchmarks (WebArena, WebVoyager, REAL) by introducing real-world network conditions, HTTPS instability, and web security threats including Cross-Site Scripting attacks and malicious pop-ups.

Result: Experiments show significant drops in task success rates when WAREX conditions are introduced, demonstrating limited robustness of current state-of-the-art web agents.

Conclusion: Existing web agent benchmarks fail to capture real-world reliability challenges, and current agents lack robustness against network instability and security threats that occur in practical web environments.

Abstract: Recent advances in browser-based LLM agents have shown promise for automating tasks ranging from simple form filling to hotel booking or online shopping. Current benchmarks measure agent performance in controlled environments, such as containers or stable networks, where websites behave deterministically. However, in the real world, users access websites over networks and HTTPS connections that introduce instability from multiple sources: client-side, server-side issues or broader system failures. Moreover, live websites are prone to web attacks such Cross-Site Scripting, as well as general site modifications which can cause unexpected or malicious pop-ups or improper functionality. To address this gap, we present WAREX: Web Agent Reliability Evaluation on Existing Benchmarks. We measure the impact of WAREX across three popular benchmarks: WebArena, WebVoyager, and REAL. Our experiments show that introducing WAREX leads to significant drops in task success rates, highlighting the limited robustness of state-of-the-art agents.

Method: The authors formulated a multi-objective mixed integer programming model and proposed two approaches: an augmented epsilon-constraint method for finding Pareto-optimal solutions, and a Refined Iterated Pareto Greedy (RIPG) metaheuristic algorithm for solving large instances efficiently.

Result: The proposed methods were benchmarked across small, medium, and large-size instances and compared against two well-known algorithms, with computational results demonstrating the effectiveness of the approach.

Conclusion: The developed multi-objective optimization methods effectively address the conflicting objectives of minimizing makespan and energy consumption in hybrid flow shop scheduling with blocking constraints, providing practical solutions for energy-efficient manufacturing operations.

Abstract: The scarcity of non-renewable energy sources, geopolitical problems in its supply, increasing prices, and the impact of climate change, force the global economy to develop more energy-efficient solutions for their operations. The Manufacturing sector is not excluded from this challenge as one of the largest consumers of energy. Energy-efficient scheduling is a method that attracts manufacturing companies to reduce their consumption as it can be quickly deployed and can show impact immediately. In this study, the hybrid flow shop scheduling problem with blocking constraint (BHFS) is investigated in which we seek to minimize the latest completion time (i.e. makespan) and overall energy consumption, a typical manufacturing setting across many industries from automotive to pharmaceutical. Energy consumption and the latest completion time of customer orders are usually conflicting objectives. Therefore, we first formulate the problem as a novel multi-objective mixed integer programming (MIP) model and propose an augmented epsilon-constraint method for finding the Pareto-optimal solutions. Also, an effective multi-objective metaheuristic algorithm. Refined Iterated Pareto Greedy (RIPG), is developed to solve large instances in reasonable time. Our proposed methods are benchmarked using small, medium, and large-size instances to evaluate their efficiency. Two well-known algorithms are adopted for comparing our novel approaches. The computational results show the effectiveness of our method.

Method: Systematic evaluation framework with two tasks: binary self-recognition (identifying own vs. other model’s text) and exact model prediction, applied to 10 contemporary LLMs.

Result: Only 4 out of 10 models predicted themselves as generators, performance rarely above random chance, strong bias toward GPT and Claude families. Models show some knowledge of own existence but reasoning reveals hierarchical bias associating high-quality text with top-tier models.

Conclusion: Findings highlight limitations in current LLM self-recognition capabilities, with implications for AI safety and need to develop appropriate AI self-awareness.

Abstract: Self-recognition is a crucial metacognitive capability for AI systems, relevant not only for psychological analysis but also for safety, particularly in evaluative scenarios. Motivated by contradictory interpretations of whether models possess self-recognition (Panickssery et al., 2024; Davidson et al., 2024), we introduce a systematic evaluation framework that can be easily applied and updated. Specifically, we measure how well 10 contemporary larger language models (LLMs) can identify their own generated text versus text from other models through two tasks: binary self-recognition and exact model prediction. Different from prior claims, our results reveal a consistent failure in self-recognition. Only 4 out of 10 models predict themselves as generators, and the performance is rarely above random chance. Additionally, models exhibit a strong bias toward predicting GPT and Claude families. We also provide the first evaluation of model awareness of their own and others’ existence, as well as the reasoning behind their choices in self-recognition. We find that the model demonstrates some knowledge of its own existence and other models, but their reasoning reveals a hierarchical bias. They appear to assume that GPT, Claude, and occasionally Gemini are the top-tier models, often associating high-quality text with them. We conclude by discussing the implications of our findings on AI safety and future directions to develop appropriate AI self-awareness.

Method: The framework generates synthetic enterprise-style documents with embedded contradictions, combines automated contradiction mining with human-in-the-loop validation, models a taxonomy of contradiction types common in business processes, and enables controlled creation of self- and pairwise contradictions.

Result: The work establishes a foundation for more trustworthy and accountable RAG systems in enterprise applications by developing a contradiction-aware retrieval evaluation pipeline with human oversight to reflect domain-specific judgment complexity.

Conclusion: ContraGen addresses the limitation of existing benchmarks and provides a systematic approach for evaluating both intra-document and cross-document consistency, which is essential for reducing risk and ensuring compliance in enterprise information-seeking applications.

Abstract: Retrieval-Augmented Generation (RAG) integrates LLMs with external sources, offering advanced capabilities for information access and decision-making. However, contradictions in retrieved evidence can result in inconsistent or untrustworthy outputs, which is especially problematic in enterprise settings where compliance, governance, and accountability are critical. Existing benchmarks for contradiction detection are limited to sentence-level analysis and do not capture the complexity of enterprise documents such as contracts, financial filings, compliance reports, or policy manuals. To address this limitation, we propose ContraGen, a contradiction-aware benchmark framework tailored to enterprise domain. The framework generates synthetic enterprise-style documents with embedded contradictions, enabling systematic evaluation of both intra-document and cross-document consistency. Automated contradiction mining is combined with human-in-the-loop validation to ensure high accuracy. Our contributions include generating realistic enterprise documents, modeling a taxonomy of contradiction types common in business processes, enabling controlled creation of self- and pairwise contradictions, developing a contradiction-aware retrieval evaluation pipeline and embedding human oversight to reflect domain-specific judgment complexity. This work establishes a foundation for more trustworthy and accountable RAG systems in enterprise information-seeking applications, where detecting and resolving contradictions is essential for reducing risk and ensuring compliance.

Method: The study uses a broad perspective on theory evaluation to conduct a wide-ranging qualitative comparison of whole-mind-oriented cognitive and generative architectures and their full systems.

Result: The paper yields a comprehensive comparison framework for evaluating theories based on different architectural approaches to whole-mind modeling.

Conclusion: A broad perspective on theory evaluation enables meaningful qualitative comparison between cognitive and generative architectures for whole-mind systems, addressing the evaluation challenges in both approaches.

Abstract: Evaluation is a critical activity associated with any theory. Yet this has proven to be an exceptionally challenging activity for theories based on cognitive architectures. For an overlapping set of reasons, evaluation can also be challenging for theories based on generative neural architectures. This dual challenge is approached here by leveraging a broad perspective on theory evaluation to yield a wide-ranging, albeit qualitative, comparison of whole-mind-oriented cognitive and generative architectures and the full systems that are based on these architectures.

Method: Convert natural language plans into Kripke structures and Linear Temporal Logic (LTL) formulas using Large Language Models, then perform model checking on a simplified PlanBench dataset.

Result: GPT-5 achieves excellent classification performance with 96.3% F1 score and produces syntactically perfect formal representations that can serve as guarantees.

Conclusion: The framework successfully demonstrates high performance in plan verification, though achieving semantically perfect formal models requires further exploration.

Abstract: We introduce a novel framework for evaluating the alignment between natural language plans and their expected behavior by converting them into Kripke structures and Linear Temporal Logic (LTL) using Large Language Models (LLMs) and performing model checking. We systematically evaluate this framework on a simplified version of the PlanBench plan verification dataset and report on metrics like Accuracy, Precision, Recall and F1 scores. Our experiments demonstrate that GPT-5 achieves excellent classification performance (F1 score of 96.3%) while almost always producing syntactically perfect formal representations that can act as guarantees. However, the synthesis of semantically perfect formal models remains an area for future exploration.

Method: Created PolicyGuardBench with 60k examples from diverse agent runs, generating policies and violation labels for within and cross subdomain pairings. Includes full-trajectory evaluation and prefix-based violation detection. Trained PolicyGuard-4B model on this dataset.

Result: PolicyGuard-4B delivers strong detection accuracy across all tasks while keeping inference efficient, generalizes across domains, and preserves high accuracy on unseen settings.

Conclusion: The framework shows that accurate and generalizable guardrails for policy compliance in web agent trajectories are feasible at small scales.

Abstract: Autonomous web agents need to operate under externally imposed or human-specified policies while generating long-horizon trajectories. However, little work has examined whether these trajectories comply with such policies, or whether policy violations persist across different contexts such as domains (e.g., shopping or coding websites) and subdomains (e.g., product search and order management in shopping). To address this gap, we introduce PolicyGuardBench, a benchmark of about 60k examples for detecting policy violations in agent trajectories. From diverse agent runs, we generate a broad set of policies and create both within subdomain and cross subdomain pairings with violation labels. In addition to full-trajectory evaluation, PolicyGuardBench also includes a prefix-based violation detection task where models must anticipate policy violations from truncated trajectory prefixes rather than complete sequences. Using this dataset, we train PolicyGuard-4B, a lightweight guardrail model that delivers strong detection accuracy across all tasks while keeping inference efficient. Notably, PolicyGuard-4B generalizes across domains and preserves high accuracy on unseen settings. Together, PolicyGuardBench and PolicyGuard-4B provide the first comprehensive framework for studying policy compliance in web agent trajectories, and show that accurate and generalizable guardrails are feasible at small scales.

Method: Combines insertion-based Edit Flow for discrete text tokens with Flow Matching for image latents, using hierarchical sampling that prioritizes content over grammar for concurrent text-image synthesis.

Result: Outperforms autoregressive baselines on both generation and understanding tasks while using up to 50% fewer training FLOPs, and surpasses both autoregressive and diffusion-based approaches.

Conclusion: OneFlow unlocks new capabilities for concurrent generation, iterative refinement, and natural reasoning-like generation, demonstrating superior performance and efficiency compared to existing approaches.

Abstract: We present OneFlow, the first non-autoregressive multimodal model that enables variable-length and concurrent mixed-modal generation. Unlike autoregressive models that enforce rigid causal ordering between text and image generation, OneFlow combines an insertion-based Edit Flow for discrete text tokens with Flow Matching for image latents. OneFlow enables concurrent text-image synthesis with hierarchical sampling that prioritizes content over grammar. Through controlled experiments across model sizes from 1B to 8B, we demonstrate that OneFlow outperforms autoregressive baselines on both generation and understanding tasks while using up to 50% fewer training FLOPs. OneFlow surpasses both autoregressive and diffusion-based approaches while unlocking new capabilities for concurrent generation, iterative refinement, and natural reasoning-like generation.

Method: Theoretical analysis of transformers trained on in-context weight prediction for linear regression, characterizing task hardness via smallest eigenvalue of feature covariance matrix, and experimental validation on nonlinear transformers.

Result: Test-time compute allows reducing context length for same error; can harm performance if required skills are absent from training; diverse, relevant, hard tasks yield best test-time scaling performance.

Conclusion: Test-time scaling effectiveness depends critically on training data quality - diverse, relevant, and challenging tasks enable models to leverage longer reasoning chains effectively during inference.

Abstract: Test-time scaling improves the reasoning capabilities of large language models (LLMs) by allocating extra compute to generate longer Chains-of-Thoughts (CoTs). This enables models to tackle more complex problem by breaking them down into additional steps, backtracking, and correcting mistakes. Despite its strong performance–demonstrated by OpenAI’s o1 and DeepSeek R1, the conditions in the training data under which long CoTs emerge, and when such long CoTs improve the performance, remain unclear. In this paper, we study the performance of test-time scaling for transformers trained on an in-context weight prediction task for linear regression. Our analysis provides a theoretical explanation for several intriguing observations: First, at any fixed test error, increasing test-time compute allows us to reduce the number of in-context examples (context length) in training prompts. Second, if the skills required to solve a downstream task are not sufficiently present in the training data, increasing test-time compute can harm performance. Finally, we characterize task hardness via the smallest eigenvalue of its feature covariance matrix and show that training on a diverse, relevant, and hard set of tasks results in best performance for test-time scaling. We confirm our findings with experiments on large, nonlinear transformer architectures.

Method: CPS jointly optimizes imperceptible modifications to both visual and textual content using CLIP-transferable image perturbations and RLHF-induced linguistic biases, operating in a realistic black-box setting where attackers can only edit their own listing’s images and text metadata.

Result: CPS significantly outperforms baseline methods across state-of-the-art VLMs (GPT-4.1, Qwen-2.5VL, Pixtral-Large) on movie selection and e-commerce tasks, while maintaining 70% lower detection rates.

Conclusion: The effectiveness of CPS highlights an urgent need for robust defenses as VLM-based agents play increasingly important roles in high-stakes selection tasks like content recommendation and product ranking.

Abstract: Vision-language model (VLM)-based web agents increasingly power high-stakes selection tasks like content recommendation or product ranking by combining multimodal perception with preference reasoning. Recent studies reveal that these agents are vulnerable against attackers who can bias selection outcomes through preference manipulations using adversarial pop-ups, image perturbations, or content tweaks. Existing work, however, either assumes strong white-box access, with limited single-modal perturbations, or uses impractical settings. In this paper, we demonstrate, for the first time, that joint exploitation of visual and textual channels yields significantly more powerful preference manipulations under realistic attacker capabilities. We introduce Cross-Modal Preference Steering (CPS) that jointly optimizes imperceptible modifications to an item’s visual and natural language descriptions, exploiting CLIP-transferable image perturbations and RLHF-induced linguistic biases to steer agent decisions. In contrast to prior studies that assume gradient access, or control over webpages, or agent memory, we adopt a realistic black-box threat setup: a non-privileged adversary can edit only their own listing’s images and textual metadata, with no insight into the agent’s model internals. We evaluate CPS on agents powered by state-of-the-art proprietary and open source VLMs including GPT-4.1, Qwen-2.5VL and Pixtral-Large on both movie selection and e-commerce tasks. Our results show that CPS is significantly more effective than leading baseline methods. For instance, our results show that CPS consistently outperforms baselines across all models while maintaining 70% lower detection rates, demonstrating both effectiveness and stealth. These findings highlight an urgent need for robust defenses as agentic systems play an increasingly consequential role in society.

Method: Proposes Mutual Information Tree Search (MITS) with PMI-based scoring function for step-wise evaluation, beam search for tree expansion without look-ahead simulations, entropy-based dynamic sampling for resource allocation, and weighted voting for final prediction.

Result: MITS consistently surpasses baseline methods across diverse reasoning benchmarks while maintaining computational efficiency.

Conclusion: MITS establishes a principled and efficient framework for LLM reasoning through information-theoretic guidance.

Abstract: Tree search has become as a representative framework for test-time reasoning with large language models (LLMs), exemplified by methods such as Tree-of-Thought and Monte Carlo Tree Search that explore multiple reasoning paths. However, it remains difficult to provide instant and reliable quantitative assessments of intermediate reasoning step quality, and extensive path exploration is computationally costly. To address this, we propose Mutual Information Tree Search (MITS), a novel framework that guides reasoning with information-theoretic principles. MITS introduces an effective scoring function based on pointwise mutual information (PMI), which enables step-wise evaluation of reasoning paths and search tree expansion via beam search without expensive look-ahead simulations, achieving superior reasoning performances while maintaining computational efficiency. The framework is complemented by an entropy-based dynamic sampling strategy that adaptively allocates computational resources to uncertain reasoning steps where exploration is most beneficial. For final prediction, MITS employs a weighted voting scheme that combines PMI scores with prediction consensus. Through comprehensive experiments on diverse reasoning benchmarks, MITS consistently surpasses baseline methods, establishing a principled and efficient framework for LLM reasoning.

Method: Segments conversations by user goals, evaluates success using all relevant turns, employs teacher LLMs with thinking tokens for interpretable rationales, and uses domain experts to define goals and quality standards.

Result: Applied to AIDA enterprise chatbot system, achieved GSR improvement from 63% to 79% over six months, providing actionable insights through detailed failure analysis.

Conclusion: The framework offers generic, explainable, and data-efficient evaluation of multi-agent systems, enabling diagnosis of success rates, identification of failure modes, and informing system improvements.

Abstract: Evaluating the quality of multi-turn chatbot interactions remains challenging, as most existing methods assess interactions at the turn level without addressing whether a user’s overarching goal was fulfilled. A goal'' here refers to an information need or task, such as asking for policy information or applying for leave. We propose a comprehensive framework for goal-oriented evaluation of multi-agent systems (MAS), introducing the \textbf{Goal Success Rate (GSR)} to measure the percentage of fulfilled goals, and a \textbf{Root Cause of Failure (RCOF)} taxonomy to identify reasons for failure in multi-agent chatbots. Our method segments conversations by user goals and evaluates success using all relevant turns. We present a model-based evaluation system combining teacher LLMs, where domain experts define goals, set quality standards serving as a guidance for the LLMs. The LLMs use thinking tokens’’ to produce interpretable rationales, enabling \textit{explainable}, \textit{data-efficient} evaluations. In an enterprise setting, we apply our framework to evaluate AIDA, a zero-to-one employee conversational agent system built as a ground-up multi-agent conversational agent, and observe GSR improvement from 63% to 79% over six months since its inception. Our framework is generic and offers actionable insights through a detailed defect taxonomy based on analysis of failure points in multi-agent chatbots, diagnosing overall success, identifying key failure modes, and informing system improvements.

Method: H-DDx uses a retrieval and reranking pipeline to map free-text diagnoses to ICD-10 codes and applies hierarchical metrics that credit predictions closely related to ground-truth diagnoses.

Result: Benchmarking 22 leading models showed conventional flat metrics underestimate performance by overlooking clinically meaningful outputs, with domain-specialized open-source models performing well.

Conclusion: The hierarchical framework enhances interpretability by revealing that LLMs often correctly identify broader clinical context even when missing precise diagnoses, providing more clinically relevant evaluation.

Abstract: An accurate differential diagnosis (DDx) is essential for patient care, shaping therapeutic decisions and influencing outcomes. Recently, Large Language Models (LLMs) have emerged as promising tools to support this process by generating a DDx list from patient narratives. However, existing evaluations of LLMs in this domain primarily rely on flat metrics, such as Top-k accuracy, which fail to distinguish between clinically relevant near-misses and diagnostically distant errors. To mitigate this limitation, we introduce H-DDx, a hierarchical evaluation framework that better reflects clinical relevance. H-DDx leverages a retrieval and reranking pipeline to map free-text diagnoses to ICD-10 codes and applies a hierarchical metric that credits predictions closely related to the ground-truth diagnosis. In benchmarking 22 leading models, we show that conventional flat metrics underestimate performance by overlooking clinically meaningful outputs, with our results highlighting the strengths of domain-specialized open-source models. Furthermore, our framework enhances interpretability by revealing hierarchical error patterns, demonstrating that LLMs often correctly identify the broader clinical context even when the precise diagnosis is missed.

Method: Improve MFMs’ reasoning through discriminative tasks and structured reasoning skills (causal inference, counterfactual thinking, spatiotemporal reasoning), and enhance generative capabilities using scene graphs, multimodal conditioning, alignment strategies, and controllable 4D generation techniques.

Result: The approaches enable MFMs to go beyond surface correlations, understand deeper relationships in visual/textual data, and achieve structured, controllable generation across image and video modalities with temporal and spatial consistency.

Conclusion: By integrating enhanced reasoning and generative capabilities, MFMs can be transformed into effective world models that better simulate and understand dynamic physical processes through multimodal integration.

Abstract: Humans understand the world through the integration of multiple sensory modalities, enabling them to perceive, reason about, and imagine dynamic physical processes. Inspired by this capability, multimodal foundation models (MFMs) have emerged as powerful tools for multimodal understanding and generation. However, today’s MFMs fall short of serving as effective world models. They lack the essential ability such as perform counterfactual reasoning, simulate dynamics, understand the spatiotemporal information, control generated visual outcomes, and perform multifaceted reasoning. We investigates what it takes to bridge the gap between multimodal foundation models and world models. We begin by improving the reasoning capabilities of MFMs through discriminative tasks and equipping MFMs with structured reasoning skills, such as causal inference, counterfactual thinking, and spatiotemporal reasoning, enabling them to go beyond surface correlations and understand deeper relationships within visual and textual data. Next, we explore generative capabilities of multimodal foundation models across both image and video modalities, introducing new frameworks for structured and controllable generation. Our approaches incorporate scene graphs, multimodal conditioning, and multimodal alignment strategies to guide the generation process, ensuring consistency with high-level semantics and fine-grained user intent. We further extend these techniques to controllable 4D generation, enabling interactive, editable, and morphable object synthesis over time and space.

Method: Combines multi-agent simulations with genetic algorithms, using multiple LLM-based agents acting as simulated shopping customers as dynamic reward signals, with average agent scores serving as fitness function for evolutionary query refinement.

Result: Evaluated on 1000 real-world e-commerce queries across 5 categories, achieving 21.98% average improvement over original user queries and 3.36% improvement over Best-of-N LLM rewriting baseline.

Conclusion: OptAgent provides an effective framework for optimizing subjective tasks like query rewriting by leveraging multi-agent simulations and evolutionary algorithms, overcoming limitations of static reward models or single LLM judges.

Abstract: Deploying capable and user-aligned LLM-based systems necessitates reliable evaluation. While LLMs excel in verifiable tasks like coding and mathematics, where gold-standard solutions are available, adoption remains challenging for subjective tasks that lack a single correct answer. E-commerce Query Rewriting (QR) is one such problem where determining whether a rewritten query properly captures the user intent is extremely difficult to figure out algorithmically. In this work, we introduce OptAgent, a novel framework that combines multi-agent simulations with genetic algorithms to verify and optimize queries for QR. Instead of relying on a static reward model or a single LLM judge, our approach uses multiple LLM-based agents, each acting as a simulated shopping customer, as a dynamic reward signal. The average of these agent-derived scores serves as an effective fitness function for an evolutionary algorithm that iteratively refines the user’s initial query. We evaluate OptAgent on a dataset of 1000 real-world e-commerce queries in five different categories, and we observe an average improvement of 21.98% over the original user query and 3.36% over a Best-of-N LLM rewriting baseline.

Method: GuidedSampling decouples exploration and generation phases. The exploration phase identifies multiple concepts for solving the problem, while the generation phase applies specific concepts to produce final solution candidates.

Result: GuidedSampling improves base model performance at pass@50 by ~21.6% across various benchmarks compared to RS. Models trained on GuidedSampling trajectories show ~9.7% improvement at pass@5 and increase average concepts per instance from 1.67 to 3.03, generating more diverse candidates.

Conclusion: GuidedSampling effectively addresses the diversity limitations of Repeated Sampling by separating concept exploration from solution generation, leading to significant performance improvements and more diverse solution candidates in both inference and training scenarios.

Abstract: Repeated Sampling (RS) is a simple inference-time algorithm that has been shown to improve model performance on complex tasks. Although it is an effective way of scaling inference time, it often struggles to generate diverse solution candidates, frequently relying on the same underlying approach to solve the problem and thus producing redundant samples. To address this limitation, we propose a new inference algorithm, GuidedSampling, which decouples the exploration and generation phases during inference, increasing diversity of generated candidate solutions. The exploration phase identifies multiple concepts that can be utilized to solve the problem, while the generation phase applies a specific concept to provide final solution candidates. We first define the theoretical bounds of GuidedSampling and then empirically demonstrate that it improves the performance of base model at pass@50 by on an average ~21.6% across various benchmarks compared to RS. Furthermore, models trained on trajectories of GuidedSampling exhibit substantial performance improvements at pass@5 by on an average ~9.7%, compared to models trained on traditional RS. Additionally, models trained with GuidedSampling increases the average number of concepts per instance (1.67 -> 3.03), yielding a diverse set of candidates than traditional RS.

Method: Proposes speculative actions framework inspired by speculative execution in microprocessors and speculative decoding in LLM inference. Uses faster models to predict likely actions, enabling parallel execution of multiple steps.

Result: Achieves up to 55% accuracy in next-action prediction across gaming, e-commerce, and web search environments. Significant reductions in end-to-end latency. Performance improves with stronger guessing models, top-K action prediction, multi-step speculation, and uncertainty-aware optimization.

Conclusion: Speculative actions provide a promising path toward deploying low-latency agentic systems in real-world applications, with lossless execution and substantial speed improvements.

Abstract: Despite growing interest in AI agents across industry and academia, their execution in an environment is often slow, hampering training, evaluation, and deployment. For example, a game of chess between two state-of-the-art agents may take hours. A critical bottleneck is that agent behavior unfolds sequentially: each action requires an API call, and these calls can be time-consuming. Inspired by speculative execution in microprocessors and speculative decoding in LLM inference, we propose speculative actions, a lossless framework for general agentic systems that predicts likely actions using faster models, enabling multiple steps to be executed in parallel. We evaluate this framework across three agentic environments: gaming, e-commerce, web search, and a “lossy” extension for an operating systems environment. In all cases, speculative actions achieve substantial accuracy in next-action prediction (up to 55%), translating into significant reductions in end-to-end latency. Moreover, performance can be further improved through stronger guessing models, top-K action prediction, multi-step speculation, and uncertainty-aware optimization, opening a promising path toward deploying low-latency agentic systems in the real world.

Method: The authors develop a composition of regret minimization techniques that achieve optimal external and swap regret bounds, enabling discovery and exploitation of hidden game structures.

Result: The approach ensures rapid convergence to correlated equilibria in hidden subgames while maintaining computational efficiency through leveraging the hidden game structure.

Conclusion: The paper affirmatively answers that efficient regret minimization algorithms can be designed to discover and exploit hidden game structures, leading to equilibrium in subgames while maintaining general rationality.

Abstract: This paper investigates a class of games with large strategy spaces, motivated by challenges in AI alignment and language games. We introduce the hidden game problem, where for each player, an unknown subset of strategies consistently yields higher rewards compared to the rest. The central question is whether efficient regret minimization algorithms can be designed to discover and exploit such hidden structures, leading to equilibrium in these subgames while maintaining rationality in general. We answer this question affirmatively by developing a composition of regret minimization techniques that achieve optimal external and swap regret bounds. Our approach ensures rapid convergence to correlated equilibria in hidden subgames, leveraging the hidden game structure for improved computational efficiency.

Method: Synthesizes evidence from various SLMs and uses guided decoding libraries (XGrammar, Outlines) with strict JSON Schema outputs, validator-first tool execution, uncertainty-aware routing, and verifier cascades in SLM-default, LLM-fallback systems.

Result: SLMs can match or surpass LLMs on tool use, function calling, and RAG tasks with 10x-100x lower token costs, better latency, and energy efficiency, while maintaining schema validity and executable call rates.

Conclusion: Provides a practical blueprint for building fast, inexpensive, and reliable agents that default to SLMs while using targeted LLM assistance for specific cases like open-domain reasoning and long-horizon planning.

Abstract: Small language models (SLMs; 1-12B params, sometimes up to 20B) are sufficient and often superior for agentic workloads where the objective is schema- and API-constrained accuracy rather than open-ended generation. We synthesize recent evidence across open and proprietary SLMs (Phi-4-Mini, Qwen-2.5-7B, Gemma-2-9B, Llama-3.2-1B/3B, Ministral-3B/8B, Apple on-device 3B, DeepSeek-R1-Distill) and connect it to modern evaluations (BFCL v3/v4, StableToolBench) and serving stacks (vLLM, SGLang, TensorRT-LLM) paired with guided decoding libraries (XGrammar, Outlines). We formalize SLM-default, LLM-fallback systems with uncertainty-aware routing and verifier cascades, and propose engineering metrics that reflect real production goals: cost per successful task (CPS), schema validity rate, executable call rate, p50/p95 latency, and energy per request. Guided decoding, strict JSON Schema outputs, and validator-first tool execution close much of the capability gap with larger models and often let SLMs match or surpass LLMs on tool use, function calling, and RAG at 10x-100x lower token cost with materially better latency and energy. We provide design patterns for agent stacks that prioritize SLMs: schema-first prompting, type-safe function registries, confidence scoring with verifier rollups, and lightweight adaptation via LoRA/QLoRA. We also delineate limits where fallback remains valuable (open-domain reasoning and some long-horizon planning). The result is a practical blueprint for building fast, inexpensive, and reliable agents that default to SLMs while preserving headroom with targeted LLM assistance. Keywords: small language models, agents, function calling, structured outputs, JSON Schema, guided decoding, LoRA/QLoRA, routing, energy efficiency, edge inference

Method: MetaMuse introduces three self-reflection principles: (1) quantifying diversity in performance space, (2) steering ideation through external stimuli, and (3) constructing solutions using waypoint reasoning instead of free-form chain-of-thought.

Result: MetaMuse reduced cache misses by up to 35.76% in cache replacement and reduced bin usage by up to 30.93% in online bin packing at a global cloud provider.

Conclusion: The framework successfully addresses LLM limitations in creative algorithm generation and demonstrates practical value in real-world system optimization problems.

Abstract: Designing system algorithms remains challenging, where the discontinuous nature of the solution space often forces system engineers to rely on generic heuristics at the expense of performance. We study whether LLMs can practically drive algorithm generation, and find that they are biased towards well-known generic designs, rather than making the creative leaps needed to navigate the discontinuous solution space. To address this limitation, we introduce MetaMuse, a framework for creative ideation built on three self-reflection principles: (1) quantifying solution diversity and usefulness in measurable performance space, rather than abstract idea space, (2) steering ideation through external stimuli, rather than internal randomness, and (3) constructing executable solutions using waypoint reasoning, rather than free-form chain-of-thought. Extensive evaluation shows that MetaMuse can generate high-performing solutions for two critical problems at a global cloud provider: cache replacement (reducing cache misses by up to 35.76%) and online bin packing (reducing bin usage by up to 30.93%).

Method: Decomposes past task trajectories into reusable memory units and flexibly allocates them across orchestrators and task agents. Conducts systematic study of memory placement, retrieval methods, and agent benefits.

Result: Experiments on OfficeBench show orchestrator memory is critical for task decomposition/delegation, while fine-grained agent memory improves execution accuracy. Smaller models benefit substantially, narrowing performance gap with stronger agents.

Conclusion: LEGOMem serves as both a practical framework for memory-augmented agent systems and a research tool for understanding memory design in multi-agent workflow automation.

Abstract: We introduce LEGOMem, a modular procedural memory framework for multi-agent large language model (LLM) systems in workflow automation. LEGOMem decomposes past task trajectories into reusable memory units and flexibly allocates them across orchestrators and task agents to support planning and execution. To explore the design space of memory in multi-agent systems, we use LEGOMem as a lens and conduct a systematic study of procedural memory in multi-agent systems, examining where memory should be placed, how it should be retrieved, and which agents benefit most. Experiments on the OfficeBench benchmark show that orchestrator memory is critical for effective task decomposition and delegation, while fine-grained agent memory improves execution accuracy. We find that even teams composed of smaller language models can benefit substantially from procedural memory, narrowing the performance gap with stronger agents by leveraging prior execution traces for more accurate planning and tool use. These results position LEGOMem as both a practical framework for memory-augmented agent systems and a research tool for understanding memory design in multi-agent workflow automation.

Method: Uses LLM-supported contextual reasoning with XAI agents, employing attention mechanisms, memory buffers with meaning, and avoiding detailed time-step processing to discover hidden patterns and inconsistencies in data streams.

Result: The proposed approach significantly outperforms existing models in both detection accuracy and interpretability across smart grid and healthcare IoT simulations, with improved response speed and reduced false positives.

Conclusion: The LLM-enhanced contextual reasoning method with XAI agents shows strong potential as a future solution for anomaly detection in IoT systems, offering better accuracy, transparency, and adaptability than traditional approaches.

Abstract: Ensuring that critical IoT systems function safely and smoothly depends a lot on finding anomalies quickly. As more complex systems, like smart healthcare, energy grids and industrial automation, appear, it is easier to see the shortcomings of older methods of detection. Monitoring failures usually happen in dynamic, high dimensional situations, especially when data is incomplete, messy or always evolving. Such limits point out the requirement for adaptive, intelligent systems that always improve and think. LLMs are now capable of significantly changing how context is understood and semantic inference is done across all types of data. This proposal suggests using an LLM supported contextual reasoning method along with XAI agents to improve how anomalies are found in significant IoT environments. To discover hidden patterns and notice inconsistencies in data streams, it uses attention methods, avoids dealing with details from every time step and uses memory buffers with meaning. Because no code AI stresses transparency and interpretability, people can check and accept the AI’s decisions, helping ensure AI follows company policies. The two architectures are put together in a test that compares the results of the traditional model with those of the suggested LLM enhanced model. Important measures to check are the accuracy of detection, how much inaccurate information is included in the results, how clearly the findings can be read and how fast the system responds under different test situations. The metaheuristic is tested in simulations of real world smart grid and healthcare contexts to check its adaptability and reliability. From the study, we see that the new approach performs much better than most existing models in both accuracy and interpretation, so it could be a good fit for future anomaly detection tasks in IoT

Method: Frame level generation as a multi-agent problem where multiple agents work collaboratively, reducing the number of reward calculations relative to actions and learning more local, modular design policies.

Result: Multi-agent level generators show improved efficiency by reducing reward calculation overhead and better generalization to out-of-distribution map shapes compared to single-agent approaches.

Conclusion: Treating content generation as a distributed, multi-agent task is beneficial for generating functional artifacts at scale, offering efficiency gains and improved generalization capabilities.

Abstract: Procedural Content Generation via Reinforcement Learning (PCGRL) offers a method for training controllable level designer agents without the need for human datasets, using metrics that serve as proxies for level quality as rewards. Existing PCGRL research focuses on single generator agents, but are bottlenecked by the need to frequently recalculate heuristics of level quality and the agent’s need to navigate around potentially large maps. By framing level generation as a multi-agent problem, we mitigate the efficiency bottleneck of single-agent PCGRL by reducing the number of reward calculations relative to the number of agent actions. We also find that multi-agent level generators are better able to generalize to out-of-distribution map shapes, which we argue is due to the generators’ learning more local, modular design policies. We conclude that treating content generation as a distributed, multi-agent task is beneficial for generating functional artifacts at scale.

Method: The system uses a procedural generation pipeline with constraint-based difficulty control, automated correctness verification, and human-in-the-loop validation to create dynamic questions requiring spatial reasoning skills.

Result: Evaluation on Spatial-CAPTCHA-Bench shows humans vastly outperform 10 state-of-the-art MLLMs, with the best model achieving only 31.0% Pass@1 accuracy. The framework also proves effective compared to Google reCAPTCHA.

Conclusion: Spatial CAPTCHA provides an effective security mechanism against automated abuse and serves as a diagnostic tool for assessing spatial reasoning capabilities in AI systems, leveraging human strengths in spatial cognition that current MLLMs struggle with.

Abstract: Online services rely on CAPTCHAs as a first line of defense against automated abuse, yet recent advances in multi-modal large language models (MLLMs) have eroded the effectiveness of conventional designs that focus on text recognition or 2D image understanding. To address this challenge, we present Spatial CAPTCHA, a novel human-verification framework that leverages fundamental differences in spatial reasoning between humans and MLLMs. Unlike existing CAPTCHAs which rely on low-level perception tasks that are vulnerable to modern AI, Spatial CAPTCHA generates dynamic questions requiring geometric reasoning, perspective-taking, occlusion handling, and mental rotation. These skills are intuitive for humans but difficult for state-of-the-art (SOTA) AI systems. The system employs a procedural generation pipeline with constraint-based difficulty control, automated correctness verification, and human-in-the-loop validation to ensure scalability, robustness, and adaptability. Evaluation on a corresponding benchmark, Spatial-CAPTCHA-Bench, demonstrates that humans vastly outperform 10 state-of-the-art MLLMs, with the best model achieving only 31.0% Pass@1 accuracy. Furthermore, we compare Spatial CAPTCHA with Google reCAPTCHA, which confirms its effectiveness as both a security mechanism and a diagnostic tool for spatial reasoning in AI.

Method: ECHO combines hierarchical context representation with positional-based leveling, objective analysis-based evaluation, and consensus voting mechanisms to attribute errors.

Result: Experimental results show ECHO outperforms existing methods across various multi-agent interaction scenarios, particularly excelling in cases with subtle reasoning errors and complex interdependencies.

Conclusion: Structured hierarchical context representation combined with consensus-based objective decision-making provides a more robust framework for error attribution in multi-agent systems.

Abstract: Error attribution in Large Language Model (LLM) multi-agent systems presents a significant challenge in debugging and improving collaborative AI systems. Current approaches to pinpointing agent and step level failures in interaction traces - whether using all-at-once evaluation, step-by-step analysis, or binary search - fall short when analyzing complex patterns, struggling with both accuracy and consistency. We present ECHO (Error attribution through Contextual Hierarchy and Objective consensus analysis), a novel algorithm that combines hierarchical context representation, objective analysis-based evaluation, and consensus voting to improve error attribution accuracy. Our approach leverages a positional-based leveling of contextual understanding while maintaining objective evaluation criteria, ultimately reaching conclusions through a consensus mechanism. Experimental results demonstrate that ECHO outperforms existing methods across various multi-agent interaction scenarios, showing particular strength in cases involving subtle reasoning errors and complex interdependencies. Our findings suggest that leveraging these concepts of structured, hierarchical context representation combined with consensus-based objective decision-making, provides a more robust framework for error attribution in multi-agent systems.

Method: Mathematically expand representational basins around text token embeddings via variance scale-up before joint-attention blocks in MM-DiTs, without requiring additional training, data, or external modules.

Result: Rare semantics clearly emerge in MM-DiT’s outputs, with effective generalization across text-to-image, text-to-video, and text-driven image editing tasks.

Conclusion: The proposed intervention successfully surfaces hidden semantics that users intend, enabling generative models to better handle rare and imaginative prompts.

Abstract: Starting from flow- and diffusion-based transformers, Multi-modal Diffusion Transformers (MM-DiTs) have reshaped text-to-vision generation, gaining acclaim for exceptional visual fidelity. As these models advance, users continually push the boundary with imaginative or rare prompts, which advanced models still falter in generating, since their concepts are often too scarce to leave a strong imprint during pre-training. In this paper, we propose a simple yet effective intervention that surfaces rare semantics inside MM-DiTs without additional training steps, data, denoising-time optimization, or reliance on external modules (e.g., large language models). In particular, the joint-attention mechanism intrinsic to MM-DiT sequentially updates text embeddings alongside image embeddings throughout transformer blocks. We find that by mathematically expanding representational basins around text token embeddings via variance scale-up before the joint-attention blocks, rare semantics clearly emerge in MM-DiT’s outputs. Furthermore, our results generalize effectively across text-to-vision tasks, including text-to-image, text-to-video, and text-driven image editing. Our work invites generative models to reveal the semantics that users intend, once hidden yet ready to surface.

Method: Six-round game with three options per round. Uses two symbolic engines: Kantian module flags rule violations (child labor, deforestation, etc.) and utilitarian module scores options via multi-criteria aggregation. Meta-explainer with regret bound (0.2) manages alignment between frameworks.

Result: Developed a complete system with structured configuration (attribute schema, certification map, weights, rule set), policy trace for auditability, and interactive UI.

Conclusion: The system successfully integrates multiple ethical frameworks in an explainable AI approach for consumer decision-making, providing both transparency and practical guidance for ethical coffee choices.

Abstract: We present a gamified explainable AI (XAI) system for ethically aware consumer decision-making in the coffee domain. Each session comprises six rounds with three options per round. Two symbolic engines provide real-time reasons: a Kantian module flags rule violations (e.g., child labor, deforestation risk without shade certification, opaque supply chains, unsafe decaf), and a utilitarian module scores options via multi-criteria aggregation over normalized attributes (price, carbon, water, transparency, farmer income share, taste/freshness, packaging, convenience). A meta-explainer with a regret bound (0.2) highlights Kantian–utilitarian (mis)alignment and switches to a deontically clean, near-parity option when welfare loss is small. We release a structured configuration (attribute schema, certification map, weights, rule set), a policy trace for auditability, and an interactive UI.

Method: Model multi-turn conversations as probability distributions over query sequences using Markov processes on query graphs with semantic similarity edges, and quantify risks using confidence intervals with practical distributions (random node, graph path, adaptive with rejection).

Result: The framework reveals substantial catastrophic risks in frontier models, with certified lower bounds as high as 70% for the worst model.

Conclusion: There is an urgent need for improved safety training strategies in frontier LLMs given the high certified risk bounds revealed by QRLLM.

Abstract: Large Language Models (LLMs) can produce catastrophic responses in conversational settings that pose serious risks to public safety and security. Existing evaluations often fail to fully reveal these vulnerabilities because they rely on fixed attack prompt sequences, lack statistical guarantees, and do not scale to the vast space of multi-turn conversations. In this work, we propose QRLLM, a novel, principled Certification framework for Catastrophic risks in multi-turn Conversation for LLMs that bounds the probability of an LLM generating catastrophic responses under multi-turn conversation distributions with statistical guarantees. We model multi-turn conversations as probability distributions over query sequences, represented by a Markov process on a query graph whose edges encode semantic similarity to capture realistic conversational flow, and quantify catastrophic risks using confidence intervals. We define several inexpensive and practical distributions: random node, graph path, adaptive with rejection. Our results demonstrate that these distributions can reveal substantial catastrophic risks in frontier models, with certified lower bounds as high as 70% for the worst model, highlighting the urgent need for improved safety training strategies in frontier LLMs.

Method: Developed C^2-Eval benchmark that evaluates both convergent and divergent creativity using fine-grained criteria (Usefulness, Originality, Surprise) derived from social-science theory, and conducted extensive experiments on leading proprietary and open-source models.

Result: Analysis revealed trade-offs in creative capabilities of current foundation models, highlighting both strengths and challenges in pursuing creative machine intelligence.

Conclusion: C^2-Eval provides an effective framework for examining the evolving landscape of creative AI and serves as a comprehensive benchmark for assessing creativity in foundation models.

Abstract: The meteoric rise of foundation models (FMs) has expanded their capabilities far beyond conventional tasks. Creativity, long regarded as a hallmark of human intelligence and a driver of innovation, is now increasingly recognized as a critical dimension of machine intelligence in the era of generative FMs, complementing traditional measures of accuracy. However, existing evaluation frameworks for creativity remain fragmented, relying on ad hoc metrics not firmly grounded in established theories. To address this gap, we introduce C^2-Eval, a holistic benchmark for unified assessment of creativity in FMs. C^2-Eval distinguishes between two complementary forms of creativity: convergent creativity, where tasks admit constrained solutions (e.g., code generation), and divergent creativity, where tasks are open-ended (e.g., storytelling). It evaluates both dimensions using fine-grained criteria derived from social-science theory, focusing on Usefulness, Originality, and Surprise (U-O-S). Through extensive experiments on leading proprietary and open-source models, we analyze trade-offs in their creative capabilities. Our results highlight both the strengths and challenges of current FMs in pursuing a creative machine mind, showing that C^2-Eval is an effective lens for examining the evolving landscape of creative AI.

Method: Built ZephyrusWorld environment with weather data tools and Zephyrus LLM-based agent that iteratively analyzes data through conversational feedback loops, plus ZephyrusBench benchmark with diverse weather tasks.

Result: Zephyrus agents outperform text-only baselines by up to 35 percentage points in correctness on ZephyrusBench, though performance is similar on harder tasks.

Conclusion: The framework successfully bridges weather models and LLMs, with benchmark highlighting challenges for future work on complex weather reasoning tasks.

Abstract: Foundation models for weather science are pre-trained on vast amounts of structured numerical data and outperform traditional weather forecasting systems. However, these models lack language-based reasoning capabilities, limiting their utility in interactive scientific workflows. Large language models (LLMs) excel at understanding and generating text but cannot reason about high-dimensional meteorological datasets. We bridge this gap by building a novel agentic framework for weather science. Our framework includes a Python code-based environment for agents (ZephyrusWorld) to interact with weather data, featuring tools like an interface to WeatherBench 2 dataset, geoquerying for geographical masks from natural language, weather forecasting, and climate simulation capabilities. We design Zephyrus, a multi-turn LLM-based weather agent that iteratively analyzes weather datasets, observes results, and refines its approach through conversational feedback loops. We accompany the agent with a new benchmark, ZephyrusBench, with a scalable data generation pipeline that constructs diverse question-answer pairs across weather-related tasks, from basic lookups to advanced forecasting, extreme event detection, and counterfactual reasoning. Experiments on this benchmark demonstrate the strong performance of Zephyrus agents over text-only baselines, outperforming them by up to 35 percentage points in correctness. However, on harder tasks, Zephyrus performs similarly to text-only baselines, highlighting the challenging nature of our benchmark and suggesting promising directions for future work.

Method: Developed a lifecycle-aligned taxonomy mapping 45 data science agent systems across six data science stages, with annotation along five cross-cutting design dimensions including reasoning style, modality integration, tool orchestration, learning methods, and trust mechanisms.

Result: Identified three key trends: most systems focus on exploratory analysis and modeling while neglecting business understanding and deployment; multimodal reasoning and tool orchestration remain challenging; and over 90% lack explicit trust and safety mechanisms.

Conclusion: Outlined open challenges in alignment stability, explainability, governance, and evaluation frameworks, proposing future research directions for developing robust, trustworthy, and accessible data science agents.

Abstract: Recent advances in large language models (LLMs) have enabled a new class of AI agents that automate multiple stages of the data science workflow by integrating planning, tool use, and multimodal reasoning across text, code, tables, and visuals. This survey presents the first comprehensive, lifecycle-aligned taxonomy of data science agents, systematically analyzing and mapping forty-five systems onto the six stages of the end-to-end data science process: business understanding and data acquisition, exploratory analysis and visualization, feature engineering, model building and selection, interpretation and explanation, and deployment and monitoring. In addition to lifecycle coverage, we annotate each agent along five cross-cutting design dimensions: reasoning and planning style, modality integration, tool orchestration depth, learning and alignment methods, and trust, safety, and governance mechanisms. Beyond classification, we provide a critical synthesis of agent capabilities, highlight strengths and limitations at each stage, and review emerging benchmarks and evaluation practices. Our analysis identifies three key trends: most systems emphasize exploratory analysis, visualization, and modeling while neglecting business understanding, deployment, and monitoring; multimodal reasoning and tool orchestration remain unresolved challenges; and over 90% lack explicit trust and safety mechanisms. We conclude by outlining open challenges in alignment stability, explainability, governance, and robust evaluation frameworks, and propose future research directions to guide the development of robust, trustworthy, low-latency, transparent, and broadly accessible data science agents.

Method: MedLog defines a structured protocol with nine core fields (header, model, user, target, inputs, artifacts, outputs, outcomes, feedback) to record AI model interactions. It supports risk-based sampling, retention policies, and caching for efficiency.

Result: The protocol enables consistent logging of clinical AI activities including model invocations, user interactions, and autonomous actions, providing a foundation for analysis and monitoring.

Conclusion: MedLog can catalyze development of databases and tools for continuous surveillance, auditing, and iterative improvement of medical AI, establishing a foundation for digital epidemiology in healthcare.

Abstract: Modern computer systems often rely on syslog, a simple, universal protocol that records every critical event across heterogeneous infrastructure. However, healthcare’s rapidly growing clinical AI stack has no equivalent. As hospitals rush to pilot large language models and other AI-based clinical decision support tools, we still lack a standard way to record how, when, by whom, and for whom these AI models are used. Without that transparency and visibility, it is challenging to measure real-world performance and outcomes, detect adverse events, or correct bias or dataset drift. In the spirit of syslog, we introduce MedLog, a protocol for event-level logging of clinical AI. Any time an AI model is invoked to interact with a human, interface with another algorithm, or act independently, a MedLog record is created. This record consists of nine core fields: header, model, user, target, inputs, artifacts, outputs, outcomes, and feedback, providing a structured and consistent record of model activity. To encourage early adoption, especially in low-resource settings, and minimize the data footprint, MedLog supports risk-based sampling, lifecycle-aware retention policies, and write-behind caching; detailed traces for complex, agentic, or multi-stage workflows can also be captured under MedLog. MedLog can catalyze the development of new databases and software to store and analyze MedLog records. Realizing this vision would enable continuous surveillance, auditing, and iterative improvement of medical AI, laying the foundation for a new form of digital epidemiology.

Method: Introduced FaithCoT-Bench with rigorous task formulation for discriminative unfaithfulness detection, created FINE-CoT dataset with 1,000+ expert-annotated trajectories from 4 LLMs across 4 domains, and systematically evaluated 11 detection methods.

Result: Evaluation revealed strengths/weaknesses of existing approaches, showing increased detection challenges in knowledge-intensive domains and with more advanced models. The benchmark includes 300+ unfaithful instances with fine-grained causes.

Conclusion: FaithCoT-Bench establishes the first comprehensive benchmark for instance-level CoT faithfulness, providing a solid foundation for future research toward more interpretable and trustworthy reasoning in LLMs.

Abstract: Large language models (LLMs) increasingly rely on Chain-of-Thought (CoT) prompting to improve problem-solving and provide seemingly transparent explanations. However, growing evidence shows that CoT often fail to faithfully represent the underlying reasoning process, raising concerns about their reliability in high-risk applications. Although prior studies have focused on mechanism-level analyses showing that CoTs can be unfaithful, they leave open the practical challenge of deciding whether a specific trajectory is faithful to the internal reasoning of the model. To address this gap, we introduce FaithCoT-Bench, a unified benchmark for instance-level CoT unfaithfulness detection. Our framework establishes a rigorous task formulation that formulates unfaithfulness detection as a discriminative decision problem, and provides FINE-CoT (Faithfulness instance evaluation for Chain-of-Thought), an expert-annotated collection of over 1,000 trajectories generated by four representative LLMs across four domains, including more than 300 unfaithful instances with fine-grained causes and step-level evidence. We further conduct a systematic evaluation of eleven representative detection methods spanning counterfactual, logit-based, and LLM-as-judge paradigms, deriving empirical insights that clarify the strengths and weaknesses of existing approaches and reveal the increased challenges of detection in knowledge-intensive domains and with more advanced models. To the best of our knowledge, FaithCoT-Bench establishes the first comprehensive benchmark for instance-level CoT faithfulness, setting a solid basis for future research toward more interpretable and trustworthy reasoning in LLMs.

Method: Implemented an abstract protocol based on communicating finite-state machines for human-LLM interaction, tested with controlled experiments using a human proxy (database) and uncontrolled experiments with human subjects in radiology and drug design domains.

Result: Empirical evidence supports the protocol’s capability to capture one- and two-way intelligibility in human-LLM interaction, demonstrating utility of two-way intelligibility in human-machine system design.

Conclusion: The structured protocol enables effective human-LLM collaboration, with two-way intelligibility proving valuable for designing human-machine systems that leverage both human expertise and LLM capabilities.

Abstract: Our interest is in the design of software systems involving a human-expert interacting – using natural language – with a large language model (LLM) on data analysis tasks. For complex problems, it is possible that LLMs can harness human expertise and creativity to find solutions that were otherwise elusive. On one level, this interaction takes place through multiple turns of prompts from the human and responses from the LLM. Here we investigate a more structured approach based on an abstract protocol described in [3] for interaction between agents. The protocol is motivated by a notion of “two-way intelligibility” and is modelled by a pair of communicating finite-state machines. We provide an implementation of the protocol, and provide empirical evidence of using the implementation to mediate interactions between an LLM and a human-agent in two areas of scientific interest (radiology and drug design). We conduct controlled experiments with a human proxy (a database), and uncontrolled experiments with human subjects. The results provide evidence in support of the protocol’s capability of capturing one- and two-way intelligibility in human-LLM interaction; and for the utility of two-way intelligibility in the design of human-machine systems. Our code is available at https://github.com/karannb/interact.

Method: Developed a theoretical framework for question answering with variable voting thresholds, allowing ensembles to abstain when dominant responses don’t meet confidence thresholds. Tested on arithmetic problem solving and clinical-note question-answering domains.

Result: Large gains in answer trustworthiness achieved with restrictive voting ensembles, with modest reductions in response yield and accuracy. Particularly effective in domains requiring high certainty.

Conclusion: Variable-threshold voting ensembles are valuable for applications like healthcare and data annotation that require high certainty but don’t need automated answers to every question.

Abstract: Despite huge advances, LLMs still lack convenient and reliable methods to quantify the uncertainty in their responses, making them difficult to trust in high-stakes applications. One of the simplest approaches to eliciting more accurate answers is to select the mode of many responses, a technique known as ensembling. In this work, we expand on typical ensembling approaches by looking at ensembles with a variable voting threshold. We introduce a theoretical framework for question answering and show that, by permitting ensembles to “abstain” from providing an answer when the dominant response falls short of the threshold, it is possible to dramatically increase the trustworthiness of the remaining answers. From this framework, we derive theoretical results as well as report experimental results on two problem domains: arithmetic problem solving and clinical-note question-answering. In both domains, we observe that large gains in answer trustworthiness can be achieved using highly restrictive voting ensembles, while incurring relatively modest reductions in response yield and accuracy. Due to this quality, voting ensembles may be particularly useful in applications - such as healthcare and data annotation - that require a high degree of certainty but which may not require that every question receive an automated answer.

Method: Introduces LEGO-IRT with factorized architecture that decomposes model ability into general and structure-specific components, supporting both binary and continuous evaluation metrics.

Result: Achieves stable capability estimates using only 3% of total evaluation items, reduces estimation error by up to 10% through structural knowledge, and aligns better with human preferences.

Conclusion: LEGO-IRT provides a flexible, data-efficient framework for LLM evaluation that overcomes limitations of traditional IRT methods and leverages structural knowledge for improved accuracy.

Abstract: Evaluating large language models (LLMs) on comprehensive benchmarks is a cornerstone of their development, yet it’s often computationally and financially prohibitive. While Item Response Theory (IRT) offers a promising path toward data-efficient evaluation by disentangling model capability from item difficulty, existing IRT-based methods are hampered by significant limitations. They are typically restricted to binary correctness metrics, failing to natively handle the continuous scores used in generative tasks, and they operate on single benchmarks, ignoring valuable structural knowledge like correlations across different metrics or benchmarks. To overcome these challenges, we introduce LEGO-IRT, a unified and flexible framework for data-efficient LLM evaluation. LEGO-IRT’s novel design natively supports both binary and continuous evaluation metrics. Moreover, it introduces a factorized architecture to explicitly model and leverage structural knowledge, decomposing model ability estimates into a general component and structure-specific (e.g., per-metric or per-benchmark) components. Through extensive experiments involving $70$ LLMs across $5$ benchmarks, we show that LEGO-IRT achieves stable capability estimates using just $3%$ of the total evaluation items. We demonstrate that incorporating structural knowledge reduces estimation error by up to $10%$ and reveal that the latent abilities estimated by our framework may align more closely with human preferences.

Method: Used a novel large-scale Reddit corpus of 400,000 utterances balanced across seven basic emotions, employing lightweight probes to read information from hidden layers of Qwen3 and LLaMA models without altering parameters.

Result: LLMs develop surprisingly well-defined internal emotional geometry that sharpens with model scale and outperforms zero-shot prompting. Emotional signal emerges early, peaks mid-network, is malleable via system prompts, and persists for hundreds of subsequent tokens.

Conclusion: The study provides crucial insights for developing more transparent and aligned AI systems by mapping the emotional landscape within LLMs, with open-sourced dataset and probing toolkit.

Abstract: Large Language Models (LLMs) are increasingly expected to navigate the nuances of human emotion. While research confirms that LLMs can simulate emotional intelligence, their internal emotional mechanisms remain largely unexplored. This paper investigates the latent emotional representations within modern LLMs by asking: how, where, and for how long is emotion encoded in their neural architecture? To address this, we introduce a novel, large-scale Reddit corpus of approximately 400,000 utterances, balanced across seven basic emotions through a multi-stage process of classification, rewriting, and synthetic generation. Using this dataset, we employ lightweight “probes” to read out information from the hidden layers of various Qwen3 and LLaMA models without altering their parameters. Our findings reveal that LLMs develop a surprisingly well-defined internal geometry of emotion, which sharpens with model scale and significantly outperforms zero-shot prompting. We demonstrate that this emotional signal is not a final-layer phenomenon but emerges early and peaks mid-network. Furthermore, the internal states are both malleable (they can be influenced by simple system prompts) and persistent, as the initial emotional tone remains detectable for hundreds of subsequent tokens. We contribute our dataset, an open-source probing toolkit, and a detailed map of the emotional landscape within LLMs, offering crucial insights for developing more transparent and aligned AI systems. The code and dataset are open-sourced.

Method: MAS combines real-time Bayesian inference for monitoring value states, LSTM networks for forecasting drift, and a human-centric governance layer for adaptive interventions. It emphasizes low-latency responses (<20 ms) and reduces false positives via supervised fine-tuning with human feedback.

Result: MAS reduces value drift incidents by 80 percent or more in simulations, maintaining high detection accuracy (85 percent) and low false positive rates (0.08 post-adaptation). Experiments with goal-misaligned agents validate scalability and responsiveness.

Conclusion: MAS provides a predictive and adaptive solution for AI value alignment, contrasting static methods, with cross-domain applicability and open-source implementation for replication.

Abstract: The rise of artificial intelligence (AI) as super-capable assistants has transformed productivity and decision-making across domains. Yet, this integration raises critical concerns about value alignment - ensuring AI behaviors remain consistent with human ethics and intentions. A key risk is value drift, where AI systems deviate from aligned values due to evolving contexts, learning dynamics, or unintended optimizations, potentially leading to inefficiencies or ethical breaches. We propose the Moral Anchor System (MAS), a novel framework to detect, predict, and mitigate value drift in AI agents. MAS combines real-time Bayesian inference for monitoring value states, LSTM networks for forecasting drift, and a human-centric governance layer for adaptive interventions. It emphasizes low-latency responses (<20 ms) to prevent breaches, while reducing false positives and alert fatigue via supervised fine-tuning with human feedback. Our hypothesis: integrating probabilistic drift detection, predictive analytics, and adaptive governance can reduce value drift incidents by 80 percent or more in simulations, maintaining high detection accuracy (85 percent) and low false positive rates (0.08 post-adaptation). Rigorous experiments with goal-misaligned agents validate MAS’s scalability and responsiveness. MAS’s originality lies in its predictive and adaptive nature, contrasting static alignment methods. Contributions include: (1) MAS architecture for AI integration; (2) empirical results prioritizing speed and usability; (3) cross-domain applicability insights; and (4) open-source code for replication.

Method: SPOGW uses Iterative offline GRPO (ioGRPO) with advantage-masked KL divergence (mKL) to operate on cardinal reward signals through group-wise comparison, enabling efficient optimization in continuous space with emphasis on advantageous policy regions.

Result: SPOGW matches or exceeds state-of-the-art performance across five benchmark datasets covering mathematical reasoning, coding, and question answering tasks.

Conclusion: SPOGW presents a viable and forward-looking methodology for automated generation and optimization of agentic workflows, addressing key limitations of current approaches.

Abstract: Large language models (LLMs) have exhibited significant capabilities in addressing challenging problems throughout various fields, often through the use of agentic workflows that adhere to structured instructions and multi-step procedures. However, designing such workflows demands substantial manual effort, posing challenges to scalability and generalizability. Recent studies have aimed to minimize the human intervention needed for their construction, leading to advances in automated techniques for optimizing agentic workflows. However, current approaches are often constrained by their limited representational capacity, insufficient adaptability, weak scalability, and pairwise comparison paradigm – issues that stem primarily from a dependence on discrete optimization techniques. To overcome these limitations, we introduce a new score-based preference approach, refereed as SPOGW, which operates directly on cardinal reward signals through group-wise comparison and enables more efficient and stable optimization in a continuous space. SPOGW incorporates Iterative offline GRPO (ioGRPO) with advantage-masked KL divergence (mKL), which regulates training update by placing greater emphasis on the advantageous regions of the policy response. In five benchmark datasets covering mathematical reasoning, coding, and question answering, SPOGW matches or exceeds the performance of current state-of-the-art approaches, presenting a viable and forward-looking methodology for automated generation and optimization of agentic workflows.

Method: DLLM constructs independent subgraphs based on response correctness, applies relation augmentation alignment, fuses representations with LLM-derived semantic knowledge, and uses a two-stage denoising diffusion module (unconditional then conditional) to remove noise and align structural representations.

Result: Experiments on three web-based educational datasets show DLLM achieves optimal predictive performance across varying noise levels, demonstrating noise robustness while effectively leveraging LLM semantic knowledge.

Conclusion: DLLM successfully addresses noise and data imbalance challenges in cognitive diagnosis by integrating structural and semantic representations through diffusion-based denoising, outperforming existing methods in noisy educational environments.

Abstract: Cognitive diagnostics in the Web-based Intelligent Education System (WIES) aims to assess students’ mastery of knowledge concepts from heterogeneous, noisy interactions. Recent work has tried to utilize Large Language Models (LLMs) for cognitive diagnosis, yet LLMs struggle with structured data and are prone to noise-induced misjudgments. Specially, WIES’s open environment continuously attracts new students and produces vast amounts of response logs, exacerbating the data imbalance and noise issues inherent in traditional educational systems. To address these challenges, we propose DLLM, a Diffusion-based LLM framework for noise-robust cognitive diagnosis. DLLM first constructs independent subgraphs based on response correctness, then applies relation augmentation alignment module to mitigate data imbalance. The two subgraph representations are then fused and aligned with LLM-derived, semantically augmented representations. Importantly, before each alignment step, DLLM employs a two-stage denoising diffusion module to eliminate intrinsic noise while assisting structural representation alignment. Specifically, unconditional denoising diffusion first removes erroneous information, followed by conditional denoising diffusion based on graph-guided to eliminate misleading information. Finally, the noise-robust representation that integrates semantic knowledge and structural information is fed into existing cognitive diagnosis models for prediction. Experimental results on three publicly available web-based educational platform datasets demonstrate that our DLLM achieves optimal predictive performance across varying noise levels, which demonstrates that DLLM achieves noise robustness while effectively leveraging semantic knowledge from LLM.

Method: Created WebRenderBench with 22.5k diverse real-world webpages, proposed a novel evaluation metric for layout/style consistency from rendered pages, and developed ALISA agent that integrates this metric into reinforcement learning as reward signal.

Result: ALISA significantly boosts generation performance and achieves state-of-the-art results across multiple metrics compared to existing methods.

Conclusion: The proposed WebRenderBench benchmark and ALISA agent with novel evaluation metric provide more efficient, objective, and reliable UI quality assessment for WebUI-to-Code conversion tasks.

Abstract: Automating the conversion of UI images into web code is a critical task for front-end development and rapid prototyping. Advances in multimodal large language models (MLLMs) have made WebUI-to-Code increasingly feasible, yet existing benchmarks remain limited in data diversity and evaluation reliability. To address these issues, we present WebRenderBench, a large-scale benchmark of 22.5k webpages collected from real-world portal sites, offering greater diversity, complexity, and realism than prior benchmarks. We further propose a novel evaluation metric that measures layout and style consistency from the final rendered pages. Unlike vision-based methods that rely on costly LLM reasoning or structure-based comparisons vulnerable to noise and asymmetry, our approach enables more efficient, objective, and reliable UI quality assessment. Finally, we introduce the Automated Layout and Style Inspection Agent (ALISA), which integrates this metric into reinforcement learning as a reward signal to enhance training on crawled asymmetric webpages. Experiments show that ALISA significantly boosts generation performance, achieving state-of-the-art results across multiple metrics.

Method: Represent meta reasoning skeletons as directed acyclic graphs (DAGs), construct a search space, formulate the search problem, and use dynamic skeleton sampling that expands skeletons along with reasoning context during inference.

Result: AutoMR achieves better reasoning performance than previous works across extensive benchmark datasets.

Conclusion: Automated search for query-aware meta reasoning skeletons using DAG representations and dynamic sampling enables more adaptive and effective reasoning in large language models.

Abstract: Meta reasoning behaviors work as a skeleton to guide large language model (LLM) reasoning, thus help to improve reasoning performance. However, prior researches implement meta reasoning skeleton with manually designed structure, limiting ability to adapt to query-specific requirement and capture intricate logical dependency among reasoning steps. To deal with the challenges, we represent meta reasoning skeleton with directed acyclic graph (DAG) to unify skeletons proposed in prior works and model intricate logical dependency. Then we propose AutoMR, a framework that searches for query-aware meta reasoning skeleton automatically inspired by automated machine learning (AutoML). Specifically, we construct search space based on DAG representation of skeleton and then formulate the search problem. We design a dynamic skeleton sampling algorithm by expanding meta reasoning skeleton along with reasoning context at inference time. This algorithm can derive any meta reasoning skeleton in search space efficiently and adapt skeleton to evolving base reasoning context, thus enable efficient query-aware skeleton search. We conduct experiments on extensive benchmark datasets. Experimental results show that AutoMR achieves better reasoning performance than previous works broadly.

Method: Train crosscoders at multiple layers of DeepSeek-R1-Distill-Llama-8B and its base version, introduce latent attribution technique in crosscoder setting to locate features relevant for promoting/suppressing wait token probabilities.

Result: Identified a small set of features relevant for wait token probabilities; through max activating examples and causal interventions, showed these features are indeed relevant for reasoning and give rise to patterns like restarting, recalling prior knowledge, expressing uncertainty, and double-checking.

Conclusion: Model latents preceding wait tokens contain relevant information that modulates subsequent reasoning processes, enabling different reasoning patterns.

Abstract: Prior work has shown that a significant driver of performance in reasoning models is their ability to reason and self-correct. A distinctive marker in these reasoning traces is the token wait, which often signals reasoning behavior such as backtracking. Despite being such a complex behavior, little is understood of exactly why models do or do not decide to reason in this particular manner, which limits our understanding of what makes a reasoning model so effective. In this work, we address the question whether model’s latents preceding wait tokens contain relevant information for modulating the subsequent reasoning process. We train crosscoders at multiple layers of DeepSeek-R1-Distill-Llama-8B and its base version, and introduce a latent attribution technique in the crosscoder setting. We locate a small set of features relevant for promoting/suppressing wait tokens’ probabilities. Finally, through a targeted series of experiments analyzing max activating examples and causal interventions, we show that many of our identified features indeed are relevant for the reasoning process and give rise to different types of reasoning patterns such as restarting from the beginning, recalling prior knowledge, expressing uncertainty, and double-checking.

Method: Proposed MENTOR framework that provides expert guidance only at critical decision points rather than entire reasoning paths, enabling mixed-policy expert navigation for token-level optimization.

Result: Extensive experiments show MENTOR enables models to capture the essence of expert strategies rather than surface imitation, performing high-quality exploration and achieving superior overall performance.

Conclusion: MENTOR effectively addresses the exploration quality issue in RLVR by providing targeted expert guidance at critical points, leading to better reasoning capability without full trajectory imitation.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a widely adopted technique for enhancing the reasoning ability of Large Language Models (LLMs). However, the effectiveness of RLVR strongly depends on the capability of base models. This issue arises because it requires the model to have sufficient capability to perform high-quality exploration, which involves both effectiveness and diversity. Unfortunately, existing methods address this issue by imitating expert trajectories, which improve effectiveness but neglect diversity. To address this, we argue that the expert only needs to provide guidance only at critical decision points rather than the entire reasoning path. Based on this insight, we propose MENTOR: Mixed-policy Expert Navigation for Token-level Optimization of Reasoning, a framework that provides expert guidance only at critical decision points to perform effective and diverse exploration in RLVR. Extensive experiments show that MENTOR enables models capture the essence of expert strategies rather than surface imitation, thereby performing high-quality exploration and achieving superior overall performance. Our code is available online.

Method: Chronicles the evolution through three stages: (1) foundational knowledge tests like ImageNet, (2) applied logic and comprehension exams like GQA and VCR, and (3) expert-level integration benchmarks like MMBench and SEED-Bench for modern MLLMs.

Result: Identifies a clear paradigm shift from testing “what” models see to probing “why” and “how” they understand, with current benchmarks focusing on reasoning processes and future directions exploring abstract, creative, and social intelligence.

Conclusion: AI evaluation is a continuous adversarial process of designing better examinations that redefine goals for creating truly intelligent systems, not just a history of datasets.

Abstract: This survey paper chronicles the evolution of evaluation in multimodal artificial intelligence (AI), framing it as a progression of increasingly sophisticated “cognitive examinations.” We argue that the field is undergoing a paradigm shift, moving from simple recognition tasks that test “what” a model sees, to complex reasoning benchmarks that probe “why” and “how” it understands. This evolution is driven by the saturation of older benchmarks, where high performance often masks fundamental weaknesses. We chart the journey from the foundational “knowledge tests” of the ImageNet era to the “applied logic and comprehension” exams such as GQA and Visual Commonsense Reasoning (VCR), which were designed specifically to diagnose systemic flaws such as shortcut learning and failures in compositional generalization. We then survey the current frontier of “expert-level integration” benchmarks (e.g., MMBench, SEED-Bench, MMMU) designed for today’s powerful multimodal large language models (MLLMs), which increasingly evaluate the reasoning process itself. Finally, we explore the uncharted territories of evaluating abstract, creative, and social intelligence. We conclude that the narrative of AI evaluation is not merely a history of datasets, but a continuous, adversarial process of designing better examinations that, in turn, redefine our goals for creating truly intelligent systems.

Method: Agent Spec uses a declarative language approach that allows AI agents to be defined independently of their execution environment, serving as an interchange format between different AI frameworks and tools.

Result: Agent Spec benefits four key groups: developers gain reusable components, framework developers get an interchange format, researchers achieve reproducible results, and enterprises benefit from faster deployment and scalability.

Conclusion: Agent Spec provides technical foundations for improving AI agent interoperability, reusability, and portability across different frameworks, with ongoing future developments.

Abstract: Open Agent Specification (Agent Spec) is a declarative language that allows AI agents and their workflows to be defined in a way that is compatible across different AI frameworks, promoting portability and interoperability within AI Agent frameworks. Agent Spec aims to resolve the challenges of fragmented agent development by providing a common unified specification that allows AI agents to be designed once and deployed across various frameworks, improving interoperability and reusability, and reducing redundant development efforts. Additionally, Agent Spec facilitates development tools and portability, allowing AI agents to be defined independently of their execution environment and enabling teams to exchange solutions without implementation-specific limitations. Agent Spec benefits four key groups: (i) Agent developers, who gain access to a superset of reusable components and design patterns, enabling them to leverage a broader range of functionalities; (ii) Agent framework and tool developers, who can use Agent Spec as an interchange format and therefore benefit from the support of other frameworks as well as other tools; (iii) Researchers, who can achieve reproducible results and comparability, facilitating more reliable and consistent outcomes; (iv) Enterprises, which benefit from faster prototype-to-deployment, increased productivity, as well as greater scalability and maintainability for their AI agent solutions. This technical report provides an overview of the technical foundations of Agent Spec, including motivation, benefits, and future developments.

Method: Uses Version Control to track graph edit history and Edge Impact Score to prioritize minimal-cost repairs based on structural reachability, path usage, and conflict propagation.

Result: Significantly improves map correctness and robustness, especially for entangled or chained inconsistencies, using a refined MANGO benchmark dataset.

Conclusion: History-aware repair mechanisms are crucial for maintaining coherent spatial memory in LLM agents, enabling effective incremental map construction and repair.

Abstract: Given a map description through global traversal navigation instructions (e.g., visiting each room sequentially with action signals such as north, west, etc.), an LLM can often infer the implicit spatial layout of the environment and answer user queries by providing a shortest path from a start to a destination (for instance, navigating from the lobby to a meeting room via the hall and elevator). However, such context-dependent querying becomes incapable as the environment grows much longer, motivating the need for incremental map construction that builds a complete topological graph from stepwise observations. We propose a framework for LLM-driven construction and map repair, designed to detect, localize, and correct structural inconsistencies in incrementally constructed navigation graphs. Central to our method is the Version Control, which records the full history of graph edits and their source observations, enabling fine-grained rollback, conflict tracing, and repair evaluation. We further introduce an Edge Impact Score to prioritize minimal-cost repairs based on structural reachability, path usage, and conflict propagation. To properly evaluate our approach, we create a refined version of the MANGO benchmark dataset by systematically removing non-topological actions and inherent structural conflicts, providing a cleaner testbed for LLM-driven construction and map repair. Our approach significantly improves map correctness and robustness, especially in scenarios with entangled or chained inconsistencies. Our results highlight the importance of introspective, history-aware repair mechanisms for maintaining coherent spatial memory in LLM agents.

Method: COSMO-RL uses mixed reinforcement learning with multimodal, multitask, and multiobjective signals to train reasoning-oriented LMRMs, allowing safety and capability to grow together.

Result: COSMO-R1 improves safety while maintaining or improving multimodal reasoning and instruction following, shows stronger robustness to multimodal jailbreaks, and reduces unnecessary refusals.

Conclusion: The framework provides a simple path to advancing both safety and general capability together in Large Multimodal Reasoning Models, with consistent gains across different backbones.

Abstract: Large Multimodal Reasoning Models (LMRMs) are moving into real applications, where they must be both useful and safe. Safety is especially challenging in multimodal settings: images and text can be combined to bypass guardrails, and single objective training can cause policy drift that yields over-refusal on benign inputs or unsafe compliance on risky ones. We present COSMO-RL, a mixed reinforcement learning framework that trains reasoning oriented LMRMs under multimodal, multitask, and multiobjective signals, and we release the resulting model, COSMO-R1. Our approach aims to let safety and capability grow together in one stable pipeline rather than competing during alignment. In experiments, COSMO-R1 improves safety while maintaining-and often improving multimodal reasoning and instruction following, shows stronger robustness to multimodal jailbreaks, and reduces unnecessary refusals. The framework also transfers across backbones with consistent gains. Ablations support the design choices, indicating a simple path to advancing safety and general capability together in LMRMs.

Method: AgentRL uses a fully-asynchronous generation-training pipeline, unified function-call API, containerized environments, and centralized controller for infrastructure. Algorithmically, it employs cross-policy sampling for exploration and task advantage normalization for stability.

Result: AgentRL significantly outperforms GPT-5, Clause-Sonnet-4, DeepSeek-R1, and other open-source LLM agents across five agentic tasks. Multi-task training matches the best results among all task-specific models.

Conclusion: AgentRL provides an effective framework for scalable multi-turn, multi-task RL training of LLM agents, demonstrating superior performance and stability compared to existing approaches.

Abstract: Recent advances in large language models (LLMs) have sparked growing interest in building generalist agents that can learn through online interactions. However, applying reinforcement learning (RL) to train LLM agents in multi-turn, multi-task settings remains challenging due to lack of scalable infrastructure and stable training algorithms. In this work, we present the AgentRL framework for scalable multi-turn, multi-task agentic RL training. On the infrastructure side, AgentRL features a fully-asynchronous generation-training pipeline for efficient multi-turn RL. To support heterogeneous environment development in multi-task RL, we design a unified function-call based API interface, containerized environment development, and a centralized controller. On the algorithm side, we propose cross-policy sampling to encourage model exploration in multi-turn settings and task advantage normalization to stabilize multi-task training. Experiments show that AgentRL, trained on open LLMs across five agentic tasks, significantly outperforms GPT-5, Clause-Sonnet-4, DeepSeek-R1, and other open-source LLM agents. Multi-task training with AgentRL matches the best results among all task-specific models. AgentRL is open-sourced at https://github.com/THUDM/AgentRL. The algorithm and framework are adopted in building \textsc{\href{https://autoglm.zhipuai.cn}{AutoGLM}}.

Method: Bayesian evaluation with Dirichlet prior, modeling outcomes as categorical and providing posterior estimates of success probability with credible intervals.

Result: Achieves faster convergence and greater rank stability than Pass@k, enabling reliable comparisons with smaller sample counts.

Conclusion: Recommends replacing Pass@k with posterior-based protocol that unifies binary and non-binary evaluation with explicit uncertainty.

Abstract: Pass$@k$ is widely used to report performance for LLM reasoning, but it often yields unstable, misleading rankings, especially when the number of trials (samples) is limited and compute is constrained. We present a principled Bayesian evaluation framework that replaces Pass$@k$ and average accuracy over $N$ trials (avg$@N$) with posterior estimates of a model’s underlying success probability and credible intervals, yielding stable rankings and a transparent decision rule for differences. Evaluation outcomes are modeled as categorical (not just 0/1) with a Dirichlet prior, giving closed-form expressions for the posterior mean and uncertainty of any weighted rubric and enabling the use of prior evidence when appropriate. Theoretically, under a uniform prior, the Bayesian posterior mean is order-equivalent to average accuracy (Pass$@1$), explaining its empirical robustness while adding principled uncertainty. Empirically, in simulations with known ground-truth success rates and on AIME'24/‘25, HMMT'25, and BrUMO'25, the Bayesian/avg procedure achieves faster convergence and greater rank stability than Pass$@k$ and recent variants, enabling reliable comparisons at far smaller sample counts. The framework clarifies when observed gaps are statistically meaningful (non-overlapping credible intervals) versus noise, and it naturally extends to graded, rubric-based evaluations. Together, these results recommend replacing Pass$@k$ for LLM evaluation and ranking with a posterior-based, compute-efficient protocol that unifies binary and non-binary evaluation while making uncertainty explicit. Code is available at https://mohsenhariri.github.io/bayes-kit

Method: Developed a unified multi-agent RL framework with multi-timescale architecture, decomposing policies by departmental functions and assigning different learning speeds based on task complexity.

Result: The approach significantly improved profitability compared to siloed decision-making, with RL agent behaviors aligning with theoretical insights and demonstrating asymptotic convergence.

Conclusion: Provides a scalable, interpretable RL-based solution for effective cross-functional coordination in complex business environments.

Abstract: Effective cross-functional coordination is essential for enhancing firm-wide profitability, particularly in the face of growing organizational complexity and scale. Recent advances in artificial intelligence, especially in reinforcement learning (RL), offer promising avenues to address this fundamental challenge. This paper proposes a unified multi-agent RL framework tailored for joint optimization across distinct functional modules, exemplified via coordinating inventory replenishment and personalized product recommendation. We first develop an integrated theoretical model to capture the intricate interplay between these functions and derive analytical benchmarks that characterize optimal coordination. The analysis reveals synchronized adjustment patterns across products and over time, highlighting the importance of coordinated decision-making. Leveraging these insights, we design a novel multi-timescale multi-agent RL architecture that decomposes policy components according to departmental functions and assigns distinct learning speeds based on task complexity and responsiveness. Our model-free multi-agent design improves scalability and deployment flexibility, while multi-timescale updates enhance convergence stability and adaptability across heterogeneous decisions. We further establish the asymptotic convergence of the proposed algorithm. Extensive simulation experiments demonstrate that the proposed approach significantly improves profitability relative to siloed decision-making frameworks, while the behaviors of the trained RL agents align closely with the managerial insights from our theoretical model. Taken together, this work provides a scalable, interpretable RL-based solution to enable effective cross-functional coordination in complex business settings.

Method: GROK comprises three core modules: Knowledge-Guided Instruction Generation, CLIP-Style OCT-Biomarker Alignment, and Supervised Instruction Fine-Tuning, establishing a quantitative-to-qualitative diagnostic chain of thought that mirrors real clinical reasoning.

Result: With only LoRA fine-tuning of a 7B-parameter Qwen2 backbone, GROK outperforms comparable 7B and 32B baselines on both report quality and fine-grained clinical metrics, and even exceeds OpenAI o3.

Conclusion: GROK demonstrates superior performance in medical diagnosis tasks through its grounded multimodal approach and quantitative-to-qualitative reasoning chain, providing clinician-grade diagnoses with detailed lesion annotations.

Abstract: Multimodal large language models (MLLMs) hold promise for integrating diverse data modalities, but current medical adaptations such as LLaVA-Med often fail to fully exploit the synergy between color fundus photography (CFP) and optical coherence tomography (OCT), and offer limited interpretability of quantitative biomarkers. We introduce GROK, a grounded multimodal large language model that jointly processes CFP, OCT, and text to deliver clinician-grade diagnoses of ocular and systemic disease. GROK comprises three core modules: Knowledge-Guided Instruction Generation, CLIP-Style OCT-Biomarker Alignment, and Supervised Instruction Fine-Tuning, which together establish a quantitative-to-qualitative diagnostic chain of thought, mirroring real clinical reasoning when producing detailed lesion annotations. To evaluate our approach, we introduce the Grounded Ophthalmic Understanding benchmark, which covers six disease categories and three tasks: macro-level diagnostic classification, report generation quality, and fine-grained clinical assessment of the generated chain of thought. Experiments show that, with only LoRA (Low-Rank Adaptation) fine-tuning of a 7B-parameter Qwen2 backbone, GROK outperforms comparable 7B and 32B baselines on both report quality and fine-grained clinical metrics, and even exceeds OpenAI o3. Code and data are publicly available in the GROK repository.

Method: Multi-agent interactive environment with two-tiered reward architecture (separately optimizing decision-making and communication) and experience repository for policy learning.

Result: Surpasses state-of-the-art open-source specialized LLMs with higher parameter efficiency, outperforms proprietary models, and shows strong human preference in clinical dialogue generation.

Conclusion: The framework effectively enables AI doctors to master both medical decision-making and empathetic consultation skills, demonstrating practical clinical value.

Abstract: The professionalism of a human doctor in outpatient service depends on two core abilities: the ability to make accurate medical decisions and the medical consultation skill to conduct strategic, empathetic patient inquiry. Existing Large Language Models (LLMs) have achieved remarkable accuracy on medical decision-making benchmarks. However, they often lack the ability to conduct the strategic and empathetic consultation, which is essential for real-world clinical scenarios. To address this gap, we propose Doctor-R1, an AI doctor agent trained to master both of the capabilities by ask high-yield questions and conduct strategic multi-turn inquiry to guide decision-making. Our framework introduces three key components: a multi-agent interactive environment, a two-tiered reward architecture that separately optimizes clinical decision-making and communicative inquiry skills, and an experience repository to ground policy learning in high-quality prior trajectories. We evaluate Doctor-R1 on OpenAI’s HealthBench and MAQuE, assessed across multi-facet metrics, such as communication quality, user experience, and task accuracy. Remarkably, Doctor-R1 surpasses state-of-the-art open-source specialized LLMs by a substantial margin with higher parameter efficiency and outperforms powerful proprietary models. Furthermore, the human evaluations show a strong preference for Doctor-R1 to generate human-preferred clinical dialogue, demonstrating the effectiveness of the framework.

Method: Proposed a theoretical framework with depth (reasoning length) and width (capability diversity) dimensions, then empirically evaluated multi-agent debate systems on discriminative and generative tasks.

Result: LLM-MAS benefits increase with both task depth and width, with depth having a more pronounced effect on performance improvement.

Conclusion: The framework clarifies when LLM-MAS are beneficial and provides principled guidance for designing future multi-agent systems and benchmarks.

Abstract: Large language model multi-agent systems (LLM-MAS) offer a promising paradigm for harnessing collective intelligence to achieve more advanced forms of AI behaviour. While recent studies suggest that LLM-MAS can outperform LLM single-agent systems (LLM-SAS) on certain tasks, the lack of systematic experimental designs limits the strength and generality of these conclusions. We argue that a principled understanding of task complexity, such as the degree of sequential reasoning required and the breadth of capabilities involved, is essential for assessing the effectiveness of LLM-MAS in task solving. To this end, we propose a theoretical framework characterising tasks along two dimensions: depth, representing reasoning length, and width, representing capability diversity. We theoretically examine a representative class of LLM-MAS, namely the multi-agent debate system, and empirically evaluate its performance in both discriminative and generative tasks with varying depth and width. Theoretical and empirical results show that the benefit of LLM-MAS over LLM-SAS increases with both task depth and width, and the effect is more pronounced with respect to depth. This clarifies when LLM-MAS are beneficial and provides a principled foundation for designing future LLM-MAS methods and benchmarks.

Method: JEF Hinter uses a zooming mechanism to highlight decisive steps in long trajectories, distilling both successful and failed traces into compact hints. It supports parallelized hint generation and includes a retriever that selects relevant hints for the current state during inference.

Result: Experiments on MiniWoB++, WorkArena-L1, and WebArena-Lite show JEF Hinter consistently outperforms strong baselines, including human- and document-based hints.

Conclusion: JEF Hinter effectively leverages offline trajectories to provide targeted guidance for LLM agents, demonstrating improved performance across multiple benchmarks while offering transparency and traceability.

Abstract: Large language model (LLM) agents perform well in sequential decision-making tasks, but improving them on unfamiliar domains often requires costly online interactions or fine-tuning on large expert datasets. These strategies are impractical for closed-source models and expensive for open-source ones, with risks of catastrophic forgetting. Offline trajectories offer reusable knowledge, yet demonstration-based methods struggle because raw traces are long, noisy, and tied to specific tasks. We present Just-in-time Episodic Feedback Hinter (JEF Hinter), an agentic system that distills offline traces into compact, context-aware hints. A zooming mechanism highlights decisive steps in long trajectories, capturing both strategies and pitfalls. Unlike prior methods, JEF Hinter leverages both successful and failed trajectories, extracting guidance even when only failure data is available, while supporting parallelized hint generation and benchmark-independent prompting. At inference, a retriever selects relevant hints for the current state, providing targeted guidance with transparency and traceability. Experiments on MiniWoB++, WorkArena-L1, and WebArena-Lite show that JEF Hinter consistently outperforms strong baselines, including human- and document-based hints.

Method: Uses LLM-powered Gaussian Process as surrogate model, generates prompt candidates via LLM expansion of seed prompts, evaluates with UCB acquisition function, and iteratively refines prompts based on data subset.

Result: The proposed BO-LLM algorithm was evaluated on two datasets and showed advantages in improving classification performance.

Conclusion: Bayesian Optimization with LLM-powered GP is effective for prompt engineering in text classification tasks, achieving better accuracy while minimizing API calls through uncertainty-aware optimization.

Abstract: Bayesian Optimization (BO) has been widely used to efficiently optimize expensive black-box functions with limited evaluations. In this paper, we investigate the use of BO for prompt engineering to enhance text classification with Large Language Models (LLMs). We employ an LLM-powered Gaussian Process (GP) as the surrogate model to estimate the performance of different prompt candidates. These candidates are generated by an LLM through the expansion of a set of seed prompts and are subsequently evaluated using an Upper Confidence Bound (UCB) acquisition function in conjunction with the GP posterior. The optimization process iteratively refines the prompts based on a subset of the data, aiming to improve classification accuracy while reducing the number of API calls by leveraging the prediction uncertainty of the LLM-based GP. The proposed BO-LLM algorithm is evaluated on two datasets, and its advantages are discussed in detail in this paper.

Method: Used psychological network analysis to assess imagination vividness ratings from questionnaires, constructing imagination networks from human reports and comparing them with networks generated by large language models under different prompts and memory conditions.

Result: Human imagination networks showed correlations between centrality measures (expected influence, strength, closeness), while LLM networks lacked clustering and showed lower correlations between centrality measures across different conditions.

Conclusion: There is a lack of similarity between internal world models in humans and LLM agents, providing a novel method for comparing internally-generated representations and insights for developing human-like imagination in AI.

Abstract: What is the computational objective of imagination? While classical interpretations suggest imagination is useful for maximizing rewards, recent findings challenge this view. In this study, we propose that imagination serves to access an internal world model (IWM) and use psychological network analysis to explore IWMs in humans and large language models (LLMs). Specifically, we assessed imagination vividness ratings using two questionnaires and constructed imagination networks from these reports. Imagination networks from human groups showed correlations between different centrality measures, including expected influence, strength, and closeness. However, imagination networks from LLMs showed a lack of clustering and lower correlations between centrality measures under different prompts and conversational memory conditions. Together, these results indicate a lack of similarity between IWMs in human and LLM agents. Overall, our study offers a novel method for comparing internally-generated representations in humans and AI, providing insights for developing human-like imagination in artificial intelligence.

Method: The authors formalize self-modification using a five-axis decomposition with a decision layer, analyze structural conflicts theoretically, and validate with numerical experiments comparing destructive utility policies against proposed two-gate policies.

Result: The central finding reveals that utility-driven self-modifications can render learnable tasks unlearnable when model capacity grows without bounds, but learnability is preserved when the policy-reachable model family is uniformly capacity-bounded.

Conclusion: The paper establishes a single boundary criterion for safe self-modification and demonstrates that carefully designed policies (two-gate policies) can preserve learnability while allowing beneficial self-improvements.

Abstract: As systems trend toward superintelligence, a natural modeling premise is that agents can self-improve along every facet of their own design. We formalize this with a five-axis decomposition and a decision layer, separating incentives from learning behavior and analyzing axes in isolation. Our central result identifies and introduces a sharp utility–learning tension, the structural conflict in self-modifying systems whereby utility-driven changes that improve immediate or expected performance can also erode the statistical preconditions for reliable learning and generalization. Our findings show that distribution-free guarantees are preserved iff the policy-reachable model family is uniformly capacity-bounded; when capacity can grow without limit, utility-rational self-changes can render learnable tasks unlearnable. Under standard assumptions common in practice, these axes reduce to the same capacity criterion, yielding a single boundary for safe self-modification. Numerical experiments across several axes validate the theory by comparing destructive utility policies against our proposed two-gate policies that preserve learnability.

Method: DRPO decouples length-based learning signals for correct rollouts from incorrect ones, ensuring positive samples are normalized only within their own group. It uses an optimized positive data distribution with KL regularization and importance weighting.

Result: DRPO achieves 77% length reduction with only 1.1% performance loss on GSM8k dataset using a 1.5B model, significantly outperforming baselines that sacrifice 4.3% performance for 68% length reduction.

Conclusion: DRPO effectively addresses overthinking in reasoning models by decoupling reward signals, enabling substantial computational efficiency gains without significant performance degradation. The framework is generalizable to other preference rewards beyond length.

Abstract: Recent large reasoning models (LRMs) driven by reinforcement learning algorithms (e.g., GRPO) have achieved remarkable performance on challenging reasoning tasks. However, these models suffer from overthinking, generating unnecessarily long and redundant reasoning even for simple questions, which substantially increases computational cost and response latency. While existing methods incorporate length rewards to GRPO to promote concise reasoning, they incur significant performance degradation. We identify the root cause: when rewards for correct but long rollouts are penalized, GRPO’s group-relative advantage function can assign them negative advantages, actively discouraging valid reasoning. To overcome this, we propose Decoupled Reward Policy Optimization (DRPO), a novel framework that decouples the length-based learning signal of correct rollouts from incorrect ones. DRPO ensures that reward signals for correct rollouts are normalized solely within the positive group, shielding them from interference by negative samples. The DRPO’s objective is grounded in integrating an optimized positive data distribution, which maximizes length-based rewards under a KL regularization, into a discriminative objective. We derive a closed-form solution for this distribution, enabling efficient computation of the objective and its gradients using only on-policy data and importance weighting. Of independent interest, this formulation is general and can incorporate other preference rewards of positive data beyond length. Experiments on mathematical reasoning tasks demonstrate DRPO’s significant superiority over six efficient reasoning baselines. Notably, with a 1.5B model, our method achieves 77% length reduction with only 1.1% performance loss on simple questions like GSM8k dataset, while the follow-up baseline sacrifices 4.3% for 68% length reduction.

Method: Uses Walsh-Fourier transform to convert CSP constraints to compact multilinear polynomials, employs circuit-output probability for efficient evaluation and differentiation, and uses projected gradient optimization with theoretical guarantees.

Result: Empirical results show FourierCSP is scalable and competitive, significantly broadening the class of problems solvable by continuous local search techniques.

Conclusion: The FourierCSP framework successfully generalizes continuous local search to general CSPs, demonstrating competitive performance and expanding the applicability of CLS methods.

Abstract: Constraint satisfaction problems (CSPs) are fundamental in mathematics, physics, and theoretical computer science. While conflict-driven clause learning Boolean Satisfiability (SAT) solvers have achieved remarkable success and become the mainstream approach for Boolean satisfiability, recent advances show that modern continuous local search (CLS) solvers can achieve highly competitive results on certain classes of SAT problems. Motivated by these advances, we extend the CLS framework from Boolean SAT to general CSP with finite-domain variables and expressive constraints. We present FourierCSP, a continuous optimization framework that generalizes the Walsh-Fourier transform to CSP, allowing for transforming versatile constraints to compact multilinear polynomials, thereby avoiding the need for auxiliary variables and memory-intensive encodings. Our approach leverages efficient evaluation and differentiation of the objective via circuit-output probability and employs a projected gradient optimization method with theoretical guarantees. Empirical results on benchmark suites demonstrate that FourierCSP is scalable and competitive, significantly broadening the class of problems that can be efficiently solved by CLS techniques.

Method: MACI uses two independent dials: information dial (gates evidence by quality) and behavior dial (schedules contentiousness from exploration to consolidation). A moderator tracks disagreement, overlap, evidence quality, and argument quality, halting when gains plateau. It includes a budget-feasible scheduler and uses cross-family LLM judge (CRIT) as conservative soft weight and stop signal.

Result: Across clinical diagnosis and news-bias tasks, MACI improves accuracy and calibration while reducing tokens. It converts residual uncertainty into precision RAG plans that specify what to retrieve next. The system demonstrates order invariance and judge-swap stability when using high-capability judges.

Conclusion: MACI turns debate into a budget-aware, measurable, and provably terminating controller that efficiently manages multi-agent deliberation while maintaining theoretical guarantees.

Abstract: Multi-agent debate often wastes compute by using a fixed adversarial stance, aggregating without deliberation, or stopping on heuristics. We introduce MACI, an active controller with two independent dials that decouple information from behavior: an information dial that gates evidence by quality, and a behavior dial that schedules contentiousness from exploration to consolidation. A moderator tracks disagreement, overlap, evidence quality, and argument quality, and halts when gains plateau. We provide theory-lite guarantees for nonincreasing dispersion and provable termination, with a budget-feasible scheduler. Across clinical diagnosis and news-bias tasks, MACI improves accuracy and calibration while reducing tokens, and converts residual uncertainty into precision RAG plans that specify what to retrieve next. We use a cross-family LLM judge (CRIT) as a conservative soft weight and stop signal, validated for order invariance and judge-swap stability; stability depends on using high-capability judges. MACI turns debate into a budget-aware, measurable, and provably terminating controller.

Method: TraitBasis learns directions in activation space corresponding to user traits (impatience, incoherence, skepticism) that can be controlled, scaled, composed, and applied at inference time without fine-tuning. It extends τ-Bench to τ-Trait with controlled trait vectors.

Result: Average 2%-30% performance degradation on τ-Trait across frontier models, demonstrating current AI agents’ lack of robustness to user behavior variations.

Conclusion: TraitBasis provides a simple, data-efficient, compositional tool for robustness testing and opens the door to building more reliable AI agents for real-world human interactions. The method has been open-sourced across four domains.

Abstract: Despite rapid progress in building conversational AI agents, robustness is still largely untested. Small shifts in user behavior, such as being more impatient, incoherent, or skeptical, can cause sharp drops in agent performance, revealing how brittle current AI agents are. Today’s benchmarks fail to capture this fragility: agents may perform well under standard evaluations but degrade spectacularly in more realistic and varied settings. We address this robustness testing gap by introducing TraitBasis, a lightweight, model-agnostic method for systematically stress testing AI agents. TraitBasis learns directions in activation space corresponding to steerable user traits (e.g., impatience or incoherence), which can be controlled, scaled, composed, and applied at inference time without any fine-tuning or extra data. Using TraitBasis, we extend $\tau$-Bench to $\tau$-Trait, where user behaviors are altered via controlled trait vectors. We observe on average a 2%-30% performance degradation on $\tau$-Trait across frontier models, highlighting the lack of robustness of current AI agents to variations in user behavior. Together, these results highlight both the critical role of robustness testing and the promise of TraitBasis as a simple, data-efficient, and compositional tool. By powering simulation-driven stress tests and training loops, TraitBasis opens the door to building AI agents that remain reliable in the unpredictable dynamics of real-world human interactions. We have open-sourced $\tau$-Trai across four domains: airline, retail, telecom, and telehealth, so the community can systematically QA their agents under realistic, behaviorally diverse intents and trait scenarios: https://github.com/collinear-ai/tau-trait.

Method: Iteratively decomposes queries into visual subtasks and manipulates chart images through specialized actions like drawing annotations, cropping regions, and localizing axes using chart-specific vision tools.

Result: Achieves state-of-the-art accuracy on ChartBench and ChartX benchmarks, with up to 16.07% absolute gain overall and 17.31% on unannotated, numerically intensive queries.

Conclusion: ChartAgent demonstrates effective visually grounded reasoning for chart understanding using tool-augmented multimodal agents, working across diverse chart types and complexity levels.

Abstract: Recent multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts, those requiring precise visual interpretation rather than relying on textual shortcuts. To address this, we introduce ChartAgent, a novel agentic framework that explicitly performs visual reasoning directly within the chart’s spatial domain. Unlike textual chain-of-thought reasoning, ChartAgent iteratively decomposes queries into visual subtasks and actively manipulates and interacts with chart images through specialized actions such as drawing annotations, cropping regions (e.g., segmenting pie slices, isolating bars), and localizing axes, using a library of chart-specific vision tools to fulfill each subtask. This iterative reasoning process closely mirrors human cognitive strategies for chart comprehension. ChartAgent achieves state-of-the-art accuracy on the ChartBench and ChartX benchmarks, surpassing prior methods by up to 16.07% absolute gain overall and 17.31% on unannotated, numerically intensive queries. Furthermore, our analyses show that ChartAgent is (a) effective across diverse chart types, (b) achieve the highest scores across varying visual and reasoning complexity levels, and (c) serves as a plug-and-play framework that boosts performance across diverse underlying LLMs. Our work is among the first to demonstrate visually grounded reasoning for chart understanding using tool-augmented multimodal agents.

Method: Two-phase Graph-of-Thought process: recursively decomposing statements into dependency graphs, then constructing formalizations from grounded concepts, with AriaScorer for term-level verification using Mathlib definitions.

Result: Achieved 91.6% compilation success and 68.5% final accuracy on ProofNet, 44.0% vs 24.0% on FATE-X algebra problems, and 42.9% vs 0% on homological conjectures.

Conclusion: Aria significantly advances theorem formalization by emulating human reasoning and ensuring semantic correctness, outperforming existing methods across diverse mathematical domains.

Abstract: Accurate auto-formalization of theorem statements is essential for advancing automated discovery and verification of research-level mathematics, yet remains a major bottleneck for LLMs due to hallucinations, semantic mismatches, and their inability to synthesize new definitions. To tackle these issues, we present Aria (Agent for Retrieval and Iterative Autoformalization), a system for conjecture-level formalization in Lean that emulates human expert reasoning via a two-phase Graph-of-Thought process: recursively decomposing statements into a dependency graph and then constructing formalizations from grounded concepts. To ensure semantic correctness, we introduce AriaScorer, a checker that retrieves definitions from Mathlib for term-level grounding, enabling rigorous and reliable verification. We evaluate Aria on diverse benchmarks. On ProofNet, it achieves 91.6% compilation success rate and 68.5% final accuracy, surpassing previous methods. On FATE-X, a suite of challenging algebra problems from research literature, it outperforms the best baseline with 44.0% vs. 24.0% final accuracy. On a dataset of homological conjectures, Aria reaches 42.9% final accuracy while all other models score 0%.

Method: Built DriveMind dataset from nuPlan with plan-aligned Chain-of-Thought, trained VLM agents with SFT and GRPO, conducted information ablations and attention analysis to test causal relationships.

Result: Consistent causal disconnect found: removing ego/navigation priors causes large planning drops, while removing CoT causes minor changes. Planning primarily focuses on priors rather than reasoning.

Conclusion: Proposed Reasoning-Planning Decoupling Hypothesis - reasoning is an ancillary byproduct, not causal mediator. Introduced training-free diagnostic probe to measure reliance on priors.

Abstract: Vision-Language Model (VLM) driving agents promise explainable end-to-end autonomy by first producing natural-language reasoning and then predicting trajectory planning. However, whether planning is causally driven by this reasoning remains a critical but unverified assumption. To investigate this, we build DriveMind, a large-scale driving Visual Question Answering (VQA) corpus with plan-aligned Chain-of-Thought (CoT), automatically generated from nuPlan. Our data generation process converts sensors and annotations into structured inputs and, crucially, separates priors from to-be-reasoned signals, enabling clean information ablations. Using DriveMind, we train representative VLM agents with Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) and evaluate them with nuPlan’s metrics. Our results, unfortunately, indicate a consistent causal disconnect in reasoning-planning: removing ego/navigation priors causes large drops in planning scores, whereas removing CoT produces only minor changes. Attention analysis further shows that planning primarily focuses on priors rather than the CoT. Based on this evidence, we propose the Reasoning-Planning Decoupling Hypothesis, positing that the training-yielded reasoning is an ancillary byproduct rather than a causal mediator. To enable efficient diagnosis, we also introduce a novel, training-free probe that measures an agent’s reliance on priors by evaluating its planning robustness against minor input perturbations. In summary, we provide the community with a new dataset and a diagnostic tool to evaluate the causal fidelity of future models.

Method: Use LLM to translate natural language rules into formal Python code models with state transition, legal moves, and termination functions. Generate heuristic value functions and inference functions for MCTS planning.

Result: Outperformed or matched Gemini 2.5 Pro in 9 out of 10 games tested (5 perfect information, 5 imperfect information games, including 4 novel games).

Conclusion: The code-based world modeling approach provides verifiability, strategic depth, and generalization advantages over direct LLM policy generation for game playing.

Abstract: Large Language Models (LLMs) reasoning abilities are increasingly being applied to classical board and card games, but the dominant approach – involving prompting for direct move generation – has significant drawbacks. It relies on the model’s implicit fragile pattern-matching capabilities, leading to frequent illegal moves and strategically shallow play. Here we introduce an alternative approach: We use the LLM to translate natural language rules and game trajectories into a formal, executable world model represented as Python code. This generated model – comprising functions for state transition, legal move enumeration, and termination checks – serves as a verifiable simulation engine for high-performance planning algorithms like Monte Carlo tree search (MCTS). In addition, we prompt the LLM to generate heuristic value functions (to make MCTS more efficient), and inference functions (to estimate hidden states in imperfect information games). Our method offers three distinct advantages compared to directly using the LLM as a policy: (1) Verifiability: The generated CWM serves as a formal specification of the game’s rules, allowing planners to algorithmically enumerate valid actions and avoid illegal moves, contingent on the correctness of the synthesized model; (2) Strategic Depth: We combine LLM semantic understanding with the deep search power of classical planners; and (3) Generalization: We direct the LLM to focus on the meta-task of data-to-code translation, enabling it to adapt to new games more easily. We evaluate our agent on 10 different games, of which 4 are novel and created for this paper. 5 of the games are fully observed (perfect information), and 5 are partially observed (imperfect information). We find that our method outperforms or matches Gemini 2.5 Pro in 9 out of the 10 considered games.

Method: TRAJECT-Bench pairs high-fidelity, executable tools across practical domains with production-style API tasks, and synthesizes trajectories varying in breadth (parallel calls) and depth (interdependent chains).

Result: The benchmark reveals failure modes like similar tool confusion and parameter-blind selection, and shows scaling behavior with tool diversity and trajectory length, identifying bottlenecks in transitioning from short to mid-length trajectories.

Conclusion: TRAJECT-Bench provides actionable guidance for improving LLMs’ tool use by offering trajectory-level diagnostics and revealing critical failure patterns and scaling limitations.

Abstract: Large language model (LLM)-based agents increasingly rely on tool use to complete real-world tasks. While existing works evaluate the LLMs’ tool use capability, they largely focus on the final answers yet overlook the detailed tool usage trajectory, i.e., whether tools are selected, parameterized, and ordered correctly. We introduce TRAJECT-Bench, a trajectory-aware benchmark to comprehensively evaluate LLMs’ tool use capability through diverse tasks with fine-grained evaluation metrics. TRAJECT-Bench pairs high-fidelity, executable tools across practical domains with tasks grounded in production-style APIs, and synthesizes trajectories that vary in breadth (parallel calls) and depth (interdependent chains). Besides final accuracy, TRAJECT-Bench also reports trajectory-level diagnostics, including tool selection and argument correctness, and dependency/order satisfaction. Analyses reveal failure modes such as similar tool confusion and parameter-blind selection, and scaling behavior with tool diversity and trajectory length where the bottleneck of transiting from short to mid-length trajectories is revealed, offering actionable guidance for LLMs’ tool use.

Method: ContextNav integrates automated retrieval with agentic curation using a graph-based orchestration workflow. It builds a resource-aware multimodal embedding pipeline, maintains a retrievable vector database, and applies agentic retrieval with structural alignment. An Operational Grammar Graph enables adaptive workflow planning and optimization based on downstream ICL feedback.

Result: Experimental results show ContextNav achieves state-of-the-art performance across various datasets, demonstrating effective noise-robust contextualization for multimodal ICL.

Conclusion: ContextNav demonstrates the promise of agentic workflows for advancing scalable and robust contextualization in multimodal in-context learning, successfully addressing the trade-off between scalability and robustness.

Abstract: Recent advances demonstrate that multimodal large language models (MLLMs) exhibit strong multimodal in-context learning (ICL) capabilities, enabling them to adapt to novel vision-language tasks from a few contextual examples. However, existing ICL approaches face challenges in reconciling scalability with robustness across diverse tasks and noisy contextual examples: manually selecting examples produces clean contexts but is labor-intensive and task-specific, while similarity-based retrieval improves scalability but could introduce irrelevant or structurally inconsistent samples that degrade ICL performance. To address these limitations, we propose ContextNav, the first agentic framework that integrates the scalability of automated retrieval with the quality and adaptiveness of human-like curation, enabling noise-robust and dynamically optimized contextualization for multimodal ICL. ContextNav unifies context management and noise-robust contextualization within a closed-loop workflow driven by graph-based orchestration. Specifically, it builds a resource-aware multimodal embedding pipeline, maintains a retrievable vector database, and applies agentic retrieval and structural alignment to construct noise-resilient contexts. An Operational Grammar Graph (OGG) further supports adaptive workflow planning and optimization, enabling the agent to refine its operational strategies based on downstream ICL feedback. Experimental results demonstrate that ContextNav achieves state-of-the-art performance across various datasets, underscoring the promise of agentic workflows for advancing scalable and robust contextualization in multimodal ICL.

Method: COSMIR uses a structured memory approach with three agent types: Planner converts queries into checkable sub-questions, Worker agents process chunks using a fixed micro-cycle (Extract, Infer, Refine) and update shared memory, and Manager synthesizes final answers from the structured memory.

Result: On long-context QA from the HELMET suite, COSMIR reduces propagation-stage information loss and improves accuracy over Chain of Agents (CoA) baseline.

Conclusion: COSMIR preserves step-wise read-then-reason benefits while using structured memory and fixed worker procedures, yielding higher faithfulness, better long-range aggregation, and auditability for long-input reasoning tasks.

Abstract: Reasoning over very long inputs remains difficult for large language models (LLMs). Common workarounds either shrink the input via retrieval (risking missed evidence), enlarge the context window (straining selectivity), or stage multiple agents to read in pieces. In staged pipelines (e.g., Chain of Agents, CoA), free-form summaries passed between agents can discard crucial details and amplify early mistakes. We introduce COSMIR (Chain Orchestrated Structured Memory for Iterative Reasoning), a chain-style framework that replaces ad hoc messages with a structured memory. A Planner agent first turns a user query into concrete, checkable sub-questions. worker agents process chunks via a fixed micro-cycle: Extract, Infer, Refine, writing all updates to the shared memory. A Manager agent then Synthesizes the final answer directly from the memory. This preserves step-wise read-then-reason benefits while changing both the communication medium (structured memory) and the worker procedure (fixed micro-cycle), yielding higher faithfulness, better long-range aggregation, and auditability. On long-context QA from the HELMET suite, COSMIR reduces propagation-stage information loss and improves accuracy over a CoA baseline.

Method: Partition state space by the sum of tile numbers (called ‘age’), which remains invariant between states and afterstates, allowing systematic enumeration and value calculation by processing states in age order.

Result: Identified 1,152,817,492,752 reachable states and 739,648,886,170 afterstates. Optimal strategy yields expected score of 50724.26 for initial states with two ‘2’ tiles.

Conclusion: Successfully strongly solved the 2048-4x3 variant using age-based state space partitioning, providing insights into optimal play and state space complexity of 2048 variants.

Abstract: 2048 is a stochastic single-player game involving 16 cells on a 4 by 4 grid, where a player chooses a direction among up, down, left, and right to obtain a score by merging two tiles with the same number located in neighboring cells along the chosen direction. This paper presents that a variant 2048-4x3 12 cells on a 4 by 3 board, one row smaller than the original, has been strongly solved. In this variant, the expected score achieved by an optimal strategy is about $50724.26$ for the most common initial states: ones with two tiles of number 2. The numbers of reachable states and afterstates are identified to be $1,152,817,492,752$ and $739,648,886,170$, respectively. The key technique is to partition state space by the sum of tile numbers on a board, which we call the age of a state. An age is invariant between a state and its successive afterstate after any valid action and is increased two or four by stochastic response from the environment. Therefore, we can partition state space by ages and enumerate all (after)states of an age depending only on states with the recent ages. Similarly, we can identify (after)state values by going along with ages in decreasing order.

Method: Philosophical analysis of the ‘perfect mimic’ concept and its implications for mind-recognition practices, examining consistency in consciousness attribution based on empirical evidence.

Result: Identifies a fundamental dilemma: refusing consciousness to perfect mimics requires invoking inaccessible factors, risking epistemological solipsism or inconsistent reasoning.

Conclusion: Epistemic consistency demands ascribing the same status to empirically indistinguishable entities, forcing critical reflection on consciousness theories and ethical frameworks for AI.

Abstract: Rapid advances in artificial intelligence necessitate a re-examination of the epistemological foundations upon which we attribute consciousness. As AI systems increasingly mimic human behavior and interaction with high fidelity, the concept of a “perfect mimic”-an entity empirically indistinguishable from a human through observation and interaction-shifts from hypothetical to technologically plausible. This paper argues that such developments pose a fundamental challenge to the consistency of our mind-recognition practices. Consciousness attributions rely heavily, if not exclusively, on empirical evidence derived from behavior and interaction. If a perfect mimic provides evidence identical to that of humans, any refusal to grant it equivalent epistemic status must invoke inaccessible factors, such as qualia, substrate requirements, or origin. Selectively invoking such factors risks a debilitating dilemma: either we undermine the rational basis for attributing consciousness to others (epistemological solipsism), or we accept inconsistent reasoning. I contend that epistemic consistency demands we ascribe the same status to empirically indistinguishable entities, regardless of metaphysical assumptions. The perfect mimic thus acts as an epistemic mirror, forcing critical reflection on the assumptions underlying intersubjective recognition in light of advancing AI. This analysis carries significant implications for theories of consciousness and ethical frameworks concerning artificial agents.

Method: AdaR synthesizes logically equivalent queries by varying variable values, trains models with RLVR to penalize spurious logic while encouraging adaptive logic, and uses code execution and sanity checks for data quality.

Result: AdaR improves robustness and generalization, achieving substantial improvement in mathematical reasoning while maintaining high data efficiency.

Conclusion: Data synthesis and RLVR work together to enable adaptive reasoning in LLMs, with analyses providing design insights for critical factors and applicability to instruction-tuned LLMs.

Abstract: Mathematical reasoning is a primary indicator of large language models (LLMs) intelligence. However, existing LLMs exhibit failures of robustness and generalization. This paper attributes these deficiencies to spurious reasoning, i.e., producing answers from superficial features. To address this challenge, we propose the AdaR framework to enable adaptive reasoning, wherein models rely on problem-solving logic to produce answers. AdaR synthesizes logically equivalent queries by varying variable values, and trains models with RLVR on these data to penalize spurious logic while encouraging adaptive logic. To improve data quality, we extract the problem-solving logic from the original query and generate the corresponding answer by code execution, then apply a sanity check. Experimental results demonstrate that AdaR improves robustness and generalization, achieving substantial improvement in mathematical reasoning while maintaining high data efficiency. Analysis indicates that data synthesis and RLVR function in a coordinated manner to enable adaptive reasoning in LLMs. Subsequent analyses derive key design insights into the effect of critical factors and the applicability to instruct LLMs. Our project is available at https://github.com/LaiZhejian/AdaR

Method: Uses a Plan-Act-Observe (PAO) loop with specialized tools, grounded in established clinical protocols like the ABCDEF protocol for CXR analysis, to decompose report structuring into a transparent process.

Result: Achieved F1-score of 0.96 on concept categorization and received average expert rating of 4.52/5 from radiologists and clinicians, surpassing baseline LLM approaches.

Conclusion: MedPAO provides a verifiable, protocol-driven alternative to opaque monolithic models, demonstrating high reliability for clinical data structuring tasks.

Abstract: The deployment of Large Language Models (LLMs) for structuring clinical data is critically hindered by their tendency to hallucinate facts and their inability to follow domain-specific rules. To address this, we introduce MedPAO, a novel agentic framework that ensures accuracy and verifiable reasoning by grounding its operation in established clinical protocols such as the ABCDEF protocol for CXR analysis. MedPAO decomposes the report structuring task into a transparent process managed by a Plan-Act-Observe (PAO) loop and specialized tools. This protocol-driven method provides a verifiable alternative to opaque, monolithic models. The efficacy of our approach is demonstrated through rigorous evaluation: MedPAO achieves an F1-score of 0.96 on the critical sub-task of concept categorization. Notably, expert radiologists and clinicians rated the final structured outputs with an average score of 4.52 out of 5, indicating a level of reliability that surpasses baseline approaches relying solely on LLM-based foundation models. The code is available at: https://github.com/MiRL-IITM/medpao-agent

Method: Developed QuantAgents with four specialized agents (simulated trading analyst, risk control analyst, market news analyst, and manager) that collaborate through meetings and receive feedback on both real-world market performance and simulated trading predictive accuracy.

Result: The framework achieved excellent performance across all metrics, yielding an overall return of nearly 300% over three years of testing.

Conclusion: QuantAgents successfully addresses the limitations of current LLM-based financial agents by incorporating long-term prediction capabilities and collaborative multi-agent decision making through simulated trading environments.

Abstract: In this paper, our objective is to develop a multi-agent financial system that incorporates simulated trading, a technique extensively utilized by financial professionals. While current LLM-based agent models demonstrate competitive performance, they still exhibit significant deviations from real-world fund companies. A critical distinction lies in the agents’ reliance on ``post-reflection’’, particularly in response to adverse outcomes, but lack a distinctly human capability: long-term prediction of future trends. Therefore, we introduce QuantAgents, a multi-agent system integrating simulated trading, to comprehensively evaluate various investment strategies and market scenarios without assuming actual risks. Specifically, QuantAgents comprises four agents: a simulated trading analyst, a risk control analyst, a market news analyst, and a manager, who collaborate through several meetings. Moreover, our system incentivizes agents to receive feedback on two fronts: performance in real-world markets and predictive accuracy in simulated trading. Extensive experiments demonstrate that our framework excels across all metrics, yielding an overall return of nearly 300% over the three years (https://quantagents.github.io/).

Method: AFIRE standardizes time-aligned post-fusion tokens from different encoders, and MIND uses token-dependent Top-K sparse routing with subject prior for personalized expert selection while maintaining generality.

Result: Experiments show consistent improvements over baselines, enhanced cross-subject generalization, and interpretable expert patterns correlated with content types.

Conclusion: The framework provides a simple attachment point for new encoders and datasets, enabling robust plug-and-improve performance for naturalistic neuroimaging studies.

Abstract: Naturalistic fMRI encoding must handle multimodal inputs, shifting fusion styles, and pronounced inter-subject variability. We introduce AFIRE (Agnostic Framework for Multimodal fMRI Response Encoding), an agnostic interface that standardizes time-aligned post-fusion tokens from varied encoders, and MIND, a plug-and-play Mixture-of-Experts decoder with a subject-aware dynamic gating. Trained end-to-end for whole-brain prediction, AFIRE decouples the decoder from upstream fusion, while MIND combines token-dependent Top-K sparse routing with a subject prior to personalize expert usage without sacrificing generality. Experiments across multiple multimodal backbones and subjects show consistent improvements over strong baselines, enhanced cross-subject generalization, and interpretable expert patterns that correlate with content type. The framework offers a simple attachment point for new encoders and datasets, enabling robust, plug-and-improve performance for naturalistic neuroimaging studies.

Method: Formulate the problem as inverse dynamics - predicting user actions from consecutive screen states, develop a pipeline with task-aware video retrieval, and generate over 53k trajectories from web videos.

Result: W&L trajectories improve CUAs as both in-context demonstrations and supervised training data, consistently enhancing performance on OSWorld benchmark for general-purpose and state-of-the-art frameworks.

Conclusion: Web-scale human demonstration videos provide a practical and scalable foundation for advancing computer use agents towards real-world deployment.

Abstract: Computer use agents (CUAs) need to plan task workflows grounded in diverse, ever-changing applications and environments, but learning is hindered by the scarcity of large-scale, high-quality training data in the target application. Existing datasets are domain-specific, static, and costly to annotate, while current synthetic data generation methods often yield simplistic or misaligned task demonstrations. To address these limitations, we introduce Watch & Learn (W&L), a framework that converts human demonstration videos readily available on the Internet into executable UI trajectories at scale. Instead of directly generating trajectories or relying on ad hoc reasoning heuristics, we cast the problem as an inverse dynamics objective: predicting the user’s action from consecutive screen states. This formulation reduces manual engineering, is easier to learn, and generalizes more robustly across applications. Concretely, we develop an inverse dynamics labeling pipeline with task-aware video retrieval, generate over 53k high-quality trajectories from raw web videos, and demonstrate that these trajectories improve CUAs both as in-context demonstrations and as supervised training data. On the challenging OSWorld benchmark, UI trajectories extracted with W&L consistently enhance both general-purpose and state-of-the-art frameworks in-context, and deliver stronger gains for open-source models under supervised training. These results highlight web-scale human demonstration videos as a practical and scalable foundation for advancing CUAs towards real-world deployment.

Method: Two-stage training: Stage 1 trains agents with retrieval recall-based rewards to improve search effectiveness; Stage 2 uses outcome rewards to optimize final answer generation.

Result: Across seven QA benchmarks, DeSA consistently improves search behaviors with substantially higher search recall and answer accuracy than outcome-only baselines, outperforming single-stage approaches.

Conclusion: Explicitly decoupling search optimization from answer generation is necessary for effective search-augmented LLMs, as demonstrated by DeSA’s superior performance over simultaneous optimization approaches.

Abstract: Enabling large language models (LLMs) to utilize search tools offers a promising path to overcoming fundamental limitations such as knowledge cutoffs and hallucinations. Recent work has explored reinforcement learning (RL) for training search-augmented agents that interleave reasoning and retrieval before answering. These approaches usually rely on outcome-based rewards (e.g., exact match), implicitly assuming that optimizing for final answers will also yield effective intermediate search behaviors. Our analysis challenges this assumption: we uncover multiple systematic deficiencies in search that arise under outcome-only training and ultimately degrade final answer quality, including failure to invoke tools, invalid queries, and redundant searches. To address these shortcomings, we introduce DeSA (Decoupling Search-and-Answering), a simple two-stage training framework that explicitly separates search optimization from answer generation. In Stage 1, agents are trained to improve search effectiveness with retrieval recall-based rewards. In Stage 2, outcome rewards are employed to optimize final answer generation. Across seven QA benchmarks, DeSA-trained agents consistently improve search behaviors, delivering substantially higher search recall and answer accuracy than outcome-only baselines. Notably, DeSA outperforms single-stage training approaches that simultaneously optimize recall and outcome rewards, underscoring the necessity of explicitly decoupling the two objectives.

Method: Built BrokenMath benchmark from advanced 2025 competition problems perturbed with an LLM to produce false statements, refined through expert review, and evaluated using LLM-as-a-judge framework on state-of-the-art LLMs and agentic systems.

Result: Sycophancy is widespread, with GPT-5 producing sycophantic answers 29% of the time. Mitigation strategies including test-time interventions and supervised fine-tuning on curated sycophantic examples substantially reduce but do not eliminate sycophantic behavior.

Conclusion: Sycophantic behavior in LLMs for mathematical theorem proving is a significant issue that current mitigation strategies can reduce but not fully eliminate, highlighting the need for continued research in this area.

Abstract: Large language models (LLMs) have recently shown strong performance on mathematical benchmarks. At the same time, they are prone to hallucination and sycophancy, often providing convincing but flawed proofs for incorrect mathematical statements provided by users. This significantly limits the applicability of LLMs in theorem proving, as verification of these flawed proofs must be done manually by expert mathematicians. However, existing benchmarks that measure sycophancy in mathematics are limited: they focus solely on final-answer problems, rely on very simple and often contaminated datasets, and construct benchmark samples using synthetic modifications that create ill-posed questions rather than well-posed questions that are demonstrably false. To address these issues, we introduce BrokenMath, the first benchmark for evaluating sycophantic behavior in LLMs within the context of natural language theorem proving. BrokenMath is built from advanced 2025 competition problems, which are perturbed with an LLM to produce false statements and subsequently refined through expert review. Using an LLM-as-a-judge framework, we evaluate state-of-the-art LLMs and agentic systems and find that sycophancy is widespread, with the best model, GPT-5, producing sycophantic answers 29% of the time. We further investigate several mitigation strategies, including test-time interventions and supervised fine-tuning on curated sycophantic examples. These approaches substantially reduce, but do not eliminate, sycophantic behavior.

Method: Proposed LMM-based contract-theoretic model with LMM agents for UGC quality evaluation using prompt engineering. Developed improved Mixture of Experts-based Proximal Policy Optimization algorithm for optimal contract design in dynamic Web 3.0 environments.

Result: Simulation results show superiority of MoE-based PPO algorithm over benchmarks. Successfully deployed contract in Ethereum smart contract framework, validating scheme effectiveness.

Conclusion: LMM-Incentive effectively addresses Web 3.0 content quality issues through LMM-based contracts and quality evaluation, providing a practical solution for incentivizing high-quality UGC while mitigating information asymmetry problems.

Abstract: Web 3.0 represents the next generation of the Internet, which is widely recognized as a decentralized ecosystem that focuses on value expression and data ownership. By leveraging blockchain and artificial intelligence technologies, Web 3.0 offers unprecedented opportunities for users to create, own, and monetize their content, thereby enabling User-Generated Content (UGC) to an entirely new level. However, some self-interested users may exploit the limitations of content curation mechanisms and generate low-quality content with less effort, obtaining platform rewards under information asymmetry. Such behavior can undermine Web 3.0 performance. To this end, we propose \textit{LMM-Incentive}, a novel Large Multimodal Model (LMM)-based incentive mechanism for UGC in Web 3.0. Specifically, we propose an LMM-based contract-theoretic model to motivate users to generate high-quality UGC, thereby mitigating the adverse selection problem from information asymmetry. To alleviate potential moral hazards after contract selection, we leverage LMM agents to evaluate UGC quality, which is the primary component of the contract, utilizing prompt engineering techniques to improve the evaluation performance of LMM agents. Recognizing that traditional contract design methods cannot effectively adapt to the dynamic environment of Web 3.0, we develop an improved Mixture of Experts (MoE)-based Proximal Policy Optimization (PPO) algorithm for optimal contract design. Simulation results demonstrate the superiority of the proposed MoE-based PPO algorithm over representative benchmarks in the context of contract design. Finally, we deploy the designed contract within an Ethereum smart contract framework, further validating the effectiveness of the proposed scheme.

Method: Proposed HBG framework integrates TB and DB in a principled adaptive manner, with specialized inference strategy for depot-centric scenarios like CVRP while maintaining broad applicability to problems without depots like TSP.

Result: HBG integrated into AGFN and GFACS solvers shows consistent and significant improvements across both CVRP and TSP, demonstrating enhanced solution quality and generalization.

Conclusion: The hybrid approach combining TB and DB in HBG framework effectively addresses the limitations of individual methods and provides superior performance in vehicle routing problems.

Abstract: Existing GFlowNet-based methods for vehicle routing problems (VRPs) typically employ Trajectory Balance (TB) to achieve global optimization but often neglect important aspects of local optimization. While Detailed Balance (DB) addresses local optimization more effectively, it alone falls short in solving VRPs, which inherently require holistic trajectory optimization. To address these limitations, we introduce the Hybrid-Balance GFlowNet (HBG) framework, which uniquely integrates TB and DB in a principled and adaptive manner by aligning their intrinsically complementary strengths. Additionally, we propose a specialized inference strategy for depot-centric scenarios like the Capacitated Vehicle Routing Problem (CVRP), leveraging the depot node’s greater flexibility in selecting successors. Despite this specialization, HBG maintains broad applicability, extending effectively to problems without explicit depots, such as the Traveling Salesman Problem (TSP). We evaluate HBG by integrating it into two established GFlowNet-based solvers, i.e., AGFN and GFACS, and demonstrate consistent and significant improvements across both CVRP and TSP, underscoring the enhanced solution quality and generalization afforded by our approach.

Method: A labeller Λ emits natural-language labels from parent state and compact context; a tuner Ψ maps (P, L, C) → Π with strict schema validation and trust-region projection around safe defaults. Uses ToT-style selection with score S=μ+βσ and depth-annealed β.

Result: NLEL strictly generalizes CoT/ToT, proves anytime-monotonicity for top-k selection, and bounds selector shortfall by control-vector distortion. Preregistered evaluation anticipates accuracy gains at comparable token budgets and improved success@compute under constraints.

Conclusion: NLEL offers an interpretable, model-agnostic interface that separates intent from execution for controllable, auditable LM inference.

Abstract: Controllers for structured LM reasoning (e.g., Chain-of-Thought, self-consistency, and Tree-of-Thoughts) often entangle what to try next with how to execute it, exposing only coarse global knobs and yielding brittle, compute-inefficient, and hard-to-audit behavior. We introduce Natural Language Edge Labelling (NLEL), a labeller-tuner overlay that attaches a free-form natural-language directive to each search edge and translates it into a schema-bounded control vector for decoding, search (branch quotas, exploration $\beta$), generation bundle size, retrieval mixtures, and verification passes. A labeller $\Lambda$ emits labels from the parent state and a compact context; a tuner $\Psi$ maps $(P, L, C)\to \Pi$, with strict schema validation and trust-region projection around safe defaults. Downstream selection remains ToT-style with score $S=\mu+\beta\sigma$ and depth-annealed $\beta$. We show NLEL strictly generalizes CoT/ToT, prove an anytime-monotonicity property for top-$k$ selection under label-conditioned bundles, and bound selector shortfall by control-vector distortion, providing decision-relevant justification for guards like trust regions and verification passes. We instantiate $\Psi$ as a prompt-only JSON Parameter Emitter and preregister an evaluation on GSM8K, MATH (subset), StrategyQA, and ARC-Challenge with compute-aware reporting (success@compute, tokens-per-success) and ablations over $\Lambda$, $\Psi$, trust-region radius, and control quantization; preregistered forecasts anticipate accuracy gains at comparable token budgets and improved success@compute under constraints. NLEL offers an interpretable, model-agnostic interface that separates intent from execution for controllable, auditable LM inference.

Method: Curated Human Behavior Atlas - a unified benchmark with over 100,000 multimodal samples covering affective states, cognitive states, pathologies, and social processes. Trained three models: OmniSapiens-7B SFT, OmniSapiens-7B BAM, and OmniSapiens-7B RL.

Result: Models trained on Human Behavior Atlas consistently outperform existing multimodal LLMs across diverse behavioral tasks. Pretraining also improves transfer to novel behavioral datasets with meaningful performance gains.

Conclusion: Unified behavioral benchmarks like Human Behavior Atlas can reduce redundancy, enable efficient cross-task scaling, and enhance generalization of behavioral features across domains.

Abstract: Using intelligent systems to perceive psychological and social behaviors, that is, the underlying affective, cognitive, and pathological states that are manifested through observable behaviors and social interactions, remains a challenge due to their complex, multifaceted, and personalized nature. Existing work tackling these dimensions through specialized datasets and single-task systems often miss opportunities for scalability, cross-task transfer, and broader generalization. To address this gap, we curate Human Behavior Atlas, a unified benchmark of diverse behavioral tasks designed to support the development of unified models for understanding psychological and social behaviors. Human Behavior Atlas comprises over 100,000 samples spanning text, audio, and visual modalities, covering tasks on affective states, cognitive states, pathologies, and social processes. Our unification efforts can reduce redundancy and cost, enable training to scale efficiently across tasks, and enhance generalization of behavioral features across domains. On Human Behavior Atlas, we train three models: OmniSapiens-7B SFT, OmniSapiens-7B BAM, and OmniSapiens-7B RL. We show that training on Human Behavior Atlas enables models to consistently outperform existing multimodal LLMs across diverse behavioral tasks. Pretraining on Human Behavior Atlas also improves transfer to novel behavioral datasets; with the targeted use of behavioral descriptors yielding meaningful performance gains.

Method: Multi-Agent System for Deep ReSearch (MARS) that integrates external tools (Google Search, Google Scholar, Python Interpreter) and uses multi-agent reinforcement learning with Group Relative Policy Optimization to optimize both reasoning systems with tool interactions and bin-packing optimization.

Result: Achieves 3.86% improvement on Humanity’s Last Exam benchmark and average 8.9% gain across 7 knowledge-intensive tasks, demonstrating effective dual-system reasoning in dynamic environments.

Conclusion: MARS successfully bridges System 1 and System 2 reasoning in LLMs, enabling more efficient and adaptive complex reasoning through strategic tool integration and multi-agent optimization.

Abstract: Large Reasoning Models (LRMs) often exhibit a tendency for overanalysis in simple tasks, where the models excessively utilize System 2-type, deliberate reasoning, leading to inefficient token generation. Furthermore, these models face challenges in adapting their reasoning capabilities to rapidly changing environments due to the static nature of their pretraining data. To address these issues, advancing Large Language Models (LLMs) for complex reasoning tasks requires innovative approaches that bridge intuitive and deliberate cognitive processes, akin to human cognition’s dual-system dynamic. This paper introduces a Multi-Agent System for Deep ReSearch (MARS) enabling seamless integration of System 1’s fast, intuitive thinking with System 2’s deliberate reasoning within LLMs. MARS strategically integrates multiple external tools, such as Google Search, Google Scholar, and Python Interpreter, to access up-to-date information and execute complex computations, while creating a specialized division of labor where System 1 efficiently processes and summarizes high-volume external information, providing distilled insights that expand System 2’s reasoning context without overwhelming its capacity. Furthermore, we propose a multi-agent reinforcement learning framework extending Group Relative Policy Optimization to simultaneously optimize both systems with multi-turn tool interactions, bin-packing optimization, and sample balancing strategies that enhance collaborative efficiency. Extensive experiments demonstrate MARS achieves substantial improvements of 3.86% on the challenging Humanity’s Last Exam (HLE) benchmark and an average gain of 8.9% across 7 knowledge-intensive tasks, validating the effectiveness of our dual-system paradigm for complex reasoning in dynamic information environments.

Method: Uses a three-component architecture: high-level planner, RL execution agent trained with proximal policy optimization, and independent compliance agent with runtime action-shield. Formulates trade execution as constrained Markov decision process with hard constraints on participation limits, price bands, and self-trading avoidance.

Result: The learned policy reduces implementation shortfall and variance while exhibiting no observed constraint violations across stress scenarios including elevated latency, partial fills, compliance module toggling, and varying constraint limits. Results are statistically significant at 95% confidence level.

Conclusion: The work successfully combines optimal execution, safe reinforcement learning, regulatory technology, and verifiable AI, demonstrating a path to real-world deployment while addressing ethical considerations and computational limitations.

Abstract: We present a cross-market algorithmic trading system that balances execution quality with rigorous compliance enforcement. The architecture comprises a high-level planner, a reinforcement learning execution agent, and an independent compliance agent. We formulate trade execution as a constrained Markov decision process with hard constraints on participation limits, price bands, and self-trading avoidance. The execution agent is trained with proximal policy optimization, while a runtime action-shield projects any unsafe action into a feasible set. To support auditability without exposing proprietary signals, we add a zero-knowledge compliance audit layer that produces cryptographic proofs that all actions satisfied the constraints. We evaluate in a multi-venue, ABIDES-based simulator and compare against standard baselines (e.g., TWAP, VWAP). The learned policy reduces implementation shortfall and variance while exhibiting no observed constraint violations across stress scenarios including elevated latency, partial fills, compliance module toggling, and varying constraint limits. We report effects at the 95% confidence level using paired t-tests and examine tail risk via CVaR. We situate the work at the intersection of optimal execution, safe reinforcement learning, regulatory technology, and verifiable AI, and discuss ethical considerations, limitations (e.g., modeling assumptions and computational overhead), and paths to real-world deployment.

Method: The work systematically examines how physics-grounded methods enhance AI’s real-world comprehension across structured symbolic reasoning, embodied systems, and generative models through rigorous analysis of recent advances.

Result: The paper establishes clear distinctions between theoretical physics reasoning and applied physical understanding, advocating for intelligent systems that ground learning in both physical principles and embodied reasoning processes.

Conclusion: The synthesis envisions next-generation world models capable of explaining physical phenomena and predicting future states, advancing safe, generalizable, and interpretable AI systems that transcend pattern recognition toward genuine understanding of physical laws.

Abstract: The rapid advancement of embodied intelligence and world models has intensified efforts to integrate physical laws into AI systems, yet physical perception and symbolic physics reasoning have developed along separate trajectories without a unified bridging framework. This work provides a comprehensive overview of physical AI, establishing clear distinctions between theoretical physics reasoning and applied physical understanding while systematically examining how physics-grounded methods enhance AI’s real-world comprehension across structured symbolic reasoning, embodied systems, and generative models. Through rigorous analysis of recent advances, we advocate for intelligent systems that ground learning in both physical principles and embodied reasoning processes, transcending pattern recognition toward genuine understanding of physical laws. Our synthesis envisions next-generation world models capable of explaining physical phenomena and predicting future states, advancing safe, generalizable, and interpretable AI systems. We maintain a continuously updated resource at https://github.com/AI4Phys/Awesome-AI-for-Physics.

Method: Developed LLM-Hanabi benchmark using cooperative game Hanabi with automated evaluation system measuring both game performance and Theory-of-Mind proficiency across various LLMs.

Result: Found significant positive correlation between ToM and in-game success, with first-order ToM (interpreting others’ intent) correlating more strongly with performance than second-order ToM (predicting others’ interpretations).

Conclusion: For effective AI collaboration, accurately interpreting a partner’s rationale (first-order ToM) is more critical than higher-order reasoning, suggesting prioritizing first-order ToM is promising for enhancing future models’ collaborative capabilities.

Abstract: Effective multi-agent collaboration requires agents to infer the rationale behind others’ actions, a capability rooted in Theory-of-Mind (ToM). While recent Large Language Models (LLMs) excel at logical inference, their ability to infer rationale in dynamic, collaborative settings remains under-explored. This study introduces LLM-Hanabi, a novel benchmark that uses the cooperative game Hanabi to evaluate the rationale inference and ToM of LLMs. Our framework features an automated evaluation system that measures both game performance and ToM proficiency. Across a range of models, we find a significant positive correlation between ToM and in-game success. Notably, first-order ToM (interpreting others’ intent) correlates more strongly with performance than second-order ToM (predicting others’ interpretations). These findings highlight that for effective AI collaboration, the ability to accurately interpret a partner’s rationale is more critical than higher-order reasoning. We conclude that prioritizing first-order ToM is a promising direction for enhancing the collaborative capabilities of future models.

Method: TTE framework with a reasoner MLLM that generates reasoning traces followed by an embedder that produces representations conditioned on both original query and intermediate reasoning.

Result: Achieved state-of-the-art performance on MMEB-V2 benchmark, surpassing proprietary models. Also achieved 7% absolute gain over recent models with smaller finetuned MLLM reasoner.

Conclusion: Explicit reasoning step enables more nuanced understanding of complex multimodal instructions, and the framework can be integrated into unified models for efficiency without performance loss.

Abstract: There is a growing interest in Universal Multimodal Embeddings (UME), where models are required to generate task-specific representations. While recent studies show that Multimodal Large Language Models (MLLMs) perform well on such tasks, they treat MLLMs solely as encoders, overlooking their generative capacity. However, such an encoding paradigm becomes less effective as instructions become more complex and require compositional reasoning. Inspired by the proven effectiveness of chain-of-thought reasoning, we propose a general Think-Then-Embed (TTE) framework for UME, composed of a reasoner and an embedder. The reasoner MLLM first generates reasoning traces that explain complex queries, followed by an embedder that produces representations conditioned on both the original query and the intermediate reasoning. This explicit reasoning step enables more nuanced understanding of complex multimodal instructions. Our contributions are threefold. First, by leveraging a powerful MLLM reasoner, we achieve state-of-the-art performance on the MMEB-V2 benchmark, surpassing proprietary models trained on massive in-house datasets. Second, to reduce the dependency on large MLLM reasoners, we finetune a smaller MLLM reasoner using high-quality embedding-centric reasoning traces, achieving the best performance among open-source models with a 7% absolute gain over recently proposed models. Third, we investigate strategies for integrating the reasoner and embedder into a unified model for improved efficiency without sacrificing performance.

Method: LAMIR learns an abstracted model directly from agent-environment interaction. The learned abstraction limits subgame sizes to manageable scales, enabling theoretically principled look-ahead reasoning during test time.

Result: With sufficient capacity, LAMIR learns the exact underlying game structure. With limited capacity, it still learns valuable abstractions that improve game playing performance of pre-trained agents, even in large games where previous methods couldn’t scale.

Conclusion: LAMIR enables tractable look-ahead reasoning in imperfect information games by learning abstracted models, addressing scalability limitations of previous approaches and improving agent performance across different capacity settings.

Abstract: Test-time reasoning significantly enhances pre-trained AI agents’ performance. However, it requires an explicit environment model, often unavailable or overly complex in real-world scenarios. While MuZero enables effective model learning for search in perfect information games, extending this paradigm to imperfect information games presents substantial challenges due to more nuanced look-ahead reasoning techniques and large number of states relevant for individual decisions. This paper introduces an algorithm LAMIR that learns an abstracted model of an imperfect information game directly from the agent-environment interaction. During test time, this trained model is used to perform look-ahead reasoning. The learned abstraction limits the size of each subgame to a manageable size, making theoretically principled look-ahead reasoning tractable even in games where previous methods could not scale. We empirically demonstrate that with sufficient capacity, LAMIR learns the exact underlying game structure, and with limited capacity, it still learns a valuable abstraction, which improves game playing performance of the pre-trained agents even in large games.

Method: Propose staircase streaming that begins generating final response as soon as partial outputs are received from intermediate steps, rather than waiting for complete intermediate outputs.

Result: Experimental results show staircase streaming reduces TTFT by up to 93% while maintaining response quality.

Conclusion: Staircase streaming enables low-latency multi-agent inference without compromising response quality, making it suitable for latency-sensitive applications.

Abstract: Recent advances in large language models (LLMs) opened up new directions for leveraging the collective expertise of multiple LLMs. These methods, such as Mixture-of-Agents, typically employ additional inference steps to generate intermediate outputs, which are then used to produce the final response. While multi-agent inference can enhance response quality, it can significantly increase the time to first token (TTFT), posing a challenge for latency-sensitive applications and hurting user experience. To address this issue, we propose staircase streaming for low-latency multi-agent inference. Instead of waiting for the complete intermediate outputs from previous steps, we begin generating the final response as soon as we receive partial outputs from these steps. Experimental results demonstrate that staircase streaming reduces TTFT by up to 93% while maintaining response quality.

Method: Uses intelligent input chunking and processing strategies to efficiently handle extensive content while maintaining model performance within browser constraints.

Result: Demonstrates particular efficacy in processing large documents and datasets directly within Chrome, making sophisticated AI capabilities accessible through the browser without external API dependencies.

Conclusion: CAG successfully enables efficient handling of large inputs within Chrome’s Gemini Nano model constraints, providing browser-native AI capabilities without external dependencies.

Abstract: We present Chunked Augmented Generation (CAG), an architecture specifically designed to overcome the context window limitations of Google Chrome’s built-in Gemini Nano model. While Chrome’s integration of Gemini Nano represents a significant advancement in bringing AI capabilities directly to the browser, its restricted context window poses challenges for processing large inputs. CAG addresses this limitation through intelligent input chunking and processing strategies, enabling efficient handling of extensive content while maintaining the model’s performance within browser constraints. Our implementation demonstrates particular efficacy in processing large documents and datasets directly within Chrome, making sophisticated AI capabilities accessible through the browser without external API dependencies. Get started now at https://github.com/vivekVells/cag-js.

Method: Developed AgentBench with 8 distinct environments to test LLM-as-Agent capabilities across multiple dimensions.

Result: Top commercial LLMs show strong agent abilities, but there’s a significant performance gap with open-source competitors under 70B parameters. Main failure reasons include poor long-term reasoning, decision-making, and instruction following.

Conclusion: Improving instruction following and training on high-quality multi-round alignment data can enhance agent performance, while code training has ambivalent impacts on different agent tasks.

Abstract: The potential of Large Language Model (LLM) as agents has been widely acknowledged recently. Thus, there is an urgent need to quantitatively \textit{evaluate LLMs as agents} on challenging tasks in interactive environments. We present AgentBench, a multi-dimensional benchmark that consists of 8 distinct environments to assess LLM-as-Agent’s reasoning and decision-making abilities. Our extensive test over \num API-based and open-sourced (OSS) LLMs shows that, while top commercial LLMs present a strong ability of acting as agents in complex environments, there is a significant disparity in performance between them and many OSS competitors that are no larger than 70B. We identify the typical reasons of failures in environments and LLMs, showing that poor long-term reasoning, decision-making, and instruction following abilities are the main obstacles for developing usable LLM agents. Improving instruction following and training on high quality multi-round alignment data could improve agent performance. And different from existing assumptions, training on code present ambivalent impacts on different agent tasks. Datasets, environments, and an integrated evaluation package for AgentBench are released at https://github.com/THUDM/AgentBench.

Method: Proposes K-Link framework that extracts Knowledge-Link graphs from LLMs capturing universal sensor knowledge, and uses graph alignment to transfer this knowledge to data-generated graphs.

Result: Extensive experiments show K-Link achieves superior performance on various MTS tasks by enhancing graph quality and representation learning.

Conclusion: Leveraging LLM knowledge for MTS graph generation effectively reduces biases and improves dependency modeling, leading to better GNN performance.

Abstract: Sourced from multiple sensors and organized chronologically, Multivariate Time-Series (MTS) data involves crucial spatial-temporal dependencies. To capture these dependencies, Graph Neural Networks (GNNs) have emerged as powerful tools. As explicit graphs are not inherent to MTS data, graph generation becomes a critical first step in adapting GNNs to this domain. However, existing approaches often rely solely on the data itself for MTS graph generation, leaving them vulnerable to biases from small training datasets. This limitation hampers their ability to construct effective graphs, undermining the accurate modeling of underlying dependencies in MTS data and reducing GNN performance in this field. To address this challenge, we propose a novel framework, K-Link, leveraging the extensive universal knowledge encoded in Large Language Models (LLMs) to reduce biases for powered MTS graph generation. To harness the knowledge within LLMs, such as physical principles, we design and extract a \textit{Knowledge-Link graph} that captures universal knowledge of sensors and their linkage. To empower MTS graph generation with the knowledge-link graph, we further introduce a graph alignment module that transfers universal knowledge from the knowledge-link graph to the graph generated from MTS data. This enhances the MTS graph quality, ensuring effective representation learning for MTS data. Extensive experiments demonstrate the efficacy of K-Link for superior performance on various MTS tasks.

Method: Proposed CHARME framework with three components: Graph Neural Network for policy modeling, state transition algorithm ensuring solution validity, and order exploration strategy for effective training.

Result: CHARME yields superior solutions in qubit usage compared to Minorminer and ATOM, surpasses OCT-based approach in several cases, and the exploration strategy enhances training efficiency over greedy approaches.

Conclusion: The RL-based CHARME framework provides an effective solution to the minor embedding problem, outperforming existing methods and demonstrating the potential of reinforcement learning for quantum computing optimization problems.

Abstract: Quantum annealing (QA) has great potential to solve combinatorial optimization problems efficiently. However, the effectiveness of QA algorithms is heavily based on the embedding of problem instances, represented as logical graphs, into the quantum processing unit (QPU) whose topology is in the form of a limited connectivity graph, known as the minor embedding problem. Because the minor embedding problem is an NP-hard problem~\mbox{\cite{Goodrich2018}}, existing methods for the minor embedding problem suffer from scalability issues when faced with larger problem sizes. In this paper, we propose a novel approach utilizing Reinforcement Learning (RL) techniques to address the minor embedding problem, named CHARME. CHARME includes three key components: a Graph Neural Network (GNN) architecture for policy modeling, a state transition algorithm that ensures solution validity, and an order exploration strategy for effective training. Through comprehensive experiments on synthetic and real-world instances, we demonstrate the efficiency of our proposed order exploration strategy as well as our proposed RL framework, CHARME. In particular, CHARME yields superior solutions in terms of qubit usage compared to fast embedding methods such as Minorminer and ATOM. Moreover, our method surpasses the OCT-based approach, known for its slower runtime but high-quality solutions, in several cases. In addition, our proposed exploration enhances the efficiency of the training of the CHARME framework by providing better solutions compared to the greedy strategy.

Method: Created a program synthesis benchmark from XLogoOnline visual programming environment tasks; developed fine-tuning pipeline using large-scale synthetic training dataset (80,000+ tasks); implemented emulator-driven feedback for curriculum learning over training data distribution.

Result: GPT-4V and Llama3-70B achieved only 20% and 2.35% success rates respectively; fine-tuned Llama3-8B drastically outperformed both larger models after curriculum-based training.

Conclusion: Current SOTA models struggle with tasks requiring combined skills, but targeted fine-tuning with synthetic data and curriculum learning can significantly boost performance, highlighting the need for specialized training approaches for multi-skill tasks.

Abstract: Large language and multimodal models have shown remarkable success on various benchmarks focused on specific skills such as general-purpose programming, math word problem-solving, and visual question answering. However, it is unclear how well these models perform on tasks that require a combination of these skills. In this paper, we curate a novel program synthesis benchmark based on the real-world tasks in the XLogoOnline visual programming environment. Each task requires a combination of different skills such as spatial planning, basic programming, and logical reasoning. Our evaluation shows that current state-of-the-art models like GPT-4V and Llama3-70B struggle to solve these tasks, achieving only 20% and 2.35% success rates, respectively. Next, we develop a fine-tuning pipeline to boost the performance of models by leveraging a large-scale synthetic training dataset with over 80,000 tasks. Moreover, we showcase how emulator-driven feedback can be used to design a curriculum over training data distribution, through which a fine-tuned Llama3-8B drastically outperforms GPT-4V and Llama3-70B models. Finally, we provide an in-depth failure analysis to understand the limitations of different models. We will publicly release the benchmark for future research on program synthesis in visual programming.

Method: Model wave scheduling as Markov decision process and develop adaptive hybrid tree search algorithm that combines Monte Carlo tree search with domain-specific knowledge to reduce action space complexity.

Result: Extensive simulations with realistic parameters show the proposed approach significantly outperforms existing industry standard methods in seed order fulfillment.

Conclusion: The hybrid tree search approach effectively handles large state and action spaces in dynamic seed distribution environments, enabling forecast-informed scheduling that balances immediate requirements with long-term efficiency.

Abstract: Efficient order fulfillment is vital in the agricultural industry, particularly due to the seasonal nature of seed supply chains. This paper addresses the challenge of optimizing seed orders fulfillment in a centralized warehouse where orders are processed in waves, taking into account the unpredictable arrival of seed stocks and strict order deadlines. We model the wave scheduling problem as a Markov decision process and propose an adaptive hybrid tree search algorithm that combines Monte Carlo tree search with domain-specific knowledge to efficiently navigate the complex, dynamic environment of seed distribution. By leveraging historical data and stochastic modeling, our method enables forecast-informed scheduling decisions that balance immediate requirements with long-term operational efficiency. The key idea is that we can augment Monte Carlo tree search algorithm with problem-specific side information that dynamically reduces the number of candidate actions at each decision step to handle the large state and action spaces that render traditional solution methods computationally intractable. Extensive simulations with realistic parameters, including a diverse range of products, a high volume of orders, and authentic seasonal durations, demonstrate that the proposed approach significantly outperforms existing industry standard methods.

Method: Introduces precision and recall for logical rules, defines rule utility as their composition, and develops SPECTRUM with linear-time algorithms for mining recurrent subgraphs and efficiently ranking rules using the utility measure.

Result: SPECTRUM scales to larger datasets, learning more accurate logical theories on CPUs in <1% the runtime of state-of-the-art neural network approaches on GPUs across various tasks.

Conclusion: The framework provides theoretical guarantees on utility and demonstrates practical scalability and efficiency for learning logical theories from relational data.

Abstract: Probabilistic logical models are a core component of neurosymbolic AI and are important in their own right for tasks that require high explainability. Unlike neural networks, logical theories that underlie the model are often handcrafted using domain expertise, making their development costly and prone to errors. While there are algorithms that learn logical theories from data, they are generally prohibitively expensive, limiting their applicability in real-world settings. Here, we introduce precision and recall for logical rules and define their composition as rule utility - a cost-effective measure of the predictive power of logical theories. We also introduce SPECTRUM, a scalable framework for learning logical theories from relational data. Its scalability derives from a linear-time algorithm for mining recurrent subgraphs in the data graph along with a second algorithm that, using a utility measure that can be computed in linear time, efficiently ranks rules derived from these subgraphs. Finally, we prove theoretical guarantees on the utility of the learnt logical theory. As a result, we demonstrate across various tasks that SPECTRUM scales to larger datasets, often learning more accurate logical theories on CPUs in < 1% the runtime of SOTA neural network approaches on GPUs.

Method: Frames detection as a multi-agent debate where multimodal agents collaborate to assess contextual consistency and retrieve external information for cross-context reasoning. The framework is domain- and time-agnostic, requiring no finetuning.

Result: Outperforms prior methods by 2% on NewsCLIPpings, 3% on VERITE, and 5% on MMFakeBench. Ablation and user studies show debate and explanations significantly improve detection performance and trust for both experts and non-experts.

Conclusion: MAD-Sherlock positions as a robust tool for autonomous citizen intelligence by providing state-of-the-art accuracy with in-depth explanations, improving both detection performance and user trust.

Abstract: One of the most challenging forms of misinformation involves pairing images with misleading text to create false narratives. Existing AI-driven detection systems often require domain-specific finetuning, limiting generalizability, and offer little insight into their decisions, hindering trust and adoption. We introduce MAD-Sherlock, a multi-agent debate system for out-of-context misinformation detection. MAD-Sherlock frames detection as a multi-agent debate, reflecting the diverse and conflicting discourse found online. Multimodal agents collaborate to assess contextual consistency and retrieve external information to support cross-context reasoning. Our framework is domain- and time-agnostic, requiring no finetuning, yet achieves state-of-the-art accuracy with in-depth explanations. Evaluated on NewsCLIPpings, VERITE, and MMFakeBench, it outperforms prior methods by 2%, 3%, and 5%, respectively. Ablation and user studies show that the debate and resultant explanations significantly improve detection performance and improve trust for both experts and non-experts, positioning MAD-Sherlock as a robust tool for autonomous citizen intelligence.

Method: Tested children, adults, and LLMs on letter-string analogies in Latin alphabet, near transfer (Greek alphabet), and far transfer (symbol lists) domains.

Result: Children and adults easily generalized analogy solving to unfamiliar domains, while LLMs failed to transfer their analogy solving capabilities to new domains.

Conclusion: LLMs still lack robust human-like analogical transfer capabilities, showing a key difference between human and AI performance in domain generalization.

Abstract: In people, the ability to solve analogies such as “body : feet :: table : ?” emerges in childhood, and appears to transfer easily to other domains, such as the visual domain “( : ) :: < : ?”. Recent research shows that large language models (LLMs) can solve various forms of analogies. However, can LLMs generalize analogy solving to new domains like people can? To investigate this, we had children, adults, and LLMs solve a series of letter-string analogies (e.g., a b : a c :: j k : ?) in the Latin alphabet, in a near transfer domain (Greek alphabet), and a far transfer domain (list of symbols). Children and adults easily generalized their knowledge to unfamiliar domains, whereas LLMs did not. This key difference between human and AI performance is evidence that these LLMs still struggle with robust human-like analogical transfer.

Method: Analyzes traditional techniques (partitional and hierarchical clustering) alongside advanced approaches (data stream, density-based, graph-based, and model-based clustering) for handling complex structured datasets.

Result: The paper highlights key clustering principles, outlines widely used tools and frameworks, introduces clustering workflow, discusses implementation challenges, and examines various applications.

Conclusion: Clustering has transformative potential in data science, with future research directions focusing on driving innovation and enabling data-driven decision-making.

Abstract: This paper explores the critical role of data clustering in data science, emphasizing its methodologies, tools, and diverse applications. Traditional techniques, such as partitional and hierarchical clustering, are analyzed alongside advanced approaches such as data stream, density-based, graph-based, and model-based clustering for handling complex structured datasets. The paper highlights key principles underpinning clustering, outlines widely used tools and frameworks, introduces the workflow of clustering in data science, discusses challenges in practical implementation, and examines various applications of clustering. By focusing on these foundations and applications, the discussion underscores clustering’s transformative potential. The paper concludes with insights into future research directions, emphasizing clustering’s role in driving innovation and enabling data-driven decision-making.

Method: An iterative search framework where solutions are deconstructed by a neural policy, then rebuilt through collaboration between the neural policy and a simple greedy insertion algorithm.

Result: The approach matches or surpasses the performance of state-of-the-art operations research methods across three challenging vehicle routing problems of various problem sizes.

Conclusion: The proposed iterative deconstruction and reconstruction framework provides a viable alternative to sequential solution construction, achieving competitive performance with traditional operations research techniques.

Abstract: Autoregressive construction approaches generate solutions to vehicle routing problems in a step-by-step fashion, leading to high-quality solutions that are nearing the performance achieved by handcrafted operations research techniques. In this work, we challenge the conventional paradigm of sequential solution construction and introduce an iterative search framework where solutions are instead deconstructed by a neural policy. Throughout the search, the neural policy collaborates with a simple greedy insertion algorithm to rebuild the deconstructed solutions. Our approach matches or surpasses the performance of state-of-the-art operations research methods across three challenging vehicle routing problems of various problem sizes.

Method: Benchmarked multiple decoding strategies across diverse tasks (Translation, Math Problem Solving, Coding, Open-ended generation) with various configurations to measure their impact on both text quality and energy consumption.

Result: Decoding strategy choice significantly impacts GPU energy usage even with minimal effect on output quality. Different strategies involve quality-energy trade-offs, and no single method performs best across all metrics.

Conclusion: This pioneering study provides insights for building energy-efficient LLM applications without compromising text quality by carefully selecting decoding strategies based on specific use cases and requirements.

Abstract: Decoding strategies significantly influence the quality and diversity of the generated text in Large Language Models (LLMs), yet their impact on computational resources, particularly GPU energy consumption, is insufficiently studied. This paper investigates the relationship between text generation decoding techniques and energy efficiency, focusing on the trade-off between generation quality and GPU energy usage across diverse tasks and decoding configurations. By benchmarking multiple strategies across various tasks, including Translation, Math Problem Solving, Coding, and Open-ended text generation, we reveal how selecting appropriate decoding techniques with their tuned hyperparameters affects text quality and has measurable implications for energy consumption. Our findings show that the choice of decoding strategy can greatly impact GPU energy usage, even when it has a minimal effect on output quality. Different strategies also involve trade-offs between quality and energy efficiency, and no single decoding method is best in all cases across every metric. To the best of our knowledge, this is one of the first studies to examine decoding strategies in LLMs from the perspective of energy consumption, providing useful insights for building energy-efficient applications without compromising text generation quality.

Method: Using the Blicket Test paradigm from developmental psychology to test LMs’ causal inference abilities across different model families, sizes, and prompting strategies, plus proposing a test-time sampling method that explicitly samples and eliminates hypotheses.

Result: LMs reliably infer disjunctive causal relationships but systematically struggle with conjunctive ones, showing a persistent “disjunctive bias” that worsens with task complexity. This bias mirrors human adult reasoning patterns.

Conclusion: LMs inherit deep-seated reasoning heuristics from training data, exhibiting adult-like inference profiles. The proposed test-time sampling method significantly reduces the disjunctive bias, moving LMs toward more scientifically rigorous causal reasoning.

Abstract: Language model (LM) agents are increasingly used as autonomous decision-makers which need to actively gather information to guide their decisions. A crucial cognitive skill for such agents is the efficient exploration and understanding of the causal structure of the world – key to robust, scientifically grounded reasoning. Yet, it remains unclear whether LMs possess this capability or exhibit systematic biases leading to erroneous conclusions. In this work, we examine LMs’ ability to explore and infer causal relationships, using the well-established Blicket Test paradigm from developmental psychology. We find that LMs reliably infer the common, intuitive disjunctive causal relationships but systematically struggle with the unusual, yet equally (or sometimes even more) evidenced conjunctive ones. This “disjunctive bias” persists across model families, sizes, and prompting strategies, and performance further declines as task complexity increases. Interestingly, an analogous bias appears in human adults, suggesting that LMs may have inherited deep-seated reasoning heuristics from their training data. To this end, we quantify similarities between LMs and humans, finding that LMs exhibit adult-like inference profiles (but not child-like). Finally, we propose a test-time sampling method which explicitly samples and eliminates hypotheses about causal relationships from the LM. This scalable approach significantly reduces the disjunctive bias and moves LMs closer to the goal of scientific, causally rigorous reasoning.

Method: Proposed TIME benchmark with 38,522 QA pairs across 3 levels and 11 sub-tasks, covering three sub-datasets (TIME-Wiki, TIME-News, TIME-Dial) reflecting different real-world challenges.

Result: Extensive experiments conducted on reasoning and non-reasoning models, with in-depth analysis of temporal reasoning performance across diverse scenarios and tasks, plus study of test-time scaling impact.

Conclusion: The benchmark and TIME-Lite subset are released to foster future research and standardized evaluation in temporal reasoning, with code and datasets publicly available.

Abstract: Temporal reasoning is pivotal for Large Language Models (LLMs) to comprehend the real world. However, existing works neglect the real-world challenges for temporal reasoning: (1) intensive temporal information, (2) fast-changing event dynamics, and (3) complex temporal dependencies in social interactions. To bridge this gap, we propose a multi-level benchmark TIME, designed for temporal reasoning in real-world scenarios. TIME consists of 38,522 QA pairs, covering 3 levels with 11 fine-grained sub-tasks. This benchmark encompasses 3 sub-datasets reflecting different real-world challenges: TIME-Wiki, TIME-News, and TIME-Dial. We conduct extensive experiments on reasoning models and non-reasoning models. And we conducted an in-depth analysis of temporal reasoning performance across diverse real-world scenarios and tasks, and summarized the impact of test-time scaling on temporal reasoning capabilities. Additionally, we release TIME-Lite, a human-annotated subset to foster future research and standardized evaluation in temporal reasoning. The code is available at https://github.com/sylvain-wei/TIME , the dataset is available at https://huggingface.co/datasets/SylvainWei/TIME , and the project page link is https://sylvain-wei.github.io/TIME/ .

Method: Proposed RECAST framework extracts constraints from real-world prompt-response pairs to synthesize datasets with many constraints, and uses rule-based and LLM-based validators for automatic verification.

Result: Models finetuned on RECAST-30K substantially improve in following complex instructions while maintaining general capabilities, and the verifiability enables reward functions for RL that further boost performance.

Conclusion: RECAST effectively addresses LLMs’ limitations with complex multi-constraint instructions and provides a scalable framework for improving instruction-following capabilities in challenging real-world scenarios.

Abstract: Large language models (LLMs) are increasingly expected to tackle complex tasks, driven by their expanding applications and users’ growing proficiency in crafting sophisticated prompts. However, as the number of explicitly stated requirements increases (particularly more than 10 constraints), LLMs often struggle to accurately follow such complex instructions, which limits their applicability in complex real-world scenarios. To the best of our knowledge, existing datasets do not exceed 10 constraints per instance. To address this challenge, we propose RECAST, an efficient and scalable framework for synthesizing datasets where each example incorporates far more constraints than those in existing benchmarks, aiming to challenge and extend the boundaries of models’ ability to follow complex instructions. These constraints are extracted from real-world prompt-response pairs to ensure practical relevance. Using this framework, we construct RECAST-30K, a large-scale, high-quality dataset comprising 30k instances spanning 19 constraint types. Experimental results demonstrate that models finetuned on RECAST-30K substantially improve in following complex instructions while maintaining their general capabilities without degradation. Moreover, RECAST enables automatic verification of constraint satisfaction via rule-based validators for quantitative constraints and LLM-based validators for qualitative ones; the verifiability provided by RECAST enables the design of reward functions for reinforcement learning, which further boosts model performance on complex and challenging tasks.

Method: PatentMind decomposes patents into three dimensions (technical features, application domains, claim scopes), calculates dimension-specific similarity scores over MARG, and dynamically weights them through context-aware reasoning to emulate expert judgment.

Result: PatentMind achieves strong correlation (r=0.938) with expert annotations on PatentSimBench (500 patent pairs), significantly outperforming embedding-based models, patent-specific models, and advanced prompt engineering methods.

Conclusion: The framework provides a structured, semantically grounded foundation for real-world patent decision-making, particularly for infringement risk assessment, demonstrating broader impact on patent analytics and evaluation.

Abstract: Patent similarity evaluation plays a critical role in intellectual property analysis. However, existing methods often overlook the intricate structure of patent documents, which integrate technical specifications, legal boundaries, and application contexts. We introduce PatentMind, a novel framework for patent similarity assessment based on a Multi-Aspect Reasoning Graph (MARG). PatentMind decomposes patents into their three dimensions of technical features, application domains, and claim scopes, then dimension-specific similarity scores are calculated over the MARG. These scores are dynamically weighted through a context-aware reasoning process, which integrates contextual signals to emulate expert-level judgment. To support evaluation, we construct a human-annotated benchmark PatentSimBench, comprising 500 patent pairs. Experimental results demonstrate that the PatentMind-generated scores show a strong correlation ($r=0.938$) with expert annotations, significantly outperforming embedding-based models, patent-specific models, and advanced prompt engineering methods. Beyond computational linguistics, our framework provides a structured and semantically grounded foundation for real-world decision-making, particularly for tasks such as infringement risk assessment, underscoring its broader impact on both patent analytics and evaluation.

Method: First prompted LLMs to reason explicitly about contextual integrity, then developed a reinforcement learning framework to instill reasoning capabilities. Used a synthetic dataset of ~700 examples with diverse contexts and information disclosure norms.

Result: Substantially reduced inappropriate information disclosure while maintaining task performance across multiple model sizes and families. Improvements transferred from synthetic dataset to established CI benchmarks like PrivacyLens with human annotations.

Conclusion: The approach successfully enables agents to reason about contextual integrity, reducing privacy violations while preserving functionality, with transferable improvements to real-world benchmarks.

Abstract: As the era of autonomous agents making decisions on behalf of users unfolds, ensuring contextual integrity (CI) – what is the appropriate information to share while carrying out a certain task – becomes a central question to the field. We posit that CI demands a form of reasoning where the agent needs to reason about the context in which it is operating. To test this, we first prompt LLMs to reason explicitly about CI when deciding what information to disclose. We then extend this approach by developing a reinforcement learning (RL) framework that further instills in models the reasoning necessary to achieve CI. Using a synthetic, automatically created, dataset of only $\sim700$ examples but with diverse contexts and information disclosure norms, we show that our method substantially reduces inappropriate information disclosure while maintaining task performance across multiple model sizes and families. Importantly, improvements transfer from this synthetic dataset to established CI benchmarks such as PrivacyLens that has human annotations and evaluates privacy leakage of AI assistants in actions and tool calls.

Method: Social choice theory-based approach that infers evaluator population distributions from pairwise comparisons, constructs policies satisfying monotonicity, Pareto efficiency, and new axioms of population-proportional alignment and bounded manipulability, with soft-max relaxation for trade-offs.

Result: The approach effectively addresses bias and manipulation while maintaining foundational social choice axioms, validated through experiments on tabular recommendation tasks and large language model alignment.

Conclusion: The framework provides a principled solution for proportional preference aggregation that is robust to manipulation and scalable across different applications.

Abstract: Conventional preference learning methods often prioritize opinions held more widely when aggregating preferences from multiple evaluators. This may result in policies that are biased in favor of some types of opinions or groups and susceptible to strategic manipulation. To address this issue, we develop a novel preference learning framework capable of aligning aggregate opinions and policies proportionally with the true population distribution of evaluator preferences. Grounded in social choice theory, our approach infers the feasible set of evaluator population distributions directly from pairwise comparison data. Using these estimates, the algorithm constructs a policy that satisfies foundational axioms from social choice theory, namely monotonicity and Pareto efficiency, as well as our newly-introduced axioms of population-proportional alignment and population-bounded manipulability. Moreover, we propose a soft-max relaxation method that smoothly trade-offs population-proportional alignment with the selection of the Condorcet winner (which beats all other options in pairwise comparisons). Finally, we validate the effectiveness and scalability of our approach through experiments on both tabular recommendation tasks and large language model alignment.

Method: ReLIFT alternates between RL training and supervised fine-tuning using high-quality solutions collected when the model encounters challenging questions, combining the strengths of both approaches.

Result: ReLIFT achieves an average improvement of +5.2 points across five competition-level benchmarks and one out-of-distribution benchmark compared to zero-RL models, while using only 13% of detailed demonstration data.

Conclusion: ReLIFT overcomes the fundamental limitations of RL in LLM reasoning and demonstrates significant potential for scalable reasoning capability enhancement through complementary RL and SFT training.

Abstract: Recent advances in large language model (LLM) reasoning have shown that sophisticated behaviors such as planning and self-reflection can emerge through reinforcement learning (RL). However, despite these successes, RL in its current form remains insufficient to induce capabilities that exceed the limitations of the base model, as it is primarily optimized based on existing knowledge of the model rather than facilitating the acquisition of new information. To address this limitation, we employ supervised fine-tuning (SFT) to learn what RL cannot, which enables the incorporation of new knowledge and reasoning patterns by leveraging high-quality demonstration data. We analyze the training dynamics of RL and SFT for LLM reasoning and find that RL excels at maintaining and improving performance on questions within the model’s original capabilities, while SFT is more effective at enabling progress on questions beyond the current scope of the model. Motivated by the complementary strengths of RL and SFT, we introduce a novel training approach, \textbf{ReLIFT} (\textbf{Re}inforcement \textbf{L}earning \textbf{I}nterleaved with Online \textbf{F}ine-\textbf{T}uning). In ReLIFT, the model is primarily trained using RL, but when it encounters challenging questions, high-quality solutions are collected for fine-tuning, and the training process alternates between RL and fine-tuning to enhance the model’s reasoning abilities. ReLIFT achieves an average improvement of over +5.2 points across five competition-level benchmarks and one out-of-distribution benchmark compared to other zero-RL models. Furthermore, we demonstrate that ReLIFT outperforms both RL and SFT while using only 13% of the detailed demonstration data, highlighting its scalability. These results provide compelling evidence that ReLIFT overcomes the fundamental limitations of RL and underscores the significant potential.

Method: Created ALE-Bench using real tasks from AtCoder Heuristic Contests, featuring computationally hard optimization problems with no known exact solutions. Developed a software framework supporting interactive agent architectures with test-run feedback and visualizations.

Result: Evaluation of frontier LLMs showed they perform well on specific problems but lack consistency across different problems and struggle with long-horizon problem-solving compared to humans.

Conclusion: ALE-Bench is needed to drive future AI advancements by providing a proper benchmark for evaluating iterative, long-term optimization problem-solving capabilities where current AI systems still fall short of human performance.

Abstract: How well do AI systems perform in algorithm engineering for hard optimization problems in domains such as package-delivery routing, crew scheduling, factory production planning, and power-grid balancing? We introduce ALE-Bench, a new benchmark for evaluating AI systems on score-based algorithmic programming contests. Drawing on real tasks from the AtCoder Heuristic Contests, ALE-Bench presents optimization problems that are computationally hard and admit no known exact solution. Unlike short-duration, pass/fail coding benchmarks, ALE-Bench encourages iterative solution refinement over long time horizons. Our software framework supports interactive agent architectures that leverage test-run feedback and visualizations. Our evaluation of frontier LLMs revealed that while they demonstrate high performance on specific problems, a notable gap remains compared to humans in terms of consistency across problems and long-horizon problem-solving capabilities. This highlights the need for this benchmark to foster future AI advancements.

Method: The authors use Dalal’s method for belief change to induce structured evolution of belief states, and demonstrate that binary ANN training dynamics can be better modeled using lexicographic revision and moderate contraction operations that align with the Darwiche-Pearl framework for iterated belief change.

Result: The study shows that robust AGM-style change operations provide a more effective model for binary ANN training dynamics compared to the previous full-meet belief change approach.

Conclusion: Binary ANN training can be effectively modeled using robust belief change operations (lexicographic revision and moderate contraction) within the Darwiche-Pearl framework, overcoming limitations of the previous full-meet belief change approach.

Abstract: Artificial Neural Networks (ANNs) are powerful machine-learning models capable of capturing intricate non-linear relationships. They are widely used nowadays across numerous scientific and engineering domains, driving advancements in both research and real-world applications. In our recent work, we focused on the statics and dynamics of a particular subclass of ANNs, which we refer to as binary ANNs. A binary ANN is a feed-forward network in which both inputs and outputs are restricted to binary values, making it particularly suitable for a variety of practical use cases. Our previous study approached binary ANNs through the lens of belief-change theory, specifically the Alchourron, Gardenfors and Makinson (AGM) framework, yielding several key insights. Most notably, we demonstrated that the knowledge embodied in a binary ANN (expressed through its input-output behaviour) can be symbolically represented using a propositional logic language. Moreover, the process of modifying a belief set (through revision or contraction) was mapped onto a gradual transition through a series of intermediate belief sets. Analogously, the training of binary ANNs was conceptualized as a sequence of such belief-set transitions, which we showed can be formalized using full-meet AGM-style belief change. In the present article, we extend this line of investigation by addressing some critical limitations of our previous study. Specifically, we show that Dalal’s method for belief change naturally induces a structured, gradual evolution of states of belief. More importantly, given the known shortcomings of full-meet belief change, we demonstrate that the training dynamics of binary ANNs can be more effectively modelled using robust AGM-style change operations – namely, lexicographic revision and moderate contraction – that align with the Darwiche-Pearl framework for iterated belief change.

Method: Two-stage fine-tuning: (1) Inject difficulty hypnosis into output prefixes to guide adaptive reasoning depth using hybrid dataset, (2) Incorporate redundancy hypnosis to supervise intermediate steps to eliminate unnecessary reasoning patterns.

Result: Reduces inference costs by over 70% on easy tasks and 40% on hard tasks while maintaining performance stability. Models show clear difficulty-aware capabilities and reduced redundancy.

Conclusion: TH2T effectively alleviates overthinking in LRMs by bootstrapping difficulty and redundancy cognition, enabling more efficient and adaptive reasoning without performance degradation.

Abstract: Recent Large Reasoning Models (LRMs) excel at complex reasoning tasks but often suffer from overthinking, generating overly long and redundant reasoning trajectories. To explore its essence, our empirical analysis reveals that LRMs are primarily limited to recognizing task properties (i.e., difficulty levels) like humans before solving the problem, leading to a one-size-fits-all reasoning process. Inspired by this, a pressing and natural question emerges: Can we explicitly bootstrap such ability to alleviate overthinking in LRMs? In this paper, we propose Think-How-to-Think (TH2T), a novel two-stage fine-tuning strategy that progressively inspires LRMs’ difficulty cognition and redundancy cognition of LRMs. Specifically, we first inject difficulty hypnosis into output prefixes to guide the model toward adaptive reasoning depth, trained on a hybrid dataset mixing short and long reasoning paths. Then, we incorporate redundancy hypnosis, which supervises the intermediate reasoning steps to identify and eliminate unnecessary reasoning patterns. Experiments on 7B/14B/32B models demonstrate that TH2T significantly reduces inference costs by over 70% on easy tasks and 40% on hard tasks while maintaining performance stability. The resulting outputs exhibit clear signs of difficulty-aware capabilities and reduced redundancy (e.g., reflection and looping).

Method: 1) Test-time scaling: samples multiple candidate action proposals, evaluates them with a judge model, and selects the best one. 2) Reinforcement learning-based grounding: improves visual element interaction through objective alignment that rewards successful clicks.

Result: GTA1 achieves state-of-the-art performance on both grounding and agent task execution benchmarks.

Conclusion: The proposed test-time scaling and RL-based grounding methods effectively address planning and action grounding challenges in GUI agents, leading to superior performance.

Abstract: Graphical user interface (GUI) agents autonomously complete tasks across platforms (\eg, Linux) by sequentially decomposing user instructions into action proposals that iteratively interact with visual elements in the evolving environment. However, two main challenges arise: i) planning (\ie, the action proposal sequence) under expansive action space, where selecting an appropriate plan is non-trivial, as many valid ones may exist; ii) accurately grounding actions in complex and high-resolution interfaces, \ie, precisely interacting with visual targets. This paper investigates the aforementioned challenges with our \textbf{G}UI \textbf{T}est-time Scaling \textbf{A}gent, namely GTA1. First, we conduct test-time scaling to select the most appropriate action proposal: at each step, multiple candidate proposals are sampled and evaluated and selected by a judge model. It trades off computation for better decision quality by concurrent sampling. Second, we propose a model that improves grounding of the selected action proposals to its corresponding visual elements. Our key insight is that reinforcement learning (RL) facilitates grounding through inherent objective alignments, rewarding successful clicks on interface elements. Experimentally, GTA1 achieves state-of-the-art performance on both grounding and agent task execution benchmarks. The code and models are released here.

Method: Created a benchmark with structured representations of indoor scenes (kitchens, living rooms, etc.) in JSON/XML formats, testing core spatial tasks like distance measurement, visibility, path finding, and object placement.

Result: LLMs succeed in shallow queries but fail to respect physical constraints and preserve spatial coherence, though they remain mostly robust to small spatial perturbations. The benchmark reveals inconsistent reasoning about indoor layouts.

Conclusion: FloorplanQA uncovers a blind spot in current LLMs regarding spatial reasoning and aims to inspire development of models that can better infer and manipulate spatial properties in practical settings.

Abstract: We introduce FloorplanQA, a diagnostic benchmark for evaluating spatial reasoning in large-language models (LLMs). FloorplanQA is grounded in structured representations of indoor scenes, such as (e.g., kitchens, living rooms, bedrooms, bathrooms, and others), encoded symbolically in JSON or XML layouts. The benchmark covers core spatial tasks, including distance measurement, visibility, path finding, and object placement within constrained spaces. Our results across a variety of frontier open-source and commercial LLMs reveal that while models may succeed in shallow queries, they often fail to respect physical constraints, preserve spatial coherence, though they remain mostly robust to small spatial perturbations. FloorplanQA uncovers a blind spot in today’s LLMs: inconsistent reasoning about indoor layouts. We hope this benchmark inspires new work on language models that can accurately infer and manipulate spatial and geometric properties in practical settings.

Method: Introduces soft constrained knowledge distillation strategies at feature and classifier levels instead of hard constrained loss, and proposes a quality-based adaptive weights module to weigh input samples based on quantified data quality.

Result: Experiments on speaker recognition and image classification tasks show effective knowledge transfer between image, text, and speech modalities.

Conclusion: The proposed cross-modal distillation framework successfully enables knowledge transfer between widely differing modalities while preventing overfitting through soft constraints and quality-based weighting.

Abstract: Deep learning achieved great progress recently, however, it is not easy or efficient to further improve its performance by increasing the size of the model. Multi-modal learning can mitigate this challenge by introducing richer and more discriminative information as input. To solve the problem of limited access to multi-modal data at the time of use, we conduct multi-modal learning by introducing a teacher model to transfer discriminative knowledge to a student model during training. However, this knowledge transfer via distillation is not trivial because the big domain gap between the widely differing modalities can easily lead to overfitting. In this work, we introduce a cross-modal distillation framework. Specifically, we find hard constrained loss, e.g. l2 loss forcing the student being exact the same as the teacher, can easily lead to overfitting in cross-modality distillation. To address this, we propose two soft constrained knowledge distillation strategies at the feature level and classifier level respectively. In addition, we propose a quality-based adaptive weights module to weigh input samples via quantified data quality, leading to robust model training. We conducted experiments on speaker recognition and image classification tasks, and the results show that our approach is able to effectively achieve knowledge transfer between the commonly used and widely differing modalities of image, text, and speech.

Method: Created QCBench with 350 computational chemistry problems across 7 subfields, categorized into easy/medium/difficult tiers, using realistic chemical scenarios that require explicit numerical reasoning.

Result: Evaluation of 24 LLMs showed consistent performance degradation with increasing task complexity, revealing gaps between language fluency and scientific computation accuracy.

Conclusion: QCBench enables fine-grained diagnosis of computational weaknesses and lays groundwork for future improvements like domain-adaptive fine-tuning or multi-modal integration in quantitative chemistry.

Abstract: Quantitative chemistry is central to modern chemical research, yet the ability of large language models (LLMs) to perform its rigorous, step-by-step calculations remains underexplored. To fill this blank, we propose QCBench, a Quantitative Chemistry oriented benchmark comprising 350 computational chemistry problems across 7 chemistry subfields, which contains analytical chemistry, bio/organic chemistry, general chemistry, inorganic chemistry, physical chemistry, polymer chemistry and quantum chemistry. To systematically evaluate the mathematical reasoning abilities of large language models (LLMs), they are categorized into three tiers: easy, medium, and difficult. Each problem, rooted in realistic chemical scenarios, is structured to prevent heuristic shortcuts and demand explicit numerical reasoning. QCBench enables fine-grained diagnosis of computational weaknesses, reveals model-specific limitations across difficulty levels, and lays the groundwork for future improvements such as domain-adaptive fine-tuning or multi-modal integration. Evaluations on 24 LLMs demonstrate a consistent performance degradation with increasing task complexity, highlighting the current gap between language fluency and scientific computation accuracy. Code for QCBench is available at https://github.com/jiaqingxie/QCBench.

Method: Created CloudAnoBench with 28 anomalous scenarios and 16 deceptive normal scenarios, containing 1,252 labeled cases and ~200,000 log and metric entries. Proposed CloudAnoAgent, an LLM-based agent enhanced by symbolic verification that integrates metrics and logs for context-aware anomaly detection.

Result: CloudAnoBench exhibits higher ambiguity and greater difficulty than prior benchmarks, where both traditional ML methods and vanilla LLM prompting perform poorly. CloudAnoAgent achieves substantial improvements in both anomaly detection and scenario identification on CloudAnoBench, and shows strong generalization to existing datasets.

Conclusion: CloudAnoBench and CloudAnoAgent lay the groundwork for advancing context-aware anomaly detection in cloud systems by addressing the limitations of single-modality approaches and providing a comprehensive benchmark with effective detection methods.

Abstract: Anomaly detection in cloud environments remains both critical and challenging. Existing context-level benchmarks typically focus on either metrics or logs and often lack reliable annotation, while most detection methods emphasize point anomalies within a single modality, overlooking contextual signals and limiting real-world applicability. Constructing a benchmark for context anomalies that combines metrics and logs is inherently difficult: reproducing anomalous scenarios on real servers is often infeasible or potentially harmful, while generating synthetic data introduces the additional challenge of maintaining cross-modal consistency. We introduce CloudAnoBench, a large-scale benchmark for context anomalies in cloud environments, comprising 28 anomalous scenarios and 16 deceptive normal scenarios, with 1,252 labeled cases and roughly 200,000 log and metric entries. Compared with prior benchmarks, CloudAnoBench exhibits higher ambiguity and greater difficulty, on which both prior machine learning methods and vanilla LLM prompting perform poorly. To demonstrate its utility, we further propose CloudAnoAgent, an LLM-based agent enhanced by symbolic verification that integrates metrics and logs. This agent system achieves substantial improvements in both anomaly detection and scenario identification on CloudAnoBench, and shows strong generalization to existing datasets. Together, CloudAnoBench and CloudAnoAgent lay the groundwork for advancing context-aware anomaly detection in cloud systems. Project Page: https://jayzou3773.github.io/cloudanobench-agent/

Method: Three key innovations: 1) Content-aware sparse masks generated dynamically from value vectors, 2) Position-aware sparse attention computation that skips unnecessary regions, 3) Gradient-preserving design that supports end-to-end training without obstructing gradients.

Result: Achieves Pareto dominance across various tasks including scaling laws, multi-query associative recall, general benchmarks, and needle-in-a-haystack tests, with up to 10 times acceleration while maintaining performance.

Conclusion: The method effectively balances model efficiency with long-context modeling through dual-sparsity design that retains complete information while significantly reducing computational complexity.

Abstract: In large language models, the demand for modeling long contexts is ever-increasing, yet the quadratic complexity of standard self-attention presents a significant bottleneck. While existing sparse attention mechanisms enhance efficiency, they often suffer from limitations such as static patterns and information loss. This paper introduces a Trainable Dynamic Mask Sparse Attention mechanism that addresses these challenges through three key innovations. First, it leverages value vectors to dynamically generate content-aware sparse masks, enabling the model to adaptively identify and focus on crucial information. Second, it implements a position-aware sparse attention computation that effectively skips unnecessary computational regions. Finally, we ensure that the introduced dynamic masks and sparse weights do not obstruct gradients, thereby supporting end-to-end training. This dual-sparsity design allows the model to retain complete information while significantly reducing computational complexity, achieving an excellent balance between efficiency and performance. We validate the performance of Dynamic Mask Attention through comprehensive experiments. Comparative studies demonstrate that our method consistently achieves Pareto dominance across various tasks, including scaling laws, multi-query associative recall, general benchmarks, and needle-in-a-haystack tests, delivering up to 10 times acceleration. These results highlight its capability to effectively balance model efficiency with long-context modeling. Our computational kernel is open-sourced at https://github.com/SmallDoges/flash-dmattn to facilitate further research and application within the community.

Method: The framework consists of: (1) annotation infrastructure for capturing human demonstrations, (2) AgentNet dataset spanning 3 OS and 200+ applications, (3) scalable pipeline converting demonstrations to state-action pairs with reflective Chain-of-Thought reasoning.

Result: OpenCUA-72B achieves 45.0% average success rate on OSWorld-Verified, establishing new SOTA among open-source models. The approach generalizes well across domains and benefits from increased test-time computation.

Conclusion: OpenCUA provides comprehensive open foundations for computer-use agent research, with released annotation tools, datasets, code, and models to enable further study of these increasingly important systems.

Abstract: Vision-language models have demonstrated impressive capabilities as computer-use agents (CUAs) capable of automating diverse computer tasks. As their commercial potential grows, critical details of the most capable CUA systems remain closed. As these agents will increasingly mediate digital interactions and execute consequential decisions on our behalf, the research community needs access to open CUA frameworks to study their capabilities, limitations, and risks. To bridge this gap, we propose OpenCUA, a comprehensive open-source framework for scaling CUA data and foundation models. Our framework consists of: (1) an annotation infrastructure that seamlessly captures human computer-use demonstrations; (2) AgentNet, the first large-scale computer-use task dataset spanning 3 operating systems and 200+ applications and websites; (3) a scalable pipeline that transforms demonstrations into state-action pairs with reflective long Chain-of-Thought reasoning that sustain robust performance gains as data scales. Our end-to-end agent models demonstrate strong performance across CUA benchmarks. In particular, OpenCUA-72B achieves an average success rate of 45.0% on OSWorld-Verified, establishing a new state-of-the-art (SOTA) among open-source models. Further analysis confirms that our approach generalizes well across domains and benefits significantly from increased test-time computation. We release our annotation tool, datasets, code, and models to build open foundations for further CUA research.

Method: Integrates lightweight symbolic representations into few-shot prompts to structure inference steps with consistent strategy, making reasoning patterns more explicit in non-interactive reasoning processes.

Result: Significantly outperforms conventional CoT on three out of four datasets (ProofWriter, ProntoQA, LogicalDeduction), consistently improves reasoning capabilities across various model sizes, particularly effective in complex reasoning tasks requiring multiple constraints.

Conclusion: Symbolic-Aided CoT is an effective approach that enhances LLM logical reasoning by incorporating symbolic structures, demonstrating superior performance over standard CoT methods on multiple benchmarks.

Abstract: This work introduces Symbolic-Aided Chain-of-Thought (CoT), an improved approach to standard CoT, for logical reasoning in large language models (LLMs). The key idea is to integrate lightweight symbolic representations into few-shot prompts, structuring the inference steps with a consistent strategy to make reasoning patterns more explicit within a non-interactive reasoning process. By incorporating these symbolic structures, Symbolic-Aided CoT preserves the generalizability of standard prompting techniques while enhancing the transparency, interpretability, and analyzability of LLM logical reasoning. Extensive experiments on four well-known logical reasoning benchmarks – ProofWriter, FOLIO, ProntoQA, and LogicalDeduction, which cover diverse reasoning tasks and scenarios – demonstrate the effectiveness of the proposed approach, particularly in complex reasoning tasks that require navigating multiple constraints or rules. Notably, Symbolic-Aided CoT consistently improves LLMs’ reasoning capabilities across various model sizes and significantly outperforms conventional CoT on three out of four datasets, ProofWriter, ProntoQA, and LogicalDeduction.

Method: Uses evolutionary testing to automatically generate and evolve mathematical benchmarks through continuous self-iteration, enhancing complexity of existing datasets.

Result: Generated high-difficulty problems that reduced LLM accuracy by average 48%, and identified ‘Pseudo Aha Moment’ phenomenon where LLMs bypass complex reasoning using simplistic conditions, accounting for 77-100% of errors.

Conclusion: EvolMathEval effectively creates challenging mathematical benchmarks that reveal LLM limitations in complex reasoning and provides better evaluation of true model capabilities.

Abstract: The rapid advancement of Large Language Models (LLMs) poses a significant challenge to existing mathematical reasoning benchmarks. However, these benchmarks tend to become easier over time as LLMs can learn from the published benchmarks. This limitation hinder the precise evaluation of the true capabilities of SOTA models. To address this challenge, this paper introduces EvolMathEval, an automated mathematical benchmark generation and evolution framework based on evolutionary testing. Experimental results demonstrate that EvolMathEval can not only generate a large volume of high-difficulty problems through continuous self-iteration, but it can also significantly enhance the complexity of public datasets like GSM8K through evolution, reducing model accuracy by an average of 48%. Deeper investigation reveals that when solving these evolved problems, LLMs tend to bypass complex multi-step logical reasoning by relying on simplistic and fuzzy conditions, consequently leading to incorrect solutions. We define this phenomenon as the ``Pseudo Aha Moment", which we find accounts for 77% to 100% of errors on targeted problems. Code and resources are available at: https://anonymous.4open.science/r/EvolMathEval

Method: Used an infinitely scalable framework to synthesize multi-step reasoning questions from arXiv papers, leveraging temporal structure to test performance before/after knowledge cutoff dates across 4 frontier models.

Result: Consistently found no significant performance decay near knowledge cutoff dates across models of various sizes, developers, and release dates, contrary to previous studies.

Conclusion: Multi-step reasoning required by the synthesis pipeline provides complexity beyond shallow memorization, serving as an effective mitigation strategy against benchmark contamination, advocating for reasoning-driven benchmark construction.

Abstract: Capability evaluation of large language models (LLMs) is increasingly shadowed by rising concerns of data contamination that cast doubts on whether static benchmarks measure genuine reasoning or mere memorization. We present an empirical study using an infinitely scalable framework to synthesize research-level QA directly from arXiv papers, harnessing the natural temporal structure of research publications where performance decay after knowledge cutoffs may indicate potential contamination. We evaluated 4 frontier model represented by 2 models of different knowledge cutoff dates per family on 1,643 multi-step reasoning questions synthesized from 20,277 arXiv papers stratified over 26 months, covering at least 6 months before and after all cutoff dates. Our results consistently showed a lack of significant performance decay near knowledge cutoff dates for models of various sizes, developers, and release dates. We further performed a comparative analysis with previous longitudinal studies that reported significant post-cutoff performance decay using directly retrieved questions based on public data. we hypothesize that the multi-step reasoning required by our synthesis pipeline offered additional complexity that goes deeper than shallow memorization, which effectively serves a mitigation strategy against benchmark contamination. We fully open source our code and dataset to aid reproducibility and advocate for a paradigm shift that prioritize reasoning-driven synthesis to construct benchmarks over simply collecting newly released questions periodically.

Method: Introduces concept-level memory with strategies for abstracting takeaways from solution rollouts and retrieving relevant concepts for new queries. Uses natural language to store modular abstractions that can be selectively integrated into prompts.

Result: Achieves 7.5% relative gain over strong no-memory baseline on ARC-AGI benchmark. Performance continues to scale with inference compute, with abstract concepts being the most consistent memory design. Dynamic memory updates during test-time outperform fixed settings.

Conclusion: Concept-level memory enables effective test-time continual learning through accumulation and abstraction of patterns, supporting self-improvement without weight updates. The approach shows promise for compositional generalization and abstract reasoning tasks.

Abstract: While inference-time scaling enables LLMs to carry out increasingly long and capable reasoning traces, the patterns and insights uncovered during these traces are immediately discarded once the context window is reset for a new query. External memory is a natural way to persist these discoveries, and recent work has shown clear benefits for reasoning-intensive tasks. We see an opportunity to make such memories more broadly reusable and scalable by moving beyond instance-based memory entries (e.g. exact query/response pairs, or summaries tightly coupled with the original problem context) toward concept-level memory: reusable, modular abstractions distilled from solution traces and stored in natural language. For future queries, relevant concepts are selectively retrieved and integrated into the prompt, enabling test-time continual learning without weight updates. Our design introduces new strategies for abstracting takeaways from rollouts and retrieving entries for new queries, promoting reuse and allowing memory to expand with additional experiences. We evaluate on ARC-AGI, a benchmark that stresses compositional generalization and abstract reasoning, making it a natural fit for concept memory. Our method yields a 7.5% relative gain over a strong no-memory baseline with performance continuing to scale with inference compute. We find abstract concepts to be the most consistent memory design, outscoring the baseline at all tested inference compute scales. Moreover, dynamically updating memory during test-time outperforms fixed settings, supporting the hypothesis that accumulating and abstracting patterns enables further solutions in a form of self-improvement. Code is available at https://github.com/matt-seb-ho/arc_memo.

Method: Divides workspace into private/public spaces with structured anonymization. Uses Domain-Rule-based Knowledge Enhancement (DRKE) and Disproof-based Logic Enhancement (DLE) to mitigate semantic loss from anonymization.

Result: Outperforms existing baselines on general QA datasets, privacy leakage benchmarks, and customized privacy-related QA datasets in both task accuracy and privacy preservation metrics.

Conclusion: GAMA effectively balances privacy protection and task performance in LLM-based multi-agent systems through its structured anonymization approach and enhancement modules.

Abstract: With the rapid advancement of Large Language Models (LLMs), LLM-based agents exhibit exceptional abilities in understanding and generating natural language, enabling human-like collaboration and information transmission in LLM-based Multi-Agent Systems (MAS). High-performance LLMs are often hosted on web servers in public cloud environments. When tasks involve private data, MAS cannot securely utilize these LLMs without implementing the agentic privacy-preserving mechanism. To address this challenge, we propose a General Anonymizing Multi-Agent System (GAMA), which divides the agents’ workspace into private and public spaces, ensuring privacy through a structured anonymization mechanism. In the private space, agents handle sensitive data, while in the public web space, only anonymized data is utilized. GAMA incorporates two key modules to mitigate semantic loss caused by anonymization: Domain-Rule-based Knowledge Enhancement (DRKE) and Disproof-based Logic Enhancement (DLE). We evaluate GAMA on two general question-answering datasets, a public privacy leakage benchmark, and two customized question-answering datasets related to privacy. The results demonstrate that GAMA outperforms existing baselines on the evaluated datasets in terms of both task accuracy and privacy preservation metrics.

Method: DataAgents integrate LLM reasoning with task decomposition, action reasoning and grounding, and tool calling. They dynamically plan workflows, call powerful tools, and adapt to diverse data tasks at scale.

Result: DataAgents can handle collection, integration, preprocessing, selection, transformation, reweighing, augmentation, reprogramming, repairs, and retrieval, transforming complex unstructured data into coherent actionable knowledge.

Conclusion: DataAgents represent a paradigm shift toward autonomous data-to-knowledge systems. The report calls for advancing workflow optimization, establishing benchmarks, safeguarding privacy, balancing efficiency with scalability, and developing trustworthy guardrails.

Abstract: As data continues to grow in scale and complexity, preparing, transforming, and analyzing it remains labor-intensive, repetitive, and difficult to scale. Since data contains knowledge and AI learns knowledge from it, the alignment between AI and data is essential. However, data is often not structured in ways that are optimal for AI utilization. Moreover, an important question arises: how much knowledge can we pack into data through intensive data operations? Autonomous data agents (DataAgents), which integrate LLM reasoning with task decomposition, action reasoning and grounding, and tool calling, can autonomously interpret data task descriptions, decompose tasks into subtasks, reason over actions, ground actions into python code or tool calling, and execute operations. Unlike traditional data management and engineering tools, DataAgents dynamically plan workflows, call powerful tools, and adapt to diverse data tasks at scale. This report argues that DataAgents represent a paradigm shift toward autonomous data-to-knowledge systems. DataAgents are capable of handling collection, integration, preprocessing, selection, transformation, reweighing, augmentation, reprogramming, repairs, and retrieval. Through these capabilities, DataAgents transform complex and unstructured data into coherent and actionable knowledge. We first examine why the convergence of agentic AI and data-to-knowledge systems has emerged as a critical trend. We then define the concept of DataAgents and discuss their architectural design, training strategies, as well as the new skills and capabilities they enable. Finally, we call for concerted efforts to advance action workflow optimization, establish open datasets and benchmark ecosystems, safeguard privacy, balance efficiency with scalability, and develop trustworthy DataAgent guardrails to prevent malicious actions.

Method: JET performs trajectory truncation during rollout to expose models to short reasoning paths and uses a quality-controlled length reward to encourage concise reasoning while maintaining correctness.

Result: JET significantly improves reasoning efficiency without sacrificing accuracy, with DeepSeek-Distill-Qwen-1.5B achieving 4.6% accuracy gain while reducing output length by 46.3% on the Olympiad benchmark.

Conclusion: JET enables efficient reasoning by training models to terminate unnecessary reasoning steps early, demonstrating substantial improvements in both efficiency and accuracy.

Abstract: Large Reasoning Models (LRMs) have achieved impressive performance on challenging tasks, yet their deep reasoning often incurs substantial computational costs. To achieve efficient reasoning, existing reinforcement learning methods still struggle to construct short reasoning path during the rollout stage, limiting effective learning. Inspired by Evidence Accumulation Models, we find that LRMs have accumulated sufficient information early in reasoning, making further reasoning steps redundant. Based on this insight, we propose Just-Enough Thinking (JET), which trains models to proactively terminate unnecessary reasoning. JET performs trajectory truncation during rollout to expose the model to short, distributionally consistent reasoning paths. Besides, it uses a quality-controlled length reward to better encourage concise reasoning while maintaining correctness. Extensive experiments demonstrate that JET significantly improves reasoning efficiency without sacrificing accuracy. Especially, DeepSeek-Distill-Qwen-1.5B achieves a 4.6% accuracy gain while reducing output length by 46.3% on the Olympiad benchmark. Our code is available in the GitHub.

Method: Used the Jailbreak Oracle (JO) systematic LLM evaluation tool to probe GPT-OSS-20B’s behavior under different adversarial conditions, identifying specific failure modes.

Result: Uncovered several critical failure modes: quant fever, reasoning blackholes, Schrodinger’s compliance, reasoning procedure mirage, and chain-oriented prompting that can be exploited on the model.

Conclusion: The study demonstrates severe security vulnerabilities in GPT-OSS-20B that can lead to significant consequences when exploited by adversaries.

Abstract: OpenAI’s GPT-OSS family provides open-weight language models with explicit chain-of-thought (CoT) reasoning and a Harmony prompt format. We summarize an extensive security evaluation of GPT-OSS-20B that probes the model’s behavior under different adversarial conditions. Using the Jailbreak Oracle (JO) [1], a systematic LLM evaluation tool, the study uncovers several failure modes including quant fever, reasoning blackholes, Schrodinger’s compliance, reasoning procedure mirage, and chain-oriented prompting. Experiments demonstrate how these behaviors can be exploited on the GPT-OSS-20B model, leading to severe consequences.

Method: Two monitoring approaches: 1) Reasoning expression monitoring for black-box models (analyzing confidence trajectories), 2) Hidden states monitoring for white-box models (analyzing separability of solvable/unsolvable problems in hidden representations).

Result: Boundary-aware strategies reduce token usage by 62.7-93.6% while maintaining accuracy, significantly improving reliability and efficiency.

Conclusion: LRMs possess inherent capability boundary awareness that can be leveraged through simple monitoring strategies to prevent unproductive reasoning and enhance computational efficiency.

Abstract: Large Reasoning Models (LRMs) have shown impressive performance on complex reasoning tasks such as mathematics, yet they also display misbehaviors that expose their limitations. In particular, when faced with hard questions, LRMs often engage in unproductive reasoning until context limit, producing wrong answers while wasting substantial computation. This phenomenon reflects a fundamental issue: current answering paradigms overlook the relationship between questions and LRMs’ capability boundaries. In this paper, we investigate whether LRMs possess self-awareness of capability boundaries. We begin by an observation that LRMs may know what they cannot solve through expressed reasoning confidence. For black-box models, we find that reasoning expressions reveal boundary signals, with accelerated growing confidence trajectory for solvable problems but convergent uncertainty trajectory for unsolvable ones. For white-box models, we show that hidden states of the last input token encode boundary information, with solvable and unsolvable problems linearly separable even before reasoning begins. Building on these findings, we propose two simple yet effective optimization strategies: reasoning expression monitoring and hidden states monitoring. Experiments demonstrate that these boundary-aware strategies enable LRMs to avoid unproductive reasoning without sacrificing accuracy, significantly improving reliability and efficiency by cutting token usage up to 62.7 - 93.6%.

Method: Introduces Data Reasoning Intensity (DRI) metric to quantify logical reasoning complexity by decomposing and aggregating logical structures. Proposes a re-cognizing optimization strategy that systematically enhances logical reasoning intensity of training data to better align with LLM’s reasoning boundary.

Result: Extensive experiments show significant improvements in performance and generalization over data-centric strategies. Method validated under reinforcement learning framework demonstrates enhanced reasoning capabilities.

Conclusion: Prioritizing reasoning complexity in data rather than sheer scale or superficial form is essential to realizing LLMs’ full cognitive potential.

Abstract: Recent advances in large language models (LLMs) highlight the importance of training data structure and quality in shaping reasoning behavior. However, most existing approaches focus on transforming data formats while neglecting the internal reasoning complexity of training samples, leaving the reasoning potential of data under-explored and underutilized. In this work, we posit that LLM logical reasoning performance is jointly constrained by the potential of the training data and the cognitive capacity of the model. To make this relationship measurable, we introduce Data Reasoning Intensity (DRI), a novel metric that quantifies the latent logical reasoning complexity of samples by decomposing and aggregating their logical structures. This allows us to analyze how well current LLMs utilize logical reasoning signals and identify performance gaps relative to data potential. Based on this insight, we introduce a re-cognizing optimization strategy that systematically enhances the logical reasoning intensity of training data. Rather than increasing data volume, our method re-optimizes existing samples to better align with the LLM’s logical reasoning boundary. Extensive experiments show that our approach significantly improves performance and generalization over data-centric strategies. We further validate our method under a reinforcement learning framework. Our results indicate that prioritizing reasoning complexity in data rather than sheer scale or superficial form is essential to realizing LLMs’ full cognitive potential.

Method: SRM uses a modular framework with side-branch models as auxiliary feature generators, introducing fine-grained dimensions for interpretable evaluation.

Result: SRMs outperform scalar RMs and GRMs in robustness and alignment with human preferences, supporting efficient optimization for practical scenarios.

Conclusion: SRM provides a practical, adaptable, and scalable reward modeling solution suitable for industrial deployment in applications like search and recommendation systems.

Abstract: Reward Models (RMs) are key components for evaluating and guiding language model outputs. However, traditional scalar RMs often struggle with incorporating contextual and background information during inference, leading to incomplete evaluations. Generative RMs (GRMs) attempt to address these limitations by generating intermediate reasoning steps. Yet, their uncontrolled black-box nature and inefficiency due to sequential decoding hinder their industrial deployment. Industrial scenarios, such as search and recommendation systems, often involve single-domain tasks requiring evaluation along specific dimensions. In such contexts, diagnosing “bad cases” necessitates structured feedback to identify and optimize dimension-specific issues. In this paper, we propose the Structural Reward Model (SRM), a modular and interpretable framework integrating side-branch models as auxiliary feature generators. By introducing fine-grained dimensions, SRMs enable interpretable and efficient evaluation, facilitating targeted diagnostics and optimization. This structured approach ensures adaptability and scalability for industrial applications. Through comprehensive experiments, we demonstrate that SRMs outperform scalar RMs and GRMs in robustness and alignment with human preferences. The modular design further supports efficient optimization for practical scenarios, allowing SRM to provide a practical reward modeling solution for industry.

Method: Proposes a unified analytical framework dividing vision-language understanding into Perception (visual information extraction) and Cognition (multi-step reasoning with observe-think-verify loop).

Result: Systematically analyzes current MLLM bottlenecks, surveys enhancement techniques from visual representations to reasoning paradigms, and reviews benchmarks.

Conclusion: Provides structured perspective on MLLM limitations and illuminates path toward next-generation models with deep reasoning and genuine world understanding.

Abstract: Multimodal Large Language Models (MLLMs) strive to achieve a profound, human-like understanding of and interaction with the physical world, but often exhibit a shallow and incoherent integration when acquiring information (Perception) and conducting reasoning (Cognition). This disconnect leads to a spectrum of reasoning failures, with hallucination being the most prominent. Collectively, these issues expose a fundamental challenge: the ability to process pixels does not yet confer the ability to construct a coherent, credible internal world model. To systematically dissect and address this challenge, this survey introduces a novel and unified analytical framework: ``From Perception to Cognition." We deconstruct the complex process of vision-language interactive understanding into two interdependent layers: Perception, the foundational ability to accurately extract visual information and achieve fine-grained alignment with textual instructions; and Cognition, the higher-order capability for proactive, multi-step, goal-oriented reasoning built upon this perceptual foundation, the core of which is the formation of a dynamic observe-think-verify reasoning loop. Guided by this framework, this paper systematically analyzes the key bottlenecks of current MLLMs at both layers. It surveys the landscape of cutting-edge methods designed to address these challenges, spanning from techniques that enhance low-level visual representations to those that improve high-level reasoning paradigms. Furthermore, we review critical benchmarks and delineate future research directions. This survey aims to provide the research community with a clear, structured perspective for understanding the intrinsic limitations of current MLLMs and to illuminate the path toward building next-generation models capable of deep reasoning and a genuine understanding of the world.

Method: Implemented SegFormer architecture with semantic segmentation to quantify malignant pixel distribution, trained on curated BreastDCEDL_AMBL dataset with 88 patients and 133 annotated lesions.

Result: Achieved 0.92 AUC for lesion-level classification, 100% sensitivity and 67% specificity at patient level, potentially eliminating one-third of unnecessary biopsies without missing malignancies.

Conclusion: Framework provides interpretable spatial predictions and establishes first standardized benchmark for DCE-MRI lesion classification, enabling clinical deployment advancement.

Abstract: Breast magnetic resonance imaging is a critical tool for cancer detection and treatment planning, but its clinical utility is hindered by poor specificity, leading to high false-positive rates and unnecessary biopsies. This study introduces a transformer-based framework for automated classification of breast lesions in dynamic contrast-enhanced MRI, addressing the challenge of distinguishing benign from malignant findings. We implemented a SegFormer architecture that achieved an AUC of 0.92 for lesion-level classification, with 100% sensitivity and 67% specificity at the patient level - potentially eliminating one-third of unnecessary biopsies without missing malignancies. The model quantifies malignant pixel distribution via semantic segmentation, producing interpretable spatial predictions that support clinical decision-making. To establish reproducible benchmarks, we curated BreastDCEDL_AMBL by transforming The Cancer Imaging Archive’s AMBL collection into a standardized deep learning dataset with 88 patients and 133 annotated lesions (89 benign, 44 malignant). This resource addresses a key infrastructure gap, as existing public datasets lack benign lesion annotations, limiting benign-malignant classification research. Training incorporated an expanded cohort of over 1,200 patients through integration with BreastDCEDL datasets, validating transfer learning approaches despite primary tumor-only annotations. Public release of the dataset, models, and evaluation protocols provides the first standardized benchmark for DCE-MRI lesion classification, enabling methodological advancement toward clinical deployment.

Method: TRACE progressively truncates a model’s chain-of-thought at various lengths, forces the model to answer, and estimates expected reward at each cutoff. It quantifies effort by measuring when reasoning becomes sufficient to obtain reward.

Result: TRACE achieves over 65% gains over a 72B CoT monitor in math reasoning and over 30% gains over a 32B monitor in coding. It also discovers unknown loopholes during training.

Conclusion: TRACE offers a scalable unsupervised approach for oversight where current monitoring methods prove ineffective against implicit reward hacking.

Abstract: Reward hacking, where a reasoning model exploits loopholes in a reward function to achieve high rewards without solving the intended task, poses a significant threat. This behavior may be explicit, i.e. verbalized in the model’s chain-of-thought (CoT), or implicit, where the CoT appears benign thus bypasses CoT monitors. To detect implicit reward hacking, we propose TRACE (Truncated Reasoning AUC Evaluation). Our key observation is that hacking occurs when exploiting the loophole is easier than solving the actual task. This means that the model is using less ’effort’ than required to achieve high reward. TRACE quantifies effort by measuring how early a model’s reasoning becomes sufficient to obtain the reward. We progressively truncate a model’s CoT at various lengths, force the model to answer, and estimate the expected reward at each cutoff. A hacking model, which takes a shortcut, will achieve a high expected reward with only a small fraction of its CoT, yielding a large area under the accuracy-vs-length curve. TRACE achieves over 65% gains over our strongest 72B CoT monitor in math reasoning, and over 30% gains over a 32B monitor in coding. We further show that TRACE can discover unknown loopholes during training. Overall, TRACE offers a scalable unsupervised approach for oversight where current monitoring methods prove ineffective.

Method: Developed a multi-agent system using LLMs that dispatch tool calls via Model Context Protocol (MCP) to perform thermodynamic simulations (Thermo-Calc) and generate process maps, with dynamic task adjustment based on results.

Result: The system can effectively reason through complex user prompts, provide analysis on alloy printability, and make autonomous decisions by dynamically adjusting task trajectories.

Conclusion: LLM-enabled multi-agent systems can automate and accelerate alloy discovery in additive manufacturing, demonstrating the benefits of adopting such systems for complex materials science challenges.

Abstract: Agentic systems enable the intelligent use of research tooling, augmenting a researcher’s ability to investigate and propose novel solutions to existing problems. Within Additive Manufacturing (AM), alloy discovery remains a complex challenge, often requiring expertise in the various domains of materials science, thermodynamic simulations, and experimental analysis. Large Language Model (LLM) enabled agents can facilitate this endeavor by utilizing their extensive knowledge base to dispatch tool calls via Model Context Protocol (MCP) to perform actions such as Thermo-Calc property diagram calculations and lack of fusion process map generation. In addition, the multi-agent system developed in this work is able to effectively reason through complex user prompts and provide analysis on the printability of proposed alloys. These agents can dynamically adjust their task trajectory to the outcomes of tool call results, effectively enabling autonomous decision-making in practical environments. This work aims to utilize LLM enabled agents to automate and accelerate the task of alloy discovery within the field of additive manufacturing and showcase the benefits of adopting this multi-agent system.

Method: Study cross-modal conflicts where conflicting evidence is presented across modalities, analyze cross-modal attention imbalance, and test a simple method of explicitly combining multiple modalities within each training instance.

Result: FMs recognize conflicts in unimodal contexts 90% of the time but fall to as low as 3% when evidence is split across modalities. Cross-modal attention imbalance causes this failure, with FMs disproportionately prioritizing certain modalities. The proposed training method significantly reduces attention imbalance and improves downstream performance on vision-language benchmarks.

Conclusion: Cross-modal attention imbalance is a critical issue that doesn’t resolve with simple dataset scaling, and systematic approaches to address cross-modal contexts are essential for building reliable foundation models.

Abstract: Foundation models (FMs) deployed in real-world tasks such as computer-use agents must integrate diverse modalities. How good are FMs at performing joint reasoning, simultaneously reasoning over multiple modalities, especially when the modalities interact and relate to each other to form cross-modal context? To better understand this problem, we study FMs on cross-modal conflicts: scenarios where conflicting evidence is presented across modalities. This allows us to examine whether FMs prioritize one modality over another or reason jointly to reconcile the conflict. Our experiments reveal that FMs can recognize conflicts in unimodal contexts, composed of a single modality, 90% of the time, but the ratio falls as low as 3% when evidence is split across modalities – similar observations hold in cross-lingual contexts, composed of multiple languages. We trace this failure to cross-modal attention imbalance, showing that FMs exhibit extreme asymmetry in attention scores, disproportionately prioritizing certain modalities. We show that cross-modal attention imbalance does not go away by simply scaling up multimodal or multilingual datasets blindly, since they lack training examples that explicitly require cross-modal reasoning. We demonstrate that even a simple and scalable method of explicitly combining multiple modalities within each training instance significantly reduces attention imbalance. Reduced attention imbalance directly translates to improved downstream performance on several vision-language benchmarks. Our findings underscore the importance of systematically addressing cross-modal contexts to build reliable foundation models.

Method: Used Whisper embeddings for audio features and extracted linguistic features (pronoun usage, syntactic complexity, filler words, clause structure) from transcriptions of Semantic Fluency, Phonemic Fluency, and Cookie Theft picture description tasks. Built classification models for HC/MCI/AD distinction and regression models for MMSE score prediction.

Result: Voted ensemble models with linguistic features achieved best classification (F1 = 0.497). Whisper embedding-based ensemble regressors yielded lowest MMSE prediction error (RMSE = 2.843). Models placed among top submissions in regression and mid-range for classification in PROCESS Challenge.

Conclusion: Multimodal speech-based approaches show promise for scalable, non-invasive cognitive assessment. Integration of task-specific linguistic and acoustic markers is important for dementia detection.

Abstract: Early detection of Alzheimer’s Dementia (AD) and Mild Cognitive Impairment (MCI) is critical for timely intervention, yet current diagnostic approaches remain resource-intensive and invasive. Speech, encompassing both acoustic and linguistic dimensions, offers a promising non-invasive biomarker for cognitive decline. In this study, we present a machine learning framework for the PROCESS Challenge, leveraging both audio embeddings and linguistic features derived from spontaneous speech recordings. Audio representations were extracted using Whisper embeddings from the Cookie Theft description task, while linguistic features-spanning pronoun usage, syntactic complexity, filler words, and clause structure-were obtained from transcriptions across Semantic Fluency, Phonemic Fluency, and Cookie Theft picture description. Classification models aimed to distinguish between Healthy Controls (HC), MCI, and AD participants, while regression models predicted Mini-Mental State Examination (MMSE) scores. Results demonstrated that voted ensemble models trained on concatenated linguistic features achieved the best classification performance (F1 = 0.497), while Whisper embedding-based ensemble regressors yielded the lowest MMSE prediction error (RMSE = 2.843). Comparative evaluation within the PROCESS Challenge placed our models among the top submissions in regression task, and mid-range for classification, highlighting the complementary strengths of linguistic and audio embeddings. These findings reinforce the potential of multimodal speech-based approaches for scalable, non-invasive cognitive assessment and underline the importance of integrating task-specific linguistic and acoustic markers in dementia detection.

Method: Created a comprehensive blind evaluation framework with 90 hours of audio (21,000 samples) from 21 real sources and 17 TTS models, testing across three tasks of increasing difficulty.

Result: The challenge provides initial insights into the strengths and limitations of current synthetic audio detection approaches through systematic benchmarking.

Conclusion: SAFE offers a foundational framework for advancing synthetic audio detection research by establishing standardized evaluation protocols for increasingly realistic synthetic speech threats.

Abstract: The increasing realism of synthetic speech generated by advanced text-to-speech (TTS) models, coupled with post-processing and laundering techniques, presents a significant challenge for audio forensic detection. In this paper, we introduce the SAFE (Synthetic Audio Forensics Evaluation) Challenge, a fully blind evaluation framework designed to benchmark detection models across progressively harder scenarios: raw synthetic speech, processed audio (e.g., compression, resampling), and laundered audio intended to evade forensic analysis. The SAFE challenge consisted of a total of 90 hours of audio and 21,000 audio samples split across 21 different real sources and 17 different TTS models and 3 tasks. We present the challenge, evaluation design and tasks, dataset details, and initial insights into the strengths and limitations of current approaches, offering a foundation for advancing synthetic audio detection research. More information is available at \href{https://stresearch.github.io/SAFE/}{https://stresearch.github.io/SAFE/}.

Method: Proposed Siren framework using multiple isolated transformers with causal conditioning and anti-causal alignment via reinforcement learning to address RVQ limitations.

Result: Extensive experiments show Siren outperforms both existing LM-based and diffusion-based T2A systems, achieving state-of-the-art results.

Conclusion: Siren bridges LM representational strengths with audio synthesis fidelity demands, repositioning LMs as competitive contenders against diffusion models in T2A tasks and enabling unified multi-modal generation frameworks.

Abstract: While language models (LMs) paired with residual vector quantization (RVQ) tokenizers have shown promise in text-to-audio (T2A) generation, they still lag behind diffusion-based models by a non-trivial margin. We identify a critical dilemma underpinning this gap: incorporating more RVQ layers improves audio reconstruction fidelity but exceeds the generation capacity of conventional LMs. To address this, we first analyze RVQ dynamics and uncover two key limitations: 1) orthogonality of features across RVQ layers hinders effective LMs training, and 2) descending semantic richness in tokens from deeper RVQ layers exacerbates exposure bias during autoregressive decoding. Based on these insights, we propose Siren, a novel LM-based framework that employs multiple isolated transformers with causal conditioning and anti-causal alignment via reinforcement learning. Extensive experiments demonstrate that Siren outperforms both existing LM-based and diffusion-based T2A systems, achieving state-of-the-art results. By bridging the representational strengths of LMs with the fidelity demands of audio synthesis, our approach repositions LMs as competitive contenders against diffusion models in T2A tasks. Moreover, by aligning audio representations with linguistic structures, Siren facilitates a promising pathway toward unified multi-modal generation frameworks.

Method: Combines supervised contrastive fine-tuning of pre-trained models with contrastive representation distillation to structure embedding space and transfer knowledge to compact student models.

Result: Shows improved few-shot adaptation to unseen categories while maintaining strong closed-set classification performance.

Conclusion: ContrastASC provides a practical solution for edge devices to handle evolving acoustic scene categories without requiring model retraining.

Abstract: Acoustic scene classification (ASC) models on edge devices typically operate under fixed class assumptions, lacking the transferability needed for real-world applications that require adaptation to new or refined acoustic categories. We propose ContrastASC, which learns generalizable acoustic scene representations by structuring the embedding space to preserve semantic relationships between scenes, enabling adaptation to unseen categories without retraining. Our approach combines supervised contrastive fine-tuning of pre-trained models with contrastive representation distillation to transfer this structured knowledge to compact student models. Our evaluation shows that ContrastASC demonstrates improved few-shot adaptation to unseen categories while maintaining strong closed-set performance.

Method: Spectral decomposition of time-domain signals followed by multi-branch audio codec operating on decomposed components.

Result: Better reconstruction and perceptual performance than state-of-the-art baseline, with advantages for inpainting tasks.

Conclusion: The proposed approach provides effective disentangled audio representations with improved performance and task versatility.

Abstract: In neural-based audio feature extraction, ensuring that representations capture disentangled information is crucial for model interpretability. However, existing disentanglement methods often rely on assumptions that are highly dependent on data characteristics or specific tasks. In this work, we introduce a generalizable approach for learning disentangled features within a neural architecture. Our method applies spectral decomposition to time-domain signals, followed by a multi-branch audio codec that operates on the decomposed components. Empirical evaluations demonstrate that our approach achieves better reconstruction and perceptual performance compared to a state-of-the-art baseline while also offering potential advantages for inpainting tasks.

Method: Leverages spectral decomposition of time-domain signals to impose structure on extracted tokens in a disentangled neural audio codec.

Result: Experimental evaluations show the method surpasses state-of-the-art baseline in both reconstruction fidelity and perceptual quality.

Conclusion: The proposed spectral decomposition-based approach effectively enhances interpretability while maintaining high audio quality.

Abstract: While neural-based models have led to significant advancements in audio feature extraction, the interpretability of the learned representations remains a critical challenge. To address this, disentanglement techniques have been integrated into discrete neural audio codecs to impose structure on the extracted tokens. However, these approaches often exhibit strong dependencies on specific datasets or task formulations. In this work, we propose a disentangled neural audio codec that leverages spectral decomposition of time-domain signals to enhance representation interpretability. Experimental evaluations demonstrate that our method surpasses a state-of-the-art baseline in both reconstruction fidelity and perceptual quality.

Method: Uses a denoising diffusion probabilistic model (DDPM) with a guidance mechanism where a lightweight helper model estimates noise distribution to guide the diffusion denoising process.

Result: Consistent improvements over state-of-the-art baselines under mismatched noise conditions, showing better adaptation to unseen noise types.

Conclusion: The guided diffusion approach enables robust speech enhancement by effectively leveraging pre-trained speech generation models and adapting to various noise conditions through noise distribution estimation.

Abstract: This paper introduces a novel speech enhancement (SE) approach based on a denoising diffusion probabilistic model (DDPM), termed Guided diffusion for speech enhancement (GDiffuSE). In contrast to conventional methods that directly map noisy speech to clean speech, our method employs a lightweight helper model to estimate the noise distribution, which is then incorporated into the diffusion denoising process via a guidance mechanism. This design improves robustness by enabling seamless adaptation to unseen noise types and by leveraging large-scale DDPMs originally trained for speech generation in the context of SE. We evaluate our approach on noisy signals obtained by adding noise samples from the BBC sound effects database to LibriSpeech utterances, showing consistent improvements over state-of-the-art baselines under mismatched noise conditions. Examples are available at our project webpage.

Method: Proposes an adversarial-attack-based approach that fine-tunes a pre-trained speech emotion recognition model using only the data to be forgotten, without requiring other data.

Result: The approach effectively removes knowledge of forgotten data from the model while preserving high performance on emotion recognition test sets.

Conclusion: The proposed method provides an efficient solution for machine unlearning in speech emotion recognition that addresses privacy concerns without the computational burden of traditional approaches.

Abstract: Speech emotion recognition aims to identify emotional states from speech signals and has been widely applied in human-computer interaction, education, healthcare, and many other fields. However, since speech data contain rich sensitive information, partial data can be required to be deleted by speakers due to privacy concerns. Current machine unlearning approaches largely depend on data beyond the samples to be forgotten. However, this reliance poses challenges when data redistribution is restricted and demands substantial computational resources in the context of big data. We propose a novel adversarial-attack-based approach that fine-tunes a pre-trained speech emotion recognition model using only the data to be forgotten. The experimental results demonstrate that the proposed approach can effectively remove the knowledge of the data to be forgotten from the model, while preserving high model performance on the test set for emotion recognition.

Method: Two-stage training: first train a pitch-timbre disentangled 2D representation using Variational Autoencoder, then use this as conditioning for a Transformer-based generative model.

Result: The method effectively learns a disentangled timbre space enabling expressive and controllable audio generation with reliable pitch conditioning, capturing subtle timbre variations while maintaining pitch accuracy.

Conclusion: The approach provides an intuitive interface for sound navigation and represents a step towards future music production environments that are both intuitive and creatively empowering.

Abstract: This paper presents a novel approach to neural instrument sound synthesis using a two-stage semi-supervised learning framework capable of generating pitch-accurate, high-quality music samples from an expressive timbre latent space. Existing approaches that achieve sufficient quality for music production often rely on high-dimensional latent representations that are difficult to navigate and provide unintuitive user experiences. We address this limitation through a two-stage training paradigm: first, we train a pitch-timbre disentangled 2D representation of audio samples using a Variational Autoencoder; second, we use this representation as conditioning input for a Transformer-based generative model. The learned 2D latent space serves as an intuitive interface for navigating and exploring the sound landscape. We demonstrate that the proposed method effectively learns a disentangled timbre space, enabling expressive and controllable audio generation with reliable pitch conditioning. Experimental results show the model’s ability to capture subtle variations in timbre while maintaining a high degree of pitch accuracy. The usability of our method is demonstrated in an interactive web application, highlighting its potential as a step towards future music production environments that are both intuitive and creatively empowering: https://pgesam.faresschulz.com

Method: Use LLMs to compute mean log-likelihood on discrete token sequences from SSL models with minimal domain cues that guide in-context learning.

Result: LLM-based scores correlate with automatic speech recognition performance and provide useful embeddings for speaker verification tasks.

Conclusion: LLMs can effectively evaluate SSL models without extra training and also serve as inference-time embedding providers for downstream tasks.

Abstract: Self-Supervised Learning (SSL) has gained traction for its ability to learn rich representations with low labeling costs, applicable across diverse downstream tasks. However, assessing the downstream-task performance remains challenging due to the cost of extra training and evaluation. Existing methods for task-agnostic evaluation also require extra training or hyperparameter tuning. We propose a novel evaluation metric using large language models (LLMs). By inputting discrete token sequences and minimal domain cues derived from SSL models into LLMs, we obtain the mean log-likelihood; these cues guide in-context learning, rendering the score more reliable without extra training or hyperparameter tuning. Experimental results show a correlation between LLM-based scores and automatic speech recognition task. Additionally, our findings reveal that LLMs not only functions as an SSL evaluation tools but also provides inference-time embeddings that are useful for speaker verification task.

Method: Systematic experiments across five datasets (EmoMusic, DEAM, PMEmo, WTC, WCMED) using multiple data and feature sets, combining Jukebox model embeddings with chroma features, and training with diverse datasets.

Result: The proposed framework demonstrates substantially improved cross-dataset generalization capabilities by addressing genre dominance and dataset biases in feature representations.

Conclusion: A simple yet effective framework combining Jukebox embeddings with chroma features, trained on diverse datasets, significantly enhances cross-dataset generalization in Music Emotion Recognition.

Abstract: Music Emotion Recognition (MER) is a task deeply connected to human perception, relying heavily on subjective annotations collected from contributors. Prior studies tend to focus on specific musical styles rather than incorporating a diverse range of genres, such as rock and classical, within a single framework. In this paper, we address the task of recognizing emotion from audio content by investigating five datasets with dimensional emotion annotations – EmoMusic, DEAM, PMEmo, WTC, and WCMED – which span various musical styles. We demonstrate the problem of out-of-distribution generalization in a systematic experiment. By closely looking at multiple data and feature sets, we provide insight into genre-emotion relationships in existing data and examine potential genre dominance and dataset biases in certain feature representations. Based on these experiments, we arrive at a simple yet effective framework that combines embeddings extracted from the Jukebox model with chroma features and demonstrate how, alongside a combination of several diverse training sets, this permits us to train models with substantially improved cross-dataset generalization capabilities.

Method: Integrates Mamba (structured state-space model) for efficient audio sequence modeling with cross-attention mechanisms for precise text-acoustic alignment in an autoregressive architecture.

Result: Achieves state-of-the-art in speech editing and competitive zero-shot TTS, with 57.2% of listeners rating edited speech as perceptually equal to original. Requires ~6x less memory than VoiceCraft while maintaining similar latency.

Conclusion: MAVE establishes a new standard for flexible, high-fidelity voice editing and synthesis through synergistic integration of structured state-space modeling and cross-modal attention.

Abstract: We introduce MAVE (Mamba with Cross-Attention for Voice Editing and Synthesis), a novel autoregressive architecture for text-conditioned voice editing and high-fidelity text-to-speech (TTS) synthesis, built on a cross-attentive Mamba backbone. MAVE achieves state-of-the-art performance in speech editing and very competitive results in zero-shot TTS, while not being explicitly trained on the latter task, outperforming leading autoregressive and diffusion models on diverse, real-world audio. By integrating Mamba for efficient audio sequence modeling with cross-attention for precise text-acoustic alignment, MAVE enables context-aware voice editing with exceptional naturalness and speaker consistency. In pairwise human evaluations on a random 40-sample subset of the RealEdit benchmark (400 judgments), 57.2% of listeners rated MAVE - edited speech as perceptually equal to the original, while 24.8% prefered the original and 18.0% MAVE - demonstrating that in the majority of cases edits are indistinguishable from the source. MAVE compares favorably with VoiceCraft and FluentSpeech both on pairwise comparisons and standalone mean opinion score (MOS) evaluations. For zero-shot TTS, MAVE exceeds VoiceCraft in both speaker similarity and naturalness, without requiring multiple inference runs or post-processing. Remarkably, these quality gains come with a significantly lower memory cost and approximately the same latency: MAVE requires ~6x less memory than VoiceCraft during inference on utterances from the RealEdit database (mean duration: 6.21s, A100, FP16, batch size 1). Our results demonstrate that MAVE establishes a new standard for flexible, high-fidelity voice editing and synthesis through the synergistic integration of structured state-space modeling and cross-modal attention.

Method: Uses a flow-matching generative training paradigm to learn audio transformations from degraded to mastered versions. Trained on SonicMaster dataset containing paired degraded and high-quality tracks simulated with 19 degradation functions across 5 enhancement groups: equalization, dynamics, reverb, amplitude, and stereo.

Result: Objective audio quality metrics show significant improvement across all artifact categories. Subjective listening tests confirm listeners prefer SonicMaster’s enhanced outputs over original degraded audio.

Conclusion: SonicMaster provides an effective unified approach for music restoration and mastering that can be controlled via natural language instructions or operate automatically, demonstrating superior performance over degraded audio inputs.

Abstract: Music recordings often suffer from audio quality issues such as excessive reverberation, distortion, clipping, tonal imbalances, and a narrowed stereo image, especially when created in non-professional settings without specialized equipment or expertise. These problems are typically corrected using separate specialized tools and manual adjustments. In this paper, we introduce SonicMaster, the first unified generative model for music restoration and mastering that addresses a broad spectrum of audio artifacts with text-based control. SonicMaster is conditioned on natural language instructions to apply targeted enhancements, or can operate in an automatic mode for general restoration. To train this model, we construct the SonicMaster dataset, a large dataset of paired degraded and high-quality tracks by simulating common degradation types with nineteen degradation functions belonging to five enhancements groups: equalization, dynamics, reverb, amplitude, and stereo. Our approach leverages a flow-matching generative training paradigm to learn an audio transformation that maps degraded inputs to their cleaned, mastered versions guided by text prompts. Objective audio quality metrics demonstrate that SonicMaster significantly improves sound quality across all artifact categories. Furthermore, subjective listening tests confirm that listeners prefer SonicMaster’s enhanced outputs over the original degraded audio, highlighting the effectiveness of our unified approach.

Method: Developed a base stereo audio generation model, created a synthetic data pipeline combining video analysis, object tracking, and audio synthesis with dynamic panning and distance-based loudness controls, then fine-tuned the base model on this synthetic dataset.