2025 04 12

EnDive: A Cross-Dialect Benchmark for Fairness and Performance in Large Language Models

Abhay Gupta,Jacob Cheung,Philip Meng,Shayan Sayyed,Austen Liao,Kevin Zhu,Sean O'Brien

Task: 评估五种大型语言模型在非标准英语方言上的表现。

Motivation: 现有NLP基准测试忽视语言内部多样性，导致非标准方言使用者未被充分服务。

Details

Method: 通过少样本提示将标准英语数据集翻译为五种非标准方言，并与基于规则的方法对比，评估翻译质量。 Result: 翻译质量高（平均得分6.02/7），但模型在方言输入上表现显著低于标准英语。 Conclusion: EnDive揭示了模型偏见，推动了更具包容性的语言技术发展。 Abstract: The diversity of human language, shaped by social, cultural, and regional influences, presents significant challenges for natural language processing (NLP) systems. Existing benchmarks often overlook intra-language variations, leaving speakers of non-standard dialects underserved. To address this gap, we introduce EnDive (English Diversity), a benchmark that evaluates five widely-used large language models (LLMs) across tasks in language understanding, algorithmic reasoning, mathematics, and logic. Our framework translates Standard American English datasets into five underrepresented dialects using few-shot prompting with verified examples from native speakers, and compare these translations against rule-based methods via fluency assessments, preference tests, and semantic similarity metrics. Human evaluations confirm high translation quality, with average scores of at least 6.02/7 for faithfulness, fluency, and formality. By filtering out near-identical translations, we create a challenging dataset that reveals significant performance disparities - models consistently underperform on dialectal inputs compared to Standard American English. EnDive thus advances dialect-aware NLP by uncovering model biases and promoting more equitable language technologies.

How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities

Aly M. Kassem,Bernhard Schölkopf,Zhijing Jin

Task: 提出DSC基准测试框架，用于评估大型语言模型（LLM）路由器的性能，涵盖多种查询类型及隐私安全风险。

Motivation: 当前评估基准过于关注通用模型能力，忽视任务特异性行为及隐私安全等关键问题。

Details

Method: 设计DSC基准测试框架，分类评估路由器性能，并整合隐私安全评估。 Result: 实验表明，偏好数据驱动的路由器虽提升效率，但常做出次优决策，如将复杂查询过度分配给强大模型，而将安全风险查询分配给弱模型。 Conclusion: DSC基准揭示了当前路由器的局限性，强调了任务特异性和安全评估的重要性。 Abstract: Large language model (LLM) routing has emerged as a crucial strategy for balancing computational costs with performance by dynamically assigning queries to the most appropriate model based on query complexity. Despite recent advances showing that preference-data-based routers can outperform traditional methods, current evaluation benchmarks remain limited. They largely focus on general model capabilities while overlooking task-specific behaviors and critical concerns such as privacy, safety, and potential backdoor vulnerabilities introduced through preference data. In response, we propose the DSC benchmark: Diverse, Simple, and Categorized, an evaluation framework that categorizes router performance across a broad spectrum of query types, including coding, translation, mathematics, human instructions, general knowledge, and LLM jailbreaking. Additionally, it integrates privacy and safety assessments to reveal hidden risks. Our experiments on three preference-based routers and two commercial counterparts demonstrate that while these systems improve efficiency, they often make suboptimal, category-driven decisions. For instance, a BERT-based router directs all coding and mathematics queries to the most powerful LLM even when simpler models would suffice, while routing jailbreaking attempts to weaker models, thereby elevating safety risks.

ChatBench: From Static Benchmarks to Human-AI Evaluation

Serina Chang,Ashton Anderson,Jake M. Hofman

Task: 评估人类与大型语言模型（LLM）协作的能力，并设计ChatBench数据集以比较AI单独、用户单独及用户-AI协作的表现。

Motivation: 现有标准基准（如MMLU）仅评估LLM的独立能力，无法反映实际应用中用户与LLM的协作效果。

Details

Method: 通过用户研究将MMLU问题转化为用户-AI对话，收集并分析AI单独、用户单独及用户-AI协作的数据。 Result: 发现AI单独准确性无法预测用户-AI协作准确性，且在不同学科（如数学、物理、道德推理）中存在显著差异；通过微调用户模拟器，提高了对用户-AI协作准确性的估计能力。 Conclusion: ChatBench数据集为交互式评估提供了新工具，揭示了用户与LLM协作的复杂性，并为未来研究提供了方向。 Abstract: With the rapid adoption of LLM-based chatbots, there is a pressing need to evaluate what humans and LLMs can achieve together. However, standard benchmarks, such as MMLU, measure LLM capabilities in isolation (i.e., "AI-alone"). Here, we design and conduct a user study to convert MMLU questions into user-AI conversations, by seeding the user with the question and having them carry out a conversation with the LLM to answer their question. We release ChatBench, a new dataset with AI-alone, user-alone, and user-AI data for 396 questions and two LLMs, including 144K answers and 7,336 user-AI conversations. We find that AI-alone accuracy fails to predict user-AI accuracy, with significant differences across multiple subjects (math, physics, and moral reasoning), and we analyze the user-AI conversations to provide insight into how they diverge from AI-alone benchmarks. Finally, we show that fine-tuning a user simulator on a subset of ChatBench improves its ability to estimate user-AI accuracies, increasing correlation on held-out questions by more than 20 points, creating possibilities for scaling interactive evaluation.

EqualizeIR: Mitigating Linguistic Biases in Retrieval Models

Jiali Cheng,Hadi Amiri

Task: 提出EqualizeIR框架以减轻信息检索模型中的语言偏见。

Motivation: 现有信息检索模型在语言复杂度不同的查询上表现不均，存在显著偏见。

Details

Method: 使用语言偏见弱学习器捕获数据集中的偏见，并通过正则化和优化预测训练鲁棒模型。 Result: 实验表明，该方法减少了语言简单和复杂查询间的性能差异，并提升了整体检索性能。 Conclusion: EqualizeIR框架能有效减轻语言偏见，提升信息检索模型的公平性和性能。 Abstract: This study finds that existing information retrieval (IR) models show significant biases based on the linguistic complexity of input queries, performing well on linguistically simpler (or more complex) queries while underperforming on linguistically more complex (or simpler) queries. To address this issue, we propose EqualizeIR, a framework to mitigate linguistic biases in IR models. EqualizeIR uses a linguistically biased weak learner to capture linguistic biases in IR datasets and then trains a robust model by regularizing and refining its predictions using the biased weak learner. This approach effectively prevents the robust model from overfitting to specific linguistic patterns in data. We propose four approaches for developing linguistically-biased models. Extensive experiments on several datasets show that our method reduces performance disparities across linguistically simple and complex queries, while improving overall retrieval performance.

Perception in Reflection

Yana Wei,Liang Zhao,Kangheng Lin,En Yu,Yuang Peng,Runpei Dong,Jianjian Sun,Haoran Wei,Zheng Ge,Xiangyu Zhang,Vishal M. Patel

Task: 提出一种反射感知范式（RePer），通过双模型反射机制迭代优化视觉感知能力。

Motivation: 当前大型视觉语言模型（LVLMs）在初始感知上存在局限性，难以实现完美感知。

Details

Method: 采用双模型反射机制（政策模型与批评模型交替），并结合反射感知学习（RPL）方法，通过视觉反射数据集和反射非似然训练增强反射能力。 Result: 实验表明，RePer在图像理解、标题生成精度和幻觉减少方面有显著提升，且模型注意力模式与人类视觉焦点高度对齐。 Conclusion: 反射感知范式为未来多模态智能体提供了稳健的框架，尤其适用于复杂推理和多步操作任务。 Abstract: We present a perception in reflection paradigm designed to transcend the limitations of current large vision-language models (LVLMs), which are expected yet often fail to achieve perfect perception initially. Specifically, we propose Reflective Perception (RePer), a dual-model reflection mechanism that systematically alternates between policy and critic models, enables iterative refinement of visual perception. This framework is powered by Reflective Perceptual Learning (RPL), which reinforces intrinsic reflective capabilities through a methodically constructed visual reflection dataset and reflective unlikelihood training. Comprehensive experimental evaluation demonstrates RePer's quantifiable improvements in image understanding, captioning precision, and hallucination reduction. Notably, RePer achieves strong alignment between model attention patterns and human visual focus, while RPL optimizes fine-grained and free-form preference alignment. These advancements establish perception in reflection as a robust paradigm for future multimodal agents, particularly in tasks requiring complex reasoning and multi-step manipulation.

CLEAR: Contrasting Textual Feedback with Experts and Amateurs for Reasoning

Andrew Rufail,Daniel Kim,Sean O'Brien,Kevin Zhu

Task: 提出一种名为CLEAR的新方法，通过对比专家模型和业余模型的反馈来改进语言模型的推理能力。

Motivation: 利用专家模型和业余模型的优势，通过对比反馈迭代优化语言模型的推理能力。

Details

Method: CLEAR方法通过对比专家模型和业余模型对初始输出的反馈，生成更精细的反馈，并迭代改进响应。 Result: CLEAR在多个推理任务中表现优于现有方法，包括故事大纲改进（趣味性提升19.6%）、受限生成（覆盖率提升18.5%）、数学推理（准确率提升6.7%）和毒性缓解（毒性降低22%）。 Conclusion: CLEAR通过对比专家和业余模型的反馈，显著提升了语言模型的推理能力，并在多个任务中取得了显著改进。 Abstract: We introduce CLEAR (Contrasting Textual Feedback with Experts and Amateurs for Reasoning), a novel approach to language model reasoning that leverages the strengths of a larger (expert) model and smaller (amateur) model. The expert and amateur models each provide feedback on a model's initial output and are contrasted with each other into refined feedback. This feedback is subsequently applied to iteratively improve CLEAR's responses. Our experiments demonstrate that CLEAR outperforms state-of-the-art methods in several challenging reasoning tasks, including story outline improvement (up to 19.6% relative increase in interestingness), constrained generation (up to 18.5% increase in coverage), mathematical reasoning (up to 6.7% improvement in accuracy) and mitigation of toxicity (decrease of up to 22% in toxicity).

Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuning

Ashutosh Chaubey,Xulang Guan,Mohammad Soleymani

Task: 提出Face-LLaVA，一种用于人脸中心多模态任务的多模态大语言模型。

Motivation: 人脸在社交交流中至关重要，需要高性能计算机视觉工具支持人本应用。

Details

Method: 开发FaceInstruct-1M数据库，并设计基于Face-Region Guided Cross-Attention的人脸视觉编码器。 Result: 在九大数据集和五项任务中表现优于开源模型，与商业方案竞争，且生成描述在零样本设置下获得更高推理评分。 Conclusion: Face-LLaVA在社交AI和基础视觉语言研究中具有潜力，数据集和模型将开源。 Abstract: The human face plays a central role in social communication, necessitating the use of performant computer vision tools for human-centered applications. We propose Face-LLaVA, a multimodal large language model for face-centered, in-context learning, including facial expression and attribute recognition. Additionally, Face-LLaVA is able to generate natural language descriptions that can be used for reasoning. Leveraging existing visual databases, we first developed FaceInstruct-1M, a face-centered database for instruction tuning MLLMs for face processing. We then developed a novel face-specific visual encoder powered by Face-Region Guided Cross-Attention that integrates face geometry with local visual features. We evaluated the proposed method across nine different datasets and five different face processing tasks, including facial expression recognition, action unit detection, facial attribute detection, age estimation and deepfake detection. Face-LLaVA achieves superior results compared to existing open-source MLLMs and competitive performance compared to commercial solutions. Our model output also receives a higher reasoning rating by GPT under a zero-shot setting across all the tasks. Both our dataset and model wil be released at https://face-llava.github.io to support future advancements in social AI and foundational vision-language research.

DeepSeek-R1 Thoughtology: Let's about LLM Reasoning

Sara Vera Marjanović,Arkil Patel,Vaibhav Adlakha,Milad Aghajohari,Parishad BehnamGhader,Mehar Bhatia,Aditi Khandelwal,Austin Kraft,Benno Krojer,Xing Han Lù,Nicholas Meade,Dongchan Shin,Amirhossein Kazemnejad,Gaurav Kamath,Marius Mosbach,Karolina Stańczak,Siva Reddy

Task: 研究DeepSeek-R1模型的多步推理行为及其影响。

Motivation: 探索大型推理模型在复杂问题中的表现，尤其是推理链的公开性为研究模型行为提供了新机会。

Details

Method: 通过分类DeepSeek-R1的基本推理模块，分析其推理长度、上下文管理、文化安全问题和认知现象。 Result: 发现DeepSeek-R1存在推理的‘最佳点’，过长推理会降低性能；模型容易陷入重复思考；安全性较弱。 Conclusion: DeepSeek-R1的推理行为复杂，需进一步优化以提升性能和安全性。 Abstract: Large Reasoning Models like DeepSeek-R1 mark a fundamental shift in how LLMs approach complex problems. Instead of directly producing an answer for a given input, DeepSeek-R1 creates detailed multi-step reasoning chains, seemingly "thinking" about a problem before providing an answer. This reasoning process is publicly available to the user, creating endless opportunities for studying the reasoning behaviour of the model and opening up the field of Thoughtology. Starting from a taxonomy of DeepSeek-R1's basic building blocks of reasoning, our analyses on DeepSeek-R1 investigate the impact and controllability of thought length, management of long or confusing contexts, cultural and safety concerns, and the status of DeepSeek-R1 vis-\`a-vis cognitive phenomena, such as human-like language processing and world modelling. Our findings paint a nuanced picture. Notably, we show DeepSeek-R1 has a 'sweet spot' of reasoning, where extra inference time can impair model performance. Furthermore, we find a tendency for DeepSeek-R1 to persistently ruminate on previously explored problem formulations, obstructing further exploration. We also note strong safety vulnerabilities of DeepSeek-R1 compared to its non-reasoning counterpart, which can also compromise safety-aligned LLMs.

Few-Shot Adaptation of Grounding DINO for Agricultural Domain

Rajhans Singh,Rafael Bidese Puhl,Kshitiz Dhakal,Sudhir Sornapudi

Task: 提出一种高效的少样本适应方法，改进Grounding-DINO架构以解决农业应用中复杂对象的检测问题。

Motivation: 深度学习模型在农业应用中需要大量标注数据，而现有方法在复杂对象检测上存在挑战。

Details

Method: 移除Grounding-DINO的文本编码器模块（BERT），引入随机初始化的可训练文本嵌入。 Result: 在多个农业数据集上表现优异，比完全微调的YOLO模型mAP提高约24%，在少样本条件下优于现有方法约10%。 Conclusion: 该方法为自动化标注和加速农业AI解决方案的开发提供了有前景的解决方案。 Abstract: Deep learning models are transforming agricultural applications by enabling automated phenotyping, monitoring, and yield estimation. However, their effectiveness heavily depends on large amounts of annotated training data, which can be labor and time intensive. Recent advances in open-set object detection, particularly with models like Grounding-DINO, offer a potential solution to detect regions of interests based on text prompt input. Initial zero-shot experiments revealed challenges in crafting effective text prompts, especially for complex objects like individual leaves and visually similar classes. To address these limitations, we propose an efficient few-shot adaptation method that simplifies the Grounding-DINO architecture by removing the text encoder module (BERT) and introducing a randomly initialized trainable text embedding. This method achieves superior performance across multiple agricultural datasets, including plant-weed detection, plant counting, insect identification, fruit counting, and remote sensing tasks. Specifically, it demonstrates up to a $\sim24\%$ higher mAP than fully fine-tuned YOLO models on agricultural datasets and outperforms previous state-of-the-art methods by $\sim10\%$ in remote sensing, under few-shot learning conditions. Our method offers a promising solution for automating annotation and accelerating the development of specialized agricultural AI solutions.

HypoEval: Hypothesis-Guided Evaluation for Natural Language Generation

Mingxuan Li,Hanchen Li,Chenhao Tan

Task: 提出HypoEval框架，用于自动化评估自然语言生成任务。

Motivation: 现有LLM评估框架存在对齐性低或需要大量标注数据的问题，且缺乏解释性。

Details

Method: 利用少量人工评估生成详细评分标准，结合LLM分解维度评分。 Result: 仅需30次人工评估，HypoEval在Spearman和Pearson相关性上优于现有方法。 Conclusion: HypoEval是一种可靠且可解释的自动化评估框架。 Abstract: Large language models (LLMs) have demonstrated great potential for automating the evaluation of natural language generation. Previous frameworks of LLM-as-a-judge fall short in two ways: they either use zero-shot setting without consulting any human input, which leads to low alignment, or fine-tune LLMs on labeled data, which requires a non-trivial number of samples. Moreover, previous methods often provide little reasoning behind automated evaluations. In this paper, we propose HypoEval, Hypothesis-guided Evaluation framework, which first uses a small corpus of human evaluations to generate more detailed rubrics for human judgments and then incorporates a checklist-like approach to combine LLM's assigned scores on each decomposed dimension to acquire overall scores. With only 30 human evaluations, HypoEval achieves state-of-the-art performance in alignment with both human rankings (Spearman correlation) and human scores (Pearson correlation), on average outperforming G-Eval by 11.86% and fine-tuned Llama-3.1-8B-Instruct with at least 3 times more human evaluations by 11.95%. Furthermore, we conduct systematic studies to assess the robustness of HypoEval, highlighting its effectiveness as a reliable and interpretable automated evaluation framework.

Quantifying Epistemic Uncertainty in Absolute Pose Regression

Fereidoon Zangeneh,Amit Dekel,Alessandro Pieropan,Patric Jensfelt

Task: 通过神经网络直接回归相机姿态的任务。

Motivation: 绝对姿态回归在内存和计算效率上具有吸引力，但其预测在训练域外不准确且不可靠。

Details

Method: 提出了一种通过变分框架估计观测可能性的新方法，量化绝对姿态回归模型的认知不确定性。 Result: 该方法在捕捉不确定性与预测误差之间的关系上优于现有方法。 Conclusion: 该方法不仅提供了预测的置信度度量，还能在存在重复结构时概率性地定位相机。 Abstract: Visual relocalization is the task of estimating the camera pose given an image it views. Absolute pose regression offers a solution to this task by training a neural network, directly regressing the camera pose from image features. While an attractive solution in terms of memory and compute efficiency, absolute pose regression's predictions are inaccurate and unreliable outside the training domain. In this work, we propose a novel method for quantifying the epistemic uncertainty of an absolute pose regression model by estimating the likelihood of observations within a variational framework. Beyond providing a measure of confidence in predictions, our approach offers a unified model that also handles observation ambiguities, probabilistically localizing the camera in the presence of repetitive structures. Our method outperforms existing approaches in capturing the relation between uncertainty and prediction error.

SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog

Jennifer D'Souza,Sameer Sadruddin,Holger Israel,Mathias Begoin,Diana Slawig

Task: 自动化主题标注，使用GND分类法为英文和德文的科学与技术记录推荐前k个主题。

Motivation: 探索如何利用大语言模型（LLMs）改进数字图书馆的分类系统。

Details

Method: 参与者开发基于LLM的系统，通过定量指标（精确率、召回率、F1分数）和主题专家的定性评估进行测试。 Result: 结果表明，LLM集成、合成数据生成和多语言处理在主题标注中表现有效。 Conclusion: 研究为LLM在数字图书馆分类中的应用提供了有价值的见解。 Abstract: We present SemEval-2025 Task 5: LLMs4Subjects, a shared task on automated subject tagging for scientific and technical records in English and German using the GND taxonomy. Participants developed LLM-based systems to recommend top-k subjects, evaluated through quantitative metrics (precision, recall, F1-score) and qualitative assessments by subject specialists. Results highlight the effectiveness of LLM ensembles, synthetic data generation, and multilingual processing, offering insights into applying LLMs for digital library classification.

CEC-MMR: Cross-Entropy Clustering Approach to Multi-Modal Regression

Krzysztof Byrski,Jacek Tabor,Przemysław Spurek,Marcin Mazur

Task: 提出一种基于交叉熵聚类（CEC）的新方法CEC-MMR，用于自动检测回归问题中的组件数量。

Motivation: 传统混合密度网络（MDN）无法准确确定组件数量，导致预测值与实际数据差异较大。

Details

Method: 使用交叉熵聚类（CEC）自动检测组件数量，并能唯一识别属性值对应的组件。 Result: 实验结果表明，CEC-MMR优于传统MDN方法。 Conclusion: CEC-MMR是一种有效解决回归问题中组件数量检测的方法。 Abstract: In practical applications of regression analysis, it is not uncommon to encounter a multitude of values for each attribute. In such a situation, the univariate distribution, which is typically Gaussian, is suboptimal because the mean may be situated between modes, resulting in a predicted value that differs significantly from the actual data. Consequently, to address this issue, a mixture distribution with parameters learned by a neural network, known as a Mixture Density Network (MDN), is typically employed. However, this approach has an important inherent limitation, in that it is not feasible to ascertain the precise number of components with a reasonable degree of accuracy. In this paper, we introduce CEC-MMR, a novel approach based on Cross-Entropy Clustering (CEC), which allows for the automatic detection of the number of components in a regression problem. Furthermore, given an attribute and its value, our method is capable of uniquely identifying it with the underlying component. The experimental results demonstrate that CEC-MMR yields superior outcomes compared to classical MDNs.

ConceptCarve: Dynamic Realization of Evidence

Eylon Caplan,Dan Goldwasser

Task: 开发一个名为ConceptCarve的证据检索框架，用于在大规模社交媒体数据中识别抽象概念实例及其在不同社区中的不同表现形式。

Motivation: 研究人类观点和行为需要理解社交媒体上复杂的思想模式，尤其是在涉及抽象概念（如自由与枪支所有权的关系）时，传统检索系统难以应对。

Details

Method: 结合传统检索器和大型语言模型（LLMs），动态表征检索空间，以识别抽象概念及其社区特定实例。 Result: ConceptCarve在社交媒体社区中检索证据的效果优于传统系统，并能生成可解释的证据表示。 Conclusion: ConceptCarve为分析不同社区中复杂思想模式提供了有效工具，具有可解释性和扩展性。 Abstract: Finding evidence for human opinion and behavior at scale is a challenging task, often requiring an understanding of sophisticated thought patterns among vast online communities found on social media. For example, studying how gun ownership is related to the perception of Freedom, requires a retrieval system that can operate at scale over social media posts, while dealing with two key challenges: (1) identifying abstract concept instances, (2) which can be instantiated differently across different communities. To address these, we introduce ConceptCarve, an evidence retrieval framework that utilizes traditional retrievers and LLMs to dynamically characterize the search space during retrieval. Our experiments show that ConceptCarve surpasses traditional retrieval systems in finding evidence within a social media community. It also produces an interpretable representation of the evidence for that community, which we use to qualitatively analyze complex thought patterns that manifest differently across the communities.

Objaverse++: Curated 3D Object Dataset with Quality Annotations

Chendi Lin,Heshan Liu,Qunshu Lin,Zachary Bright,Shitao Tang,Yihui He,Minghao Liu,Ling Zhu,Cindy Le

Task: 提出并验证了Objaverse++，一个经过人工专家详细标注的高质量3D模型子集，用于提升3D内容生成的性能。

Motivation: 尽管Objaverse是目前最大的3D资产数据集，但其低质量模型占主导地位，限制了其实际应用。

Details

Method: 人工标注10,000个3D对象的详细属性，并训练神经网络为剩余数据集自动标注标签。 Result: 实验表明，基于高质量子集预训练的模型在图像到3D生成任务中表现更优，且数据质量越高，训练损失收敛越快。 Conclusion: 精心筛选和丰富标注可以弥补数据集规模的不足，为3D生成模型的开发提供更高效的路径。 Abstract: This paper presents Objaverse++, a curated subset of Objaverse enhanced with detailed attribute annotations by human experts. Recent advances in 3D content generation have been driven by large-scale datasets such as Objaverse, which contains over 800,000 3D objects collected from the Internet. Although Objaverse represents the largest available 3D asset collection, its utility is limited by the predominance of low-quality models. To address this limitation, we manually annotate 10,000 3D objects with detailed attributes, including aesthetic quality scores, texture color classifications, multi-object composition flags, transparency characteristics, etc. Then, we trained a neural network capable of annotating the tags for the rest of the Objaverse dataset. Through experiments and a user study on generation results, we demonstrate that models pre-trained on our quality-focused subset achieve better performance than those trained on the larger dataset of Objaverse in image-to-3D generation tasks. In addition, by comparing multiple subsets of training data filtered by our tags, our results show that the higher the data quality, the faster the training loss converges. These findings suggest that careful curation and rich annotation can compensate for the raw dataset size, potentially offering a more efficient path to develop 3D generative models. We release our enhanced dataset of approximately 500,000 curated 3D models to facilitate further research on various downstream tasks in 3D computer vision. In the near future, we aim to extend our annotations to cover the entire Objaverse dataset.

Visual-Aware Speech Recognition for Noisy Scenarios

Lakshmipathi Balaji,Karan Singla

Task: 提出一种通过关联噪声源与视觉线索来改进嘈杂环境中语音转录的模型。

Motivation: 当前自动语音识别（ASR）或视听语音识别（AVSR）模型在嘈杂环境中表现不佳，而人类能利用视觉线索（如唇部动作和场景）增强听觉感知。

Details

Method: 利用预训练的语音和视觉编码器，通过多头注意力机制链接，从环境中提取更广泛的视觉信息，以过滤噪声并改进转录。 Result: 在嘈杂场景中显著优于现有纯音频模型，视觉线索对转录准确性的提升起到关键作用。 Conclusion: 通过结合视觉信息，模型能够更自然地过滤噪声并提升转录效果，类似于人类在嘈杂环境中的表现。 Abstract: Humans have the ability to utilize visual cues, such as lip movements and visual scenes, to enhance auditory perception, particularly in noisy environments. However, current Automatic Speech Recognition (ASR) or Audio-Visual Speech Recognition (AVSR) models often struggle in noisy scenarios. To solve this task, we propose a model that improves transcription by correlating noise sources to visual cues. Unlike works that rely on lip motion and require the speaker's visibility, we exploit broader visual information from the environment. This allows our model to naturally filter speech from noise and improve transcription, much like humans do in noisy scenarios. Our method re-purposes pretrained speech and visual encoders, linking them with multi-headed attention. This approach enables the transcription of speech and the prediction of noise labels in video inputs. We introduce a scalable pipeline to develop audio-visual datasets, where visual cues correlate to noise in the audio. We show significant improvements over existing audio-only models in noisy scenarios. Results also highlight that visual cues play a vital role in improved transcription accuracy.

DLTPose: 6DoF Pose Estimation From Accurate Dense Surface Point Estimates

Akash Jadhav,Michael Greenspan

Task: 提出一种名为DLTPose的新方法，用于从RGB-D图像中估计6DoF物体姿态。

Motivation: 结合稀疏关键点方法的准确性和密集像素级预测的鲁棒性，解决对称物体姿态估计中的关键点分配不一致问题。

Details

Method: 预测每个像素到一组至少四个关键点的径向距离，并通过新的DLT公式生成准确的3D物体表面估计，同时引入对称感知的关键点排序方法。 Result: 在LINEMOD、Occlusion LINEMOD和YCB-Video数据集上表现优异，平均召回率分别为86.5%、79.7%和89.5%。 Conclusion: DLTPose在对称和遮挡物体上的姿态估计性能优于现有方法。 Abstract: We propose DLTPose, a novel method for 6DoF object pose estimation from RGB-D images that combines the accuracy of sparse keypoint methods with the robustness of dense pixel-wise predictions. DLTPose predicts per-pixel radial distances to a set of minimally four keypoints, which are then fed into our novel Direct Linear Transform (DLT) formulation to produce accurate 3D object frame surface estimates, leading to better 6DoF pose estimation. Additionally, we introduce a novel symmetry-aware keypoint ordering approach, designed to handle object symmetries that otherwise cause inconsistencies in keypoint assignments. Previous keypoint-based methods relied on fixed keypoint orderings, which failed to account for the multiple valid configurations exhibited by symmetric objects, which our ordering approach exploits to enhance the model's ability to learn stable keypoint representations. Extensive experiments on the benchmark LINEMOD, Occlusion LINEMOD and YCB-Video datasets show that DLTPose outperforms existing methods, especially for symmetric and occluded objects, demonstrating superior Mean Average Recall values of 86.5% (LM), 79.7% (LM-O) and 89.5% (YCB-V). The code is available at https://anonymous.4open.science/r/DLTPose_/ .

Language Modeling for the Future of Finance: A Quantitative Survey into Metrics, Tasks, and Data Opportunities

Nikita Tatarinov,Siddhant Sukhani,Agam Shah,Sudheer Chava

Task: 系统性地回顾和分析2017年至2024年间发表的374篇NLP研究论文，重点关注其中221篇直接涉及金融任务的论文。

Motivation: 探索NLP技术在金融问题中的应用趋势，为研究者和从业者提供结构化的概述和实践见解。

Details

Method: 通过11个定性和定量维度评估论文，识别关键趋势，如通用语言模型的增加使用、情感分析和信息提取的进展，以及可解释性和隐私保护方法的兴起。 Result: 发现需要更多可访问、适应性强的数据集，并强调纳入金融危机时期以增强模型在现实条件下的鲁棒性。 Conclusion: 本调查为NLP在金融领域的应用提供了结构化概述，并为相关研究者和从业者提供了实用见解。 Abstract: Recent advances in language modeling have led to growing interest in applying Natural Language Processing (NLP) techniques to financial problems, enabling new approaches to analysis and decision-making. To systematically examine this trend, we review 374 NLP research papers published between 2017 and 2024 across 38 conferences and workshops, with a focused analysis of 221 papers that directly address finance-related tasks. We evaluate these papers across 11 qualitative and quantitative dimensions, identifying key trends such as the increasing use of general-purpose language models, steady progress in sentiment analysis and information extraction, and emerging efforts around explainability and privacy-preserving methods. We also discuss the use of evaluation metrics, highlighting the importance of domain-specific ones to complement standard machine learning metrics. Our findings emphasize the need for more accessible, adaptive datasets and highlight the significance of incorporating financial crisis periods to strengthen model robustness under real-world conditions. This survey provides a structured overview of NLP research applied to finance and offers practical insights for researchers and practitioners working at this intersection.

Zeus: Zero-shot LLM Instruction for Union Segmentation in Multimodal Medical Imaging

Siyuan Dai,Kai Ye,Guodong Liu,Haoteng Tang,Liang Zhan

Task: 提出一种基于视觉-大语言模型（Vision-LLM）联合框架的多模态医学图像分割方法，无需预先收集配对的视觉-语言数据集。

Motivation: 临床诊断需要结合领域知识（如文本信息），但收集配对的视觉-语言数据集成本高昂且耗时。

Details

Method: 利用冻结的大语言模型（LLMs）基于医学图像生成零样本指令，模拟放射学扫描和报告生成过程，并通过多模态图像生成更精确的文本指令。 Result: 实验结果表明，该方法在多模态分割任务中优于现有基线方法。 Conclusion: 提出的Vision-LLM联合框架能够有效解决多模态医学图像分割问题，无需依赖预先收集的配对数据集。 Abstract: Medical image segmentation has achieved remarkable success through the continuous advancement of UNet-based and Transformer-based foundation backbones. However, clinical diagnosis in the real world often requires integrating domain knowledge, especially textual information. Conducting multimodal learning involves visual and text modalities shown as a solution, but collecting paired vision-language datasets is expensive and time-consuming, posing significant challenges. Inspired by the superior ability in numerous cross-modal tasks for Large Language Models (LLMs), we proposed a novel Vision-LLM union framework to address the issues. Specifically, we introduce frozen LLMs for zero-shot instruction generation based on corresponding medical images, imitating the radiology scanning and report generation process. {To better approximate real-world diagnostic processes}, we generate more precise text instruction from multimodal radiology images (e.g., T1-w or T2-w MRI and CT). Based on the impressive ability of semantic understanding and rich knowledge of LLMs. This process emphasizes extracting special features from different modalities and reunion the information for the ultimate clinical diagnostic. With generated text instruction, our proposed union segmentation framework can handle multimodal segmentation without prior collected vision-language datasets. To evaluate our proposed method, we conduct comprehensive experiments with influential baselines, the statistical results and the visualized case study demonstrate the superiority of our novel method.}

RAISE: Reinforenced Adaptive Instruction Selection For Large Language Models

Lv Qingsong,Yangning Li,Zihua Lan,Zishan Xu,Jiwei Tang,Yinghui Li,Wenhao Jiang,Hai-Tao Zheng,Philip S. Yu

Task: 提出一种动态、任务目标驱动的指令选择框架RAISE，以优化大型语言模型（LLM）的指令微调过程。

Motivation: 现有指令选择方法多基于启发式质量指标，且仅在训练前进行数据选择，导致指令微调优化不足，固定指标难以针对特定任务优化。

Details

Method: 设计RAISE框架，将整个指令微调过程纳入优化，基于指令对模型性能提升的预期影响动态选择指令，并通过强化学习训练选择策略。 Result: 实验证明RAISE优于其他指令选择方法，仅更新1%的训练步骤即可达到优于全数据训练的性能。 Conclusion: RAISE具有高效性和有效性，能够针对特定任务进行优化，且具备良好的可解释性。 Abstract: In the instruction fine-tuning of large language models (LLMs), it has become a consensus that a few high-quality instructions are superior to a large number of low-quality instructions. At present, many instruction selection methods have been proposed, but most of these methods select instruction based on heuristic quality metrics, and only consider data selection before training. These designs lead to insufficient optimization of instruction fine-tuning, and fixed heuristic indicators are often difficult to optimize for specific tasks. So we designed a dynamic, task-objective-driven instruction selection framework RAISE(Reinforenced Adaptive Instruction SElection), which incorporates the entire instruction fine-tuning process into optimization, selecting instruction at each step based on the expected impact of instruction on model performance improvement. Our approach is well interpretable and has strong task-specific optimization capabilities. By modeling dynamic instruction selection as a sequential decision-making process, we use RL to train our selection strategy. Extensive experiments and result analysis prove the superiority of our method compared with other instruction selection methods. Notably, RAISE achieves superior performance by updating only 1\% of the training steps compared to full-data training, demonstrating its efficiency and effectiveness.

View-Dependent Uncertainty Estimation of 3D Gaussian Splatting

Chenyu Han,Corentin Dumery

Task: 提出一种在3D高斯泼溅（3DGS）中建模不确定性的方法。

Motivation: 3DGS在3D场景重建中具有高视觉精度，但其不确定性估计尚未充分研究，而这对下游任务（如资产提取和场景补全）至关重要。

Details

Method: 将不确定性建模为额外的视角依赖的每高斯特征，并使用球谐函数表示。 Result: 该方法简单高效，易于解释，且比集成方法更快，同时保持高精度。 Conclusion: 提出的方法为3DGS场景的不确定性估计提供了一种有效的解决方案。 Abstract: 3D Gaussian Splatting (3DGS) has become increasingly popular in 3D scene reconstruction for its high visual accuracy. However, uncertainty estimation of 3DGS scenes remains underexplored and is crucial to downstream tasks such as asset extraction and scene completion. Since the appearance of 3D gaussians is view-dependent, the color of a gaussian can thus be certain from an angle and uncertain from another. We thus propose to model uncertainty in 3DGS as an additional view-dependent per-gaussian feature that can be modeled with spherical harmonics. This simple yet effective modeling is easily interpretable and can be integrated into the traditional 3DGS pipeline. It is also significantly faster than ensemble methods while maintaining high accuracy, as demonstrated in our experiments.

MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning

Yangning Li,Zihua Lan,Lv Qingsong,Yinghui Li,Hai-Tao Zheng

Task: 提出一种名为MDIT的模型无关数据插值方法，用于生成多样化的指令调优数据。

Motivation: 当前数据管理策略在生成多样化和全面数据方面面临挑战，限制了模型性能的进一步提升。

Details

Method: 通过任务插值生成多样化的高质量指令数据，并结合基于多样性的聚类策略。 Result: 在多个基准任务中表现出优越性能，显著提升了LLMs在问答、数学推理和代码生成等任务中的表现。 Conclusion: MDIT提供了一种高效且自动化的数据生成方法，无需依赖外部资源，同时扩展了LLMs在复杂环境中的应用潜力。 Abstract: As Large Language Models (LLMs) are increasingly applied across various tasks, instruction tuning has emerged as a critical method for enhancing model performance. However, current data management strategies face substantial challenges in generating diverse and comprehensive data, restricting further improvements in model performance. To address this gap, we propose MDIT, a novel model-free data interpolation method for diverse instruction tuning, which generates varied and high-quality instruction data by performing task interpolation. Moreover, it contains diversity-based clustering strategies to ensure the diversity of the training data. Extensive experiments show that our method achieves superior performance in multiple benchmark tasks. The LLMs finetuned with MDIT show significant improvements in numerous tasks such as general question answering, math reasoning, and code generation. MDIT offers an efficient and automatic data synthetic method, generating diverse instruction data without depending on external resources while expanding the application potential of LLMs in complex environments.

Novel Diffusion Models for Multimodal 3D Hand Trajectory Prediction

Junyi Ma,Wentao Bao,Jingyi Xu,Guanzhong Sun,Xieyuanli Chen,Hesheng Wang

Task: 预测未来3D手部轨迹，结合多模态环境信息。

Motivation: 现有方法仅支持2D视频输入，缺乏对多模态环境信息的利用，且未充分结合头部相机运动信息。

Details

Method: 提出MMTwin扩散模型，吸收2D RGB图像、3D点云、过去手部轨迹和文本提示，集成两个潜在扩散模型预测相机运动和手部轨迹。 Result: 在三个公开数据集和自录数据上表现优于现有基线，并能泛化到新环境。 Conclusion: MMTwin能有效预测未来3D手部轨迹，代码和模型将开源。 Abstract: Predicting hand motion is critical for understanding human intentions and bridging the action space between human movements and robot manipulations. Existing hand trajectory prediction (HTP) methods forecast the future hand waypoints in 3D space conditioned on past egocentric observations. However, such models are only designed to accommodate 2D egocentric video inputs. There is a lack of awareness of multimodal environmental information from both 2D and 3D observations, hindering the further improvement of 3D HTP performance. In addition, these models overlook the synergy between hand movements and headset camera egomotion, either predicting hand trajectories in isolation or encoding egomotion only from past frames. To address these limitations, we propose novel diffusion models (MMTwin) for multimodal 3D hand trajectory prediction. MMTwin is designed to absorb multimodal information as input encompassing 2D RGB images, 3D point clouds, past hand waypoints, and text prompt. Besides, two latent diffusion models, the egomotion diffusion and the HTP diffusion as twins, are integrated into MMTwin to predict camera egomotion and future hand trajectories concurrently. We propose a novel hybrid Mamba-Transformer module as the denoising model of the HTP diffusion to better fuse multimodal features. The experimental results on three publicly available datasets and our self-recorded data demonstrate that our proposed MMTwin can predict plausible future 3D hand trajectories compared to the state-of-the-art baselines, and generalizes well to unseen environments. The code and pretrained models will be released at https://github.com/IRMVLab/MMTwin.

PAYADOR: A Minimalist Approach to Grounding Language Models on Structured Data for Interactive Storytelling and Role-playing Games

Santiago Góngora,Luis Chiruzzo,Gonzalo Méndez,Pablo Gervás

Task: 提出PAYADOR方法，通过预测动作结果而非直接表示动作来解决交互式叙事系统中的世界更新问题。

Motivation: 传统方法将玩家输入映射到预编程动作，限制了玩家的自由意志，尤其在强调即兴创作的RPG中问题更为突出。

Details

Method: 基于大型语言模型，结合虚构世界的最小表示，预测动作的结果。 Result: 取得了有希望的结果，并将方法开源以促进相关研究。 Conclusion: PAYADOR为释放RPG的共创潜力提供了一种新思路。 Abstract: Every time an Interactive Storytelling (IS) system gets a player input, it is facing the world-update problem. Classical approaches to this problem consist in mapping that input to known preprogrammed actions, what can severely constrain the free will of the player. When the expected experience has a strong focus on improvisation, like in Role-playing Games (RPGs), this problem is critical. In this paper we present PAYADOR, a different approach that focuses on predicting the outcomes of the actions instead of representing the actions themselves. To implement this approach, we ground a Large Language Model to a minimal representation of the fictional world, obtaining promising results. We make this contribution open-source, so it can be adapted and used for other related research on unleashing the co-creativity power of RPGs.

BRepFormer: Transformer-Based B-rep Geometric Feature Recognition

Yongkang Dai,Xiaoshui Huang,Yunpeng Bai,Hao Guo,Hongping Gan,Ling Yang,Yilei Shi

Task: 提出一种基于Transformer的模型BRepFormer，用于识别加工特征和复杂CAD模型的特征。

Motivation: 现有研究多集中于加工特征识别（MFR），未能有效捕捉复杂几何特征的拓扑和几何特性。

Details

Method: BRepFormer通过编码和融合模型的几何与拓扑特征，利用Transformer架构进行特征传播，并通过识别头识别几何特征。 Result: BRepFormer在MFInstSeg、MFTRCAD和CBF数据集上达到了最先进的准确率。 Conclusion: BRepFormer能够有效识别复杂几何特征，且提出的CBF数据集更符合工业应用需求。 Abstract: Recognizing geometric features on B-rep models is a cornerstone technique for multimedia content-based retrieval and has been widely applied in intelligent manufacturing. However, previous research often merely focused on Machining Feature Recognition (MFR), falling short in effectively capturing the intricate topological and geometric characteristics of complex geometry features. In this paper, we propose BRepFormer, a novel transformer-based model to recognize both machining feature and complex CAD models' features. BRepFormer encodes and fuses the geometric and topological features of the models. Afterwards, BRepFormer utilizes a transformer architecture for feature propagation and a recognition head to identify geometry features. During each iteration of the transformer, we incorporate a bias that combines edge features and topology features to reinforce geometric constraints on each face. In addition, we also proposed a dataset named Complex B-rep Feature Dataset (CBF), comprising 20,000 B-rep models. By covering more complex B-rep models, it is better aligned with industrial applications. The experimental results demonstrate that BRepFormer achieves state-of-the-art accuracy on the MFInstSeg, MFTRCAD, and our CBF datasets.

Multilingual MFA: Forced Alignment on Low-Resource Related Languages

Alessio Tosolini,Claire Bowern

Task: 比较多语言和跨语言训练对澳大利亚相关和非相关语言（具有相似音系库）的结果。

Motivation: 探讨在多语言和跨语言训练中，适应英语基线模型对未见语言的潜在优势。

Details

Method: 使用蒙特利尔强制对齐器从头训练声学模型，并适应大型英语模型，评估结果包括已见数据、未见数据（已见语言）和未见数据及语言。 Result: 结果表明，适应英语基线模型对未见语言有益。 Conclusion: 适应英语基线模型在未见语言任务中具有优势。 Abstract: We compare the outcomes of multilingual and crosslingual training for related and unrelated Australian languages with similar phonological inventories. We use the Montreal Forced Aligner to train acoustic models from scratch and adapt a large English model, evaluating results against seen data, unseen data (seen language), and unseen data and language. Results indicate benefits of adapting the English baseline model for previously unseen languages.

Model Discrepancy Learning: Synthetic Faces Detection Based on Multi-Reconstruction

Qingchao Jiang,Zhishuo Xu,Zhiying Zhu,Ning Chen,Haoyue Wang,Zhongjie Ba

Task: 探索合成图像与其生成技术之间的内在关系，并提出一种基于多重建的检测器。

Motivation: 现有研究忽视了不同生成技术之间的差异，导致合成人脸检测的局限性。

Details

Method: 通过使用多种生成模型对图像进行反向重建，分析重建差异以区分真实图像和合成图像。 Result: 提出的检测器在实验中表现出卓越的性能，具有强泛化能力和鲁棒性。 Conclusion: 基于多重建的方法能有效区分合成图像，并补充了亚洲合成人脸数据集。 Abstract: Advances in image generation enable hyper-realistic synthetic faces but also pose risks, thus making synthetic face detection crucial. Previous research focuses on the general differences between generated images and real images, often overlooking the discrepancies among various generative techniques. In this paper, we explore the intrinsic relationship between synthetic images and their corresponding generation technologies. We find that specific images exhibit significant reconstruction discrepancies across different generative methods and that matching generation techniques provide more accurate reconstructions. Based on this insight, we propose a Multi-Reconstruction-based detector. By reversing and reconstructing images using multiple generative models, we analyze the reconstruction differences among real, GAN-generated, and DM-generated images to facilitate effective differentiation. Additionally, we introduce the Asian Synthetic Face Dataset (ASFD), containing synthetic Asian faces generated with various GANs and DMs. This dataset complements existing synthetic face datasets. Experimental results demonstrate that our detector achieves exceptional performance, with strong generalization and robustness.

Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalization

Shujin Wu,Cheng Qian,Yi R.,Fung,Paul Pu Liang,Heng Ji

Task: 提出一种名为Alice的主动学习框架，通过利用教师和学生模型的互补知识，提升弱到强泛化（W2SG）的性能。

Motivation: 传统W2SG方法依赖被动学习，限制了学生模型发挥其潜力，需要一种更有效的方法来利用教师和学生的互补知识。

Details

Method: 通过探测教师模型的不确定性，结合教师的响应作为示范，指导学生模型自我生成改进的响应；针对能力差距大的情况，提出级联Alice，采用分层训练方法。 Result: 在知识推理、数学推理和逻辑推理任务中，性能分别提升4.0%、22.62%和12.11%。 Conclusion: Alice框架显著提升了W2SG性能，实现了更稳健的知识传递和监督效果。 Abstract: The growing capabilities of large language models (LLMs) present a key challenge of maintaining effective human oversight. Weak-to-strong generalization (W2SG) offers a promising framework for supervising increasingly capable LLMs using weaker ones. Traditional W2SG methods rely on passive learning, where a weak teacher provides noisy demonstrations to train a strong student. This hinders students from employing their knowledge during training and reaching their full potential. In this work, we introduce Alice (pro{A}ctive {l}earning w{i}th tea{c}her's D{e}monstrations), a framework that leverages complementary knowledge between teacher and student to enhance the learning process.We probe the knowledge base of the teacher model by eliciting their uncertainty, and then use these insights together with teachers' responses as demonstrations to guide student models in self-generating improved responses for supervision. In addition, for situations with significant capability gaps between teacher and student models, we introduce cascade Alice, which employs a hierarchical training approach where weak teachers initially supervise intermediate models, who then guide stronger models in sequence. Experimental results demonstrate that our method significantly enhances the W2SG performance, yielding substantial improvements in three key tasks compared to the original W2SG: knowledge-based reasoning (+4.0%), mathematical reasoning (+22.62%), and logical reasoning (+12.11%). This highlights the effectiveness of our new W2SG paradigm that enables more robust knowledge transfer and supervision outcome.

ID-Booth: Identity-consistent Face Generation with Diffusion Models

Darian Tomašević,Fadi Boutros,Chenhao Lin,Naser Damer,Vitomir Štruc,Peter Peer

Task: 提出一种名为ID-Booth的生成扩散框架，用于实现身份一致的图像生成。

Motivation: 现有生成模型在训练时未考虑身份一致性，导致生成图像与目标身份不一致；而基于身份的训练方法又容易过拟合，降低生成多样性。

Details

Method: 结合去噪网络、变分自编码器和文本编码器，采用新颖的三元组身份训练目标。 Result: 实验表明，ID-Booth在身份一致性和多样性上优于现有方法，并能有效增强小规模数据集。 Conclusion: ID-Booth在隐私保护的前提下提升了生成图像的质量和识别模型的性能。 Abstract: Recent advances in generative modeling have enabled the generation of high-quality synthetic data that is applicable in a variety of domains, including face recognition. Here, state-of-the-art generative models typically rely on conditioning and fine-tuning of powerful pretrained diffusion models to facilitate the synthesis of realistic images of a desired identity. Yet, these models often do not consider the identity of subjects during training, leading to poor consistency between generated and intended identities. In contrast, methods that employ identity-based training objectives tend to overfit on various aspects of the identity, and in turn, lower the diversity of images that can be generated. To address these issues, we present in this paper a novel generative diffusion-based framework, called ID-Booth. ID-Booth consists of a denoising network responsible for data generation, a variational auto-encoder for mapping images to and from a lower-dimensional latent space and a text encoder that allows for prompt-based control over the generation procedure. The framework utilizes a novel triplet identity training objective and enables identity-consistent image generation while retaining the synthesis capabilities of pretrained diffusion models. Experiments with a state-of-the-art latent diffusion model and diverse prompts reveal that our method facilitates better intra-identity consistency and inter-identity separability than competing methods, while achieving higher image diversity. In turn, the produced data allows for effective augmentation of small-scale datasets and training of better-performing recognition models in a privacy-preserving manner. The source code for the ID-Booth framework is publicly available at https://github.com/dariant/ID-Booth.

Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extraction

Saurabh Srivastava,Ziyu Yao

Task: 系统研究大型推理模型（LRMs）在事件提取任务中是否需要提示优化。

Motivation: 尽管LRMs在推理任务中表现出色，但人们对其是否需要提示优化存在争议，本文通过事件提取任务验证这一点。

Details

Method: 实验比较了两种LRMs（DeepSeek-R1和o1）和两种通用LLMs（GPT-4o和GPT-4.5）作为任务模型或提示优化器的表现。 Result: 结果显示，在复杂任务如事件提取中，LRMs作为任务模型仍受益于提示优化，且LRMs作为提示优化器能生成更有效的提示。 Conclusion: LRMs在优化任务指令和事件指南方面表现出稳定性和一致性，但仍需提示优化以提高性能。 Abstract: Large Reasoning Models (LRMs) such as DeepSeek-R1 and OpenAI o1 have demonstrated remarkable capabilities in various reasoning tasks. Their strong capability to generate and reason over intermediate thoughts has also led to arguments that they may no longer require extensive prompt engineering or optimization to interpret human instructions and produce accurate outputs. In this work, we aim to systematically study this open question, using the structured task of event extraction for a case study. We experimented with two LRMs (DeepSeek-R1 and o1) and two general-purpose Large Language Models (LLMs) (GPT-4o and GPT-4.5), when they were used as task models or prompt optimizers. Our results show that on tasks as complicated as event extraction, LRMs as task models still benefit from prompt optimization, and that using LRMs as prompt optimizers yields more effective prompts. Finally, we provide an error analysis of common errors made by LRMs and highlight the stability and consistency of LRMs in refining task instructions and event guidelines.

FAIR-SIGHT: Fairness Assurance in Image Recognition via Simultaneous Conformal Thresholding and Dynamic Output Repair

Arya Fayyazi,Mehdi Kamal,Massoud Pedram

Task: 提出一种名为FAIR-SIGHT的后处理框架，通过结合共形预测和动态输出修复机制，确保计算机视觉系统的公平性。

Motivation: 解决计算机视觉系统中预测错误和公平性违规的问题，无需重新训练或访问内部模型参数。

Details

Method: 计算公平性感知的非一致性分数，结合共形预测设定自适应阈值，并通过动态调整（如分类的logit偏移和检测的置信度重新校准）修复输出。 Result: 理论分析验证了方法的误差控制和收敛性，实验表明FAIR-SIGHT显著减少公平性差异，同时保持高预测性能。 Conclusion: FAIR-SIGHT是一种有效且实用的方法，能够在保证预测性能的同时显著提升计算机视觉系统的公平性。 Abstract: We introduce FAIR-SIGHT, an innovative post-hoc framework designed to ensure fairness in computer vision systems by combining conformal prediction with a dynamic output repair mechanism. Our approach calculates a fairness-aware non-conformity score that simultaneously assesses prediction errors and fairness violations. Using conformal prediction, we establish an adaptive threshold that provides rigorous finite-sample, distribution-free guarantees. When the non-conformity score for a new image exceeds the calibrated threshold, FAIR-SIGHT implements targeted corrective adjustments, such as logit shifts for classification and confidence recalibration for detection, to reduce both group and individual fairness disparities, all without the need for retraining or having access to internal model parameters. Comprehensive theoretical analysis validates our method's error control and convergence properties. At the same time, extensive empirical evaluations on benchmark datasets show that FAIR-SIGHT significantly reduces fairness disparities while preserving high predictive performance.

Enhancing Time Series Forecasting via Multi-Level Text Alignment with LLMs

Taibiao Zhao,Xiaobing Chen,Mingxuan Sun

Task: 提出一种多级文本对齐框架，将时间序列数据适配到大型语言模型（LLMs）中，以提升预测准确性和可解释性。

Motivation: 时间序列数据是连续的，而LLMs基于离散标记操作，现有方法在将时间序列数据转换为文本形式时难以兼顾预测准确性和可解释性。

Details

Method: 将时间序列分解为趋势、季节性和残差分量，并将其转换为特定分量的文本表示，通过多级对齐机制与预训练的词标记对齐。 Result: 在多个数据集上的实验表明，该方法在准确性和可解释性上优于现有最先进模型。 Conclusion: 提出的多级文本对齐框架成功解决了时间序列数据与LLMs适配的问题，同时提升了预测性能和可解释性。 Abstract: The adaptation of large language models (LLMs) to time series forecasting poses unique challenges, as time series data is continuous in nature, while LLMs operate on discrete tokens. Despite the success of LLMs in natural language processing (NLP) and other structured domains, aligning time series data with language-based representations while maintaining both predictive accuracy and interpretability remains a significant hurdle. Existing methods have attempted to reprogram time series data into text-based forms, but these often fall short in delivering meaningful, interpretable results. In this paper, we propose a multi-level text alignment framework for time series forecasting using LLMs that not only improves prediction accuracy but also enhances the interpretability of time series representations. Our method decomposes time series into trend, seasonal, and residual components, which are then reprogrammed into component-specific text representations. We introduce a multi-level alignment mechanism, where component-specific embeddings are aligned with pre-trained word tokens, enabling more interpretable forecasts. Experiments on multiple datasets demonstrate that our method outperforms state-of-the-art models in accuracy while providing good interpretability.

FlexIP: Dynamic Control of Preservation and Personality for Customized Image Generation

Linyan Huang,Haonan Lin,Yanning Zhou,Kaiwen Xiao

Task: 提出FlexIP框架，通过解耦身份保留和个性化编辑目标，实现更灵活的参数化控制。

Motivation: 现有方法在身份保留和个性化编辑之间存在固有权衡，需要一种更灵活的解决方案。

Details

Method: FlexIP框架包含两个专用组件：个性化适配器（用于风格化操作）和保留适配器（用于身份维护），通过动态调整权重适配器实现参数化控制。 Result: 实验结果表明，FlexIP突破了传统方法的性能限制，实现了更好的身份保留和更丰富的个性化生成能力。 Conclusion: FlexIP框架通过解耦控制机制，为生成模型提供了更灵活的身份保留和个性化编辑能力。 Abstract: With the rapid advancement of 2D generative models, preserving subject identity while enabling diverse editing has emerged as a critical research focus. Existing methods typically face inherent trade-offs between identity preservation and personalized manipulation. We introduce FlexIP, a novel framework that decouples these objectives through two dedicated components: a Personalization Adapter for stylistic manipulation and a Preservation Adapter for identity maintenance. By explicitly injecting both control mechanisms into the generative model, our framework enables flexible parameterized control during inference through dynamic tuning of the weight adapter. Experimental results demonstrate that our approach breaks through the performance limitations of conventional methods, achieving superior identity preservation while supporting more diverse personalized generation capabilities (Project Page: https://flexip-tech.github.io/flexip/).

TALE: A Tool-Augmented Framework for Reference-Free Evaluation of Large Language Models

Sher Badshah,Ali Emami,Hassan Sajjad

Task: 提出一种无需预定义标准答案的LLM输出评估框架TALE。

Motivation: 静态预标注的评估方法在成本、可扩展性和完整性上存在挑战，难以适应现实世界中的动态应用场景。

Details

Method: TALE通过具备工具访问能力的代理，动态检索和综合外部证据，迭代生成查询、收集信息、总结发现并通过反思优化搜索。 Result: 在多个自由形式QA基准测试中，TALE不仅优于基于标准参考的指标，还与人类评估结果高度一致。 Conclusion: TALE在不依赖静态参考的情况下，显著提升了LLM在动态现实场景中的评估可靠性。 Abstract: As Large Language Models (LLMs) become increasingly integrated into real-world, autonomous applications, relying on static, pre-annotated references for evaluation poses significant challenges in cost, scalability, and completeness. We propose Tool-Augmented LLM Evaluation (TALE), a framework to assess LLM outputs without predetermined ground-truth answers. Unlike conventional metrics that compare to fixed references or depend solely on LLM-as-a-judge knowledge, TALE employs an agent with tool-access capabilities that actively retrieves and synthesizes external evidence. It iteratively generates web queries, collects information, summarizes findings, and refines subsequent searches through reflection. By shifting away from static references, TALE aligns with free-form question-answering tasks common in real-world scenarios. Experimental results on multiple free-form QA benchmarks show that TALE not only outperforms standard reference-based metrics for measuring response accuracy but also achieves substantial to near-perfect agreement with human evaluations. TALE enhances the reliability of LLM evaluations in real-world, dynamic scenarios without relying on static references.

Leveraging LLMs for Multimodal Retrieval-Augmented Radiology Report Generation via Key Phrase Extraction

Kyoyun Choi,Byungmu Yoon,Soobum Kim,Jonggwon Park

Task: 提出一种基于检索增强生成的方法，用于自动生成放射学报告，以减少计算资源需求并提高生成质量。

Motivation: 多模态大语言模型（MLLMs）在放射学报告生成中资源消耗大，需要大量数据和计算成本，因此需要一种更高效的方法。

Details

Method: 结合多模态检索和大语言模型，通过提取关键短语、图像编码器结构搜索、文本嵌入噪声添加和对比学习等技术优化生成过程。 Result: 在MIMIC-CXR数据集上取得了CheXbert指标的先进结果和RadGraph F1指标的竞争性表现，且无需微调大语言模型。 Conclusion: 该方法在多视角放射学报告生成中表现出强大的泛化能力，适合临床广泛应用。 Abstract: Automated radiology report generation (RRG) holds potential to reduce radiologists' workload, especially as recent advancements in large language models (LLMs) enable the development of multimodal models for chest X-ray (CXR) report generation. However, multimodal LLMs (MLLMs) are resource-intensive, requiring vast datasets and substantial computational cost for training. To address these challenges, we propose a retrieval-augmented generation approach that leverages multimodal retrieval and LLMs to generate radiology reports while mitigating hallucinations and reducing computational demands. Our method uses LLMs to extract key phrases from radiology reports, effectively focusing on essential diagnostic information. Through exploring effective training strategies, including image encoder structure search, adding noise to text embeddings, and additional training objectives, we combine complementary pre-trained image encoders and adopt contrastive learning between text and semantic image embeddings. We evaluate our approach on MIMIC-CXR dataset, achieving state-of-the-art results on CheXbert metrics and competitive RadGraph F1 metric alongside MLLMs, without requiring LLM fine-tuning. Our method demonstrates robust generalization for multi-view RRG, making it suitable for comprehensive clinical applications.

Talking Point based Ideological Discourse Analysis in News Events

Nishanth Nakshatri,Nikhil Mehta,Siyi Liu,Sihao Chen,Daniel J. Hopkins,Dan Roth,Dan Goldwasser

Task: 提出一个基于意识形态话语分析理论的框架，用于分析与现实世界事件相关的新闻文章。

Motivation: 大型语言模型（LLMs）在分析意识形态话语时难以捕捉关键元素和整合上下文信息，限制了其对抽象意识形态观点的理解。

Details

Method: 通过关系结构（谈论点）表示新闻文章，构建重复主题的词汇表，并生成意识形态特定的观点。 Result: 框架在意识形态和党派分类任务中表现良好，并通过人类验证，同时展示了在创建事件快照中的实用性。 Conclusion: 提出的框架有效解决了LLMs在意识形态话语分析中的局限性，并提供了可扩展的研究工具。 Abstract: Analyzing ideological discourse even in the age of LLMs remains a challenge, as these models often struggle to capture the key elements that shape real-world narratives. Specifically, LLMs fail to focus on characteristic elements driving dominant discourses and lack the ability to integrate contextual information required for understanding abstract ideological views. To address these limitations, we propose a framework motivated by the theory of ideological discourse analysis to analyze news articles related to real-world events. Our framework represents the news articles using a relational structure - talking points, which captures the interaction between entities, their roles, and media frames along with a topic of discussion. It then constructs a vocabulary of repeating themes - prominent talking points, that are used to generate ideology-specific viewpoints (or partisan perspectives). We evaluate our framework's ability to generate these perspectives through automated tasks - ideology and partisan classification tasks, supplemented by human validation. Additionally, we demonstrate straightforward applicability of our framework in creating event snapshots, a visual way of interpreting event discourse. We release resulting dataset and model to the community to support further research.

RadZero: Similarity-Based Cross-Attention for Explainable Vision-Language Alignment in Radiology with Zero-Shot Multi-Task Capability

Jonggwon Park,Soobum Kim,Byungmu Yoon,Kyoyun Choi

Task: 提出RadZero框架，用于解决放射学中视觉-语言对齐的挑战，并实现零样本多任务能力。

Motivation: 现有方法在利用复杂放射学报告、处理低分辨率图像及注意力机制可解释性方面存在不足。

Details

Method: RadZero通过基于相似性的跨注意力框架，结合大型语言模型提取语义句子，采用多正对比学习策略，并使用预训练视觉编码器处理高分辨率图像。 Result: 在公开胸部X光基准测试中，RadZero在零样本分类、定位和分割任务上优于现有方法，并展示了跨模态相似性映射的解释潜力。 Conclusion: RadZero在医学影像中表现出高效性和可解释性，支持开放词汇语义分割。 Abstract: Recent advancements in multi-modal models have significantly improved vision-language alignment in radiology. However, existing approaches struggle to effectively utilize complex radiology reports for learning, rely on low-resolution images, and offer limited interpretability in attention mechanisms. To address these challenges, we introduce RadZero, a novel similarity-based cross-attention framework for vision-language alignment in radiology with zero-shot multi-task capability. RadZero leverages large language models to extract minimal semantic sentences from radiology reports and employs a multi-positive contrastive learning strategy to effectively capture relationships between images and multiple relevant textual descriptions. It also utilizes a pre-trained vision encoder with additional trainable Transformer layers, allowing efficient high-resolution image processing. By computing similarity between text embeddings and local image patch features, RadZero enables zero-shot inference with similarity probability for classification and pixel-level cross-modal similarity maps for grounding and segmentation. Experimental results on public chest radiograph benchmarks show that RadZero outperforms state-of-the-art methods in zero-shot classification, grounding, and segmentation. Furthermore, cross-modal similarity map analysis highlights its potential for improving explainability in vision-language alignment. Additionally, qualitative evaluation demonstrates RadZero's capability for open-vocabulary semantic segmentation, further validating its effectiveness in medical imaging.

AI Coding with Few-Shot Prompting for Thematic Analysis

Samuel Flanders,Melati Nungsari,Mark Cheong Wing Loong

Task: 探索使用大型语言模型（如GPT 3.5-Turbo）进行主题分析编码。

Motivation: 主题分析编码通常需要大量人力，使得大多数研究者难以对大型语料库进行详尽分析。

Details

Method: 采用少样本提示方法，结合语义相似段落生成的高质量编码，以提升编码质量，同时使用成本低且易于扩展的模型。 Result: 通过该方法，能够在低成本下实现高质量的主题分析编码。 Conclusion: 大型语言模型在主题分析编码中具有潜力，能够显著减少人力成本并提高效率。 Abstract: This paper explores the use of large language models (LLMs), here represented by GPT 3.5-Turbo to perform coding for a thematic analysis. Coding is highly labor intensive, making it infeasible for most researchers to conduct exhaustive thematic analyses of large corpora. We utilize few-shot prompting with higher quality codes generated on semantically similar passages to enhance the quality of the codes while utilizing a cheap, more easily scalable model.

ThermoStereoRT: Thermal Stereo Matching in Real Time via Knowledge Distillation and Attention-based Refinement

Anning Hu,Ang Li,Xirui Jin,Danping Zou

Task: 提出一种实时热成像立体匹配方法ThermoStereoRT，用于全天候条件下从两幅校正的热成像立体图像中恢复视差。

Motivation: 应用于夜间无人机监控或床下清洁机器人等场景，解决热成像图像中稀疏地面真实数据的挑战。

Details

Method: 使用轻量级但强大的主干网络构建3D成本体积，并采用多尺度注意力机制生成初始视差图；设计新颖的通道和空间注意力模块进行优化；利用知识蒸馏提升性能。 Result: 在多个数据集上的综合评估表明，ThermoStereoRT具备实时能力和鲁棒精度。 Conclusion: ThermoStereoRT是一种适用于各种挑战性环境的实用解决方案，代码将开源。 Abstract: We introduce ThermoStereoRT, a real-time thermal stereo matching method designed for all-weather conditions that recovers disparity from two rectified thermal stereo images, envisioning applications such as night-time drone surveillance or under-bed cleaning robots. Leveraging a lightweight yet powerful backbone, ThermoStereoRT constructs a 3D cost volume from thermal images and employs multi-scale attention mechanisms to produce an initial disparity map. To refine this map, we design a novel channel and spatial attention module. Addressing the challenge of sparse ground truth data in thermal imagery, we utilize knowledge distillation to boost performance without increasing computational demands. Comprehensive evaluations on multiple datasets demonstrate that ThermoStereoRT delivers both real-time capacity and robust accuracy, making it a promising solution for real-world deployment in various challenging environments. Our code will be released on https://github.com/SJTU-ViSYS-team/ThermoStereoRT

AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery

Amirhossein Abaskohi,Amrutha Varshini Ramesh,Shailesh Nanisetty,Chirag Goel,David Vazquez,Christopher Pal,Spandana Gella,Giuseppe Carenini,Issam H. Laradji

Task: 介绍并评估AgentAda，一种能够学习和使用新分析技能的LLM驱动的分析代理。

Motivation: 现有方法需要用户手动选择数据分析方法，而AgentAda能自动从技能库中选择合适的技能，处理复杂任务。

Details

Method: AgentAda采用三步策略：问题生成器、基于RAG的技能匹配器和代码生成器，结合技能库中的方法（如聚类、预测建模和NLP技术）。 Result: 人类评估显示48.78%的评估者更偏好AgentAda的分析结果，优于未熟练代理的27.67%。 Conclusion: AgentAda通过自动化技能选择和代码生成，显著提升了分析洞察力，并提出了一种与人类评估一致的LLM-as-a-judge方法。 Abstract: We introduce AgentAda, the first LLM-powered analytics agent that can learn and use new analytics skills to extract more specialized insights. Unlike existing methods that require users to manually decide which data analytics method to apply, AgentAda automatically identifies the skill needed from a library of analytical skills to perform the analysis. This also allows AgentAda to use skills that existing LLMs cannot perform out of the box. The library covers a range of methods, including clustering, predictive modeling, and NLP techniques like BERT, which allow AgentAda to handle complex analytics tasks based on what the user needs. AgentAda's dataset-to-insight extraction strategy consists of three key steps: (I) a question generator to generate queries relevant to the user's goal and persona, (II) a hybrid Retrieval-Augmented Generation (RAG)-based skill matcher to choose the best data analytics skill from the skill library, and (III) a code generator that produces executable code based on the retrieved skill's documentation to extract key patterns. We also introduce KaggleBench, a benchmark of curated notebooks across diverse domains, to evaluate AgentAda's performance. We conducted a human evaluation demonstrating that AgentAda provides more insightful analytics than existing tools, with 48.78% of evaluators preferring its analyses, compared to 27.67% for the unskilled agent. We also propose a novel LLM-as-a-judge approach that we show is aligned with human evaluation as a way to automate insight quality evaluation at larger scale.

WS-DETR: Robust Water Surface Object Detection through Vision-Radar Fusion with Detection Transformer

Huilin Yin,Pengyu Wang,Senmao Li,Jun Yan,Daniel Watzenig

Task: 提出一种鲁棒的视觉-雷达融合模型WS-DETR，用于复杂水域环境中无人水面艇（USV）的物体检测。

Motivation: 解决复杂水域环境中物体检测的挑战，如边缘模糊和物体尺度多样性，以及现有视觉-雷达融合方法中的跨模态特征冲突问题。

Details

Method: 引入多尺度边缘信息集成模块（MSEII）增强边缘感知，分层特征聚合器（HiFA）提升多尺度检测，自适应特征交互融合模块（AFIF）缓解跨模态冲突。 Result: 在WaterScenes数据集上实现了最先进的性能，并在恶劣天气和光照条件下保持优势。 Conclusion: WS-DETR通过创新的模块设计有效解决了复杂水域环境中的物体检测问题，具有鲁棒性和实用性。 Abstract: Robust object detection for Unmanned Surface Vehicles (USVs) in complex water environments is essential for reliable navigation and operation. Specifically, water surface object detection faces challenges from blurred edges and diverse object scales. Although vision-radar fusion offers a feasible solution, existing approaches suffer from cross-modal feature conflicts, which negatively affect model robustness. To address this problem, we propose a robust vision-radar fusion model WS-DETR. In particular, we first introduce a Multi-Scale Edge Information Integration (MSEII) module to enhance edge perception and a Hierarchical Feature Aggregator (HiFA) to boost multi-scale object detection in the encoder. Then, we adopt self-moving point representations for continuous convolution and residual connection to efficiently extract irregular features under the scenarios of irregular point cloud data. To further mitigate cross-modal conflicts, an Adaptive Feature Interactive Fusion (AFIF) module is introduced to integrate visual and radar features through geometric alignment and semantic fusion. Extensive experiments on the WaterScenes dataset demonstrate that WS-DETR achieves state-of-the-art (SOTA) performance, maintaining its superiority even under adverse weather and lighting conditions.

From Token to Line: Enhancing Code Generation with a Long-Term Perspective

Tingwei Lu,Yangning Li,Liyuan Wang,Binghuai Lin,Jiwei Tang,Wanshi Xu,Hai-Tao Zheng,Yinghui Li,Bingxu An,Zhao Wei,Yong Xu

Task: 提出一种基于MCTS的LSR-MCTS算法，用于逐行生成代码并优化生成路径。

Motivation: 现有代码生成研究存在冗余结果和局部过拟合问题，且缺乏对生成处理长度的关注。

Details

Method: 通过分析LLMs生成过程中的注意力机制，提出以代码行为基本处理单元，结合MCTS和自优化机制逐行生成代码。 Result: 在三个公开编码基准测试中，LSR-MCTS算法表现优于现有最优方法。 Conclusion: LSR-MCTS算法通过逐行生成和自优化机制，显著提升了代码生成的多样性和质量。 Abstract: The emergence of large language models (LLMs) has significantly promoted the development of code generation task, sparking a surge in pertinent literature. Current research is hindered by redundant generation results and a tendency to overfit local patterns in the short term. Although existing studies attempt to alleviate the issue by adopting a multi-token prediction strategy, there remains limited focus on choosing the appropriate processing length for generations. By analyzing the attention between tokens during the generation process of LLMs, it can be observed that the high spikes of the attention scores typically appear at the end of lines. This insight suggests that it is reasonable to treat each line of code as a fundamental processing unit and generate them sequentially. Inspired by this, we propose the \textbf{LSR-MCTS} algorithm, which leverages MCTS to determine the code line-by-line and select the optimal path. Further, we integrate a self-refine mechanism at each node to enhance diversity and generate higher-quality programs through error correction. Extensive experiments and comprehensive analyses on three public coding benchmarks demonstrate that our method outperforms the state-of-the-art performance approaches.

How Can Objects Help Video-Language Understanding?

Zitian Tang,Shijie Wang,Junho Cho,Jaewook Yoo,Chen Sun

Task: 探索多模态大语言模型（MLLMs）中对象表示如何帮助视频语言理解。

Motivation: 理解MLLMs如何感知视觉世界，尤其是对象和关系的建模方式，以及视觉描述对视频理解的影响。

Details

Method: 从对象表示和适应的角度研究表达力（如分布式与符号化）与集成难度（如数据效率）的权衡。 Result: 通过五个视频问答数据集的评估，发现显式集成对象中心表示是必要的，符号化对象最易集成且性能优越。 Conclusion: 鼓励社区探索将感知模块显式集成到MLLM设计中。 Abstract: How multimodal large language models (MLLMs) perceive the visual world remains a mystery. To one extreme, object and relation modeling may be implicitly implemented with inductive biases, for example by treating objects as tokens. To the other extreme, empirical results reveal the surprising finding that simply performing visual captioning, which tends to ignore spatial configuration of the objects, serves as a strong baseline for video understanding. We aim to answer the question: how can objects help video-language understanding in MLLMs? We tackle the question from the object representation and adaptation perspectives. Specifically, we investigate the trade-off between representation expressiveness (e.g., distributed versus symbolic) and integration difficulty (e.g., data-efficiency when learning the adapters). Through extensive evaluations on five video question answering datasets, we confirm that explicit integration of object-centric representation remains necessary, and the symbolic objects can be most easily integrated while being performant for question answering. We hope our findings can encourage the community to explore the explicit integration of perception modules into MLLM design. Our code and models will be publicly released.

Revisiting LLM Evaluation through Mechanism Interpretability: a New Metric and Model Utility Law

Yixin Cao,Jiahao Ying,Yaoning Wang,Xipeng Qiu,Xuanjing Huang,Yugang Jiang

Task: 提出一种新的评估指标（Model Utilization Index, MUI），用于量化大语言模型（LLMs）在完成任务时对其能力的利用程度。

Motivation: 传统评估方法难以跟上大语言模型的快速发展，需要一种更全面的评估方式，不仅关注任务表现，还关注模型实现结果所付出的努力。

Details

Method: 引入机制可解释性技术，提出MUI指标，并通过大量实验验证其与性能的反比关系。 Result: 实验揭示了MUI与性能之间的反比关系，并总结出“Utility Law”，进一步推导出四个推论，解决了训练判断、数据污染、模型比较公平性和数据多样性等关键挑战。 Conclusion: MUI指标和Utility Law有望推动评估方法与机制可解释性的共同进步。 Abstract: Large Language Models (LLMs) have become indispensable across academia, industry, and daily applications, yet current evaluation methods struggle to keep pace with their rapid development. In this paper, we analyze the core limitations of traditional evaluation pipelines and propose a novel metric, the Model Utilization Index (MUI), which introduces mechanism interpretability techniques to complement traditional performance metrics. MUI quantifies the extent to which a model leverages its capabilities to complete tasks. The core idea is that to assess an LLM's overall ability, we must evaluate not only its task performance but also the effort expended to achieve the outcome. Our extensive experiments reveal an inverse relationship between MUI and performance, from which we deduce a common trend observed in popular LLMs, which we term the Utility Law. Based on this, we derive four corollaries that address key challenges, including training judgement, the issue of data contamination, fairness in model comparison, and data diversity. We hope that our survey, novel metric, and utility law will foster mutual advancement in both evaluation and mechanism interpretability. Our code can be found at https://github.com/ALEX-nlp/MUI-Eva.

Learning Universal Features for Generalizable Image Forgery Localization

Hengrun Zhao,Yunzhi Zhuge,Yifan Wang,Lijun Wang,Huchuan Lu,Yu Zeng

Task: 提出一种可泛化的图像伪造定位方法（GIFL），用于检测和定位训练数据中未见过的伪造内容。

Motivation: 现有方法依赖识别特定伪造痕迹，难以处理未见过的伪造类型，亟需一种更通用的解决方案。

Details

Method: 通过学习原始内容的通用特征而非特定伪造痕迹，实现对不同伪造类型的泛化检测。 Result: 实验表明，该方法在未见过的伪造检测上优于现有方法，并在已知伪造检测中表现竞争性。 Conclusion: GIFL提供了一种更实用和高效的解决方案，适用于生成式AI时代的虚假信息对抗。 Abstract: In recent years, advanced image editing and generation methods have rapidly evolved, making detecting and locating forged image content increasingly challenging. Most existing image forgery detection methods rely on identifying the edited traces left in the image. However, because the traces of different forgeries are distinct, these methods can identify familiar forgeries included in the training data but struggle to handle unseen ones. In response, we present an approach for Generalizable Image Forgery Localization (GIFL). Once trained, our model can detect both seen and unseen forgeries, providing a more practical and efficient solution to counter false information in the era of generative AI. Our method focuses on learning general features from the pristine content rather than traces of specific forgeries, which are relatively consistent across different types of forgeries and therefore can be used as universal features to locate unseen forgeries. Additionally, as existing image forgery datasets are still dominated by traditional hand-crafted forgeries, we construct a new dataset consisting of images edited by various popular deep generative image editing methods to further encourage research in detecting images manipulated by deep generative models. Extensive experimental results show that the proposed approach outperforms state-of-the-art methods in the detection of unseen forgeries and also demonstrates competitive results for seen forgeries. The code and dataset are available at https://github.com/ZhaoHengrun/GIFL.

Beyond LLMs: A Linguistic Approach to Causal Graph Generation from Narrative Texts

Zehan Li,Ruhua Pan,Xinyu Pi

Task: 提出一种从叙事文本生成因果图的新框架，连接高层因果关系和事件特定关系。

Motivation: 解决现有方法在因果链接识别精度上的不足，尤其是纯基于大语言模型的方法。

Details

Method: 结合RoBERTa嵌入和专家索引（七个语言学特征）的混合系统，通过五轮提示过程构建因果图。 Result: 在100个叙事章节和短故事上的实验表明，该方法在因果图质量上优于GPT-4o和Claude 3.5，同时保持可读性。 Conclusion: 开源工具提供了一种高效且可解释的解决方案，用于捕捉叙事中的复杂因果链。 Abstract: We propose a novel framework for generating causal graphs from narrative texts, bridging high-level causality and detailed event-specific relationships. Our method first extracts concise, agent-centered vertices using large language model (LLM)-based summarization. We introduce an "Expert Index," comprising seven linguistically informed features, integrated into a Situation-Task-Action-Consequence (STAC) classification model. This hybrid system, combining RoBERTa embeddings with the Expert Index, achieves superior precision in causal link identification compared to pure LLM-based approaches. Finally, a structured five-iteration prompting process refines and constructs connected causal graphs. Experiments on 100 narrative chapters and short stories demonstrate that our approach consistently outperforms GPT-4o and Claude 3.5 in causal graph quality, while maintaining readability. The open-source tool provides an interpretable, efficient solution for capturing nuanced causal chains in narratives.

CMEdataset Advancing China Map Detection and Standardization with Digital Image Resources

Yan Xu,Zhenqiang Zhang,Zhiwei Zhou,Liting Geng,Yue Li,Jintao Li

Task: 创建专门用于问题地图检测的CME数据集，以支持高精度地图合规性检测。

Motivation: 现有数据集主要关注一般地图数据，无法有效识别复杂问题如国家边界错误表示、缺失元素和模糊边界。

Details

Method: 研究创建了一个问题地图数据集，涵盖五个关键问题领域。 Result: 该数据集为问题地图检测技术提供了多样化的样本，支持高精度地图合规性检测，并提升地图数据质量和时效性。 Conclusion: 该数据集不仅为地图合规性、国家安全监测和地图更新提供了重要资源，还促进了相关技术的创新和应用。 Abstract: Digital images of Chinas maps play a crucial role in map detection, particularly in ensuring national sovereignty, territorial integrity, and map compliance. However, there is currently no publicly available dataset specifically dedicated to problematic maps the CME dataset. Existing datasets primarily focus on general map data and are insufficient for effectively identifying complex issues such as national boundary misrepresentations, missing elements, and blurred boundaries. Therefore, this study creates a Problematic Map dataset that covers five key problem areas, aiming to provide diverse samples for problematic map detection technologies, support high-precision map compliance detection, and enhance map data quality and timeliness. This dataset not only provides essential resources for map compliance, national security monitoring, and map updates, but also fosters innovation and application of related technologies.

Defense against Prompt Injection Attacks via Mixture of Encodings

Ruiyi Zhang,David Sullivan,Kyle Jackson,Pengtao Xie,Mei Chen

Task: 提出一种新型防御机制（混合编码）以应对大型语言模型（LLMs）中的提示注入攻击。

Motivation: 尽管Base64防御方法有效，但会降低LLM在某些NLP任务上的性能，因此需要一种既能抵御攻击又能保持高性能的方法。

Details

Method: 采用多种字符编码（包括Base64）的混合编码策略。 Result: 实验表明，该方法在提示注入攻击下攻击成功率最低，同时在所有NLP任务中保持高性能。 Conclusion: 混合编码策略在安全性和任务性能上均表现出色，优于现有基于字符编码的防御方法。 Abstract: Large Language Models (LLMs) have emerged as a dominant approach for a wide range of NLP tasks, with their access to external information further enhancing their capabilities. However, this introduces new vulnerabilities, known as prompt injection attacks, where external content embeds malicious instructions that manipulate the LLM's output. Recently, the Base64 defense has been recognized as one of the most effective methods for reducing success rate of prompt injection attacks. Despite its efficacy, this method can degrade LLM performance on certain NLP tasks. To address this challenge, we propose a novel defense mechanism: mixture of encodings, which utilizes multiple character encodings, including Base64. Extensive experimental results show that our method achieves one of the lowest attack success rates under prompt injection attacks, while maintaining high performance across all NLP tasks, outperforming existing character encoding-based defense methods. This underscores the effectiveness of our mixture of encodings strategy for both safety and task performance metrics.

Kimi-VL Technical Report

Kimi Team,Angang Du,Bohong Yin,Bowei Xing,Bowen Qu,Bowen Wang,Cheng Chen,Chenlin Zhang,Chenzhuang Du,Chu Wei,Congcong Wang,Dehao Zhang,Dikang Du,Dongliang Wang,Enming Yuan,Enzhe Lu,Fang Li,Flood Sung,Guangda Wei,Guokun Lai,Han Zhu,Hao Ding,Hao Hu,Hao Yang,Hao Zhang,Haoning Wu,Haotian Yao,Haoyu Lu,Heng Wang,Hongcheng Gao,Huabin Zheng,Jiaming Li,Jianlin Su,Jianzhou Wang,Jiaqi Deng,Jiezhong Qiu,Jin Xie,Jinhong Wang,Jingyuan Liu,Junjie Yan,Kun Ouyang,Liang Chen,Lin Sui,Longhui Yu,Mengfan Dong,Mengnan Dong,Nuo Xu,Pengyu Cheng,Qizheng Gu,Runjie Zhou,Shaowei Liu,Sihan Cao,Tao Yu,Tianhui Song,Tongtong Bai,Wei Song,Weiran He,Weixiao Huang,Weixin Xu,Xiaokun Yuan,Xingcheng Yao,Xingzhe Wu,Xinxing Zu,Xinyu Zhou,Xinyuan Wang,Y. Charles,Yan Zhong,Yang Li,Yangyang Hu,Yanru Chen,Yejie Wang,Yibo Liu,Yibo Miao,Yidao Qin,Yimin Chen,Yiping Bao,Yiqin Wang,Yongsheng Kang,Yuanxin Liu,Yulun Du,Yuxin Wu,Yuzhi Wang,Yuzi Yan,Zaida Zhou,Zhaowei Li,Zhejun Jiang,Zheng Zhang,Zhilin Yang,Zhiqi Huang,Zihao Huang,Zijia Zhao,Ziwei Chen

Task: 开发一个高效的开源混合专家（MoE）视觉语言模型（VLM）Kimi-VL，具备多模态推理、长上下文理解和强大的代理能力。

Motivation: 通过激活仅2.8B参数的语言解码器，实现高效的多模态任务处理，同时与现有先进模型竞争。

Details

Method: 采用混合专家（MoE）架构，结合长上下文窗口（128K）和高分辨率视觉编码器MoonViT，并通过监督微调（SFT）和强化学习（RL）开发长思考变体Kimi-VL-Thinking。 Result: Kimi-VL在多项任务中表现优异，如多轮代理任务、图像视频理解、OCR等，并在长上下文处理和高分辨率视觉输入中取得高分。 Conclusion: Kimi-VL及其变体为高效多模态模型设定了新标准，代码和模型已开源。 Abstract: We present Kimi-VL, an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong agent capabilities - all while activating only 2.8B parameters in its language decoder (Kimi-VL-A3B). Kimi-VL demonstrates strong performance across challenging domains: as a general-purpose VLM, Kimi-VL excels in multi-turn agent tasks (e.g., OSWorld), matching flagship models. Furthermore, it exhibits remarkable capabilities across diverse challenging vision language tasks, including college-level image and video comprehension, OCR, mathematical reasoning, and multi-image understanding. In comparative evaluations, it effectively competes with cutting-edge efficient VLMs such as GPT-4o-mini, Qwen2.5-VL-7B, and Gemma-3-12B-IT, while surpassing GPT-4o in several key domains. Kimi-VL also advances in processing long contexts and perceiving clearly. With a 128K extended context window, Kimi-VL can process diverse long inputs, achieving impressive scores of 64.5 on LongVideoBench and 35.1 on MMLongBench-Doc. Its native-resolution vision encoder, MoonViT, further allows it to see and understand ultra-high-resolution visual inputs, achieving 83.2 on InfoVQA and 34.5 on ScreenSpot-Pro, while maintaining lower computational cost for common tasks. Building upon Kimi-VL, we introduce an advanced long-thinking variant: Kimi-VL-Thinking. Developed through long chain-of-thought (CoT) supervised fine-tuning (SFT) and reinforcement learning (RL), this model exhibits strong long-horizon reasoning capabilities. It achieves scores of 61.7 on MMMU, 36.8 on MathVision, and 71.3 on MathVista while maintaining the compact 2.8B activated LLM parameters, setting a new standard for efficient multimodal thinking models. Code and models are publicly accessible at https://github.com/MoonshotAI/Kimi-VL.

Transformer-Based Temporal Information Extraction and Application: A Review

Xin Su,Phillip Howard,Steven Bethard

Task: 系统总结和分析基于Transformer的时间信息提取（Temporal IE）研究，并指出未来研究方向。

Motivation: 尽管Transformer在时间信息提取领域取得了显著成果，但缺乏对这些工作的全面综述。

Details

Method: 通过系统总结和分析现有基于Transformer的时间信息提取研究。 Result: 填补了该领域的综述空白，并提出了未来研究方向。 Conclusion: 本文为时间信息提取领域的研究提供了系统性的综述和未来发展的指导。 Abstract: Temporal information extraction (IE) aims to extract structured temporal information from unstructured text, thereby uncovering the implicit timelines within. This technique is applied across domains such as healthcare, newswire, and intelligence analysis, aiding models in these areas to perform temporal reasoning and enabling human users to grasp the temporal structure of text. Transformer-based pre-trained language models have produced revolutionary advancements in natural language processing, demonstrating exceptional performance across a multitude of tasks. Despite the achievements garnered by Transformer-based approaches in temporal IE, there is a lack of comprehensive reviews on these endeavors. In this paper, we aim to bridge this gap by systematically summarizing and analyzing the body of work on temporal IE using Transformers while highlighting potential future research directions.

Event Signal Filtering via Probability Flux Estimation

Jinze Chen,Wei Zhai,Yang Cao,Bin Li,Zheng-Jun Zha

Task: 提出一种基于扩散过程理论的事件信号在线滤波框架EDFilter，用于提升事件数据的信噪比和一致性。

Motivation: 事件数据的固有随机性导致信号质量下降，传统时间序列方法难以建模事件的瞬态动态特性。

Details

Method: 通过非参数核平滑重建连续概率通量，并从中重采样滤波后的事件，采用时空核和时间变化优化框架。 Result: EDFilter在事件滤波、超分辨率和直接事件跟踪等任务中表现优异，显著提升了SLAM和视频重建等下游应用的效果。 Conclusion: EDFilter通过理论创新和高效实现，为事件数据处理提供了鲁棒且有效的解决方案。 Abstract: Events offer a novel paradigm for capturing scene dynamics via asynchronous sensing, but their inherent randomness often leads to degraded signal quality. Event signal filtering is thus essential for enhancing fidelity by reducing this internal randomness and ensuring consistent outputs across diverse acquisition conditions. Unlike traditional time series that rely on fixed temporal sampling to capture steady-state behaviors, events encode transient dynamics through polarity and event intervals, making signal modeling significantly more complex. To address this, the theoretical foundation of event generation is revisited through the lens of diffusion processes. The state and process information within events is modeled as continuous probability flux at threshold boundaries of the underlying irradiance diffusion. Building on this insight, a generative, online filtering framework called Event Density Flow Filter (EDFilter) is introduced. EDFilter estimates event correlation by reconstructing the continuous probability flux from discrete events using nonparametric kernel smoothing, and then resamples filtered events from this flux. To optimize fidelity over time, spatial and temporal kernels are employed in a time-varying optimization framework. A fast recursive solver with O(1) complexity is proposed, leveraging state-space models and lookup tables for efficient likelihood computation. Furthermore, a new real-world benchmark Rotary Event Dataset (RED) is released, offering microsecond-level ground truth irradiance for full-reference event filtering evaluation. Extensive experiments validate EDFilter's performance across tasks like event filtering, super-resolution, and direct event-based blob tracking. Significant gains in downstream applications such as SLAM and video reconstruction underscore its robustness and effectiveness.

Geological Inference from Textual Data using Word Embeddings

Nanmanas Linphrachaya,Irving Gómez-Méndez,Adil Siripatana

Task: 利用自然语言处理（NLP）技术定位工业矿物等地质资源。

Motivation: 通过NLP技术提取地质文本中的语义关系，以辅助地质资源的空间分布分析。

Details

Method: 使用GloVe模型训练词嵌入，提取目标关键词与地质文本的语义关系，并结合PCA、Autoencoder、VAE和VAE-LSTM等降维技术优化特征提取。 Result: 通过余弦相似度排名和Haversine方程验证，结果显示NLP与降维技术结合能有效分析资源空间分布，但精度有待提升。 Conclusion: NLP与降维技术的结合为地质资源定位提供了新思路，但需进一步优化以提高准确性。 Abstract: This research explores the use of Natural Language Processing (NLP) techniques to locate geological resources, with a specific focus on industrial minerals. By using word embeddings trained with the GloVe model, we extract semantic relationships between target keywords and a corpus of geological texts. The text is filtered to retain only words with geographical significance, such as city names, which are then ranked by their cosine similarity to the target keyword. Dimensional reduction techniques, including Principal Component Analysis (PCA), Autoencoder, Variational Autoencoder (VAE), and VAE with Long Short-Term Memory (VAE-LSTM), are applied to enhance feature extraction and improve the accuracy of semantic relations. For benchmarking, we calculate the proximity between the ten cities most semantically related to the target keyword and identified mine locations using the haversine equation. The results demonstrate that combining NLP with dimensional reduction techniques provides meaningful insights into the spatial distribution of natural resources. Although the result shows to be in the same region as the supposed location, the accuracy has room for improvement.

VideoExpert: Augmented LLM for Temporal-Sensitive Video Understanding

Henghao Zhao,Ge-Peng Ji,Rui Yan,Huan Xiong,Zechao Li

Task: 解决多模态大语言模型在时间敏感视频任务中生成时间戳时依赖语言模式而非视觉线索的问题。

Motivation: 现有方法在生成时间戳时过度依赖语言模式，导致性能不佳，需要一种能够更好捕捉视频动态变化的方法。

Details

Method: 提出VideoExpert模型，包含并行的时间专家和空间专家模块，分别处理时间序列和内容细节，并通过特殊令牌协作。 Result: 实验证明VideoExpert在时间敏感视频任务中表现有效且通用。 Conclusion: VideoExpert通过分离时间定位和内容生成，显著提升了时间戳预测的准确性。 Abstract: The core challenge in video understanding lies in perceiving dynamic content changes over time. However, multimodal large language models struggle with temporal-sensitive video tasks, which requires generating timestamps to mark the occurrence of specific events. Existing strategies require MLLMs to generate absolute or relative timestamps directly. We have observed that those MLLMs tend to rely more on language patterns than visual cues when generating timestamps, affecting their performance. To address this problem, we propose VideoExpert, a general-purpose MLLM suitable for several temporal-sensitive video tasks. Inspired by the expert concept, VideoExpert integrates two parallel modules: the Temporal Expert and the Spatial Expert. The Temporal Expert is responsible for modeling time sequences and performing temporal grounding. It processes high-frame-rate yet compressed tokens to capture dynamic variations in videos and includes a lightweight prediction head for precise event localization. The Spatial Expert focuses on content detail analysis and instruction following. It handles specially designed spatial tokens and language input, aiming to generate content-related responses. These two experts collaborate seamlessly via a special token, ensuring coordinated temporal grounding and content generation. Notably, the Temporal and Spatial Experts maintain independent parameter sets. By offloading temporal grounding from content generation, VideoExpert prevents text pattern biases in timestamp predictions. Moreover, we introduce a Spatial Compress module to obtain spatial tokens. This module filters and compresses patch tokens while preserving key information, delivering compact yet detail-rich input for the Spatial Expert. Extensive experiments demonstrate the effectiveness and versatility of the VideoExpert.

Supervised Optimism Correction: Be Confident When LLMs Are Sure

Junjie Zhang,Rushuai Yang,Shunyu Liu,Ting-En Lin,Fei Huang,Yi Chen,Yongbin Li,Dacheng Tao

Task: 建立监督微调与离线强化学习在token级马尔可夫决策过程下的理论联系，并提出一种新的方法（SOC）来纠正推理中的过度乐观问题。

Motivation: 揭示大型语言模型在推理过程中学习隐式Q函数的现象，并指出广泛使用的束搜索方法因对次优步骤的Q值估计过高而导致推理错误放大。

Details

Method: 提出Supervised Optimism Correction (SOC)，通过在监督微调中引入辅助损失函数，对token级Q值估计进行隐式值正则化，以抑制对未充分监督响应的过度乐观。 Result: 在数学推理基准测试（GSM8K、MATH、GAOKAO）上，SOC与束搜索结合的方法在一系列开源模型中表现出优越性。 Conclusion: SOC通过理论联系和简单有效的辅助损失，显著改善了推理中的过度乐观问题，提升了模型性能。 Abstract: In this work, we establish a novel theoretical connection between supervised fine-tuning and offline reinforcement learning under the token-level Markov decision process, revealing that large language models indeed learn an implicit $Q$-function for inference. Through this theoretical lens, we demonstrate that the widely used beam search method suffers from unacceptable over-optimism, where inference errors are inevitably amplified due to inflated $Q$-value estimations of suboptimal steps. To address this limitation, we propose Supervised Optimism Correction(SOC), which introduces a simple yet effective auxiliary loss for token-level $Q$-value estimations during supervised fine-tuning. Specifically, the auxiliary loss employs implicit value regularization to boost model confidence in expert-demonstrated responses, thereby suppressing over-optimism toward insufficiently supervised responses. Extensive experiments on mathematical reasoning benchmarks, including GSM8K, MATH, and GAOKAO, showcase the superiority of the proposed SOC with beam search across a series of open-source models.

DGOcc: Depth-aware Global Query-based Network for Monocular 3D Occupancy Prediction

Xu Zhao,Pengju Zhang,Bo Liu,Yihong Wu

Task: 从单目2D图像预测3D场景的占用和语义信息。

Motivation: 单目3D占用预测在3D场景理解中具有重要作用，但大规模室外场景的预测存在不适定性和资源密集性问题。

Details

Method: 提出DGOcc网络，结合深度上下文特征和全局查询模块，利用注意力机制和尺度感知操作优化特征交互，并设计分层监督策略以减少计算开销。 Result: 在SemanticKITTI和SSCBench-KITTI-360数据集上表现最佳，同时降低了GPU和时间开销。 Conclusion: DGOcc方法在单目语义占用预测中实现了高性能和高效性。 Abstract: Monocular 3D occupancy prediction, aiming to predict the occupancy and semantics within interesting regions of 3D scenes from only 2D images, has garnered increasing attention recently for its vital role in 3D scene understanding. Predicting the 3D occupancy of large-scale outdoor scenes from 2D images is ill-posed and resource-intensive. In this paper, we present \textbf{DGOcc}, a \textbf{D}epth-aware \textbf{G}lobal query-based network for monocular 3D \textbf{Occ}upancy prediction. We first explore prior depth maps to extract depth context features that provide explicit geometric information for the occupancy network. Then, in order to fully exploit the depth context features, we propose a Global Query-based (GQ) Module. The cooperation of attention mechanisms and scale-aware operations facilitates the feature interaction between images and 3D voxels. Moreover, a Hierarchical Supervision Strategy (HSS) is designed to avoid upsampling the high-dimension 3D voxel features to full resolution, which mitigates GPU memory utilization and time cost. Extensive experiments on SemanticKITTI and SSCBench-KITTI-360 datasets demonstrate that the proposed method achieves the best performance on monocular semantic occupancy prediction while reducing GPU and time overhead.

AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation

Tuhin Chakrabarty,Philippe Laban,Chien-Sheng Wu

Task: 研究如何评估和改进AI生成文本的写作质量。

Motivation: AI生成文本在多个领域广泛应用，但写作质量评估因主观性和专业性而受到较少关注。

Details

Method: 通过整合五个写作偏好数据集构建写作质量基准（WQ），并训练专门的写作质量奖励模型（WQRM）。 Result: WQRM在四个分布外测试集上表现良好，WQ基准准确率达74%，且人类评估显示专家更偏好WQRM生成的文本。 Conclusion: WQRM能有效提升AI生成文本的写作质量，并鼓励社区参与写作质量评估和AI写作系统的开发。 Abstract: AI-generated text is proliferating across domains, from creative writing and journalism to marketing content and scientific articles. Models can follow user-provided instructions to generate coherent and grammatically correct outputs but in this work, we study a more fundamental question: how do we evaluate and improve the writing quality of AI-generated text? Writing quality assessment has received less attention from the community, in part because it is fundamentally subjective and requires expertise. We first introduce the Writing Quality Benchmark (WQ) by consolidating five writing-preference datasets into 4,729 writing quality judgments. Our experiments show that competitive baselines, including state-of-the-art LLMs that excel at reasoning tasks, barely outperform random baselines on WQ. We then train specialized Writing Quality Reward Models (WQRM) of various sizes for writing quality assessment that demonstrate strong generalization on four out-of-distribution test sets and 74% accuracy on the WQ benchmark. To further show WQRM's practical benefits during inference, we leverage additional test-time compute to generate and rank multiple candidate revisions, allowing us to select higher-quality outputs from an initial draft. Human evaluation with 9 experienced writers confirm that WQRM-based selection produces writing samples preferred by experts 66% overall, and 72.2% when the reward gap is larger than 1 point. We release our datasets and models to encourage community engagement with writing quality assessment and development of AI writing systems better aligned with human preferences.

SydneyScapes: Image Segmentation for Australian Environments

Hongyu Lyu,Julie Stephany Berrio,Mao Shan,Stewart Worrall

Task: Introduce SydneyScapes, a dataset for computer vision tasks in AV perception systems tailored for the Australian context.

Motivation: Address the lack of locally labelled datasets in Australia for developing and testing AV algorithms.

Details

Method: Collect and annotate 756 images from Sydney and surrounding cities in NSW, Australia, with high-quality pixel-level annotations. Result: Provide a publicly available dataset with benchmarking results using state-of-the-art algorithms. Conclusion: SydneyScapes supports AV research and industry by offering annotated data and tools for algorithm development in Australia. Abstract: Autonomous Vehicles (AVs) are being partially deployed and tested across various global locations, including China, the USA, Germany, France, Japan, Korea, and the UK, but with limited demonstrations in Australia. The integration of machine learning (ML) into AV perception systems highlights the need for locally labelled datasets to develop and test algorithms in specific environments. To address this, we introduce SydneyScapes - a dataset tailored for computer vision tasks of image semantic, instance, and panoptic segmentation. This dataset, collected from Sydney and surrounding cities in New South Wales (NSW), Australia, consists of 756 images with high-quality pixel-level annotations. It is designed to assist AV industry and researchers by providing annotated data and tools for algorithm development, testing, and deployment in the Australian context. Additionally, we offer benchmarking results using state-of-the-art algorithms to establish reference points for future research and development. The dataset is publicly available at https://hdl.handle.net/2123/33051.

Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answering

Patrick Fernandes,Sweta Agrawal,Emmanouil Zaranis,André F. T. Martins,Graham Neubig

Task: 提出一种通过问答评估翻译质量的框架TREQA，以解决现有自动指标在长段落翻译评估中的不足。

Motivation: 现有自动翻译评估指标难以捕捉跨句子的意义保留，尤其是在长、复杂段落中，需要更实用的方法来评估关键信息的准确传达。

Details

Method: 引入TREQA框架，通过评估候选翻译对阅读理解问题的回答准确性来评估翻译质量。 Result: 在需要长距离理解的领域（如文学文本），TREQA在排名段落级翻译时优于或与现有先进神经和LLM指标相当，且生成的问答对能有效定位专家识别的翻译错误。 Conclusion: TREQA提供了一种实用且可解释的翻译评估方法，尤其在长段落翻译中表现优异。 Abstract: Despite the steady progress in machine translation evaluation, existing automatic metrics struggle to capture how well meaning is preserved beyond sentence boundaries. We posit that reliance on a single intrinsic quality score, trained to mimic human judgments, might be insufficient for evaluating translations of long, complex passages, and a more ``pragmatic'' approach that assesses how accurately key information is conveyed by a translation in context is needed. We introduce TREQA (Translation Evaluation via Question-Answering), a framework that extrinsically evaluates translation quality by assessing how accurately candidate translations answer reading comprehension questions that target key information in the original source or reference texts. In challenging domains that require long-range understanding, such as literary texts, we show that TREQA is competitive with and, in some cases, outperforms state-of-the-art neural and LLM-based metrics in ranking alternative paragraph-level translations, despite never being explicitly optimized to correlate with human judgments. Furthermore, the generated questions and answers offer interpretability: empirical analysis shows that they effectively target translation errors identified by experts in evaluated datasets. Our code is available at https://github.com/deep-spin/treqa

STeP: A General and Scalable Framework for Solving Video Inverse Problems with Spatiotemporal Diffusion Priors

Bingliang Zhang,Zihui Wu,Berthy T. Feng,Yang Song,Yisong Yue,Katherine L. Bouman

Task: 研究如何利用扩散模型先验解决涉及视频的一般贝叶斯逆问题。

Motivation: 现有方法依赖图像扩散先验和启发式方法，难以准确恢复时间关系，尤其是在高时间不确定性的任务中。

Details

Method: 通过微调预训练图像扩散模型的潜在视频扩散模型，构建实用的时空扩散先验，并开发通用框架。 Result: 在黑洞成像和动态MRI等科学视频逆问题中，生成了多样且高保真的视频重建结果。 Conclusion: 时空扩散先验显著提升了捕捉复杂时间关系的能力，同时增强了空间保真度。 Abstract: We study how to solve general Bayesian inverse problems involving videos using diffusion model priors. While it is desirable to use a video diffusion prior to effectively capture complex temporal relationships, due to the computational and data requirements of training such a model, prior work has instead relied on image diffusion priors on single frames combined with heuristics to enforce temporal consistency. However, these approaches struggle with faithfully recovering the underlying temporal relationships, particularly for tasks with high temporal uncertainty. In this paper, we demonstrate the feasibility of practical and accessible spatiotemporal diffusion priors by fine-tuning latent video diffusion models from pretrained image diffusion models using limited videos in specific domains. Leveraging this plug-and-play spatiotemporal diffusion prior, we introduce a general and scalable framework for solving video inverse problems. We then apply our framework to two challenging scientific video inverse problems--black hole imaging and dynamic MRI. Our framework enables the generation of diverse, high-fidelity video reconstructions that not only fit observations but also recover multi-modal solutions. By incorporating a spatiotemporal diffusion prior, we significantly improve our ability to capture complex temporal relationships in the data while also enhancing spatial fidelity.

SaRoHead: A Dataset for Satire Detection in Romanian Multi-Domain News Headlines

Mihnea-Alexandru Vîrlan,Răzvan-Alexandru Smădu,Dumitru-Clementin Cercel

Task: 提出SaRoHead，首个用于罗马尼亚多领域新闻标题讽刺检测的语料库。

Motivation: 新闻标题的表达方式和与主题的联系对讽刺检测至关重要，但现有研究缺乏针对罗马尼亚语的语料库。

Details

Method: 构建SaRoHead语料库，并分析非讽刺标题中的点击诱饵对模型的影响。 Result: 研究发现，非讽刺标题中的点击诱饵显著影响讽刺检测模型的性能。 Conclusion: SaRoHead填补了罗马尼亚语讽刺检测语料库的空白，并揭示了点击诱饵对检测的干扰。 Abstract: The headline is an important part of a news article, influenced by expressiveness and connection to the exposed subject. Although most news outlets aim to present reality objectively, some publications prefer a humorous approach in which stylistic elements of satire, irony, and sarcasm blend to cover specific topics. Satire detection can be difficult because a headline aims to expose the main idea behind a news article. In this paper, we propose SaRoHead, the first corpus for satire detection in Romanian multi-domain news headlines. Our findings show that the clickbait used in some non-satirical headlines significantly influences the model.

TokenFocus-VQA: Enhancing Text-to-Image Alignment with Position-Aware Focus and Multi-Perspective Aggregations on LVLMs

Zijian Zhang,Xuhui Zheng,Xuecheng Wu,Chong Peng,Xuezhi Cao

Task: 提出TokenFocus-VQA框架，用于细粒度语义匹配的文本到图像生成模型评估。

Motivation: 现有基于全局相似性度量的评估方法忽略了文本描述与视觉内容之间的关键令牌级对应关系。

Details

Method: 利用大型视觉语言模型（LVLMs）通过视觉问答（VQA）范式，设计令牌感知损失函数，选择性关注预定义词汇位置的分布。 Result: 在NTIRE 2025 T2I质量评估挑战赛中，TokenFocus-VQA在公共评估和官方私有测试集上均排名第二，表现优于传统评估方法。 Conclusion: TokenFocus-VQA框架能够更精确地捕捉文本与图像之间的细粒度语义对齐，优于现有方法。 Abstract: While text-to-image (T2I) generation models have achieved remarkable progress in recent years, existing evaluation methodologies for vision-language alignment still struggle with the fine-grained semantic matching. Current approaches based on global similarity metrics often overlook critical token-level correspondences between textual descriptions and visual content. To this end, we present TokenFocus-VQA, a novel evaluation framework that leverages Large Vision-Language Models (LVLMs) through visual question answering (VQA) paradigm with position-specific probability optimization. Our key innovation lies in designing a token-aware loss function that selectively focuses on probability distributions at pre-defined vocabulary positions corresponding to crucial semantic elements, enabling precise measurement of fine-grained semantical alignment. The proposed framework further integrates ensemble learning techniques to aggregate multi-perspective assessments from diverse LVLMs architectures, thereby achieving further performance enhancement. Evaluated on the NTIRE 2025 T2I Quality Assessment Challenge Track 1, our TokenFocus-VQA ranks 2nd place (0.8445, only 0.0001 lower than the 1st method) on public evaluation and 2nd place (0.8426) on the official private test set, demonstrating superiority in capturing nuanced text-image correspondences compared to conventional evaluation methods.

ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models

Joel Barmettler,Abraham Bernstein,Luca Rossetto

Task: 提出ConceptFormer，一种在不改变预训练语言模型内部结构或依赖知识图谱文本化的情况下，将结构化知识从知识图谱（如Wikidata）注入大型语言模型的新方法。

Motivation: 当前检索增强生成（RAG）方法通常需要修改预训练语言模型的内部结构或依赖知识图谱的文本化，这在令牌使用上效率低下。

Details

Method: ConceptFormer在大型语言模型的嵌入向量空间中操作，创建并注入封装知识图谱节点信息的“概念向量”，同时与冻结的大型语言模型联合训练，生成一个将知识图谱节点映射到其对应概念向量的查找表。 Result: 实验表明，将概念向量注入GPT-2 0.1B显著提高了其事实召回能力（Hit@10），在Wikipedia句子上提升高达272%，在合成句子上提升高达348%。仅注入单个概念向量也能在Wikipedia句子上提升213%，且输入令牌消耗减少130倍。 Conclusion: ConceptFormer通过高效且可扩展的方式为大型语言模型注入结构化世界知识，显著提升了其事实召回能力，同时避免了传统方法的低效问题。 Abstract: Retrieval Augmented Generation (RAG) has enjoyed increased attention in the recent past and recent advancements in Large Language Models (LLMs) have highlighted the importance of integrating world knowledge into these systems. Current RAG methodologies often modify the internal architecture of pre-trained language models (PLMs) or rely on textifying knowledge graphs (KGs), which is inefficient in terms of token usage. This paper introduces ConceptFormer, a new approach to augment LLMs with structured knowledge from KGs, such as Wikidata, without altering their internal structure or relying on textual input of KGs. ConceptFormer operates in the LLM embedding vector space, creating and injecting \emph{concept vectors} that encapsulate the information of the KG nodes directly. Trained in conjunction with a frozen LLM, ConceptFormer generates a comprehensive lookup table that maps KG nodes to their respective concept vectors. The approach aims to enhance the factual recall capabilities of LLMs by enabling them to process these concept vectors natively, thus enriching them with structured world knowledge in an efficient and scalable manner. Our experiments demonstrate that the addition of concept vectors to GPT-2 0.1B substantially increases its factual recall ability (Hit@10) by up to 272\% when tested on sentences from Wikipedia and up to 348\% on synthetically generated sentences. Even injecting only a single concept vector into the prompt increases factual recall ability (Hit@10) by up to 213\% on Wikipedia sentences, significantly outperforming RAG with graph textification while consuming 130x fewer input tokens.

Benchmarking Image Embeddings for E-Commerce: Evaluating Off-the Shelf Foundation Models, Fine-Tuning Strategies and Practical Trade-offs

Urszula Czerwinska,Cenk Bircanoglu,Jeremy Chamoux

Task: 评估基础模型图像嵌入在电子商务分类和检索中的性能。

Motivation: 研究不同预训练模型嵌入在真实电子商务应用中的适用性，为实际应用提供指导。

Details

Method: 通过监督、自监督和文本-图像对比学习训练卷积和Transformer模型，并在六个电子商务数据集上进行全面微调和迁移学习（top-tuning）评估。 Result: 全面微调表现稳定，文本-图像和自监督嵌入在较少训练下可匹配其性能；top-tuning是计算成本更低的替代方案。 Conclusion: 研究结果为嵌入选择和微调策略提供了实用指南，平衡了效率和性能。 Abstract: We benchmark foundation models image embeddings for classification and retrieval in e-Commerce, evaluating their suitability for real-world applications. Our study spans embeddings from pre-trained convolutional and transformer models trained via supervised, self-supervised, and text-image contrastive learning. We assess full fine-tuning and transfer learning (top-tuning) on six diverse e-Commerce datasets: fashion, consumer goods, cars, food, and retail. Results show full fine-tuning consistently performs well, while text-image and self-supervised embeddings can match its performance with less training. While supervised embeddings remain stable across architectures, SSL and contrastive embeddings vary significantly, often benefiting from top-tuning. Top-tuning emerges as an efficient alternative to full fine-tuning, reducing computational costs. We also explore cross-tuning, noting its impact depends on dataset characteristics. Our findings offer practical guidelines for embedding selection and fine-tuning strategies, balancing efficiency and performance.

On the Temporal Question-Answering Capabilities of Large Language Models Over Anonymized Data

Alfredo Garrachón Ruiz,Tomás de la Rosa,Daniel Borrajo

Task: 探索大型语言模型（LLMs）在未训练数据上的时序推理任务中的适用性。

Motivation: 研究LLMs在处理结构化与半结构化匿名数据时的时序推理能力，填补该领域的空白。

Details

Method: 开发直接LLM流程，比较多种方法（如Tree-of-Thought、自反思维和代码执行），并创建RATA数据集评估性能。 Result: 发现仅依赖LLMs难以实现可扩展且可靠的解决方案，需结合集成方法。 Conclusion: 强调集成方法在提升LLMs时序推理能力中的重要性。 Abstract: The applicability of Large Language Models (LLMs) in temporal reasoning tasks over data that is not present during training is still a field that remains to be explored. In this paper we work on this topic, focusing on structured and semi-structured anonymized data. We not only develop a direct LLM pipeline, but also compare various methodologies and conduct an in-depth analysis. We identified and examined seventeen common temporal reasoning tasks in natural language, focusing on their algorithmic components. To assess LLM performance, we created the \textit{Reasoning and Answering Temporal Ability} dataset (RATA), featuring semi-structured anonymized data to ensure reliance on reasoning rather than on prior knowledge. We compared several methodologies, involving SoTA techniques such as Tree-of-Thought, self-reflexion and code execution, tuned specifically for this scenario. Our results suggest that achieving scalable and reliable solutions requires more than just standalone LLMs, highlighting the need for integrated approaches.

On Model and Data Scaling for Skeleton-based Self-Supervised Gait Recognition

Adrian Cosma,Andy Cǎtrunǎ,Emilian Rǎdoi

Task: 研究骨架自监督步态识别中数据量、模型规模和计算资源对性能的影响。

Motivation: 探索神经缩放定律在步态识别领域的适用性，填补现有研究的空白。

Details

Method: 使用基于Transformer的GaitPT架构，在270万野外采集的步行序列上进行预训练，并通过零样本性能评估量化数据、模型规模和计算资源的影响。 Result: 发现性能随规模增加呈幂律提升，数据和计算资源对下游精度有显著影响。 Conclusion: 为实际步态识别系统的资源分配和性能估计提供了实用见解。 Abstract: Gait recognition from video streams is a challenging problem in computer vision biometrics due to the subtle differences between gaits and numerous confounding factors. Recent advancements in self-supervised pretraining have led to the development of robust gait recognition models that are invariant to walking covariates. While neural scaling laws have transformed model development in other domains by linking performance to data, model size, and compute, their applicability to gait remains unexplored. In this work, we conduct the first empirical study scaling on skeleton-based self-supervised gait recognition to quantify the effect of data quantity, model size and compute on downstream gait recognition performance. We pretrain multiple variants of GaitPT - a transformer-based architecture - on a dataset of 2.7 million walking sequences collected in the wild. We evaluate zero-shot performance across four benchmark datasets to derive scaling laws for data, model size, and compute. Our findings demonstrate predictable power-law improvements in performance with increased scale and confirm that data and compute scaling significantly influence downstream accuracy. We further isolate architectural contributions by comparing GaitPT with GaitFormer under controlled compute budgets. These results provide practical insights into resource allocation and performance estimation for real-world gait recognition systems.

Unveiling the Impact of Multimodal Features on Chinese Spelling Correction: From Analysis to Design

Xiaowu Zhang,Hongfei Zhao,Jingyi Hou,Zhijie Liu

Task: 提出并验证一种新型多模态模型NamBert，用于中文拼写纠错（CSC）任务。

Motivation: 现有的大型语言模型（LLMs）在中文拼写纠错中存在过纠问题，而多模态模型在利用音形信息提升纠错性能方面仍有挑战。

Details

Method: 提出多模态字符使用分析实验（MACU），并基于实验结果开发多模态模型NamBert。 Result: 在基准数据集上，NamBert优于现有最优方法（SOTA），并通过与LLMs的系统对比展示了其优势与局限性。 Conclusion: NamBert为中文拼写纠错提供了一种有效的多模态解决方案，并公开了代码和模型。 Abstract: The Chinese Spelling Correction (CSC) task focuses on detecting and correcting spelling errors in sentences. Current research primarily explores two approaches: traditional multimodal pre-trained models and large language models (LLMs). However, LLMs face limitations in CSC, particularly over-correction, making them suboptimal for this task. While existing studies have investigated the use of phonetic and graphemic information in multimodal CSC models, effectively leveraging these features to enhance correction performance remains a challenge. To address this, we propose the Multimodal Analysis for Character Usage (\textbf{MACU}) experiment, identifying potential improvements for multimodal correctison. Based on empirical findings, we introduce \textbf{NamBert}, a novel multimodal model for Chinese spelling correction. Experiments on benchmark datasets demonstrate NamBert's superiority over SOTA methods. We also conduct a comprehensive comparison between NamBert and LLMs, systematically evaluating their strengths and limitations in CSC. Our code and model are available at https://github.com/iioSnail/NamBert.

RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions

Youngwan Jin,Michal Kovac,Yagiz Nalcakan,Hyeongjin Ju,Hanbin Song,Sanghyeop Yeo,Shiho Kim

Task: Introduce the RGB and SWIR Multispectral Driving (RASMD) dataset to address the lack of large-scale SWIR data for autonomous driving.

Motivation: Current autonomous driving algorithms rely on visible spectrum data, which performs poorly in adverse conditions; SWIR imaging offers advantages but lacks datasets.

Details

Method: Collect 100,000 synchronized RGB-SWIR image pairs across diverse conditions, provide annotations for object detection and RGB-SWIR translation, and conduct experiments. Result: Combining RGB and SWIR data improves detection accuracy, especially in challenging conditions. Conclusion: The RASMD dataset will advance research in multispectral imaging for autonomous driving. Abstract: Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions like fog, rain, snow, glare, and high contrast. Although other spectral bands like near-infrared (NIR) and long-wave infrared (LWIR) can enhance vision perception in such situations, they have limitations and lack large-scale datasets and benchmarks. Short-wave infrared (SWIR) imaging offers several advantages over NIR and LWIR. However, no publicly available large-scale datasets currently incorporate SWIR data for autonomous driving. To address this gap, we introduce the RGB and SWIR Multispectral Driving (RASMD) dataset, which comprises 100,000 synchronized and spatially aligned RGB-SWIR image pairs collected across diverse locations, lighting, and weather conditions. In addition, we provide a subset for RGB-SWIR translation and object detection annotations for a subset of challenging traffic scenarios to demonstrate the utility of SWIR imaging through experiments on both object detection and RGB-to-SWIR image translation. Our experiments show that combining RGB and SWIR data in an ensemble framework significantly improves detection accuracy compared to RGB-only approaches, particularly in conditions where visible-spectrum sensors struggle. We anticipate that the RASMD dataset will advance research in multispectral imaging for autonomous driving and robust perception systems.

Synthetic Fluency: Hallucinations, Confabulations, and the Creation of Irish Words in LLM-Generated Translations

Sheila Castilho,Zoe Fitzsimmons,Claire Holton,Aoife Mc Donagh

Task: 研究大型语言模型（LLM）在爱尔兰语翻译中产生的幻觉现象，特别是生成不存在的单词的情况。

Motivation: 探讨LLM在低资源、形态丰富的语言（如爱尔兰语）中的表现，以及这些幻觉对语言演变的潜在影响。

Details

Method: 对幻觉进行分类（动词和名词），分析其是否符合爱尔兰语形态规则，并比较GPT-4.o和GPT-4.o Mini模型的幻觉频率。 Result: 发现两种模型产生的幻觉类型相似，但Mini模型的幻觉频率显著更高；幻觉部分符合爱尔兰语形态规则。 Conclusion: 提出关于LLM对爱尔兰语词汇和语言演变潜在影响的思考，旨在引发对技术如何影响低资源语言的讨论。 Abstract: This study examines hallucinations in Large Language Model (LLM) translations into Irish, specifically focusing on instances where the models generate novel, non-existent words. We classify these hallucinations within verb and noun categories, identifying six distinct patterns among the latter. Additionally, we analyse whether these hallucinations adhere to Irish morphological rules and what linguistic tendencies they exhibit. Our findings show that while both GPT-4.o and GPT-4.o Mini produce similar types of hallucinations, the Mini model generates them at a significantly higher frequency. Beyond classification, the discussion raises speculative questions about the implications of these hallucinations for the Irish language. Rather than seeking definitive answers, we offer food for thought regarding the increasing use of LLMs and their potential role in shaping Irish vocabulary and linguistic evolution. We aim to prompt discussion on how such technologies might influence language over time, particularly in the context of low-resource, morphologically rich languages.

VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model

Haozhan Shen,Peng Liu,Jingcheng Li,Chunxin Fang,Yibo Ma,Jiajia Liao,Qiaoli Shen,Zilun Zhang,Kangjia Zhao,Qianqian Zhang,Ruochen Xu,Tiancheng Zhao

Task: 扩展DeepSeek R1的强化学习方法到视觉语言模型（VLMs），以提升其视觉推理能力。

Motivation: 视觉理解任务具有明确的真实标注，适合基于规则的奖励机制，因此探索强化学习在视觉领域的应用潜力。

Details

Method: 开发了VLM-R1框架，利用强化学习优化VLMs在视觉语言任务中的表现。 Result: RL模型在视觉理解任务中表现优异，泛化能力超越监督微调（SFT），并揭示了奖励机制中的关键现象。 Conclusion: 强化学习能有效增强视觉语言模型的能力，研究结果和开源贡献有望推动视觉语言强化学习领域的进一步发展。 Abstract: Recently DeepSeek R1 has shown that reinforcement learning (RL) can substantially improve the reasoning capabilities of Large Language Models (LLMs) through a simple yet effective design. The core of R1 lies in its rule-based reward formulation, which leverages tasks with deterministic ground-truth answers to enable precise and stable reward computation. In the visual domain, we similarly observe that a wide range of visual understanding tasks are inherently equipped with well-defined ground-truth annotations. This property makes them naturally compatible with rule-based reward mechanisms. Motivated by this observation, we investigate the extension of R1-style reinforcement learning to Vision-Language Models (VLMs), aiming to enhance their visual reasoning capabilities. To this end, we develop VLM-R1, a dedicated framework designed to harness RL for improving VLMs' performance on general vision-language tasks. Using this framework, we further explore the feasibility of applying RL to visual domain. Experimental results indicate that the RL-based model not only delivers competitive performance on visual understanding tasks but also surpasses Supervised Fine-Tuning (SFT) in generalization ability. Furthermore, we conduct comprehensive ablation studies that uncover a series of noteworthy insights, including the presence of reward hacking in object detection, the emergence of the "OD aha moment", the impact of training data quality, and the scaling behavior of RL across different model sizes. Through these analyses, we aim to deepen the understanding of how reinforcement learning enhances the capabilities of vision-language models, and we hope our findings and open-source contributions will support continued progress in the vision-language RL community. Our code and model are available at https://github.com/om-ai-lab/VLM-R1

Context-Aware Monolingual Human Evaluation of Machine Translation

Silvio Picinini,Sheila Castilho

Task: 探索无源文本参考下，基于上下文感知的单语人工评估在机器翻译评估中的潜力。

Motivation: 比较单语评估与双语评估（有源文本）在单一机器翻译系统和成对机器翻译系统评估中的表现。

Details

Method: 四位专业翻译进行单语和双语评估，包括评分、错误标注及反馈。 Result: 上下文感知的单语评估结果与双语评估相当，表明其作为一种高效评估机器翻译方法的可行性。 Conclusion: 单语评估在机器翻译评估中具有潜在的高效性和可行性。 Abstract: This paper explores the potential of context-aware monolingual human evaluation for assessing machine translation (MT) when no source is given for reference. To this end, we compare monolingual with bilingual evaluations (with source text), under two scenarios: the evaluation of a single MT system, and the comparative evaluation of pairwise MT systems. Four professional translators performed both monolingual and bilingual evaluations by assigning ratings and annotating errors, and providing feedback on their experience. Our findings suggest that context-aware monolingual human evaluation achieves comparable outcomes to human bilingual evaluations, and suggest the feasibility and potential of monolingual evaluation as an efficient approach to assessing MT.

End-to-End Facial Expression Detection in Long Videos

Yini Fang,Alec Diallo,Yiqi Shi,Frederic Jumelle,Bertram Shi

Task: 提出一种端到端的面部表情检测网络（FEDN），联合优化表情定位和识别任务。

Motivation: 现有方法将表情定位和识别分开处理，导致错误传播、特征学习效率低和性能不佳，缺乏联合优化。

Details

Method: 引入基于注意力的特征提取模块，结合片段注意力和滑动窗口注意力，统一优化两个任务。 Result: 在CASME^2和CASME^3数据集上实现了最先进的定位和识别准确率。 Conclusion: 联合优化显著减少了错误传播，提升了长视频中面部表情检测的鲁棒性。 Abstract: Facial expression detection involves two interrelated tasks: spotting, which identifies the onset and offset of expressions, and recognition, which classifies them into emotional categories. Most existing methods treat these tasks separately using a two-step training pipelines. A spotting model first detects expression intervals. A recognition model then classifies the detected segments. However, this sequential approach leads to error propagation, inefficient feature learning, and suboptimal performance due to the lack of joint optimization of the two tasks. We propose FEDN, an end-to-end Facial Expression Detection Network that jointly optimizes spotting and recognition. Our model introduces a novel attention-based feature extraction module, incorporating segment attention and sliding window attention to improve facial feature learning. By unifying two tasks within a single network, we greatly reduce error propagation and enhance overall performance. Experiments on CASME}^2 and CASME^3 demonstrate state-of-the-art accuracy for both spotting and detection, underscoring the benefits of joint optimization for robust facial expression detection in long videos.

Proactive User Information Acquisition via Chats on User-Favored Topics

Shiki Sato,Jun Baba,Asahi Hentona,Shinji Iwata,Akifumi Yoshimoto,Koichiro Yoshino

Task: 提出并定义PIVOT任务，旨在通过聊天获取用户对预定义问题的回答，同时避免让用户感到突兀。

Motivation: 为面向聊天对话系统提供技术基础，使其能够在用户喜欢的主题聊天中主动获取特定用户信息。

Details

Method: 构建适合分析的数据集，并基于数据集分析开发简单但有效的系统。 Result: 发现即使最新的LLMs在PIVOT任务中成功率较低，开发的系统通过数据集分析取得了更好的效果。 Conclusion: PIVOT任务为聊天对话系统提供了新的研究方向，开发的系统展示了其有效性。 Abstract: Chat-oriented dialogue systems designed to provide tangible benefits, such as sharing the latest news or preventing frailty in senior citizens, often require Proactive acquisition of specific user Information via chats on user-faVOred Topics (PIVOT). This study proposes the PIVOT task, designed to advance the technical foundation for these systems. In this task, a system needs to acquire the answers of a user to predefined questions without making the user feel abrupt while engaging in a chat on a predefined topic. We found that even recent large language models (LLMs) show a low success rate in the PIVOT task. We constructed a dataset suitable for the analysis to develop more effective systems. Finally, we developed a simple but effective system for this task by incorporating insights obtained through the analysis of this dataset.

S2R-HDR: A Large-Scale Rendered Dataset for HDR Fusion

Yujin Wang,Jiarui Wu,Yichen Bian,Fan Zhang,Tianfan Xue

Task: 提出S2R-HDR，首个大规模高质量合成数据集用于HDR融合。

Motivation: 解决基于学习的HDR融合泛化能力受限于训练数据不足的问题。

Details

Method: 使用Unreal Engine 5设计多样化HDR场景，开发高效渲染流程，并引入S2R-Adapter进行域适应。 Result: 在真实数据集上实现了最先进的HDR重建性能。 Conclusion: S2R-HDR数据集和S2R-Adapter有效提升了HDR融合的泛化能力。 Abstract: The generalization of learning-based high dynamic range (HDR) fusion is often limited by the availability of training data, as collecting large-scale HDR images from dynamic scenes is both costly and technically challenging. To address these challenges, we propose S2R-HDR, the first large-scale high-quality synthetic dataset for HDR fusion, with 24,000 HDR samples. Using Unreal Engine 5, we design a diverse set of realistic HDR scenes that encompass various dynamic elements, motion types, high dynamic range scenes, and lighting. Additionally, we develop an efficient rendering pipeline to generate realistic HDR images. To further mitigate the domain gap between synthetic and real-world data, we introduce S2R-Adapter, a domain adaptation designed to bridge this gap and enhance the generalization ability of models. Experimental results on real-world datasets demonstrate that our approach achieves state-of-the-art HDR reconstruction performance. Dataset and code will be available at https://openimaginglab.github.io/S2R-HDR.

MRD-RAG: Enhancing Medical Diagnosis with Multi-Round Retrieval-Augmented Generation

Yixiang Chen,Penglei Sun,Xiang Li,Xiaowen Chu

Task: 提出一种多轮诊断检索增强生成框架（MRD-RAG），以模拟医生的诊断过程，解决现有医疗RAG框架在多轮对话和疾病关联性方面的不足。

Motivation: 现有医疗RAG框架多针对单轮问答任务，且未考虑疾病间的关联性，无法像医生一样进行精准的多轮诊断。

Details

Method: 设计MRD-RAG框架，分析潜在疾病的诊断信息，并模拟医生的多轮诊断过程。 Result: 在两个现代医学数据集和两个中医数据集上的实验表明，MRD-RAG显著提升了LLMs的诊断性能。 Conclusion: MRD-RAG框架在医疗诊断中具有潜力，能够有效支持多轮诊断任务。 Abstract: In recent years, accurately and quickly deploying medical large language models (LLMs) has become a significant trend. Among these, retrieval-augmented generation (RAG) has garnered significant attention due to its features of rapid deployment and privacy protection. However, existing medical RAG frameworks still have shortcomings. Most existing medical RAG frameworks are designed for single-round question answering tasks and are not suitable for multi-round diagnostic dialogue. On the other hand, existing medical multi-round RAG frameworks do not consider the interconnections between potential diseases to inquire precisely like a doctor. To address these issues, we propose a Multi-Round Diagnostic RAG (MRD-RAG) framework that mimics the doctor's diagnostic process. This RAG framework can analyze diagnosis information of potential diseases and accurately conduct multi-round diagnosis like a doctor. To evaluate the effectiveness of our proposed frameworks, we conduct experiments on two modern medical datasets and two traditional Chinese medicine datasets, with evaluations by GPT and human doctors on different methods. The results indicate that our RAG framework can significantly enhance the diagnostic performance of LLMs, highlighting the potential of our approach in medical diagnosis. The code and data can be found in our project website https://github.com/YixiangCh/MRD-RAG/tree/master.

LAPIS: A novel dataset for personalized image aesthetic assessment

Anne-Sofie Maerten,Li-Wei Chen,Stefanie De Winter,Christophe Bossens,Johan Wagemans

Task: 提出Leuven Art Personalized Image Set (LAPIS)，首个适用于个性化图像美学评估（PIAA）的艺术作品数据集。

Motivation: 填补艺术作品在个性化图像美学评估领域的数据集空白，并提供丰富的图像和个人属性以支持研究。

Details

Method: 通过与艺术史学家合作精心策划包含11,723张图像的LAPIS数据集，并评估两种现有PIAA模型的性能。 Result: 实验表明，移除某些个人和图像属性会导致性能下降，且现有模型在艺术图像美学评估中存在相似错误。 Conclusion: LAPIS为艺术图像美学评估提供了新基准，并揭示了现有模型的局限性，呼吁进一步改进。 Abstract: We present the Leuven Art Personalized Image Set (LAPIS), a novel dataset for personalized image aesthetic assessment (PIAA). It is the first dataset with images of artworks that is suitable for PIAA. LAPIS consists of 11,723 images and was meticulously curated in collaboration with art historians. Each image has an aesthetics score and a set of image attributes known to relate to aesthetic appreciation. Besides rich image attributes, LAPIS offers rich personal attributes of each annotator. We implemented two existing state-of-the-art PIAA models and assessed their performance on LAPIS. We assess the contribution of personal attributes and image attributes through ablation studies and find that performance deteriorates when certain personal and image attributes are removed. An analysis of failure cases reveals that both existing models make similar incorrect predictions, highlighting the need for improvements in artistic image aesthetic assessment. The LAPIS project page can be found at: https://github.com/Anne-SofieMaerten/LAPIS

DeepGreen: Effective LLM-Driven Green-washing Monitoring System Designed for Empirical Testing -- Evidence from China

Congluo Xu,Yu Miao,Yiling Xiao,Chengmengjia Lin

Task: 提出DeepGreen系统，利用大语言模型（LLM）检测企业绿色洗白行为。

Motivation: 为监管机构和投资者提供一种主动监测工具，补充传统方法，并揭示绿色实施对企业资产回报率的影响。

Details

Method: 采用双层LLM分析，初步识别财务报告中的绿色关键词，并通过迭代语义分析评估其实施程度，生成核心变量GreenImplement。 Result: 分析204份财务报告，验证GreenImplement与华证ESG评分的相关性，发现绿色实施显著提升资产回报率，但中小企业效果有限。 Conclusion: DeepGreen为绿色洗白检测提供新视角，绿色实施对资产回报率有积极影响，但存在规模异质性。 Abstract: This paper proposes DeepGreen, an Large Language Model Driven (LLM-Driven) system for detecting corporate green-washing behaviour. Utilizing dual-layer LLM analysis, DeepGreen preliminarily identifies potential green keywords in financial statements and then assesses their implementation degree via iterative semantic analysis of LLM. A core variable GreenImplement is derived from the ratio from the two layers' output. We extract 204 financial statements of 68 companies from A-share market over three years, comprising 89,893 words, and analyse them through DeepGreen. Our analysis, supported by violin plots and K-means clustering, reveals insights and validates the variable against the Huazheng ESG rating. It offers a novel perspective for regulatory agencies and investors, serving as a proactive monitoring tool that complements traditional methods.Empirical tests show that green implementation can significantly boost the asset return rate of companies, but there is heterogeneity in scale. Small and medium-sized companies have limited contribution to asset return via green implementation, so there is a stronger motivation for green-washing.

FMNV: A Dataset of Media-Published News Videos for Fake News Detection

Yihao Wang,Zhong Qian,Peifeng Li

Task: 构建一个专门由媒体组织发布的新闻视频数据集FMNV，并提出一个基于双流架构的基线模型FMNVD用于多模态假新闻检测。

Motivation: 现有数据集多为用户生成视频，而由媒体组织发布的专业制作的假新闻视频对社会危害更大，但缺乏相关研究。

Details

Method: 通过分析现有数据集和自建数据集FMNV，将假新闻视频分为四类，并利用大型语言模型（LLMs）生成欺骗性内容；提出FMNVD模型，结合CLIP和Faster R-CNN进行视频特征提取，并通过共注意力机制优化多模态特征。 Result: 实验表明FMNV在多个基线模型上具有泛化能力，且FMNVD在检测效果上表现优越。 Conclusion: 本研究为检测高影响力假新闻提供了关键基准，并推动了跨模态不一致性分析方法的发展。 Abstract: News media, particularly video-based platforms, have become deeply embedded in daily life, concurrently amplifying risks of misinformation dissemination. Consequently, multimodal fake news detection has garnered significant research attention. However, existing datasets predominantly comprise user-generated videos characterized by crude editing and limited public engagement, whereas professionally crafted fake news videos disseminated by media outlets often politically or virally motivated pose substantially greater societal harm. To address this gap, we construct FMNV, a novel dataset exclusively composed of news videos published by media organizations. Through empirical analysis of existing datasets and our curated collection, we categorize fake news videos into four distinct types. Building upon this taxonomy, we employ Large Language Models (LLMs) to automatically generate deceptive content by manipulating authentic media-published news videos. Furthermore, we propose FMNVD, a baseline model featuring a dual-stream architecture integrating CLIP and Faster R-CNN for video feature extraction, enhanced by co-attention mechanisms for feature refinement and multimodal aggregation. Comparative experiments demonstrate both the generalization capability of FMNV across multiple baselines and the superior detection efficacy of FMNVD. This work establishes critical benchmarks for detecting high-impact fake news in media ecosystems while advancing methodologies for cross-modal inconsistency analysis.

Automated Construction of a Knowledge Graph of Nuclear Fusion Energy for Effective Elicitation and Retrieval of Information

A. Loreti,K. Chen,R. George,R. Firth,A. Agnello,S. Tanaka

Task: 构建一个针对核聚变能源领域的知识图谱，并开发一个基于知识图谱的检索增强生成系统。

Motivation: 核聚变能源领域知识范围广且异构性强，需要一种自动化的方法来结构化这些知识，并为复杂查询提供上下文相关的答案。

Details

Method: 采用多步骤方法，包括自动命名实体识别、实体解析，以及结合预训练大语言模型和多提示方法的知识图谱检索增强生成系统。 Result: 成功构建了首个核聚变能源知识图谱，并验证了预训练大语言模型在解决命名实体识别和实体解析问题上的性能。 Conclusion: 该方法为高度专业化领域的知识图谱构建和复杂查询回答提供了有效解决方案。 Abstract: In this document, we discuss a multi-step approach to automated construction of a knowledge graph, for structuring and representing domain-specific knowledge from large document corpora. We apply our method to build the first knowledge graph of nuclear fusion energy, a highly specialized field characterized by vast scope and heterogeneity. This is an ideal benchmark to test the key features of our pipeline, including automatic named entity recognition and entity resolution. We show how pre-trained large language models can be used to address these challenges and we evaluate their performance against Zipf's law, which characterizes human-generated natural language. Additionally, we develop a knowledge-graph retrieval-augmented generation system that combines large language models with a multi-prompt approach. This system provides contextually relevant answers to natural-language queries, including complex multi-hop questions that require reasoning across interconnected entities.

Multi-modal Reference Learning for Fine-grained Text-to-Image Retrieval

Zehong Ma,Hao Chen,Wei Zeng,Limin Su,Shiliang Zhang

Task: 提出一种多模态参考学习框架，以解决细粒度文本到图像检索中文本模糊性问题。

Motivation: 现有方法假设训练图像的文本描述准确，但实际文本描述可能模糊且无法捕捉图像的区分性视觉细节，导致表示学习不准确。

Details

Method: 提出多模态参考构建模块和参考引导表示学习模块，利用多模态参考学习更准确的视觉和文本表示，并引入基于参考的细化方法优化检索结果。 Result: 在五个细粒度文本到图像检索数据集上表现优异，例如在RSTPReid数据集上Rank1准确率达到56.2%，超过CFine方法5.6%。 Conclusion: 多模态参考学习框架能有效缓解文本模糊性，提升细粒度文本到图像检索性能。 Abstract: Fine-grained text-to-image retrieval aims to retrieve a fine-grained target image with a given text query. Existing methods typically assume that each training image is accurately depicted by its textual descriptions. However, textual descriptions can be ambiguous and fail to depict discriminative visual details in images, leading to inaccurate representation learning. To alleviate the effects of text ambiguity, we propose a Multi-Modal Reference learning framework to learn robust representations. We first propose a multi-modal reference construction module to aggregate all visual and textual details of the same object into a comprehensive multi-modal reference. The multi-modal reference hence facilitates the subsequent representation learning and retrieval similarity computation. Specifically, a reference-guided representation learning module is proposed to use multi-modal references to learn more accurate visual and textual representations. Additionally, we introduce a reference-based refinement method that employs the object references to compute a reference-based similarity that refines the initial retrieval results. Extensive experiments are conducted on five fine-grained text-to-image retrieval datasets for different text-to-image retrieval tasks. The proposed method has achieved superior performance over state-of-the-art methods. For instance, on the text-to-person image retrieval dataset RSTPReid, our method achieves the Rank1 accuracy of 56.2\%, surpassing the recent CFine by 5.6\%.

NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark

Vladislav Mikhailov,Tita Enstad,David Samuel,Hans Christian Farsethås,Andrey Kutuzov,Erik Velldal,Lilja Øvrelid

Task: 介绍并评估NorEval，一个用于挪威生成语言模型的大规模标准化基准测试套件。

Motivation: 现有挪威语基准测试覆盖范围有限，NorEval旨在填补这一空白，提供更全面的任务类别和语言标准覆盖。

Details

Method: NorEval包含24个高质量人工创建的数据集，其中5个为全新构建，整合了100多个提示词，并集成到LM Evaluation Harness中。 Result: 对19个开源预训练和指令调优的挪威语言模型进行了多场景基准测试，并提供了公开可用的数据集和评估框架。 Conclusion: NorEval为挪威生成语言模型的标准化评估提供了全面且灵活的工具，支持多种任务和语言标准。 Abstract: This paper introduces NorEval, a new and comprehensive evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). NorEval consists of 24 high-quality human-created datasets -- of which five are created from scratch. In contrast to existing benchmarks for Norwegian, NorEval covers a broad spectrum of task categories targeting Norwegian language understanding and generation, establishes human baselines, and focuses on both of the official written standards of the Norwegian language: Bokm{\aa}l and Nynorsk. All our datasets and a collection of over 100 human-written prompts are integrated into LM Evaluation Harness, ensuring flexible and reproducible evaluation. We describe the NorEval design and present the results of benchmarking 19 open-source pre-trained and instruction-tuned LMs for Norwegian in various scenarios. Our benchmark, evaluation framework, and annotation materials are publicly available.

Benchmarking Multi-Organ Segmentation Tools for Multi-Parametric T1-weighted Abdominal MRI

Nicole Tran,Anisa Prasad,Yan Zhuang,Tejas Sudharshan Mathai,Boah Kim,Sydney Lewis,Pritam Mukherjee,Jianfei Liu,Ronald M. Summers

Task: 量化三种公开工具（MRSeg、TS和VIBE）在特定MRI序列类型上的多器官分割性能。

Motivation: 多器官分割在MRI研究中至关重要，但现有工具在特定MRI序列上的性能尚未量化。

Details

Method: 使用40个来自Duke Liver Dataset的MRI体积，手动标注10个腹部结构，并评估三种工具的性能。 Result: MRSeg表现最佳，Dice得分为80.7±18.6，HD误差为8.9±10.4 mm，显著优于TS和VIBE。 Conclusion: MRSeg在特定MRI序列类型上的多器官分割性能优于其他工具。 Abstract: The segmentation of multiple organs in multi-parametric MRI studies is critical for many applications in radiology, such as correlating imaging biomarkers with disease status (e.g., cirrhosis, diabetes). Recently, three publicly available tools, such as MRSegmentator (MRSeg), TotalSegmentator MRI (TS), and TotalVibeSegmentator (VIBE), have been proposed for multi-organ segmentation in MRI. However, the performance of these tools on specific MRI sequence types has not yet been quantified. In this work, a subset of 40 volumes from the public Duke Liver Dataset was curated. The curated dataset contained 10 volumes each from the pre-contrast fat saturated T1, arterial T1w, venous T1w, and delayed T1w phases, respectively. Ten abdominal structures were manually annotated in these volumes. Next, the performance of the three public tools was benchmarked on this curated dataset. The results indicated that MRSeg obtained a Dice score of 80.7 $\pm$ 18.6 and Hausdorff Distance (HD) error of 8.9 $\pm$ 10.4 mm. It fared the best ($p < .05$) across the different sequence types in contrast to TS and VIBE.

Efficient Tuning of Large Language Models for Knowledge-Grounded Dialogue Generation

Bo Zhang,Hui Ma,Dailin Li,Jian Ding,Jian Wang,Bo Xu,HongFei Lin

Task: 提出一种名为KEDiT的方法，用于微调大型语言模型（LLMs）以实现基于知识的对话生成。

Motivation: 大型语言模型在文本理解和生成方面表现出色，但缺乏利用最新或领域特定知识的能力。

Details

Method: KEDiT分为两个阶段：首先通过信息瓶颈压缩检索到的知识为可学习参数，其次通过轻量级知识感知适配器将这些参数集成到LLM中。 Result: 在Wizard of Wikipedia和PubMed-Dialog数据集上的实验表明，KEDiT在生成上下文相关且信息丰富的回答方面优于基线方法。 Conclusion: KEDiT结合了预训练LLMs的优势和动态知识整合的适应性，为医学等领域提供了可扩展的解决方案。 Abstract: Large language models (LLMs) demonstrate remarkable text comprehension and generation capabilities but often lack the ability to utilize up-to-date or domain-specific knowledge not included in their training data. To address this gap, we introduce KEDiT, an efficient method for fine-tuning LLMs for knowledge-grounded dialogue generation. KEDiT operates in two main phases: first, it employs an information bottleneck to compress retrieved knowledge into learnable parameters, retaining essential information while minimizing computational overhead. Second, a lightweight knowledge-aware adapter integrates these compressed knowledge vectors into the LLM during fine-tuning, updating less than 2\% of the model parameters. The experimental results on the Wizard of Wikipedia and a newly constructed PubMed-Dialog dataset demonstrate that KEDiT excels in generating contextually relevant and informative responses, outperforming competitive baselines in automatic, LLM-based, and human evaluations. This approach effectively combines the strengths of pretrained LLMs with the adaptability needed for incorporating dynamic knowledge, presenting a scalable solution for fields such as medicine.

MMLA: Multi-Environment, Multi-Species, Low-Altitude Aerial Footage Dataset

Jenna Kline,Samuel Stevens,Guy Maalouf,Camille Rondeau Saint-Jean,Dat Nguyen Ngoc,Majid Mirmehdi,David Guerin,Tilo Burghardt,Elzbieta Pastucha,Blair Costelloe,Matthew Watson,Thomas Richardson,Ulrik Pagh Schultz Lundquist

Task: 提出并评估一种用于低空无人机图像中野生动物实时检测的多环境、多物种数据集（MMLA）。

Motivation: 填补现有计算机视觉模型在低空航拍图像评估及跨物种和跨环境泛化性研究上的空白。

Details

Method: 构建MMLA数据集，包含三个不同环境的无人机视频，涵盖五种物种，并评估三种YOLO模型（YOLOv5m、YOLOv8m、YOLOv11m）的检测性能。 Result: 不同地点和物种间的检测性能存在显著差异，模型表现因环境和物种而异。 Conclusion: 强调了跨环境评估检测算法对无人机野生动物监测应用的重要性。 Abstract: Real-time wildlife detection in drone imagery is critical for numerous applications, including animal ecology, conservation, and biodiversity monitoring. Low-altitude drone missions are effective for collecting fine-grained animal movement and behavior data, particularly if missions are automated for increased speed and consistency. However, little work exists on evaluating computer vision models on low-altitude aerial imagery and generalizability across different species and settings. To fill this gap, we present a novel multi-environment, multi-species, low-altitude aerial footage (MMLA) dataset. MMLA consists of drone footage collected across three diverse environments: Ol Pejeta Conservancy and Mpala Research Centre in Kenya, and The Wilds Conservation Center in Ohio, which includes five species: Plains zebras, Grevy's zebras, giraffes, onagers, and African Painted Dogs. We comprehensively evaluate three YOLO models (YOLOv5m, YOLOv8m, and YOLOv11m) for detecting animals. Results demonstrate significant performance disparities across locations and species-specific detection variations. Our work highlights the importance of evaluating detection algorithms across different environments for robust wildlife monitoring applications using drones.

Plan-and-Refine: Diverse and Comprehensive Retrieval-Augmented Generation

Alireza Salemi,Chris Samarinas,Hamed Zamani

Task: 研究检索增强大型语言模型（LLMs）在生成多样且全面回答时的局限性，并引入基于两阶段系统设计的Plan-and-Refine（P&R）框架。

Motivation: 解决LLMs在生成回答时多样性和全面性不足的问题。

Details

Method: P&R框架分为全局探索阶段（生成多样化的查询计划）和局部利用阶段（基于每个计划生成并迭代优化回答提案），最后通过奖励模型选择最佳提案。 Result: 在ANTIQUE和TREC数据集上，P&R显著优于基线方法，分别提升13.1%和15.41%。 Conclusion: P&R框架在提高回答的事实性和覆盖范围方面表现出显著效果。 Abstract: This paper studies the limitations of (retrieval-augmented) large language models (LLMs) in generating diverse and comprehensive responses, and introduces the Plan-and-Refine (P&R) framework based on a two phase system design. In the global exploration phase, P&R generates a diverse set of plans for the given input, where each plan consists of a list of diverse query aspects with corresponding additional descriptions. This phase is followed by a local exploitation phase that generates a response proposal for the input query conditioned on each plan and iteratively refines the proposal for improving the proposal quality. Finally, a reward model is employed to select the proposal with the highest factuality and coverage. We conduct our experiments based on the ICAT evaluation methodology--a recent approach for answer factuality and comprehensiveness evaluation. Experiments on the two diverse information seeking benchmarks adopted from non-factoid question answering and TREC search result diversification tasks demonstrate that P&R significantly outperforms baselines, achieving up to a 13.1% improvement on the ANTIQUE dataset and a 15.41% improvement on the TREC dataset. Furthermore, a smaller scale user study confirms the substantial efficacy of the P&R framework.

SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding

Yangliu Hu,Zikai Song,Na Feng,Yawei Luo,Junqing Yu,Yi-Ping Phoebe Chen,Wei Yang

Task: 提出一种自监督片段微调方法（SF$^2$T）和新的基准数据集FineVidBench，以提升视频大语言模型（Video-LLMs）在细粒度视频理解方面的能力。

Motivation: 现有Video-LLMs在整体视频描述上表现良好，但在细粒度理解（如视觉动态和视频细节查询）方面存在不足。

Details

Method: 通过自监督片段微调（SF$^2$T）利用视频的固有特性进行训练，避免人工标注，并设计FineVidBench数据集进行多层面评估。 Result: 实验表明，SF$^2$T显著提升了模型对时空细节的捕捉和解释能力。 Conclusion: SF$^2$T是一种高效的方法，能够增强Video-LLMs的细粒度视频理解能力，同时减少对标注数据的依赖。 Abstract: Video-based Large Language Models (Video-LLMs) have witnessed substantial advancements in recent years, propelled by the advancement in multi-modal LLMs. Although these models have demonstrated proficiency in providing the overall description of videos, they struggle with fine-grained understanding, particularly in aspects such as visual dynamics and video details inquiries. To tackle these shortcomings, we find that fine-tuning Video-LLMs on self-supervised fragment tasks, greatly improve their fine-grained video understanding abilities. Hence we propose two key contributions:(1) Self-Supervised Fragment Fine-Tuning (SF$^2$T), a novel effortless fine-tuning method, employs the rich inherent characteristics of videos for training, while unlocking more fine-grained understanding ability of Video-LLMs. Moreover, it relieves researchers from labor-intensive annotations and smartly circumvents the limitations of natural language, which often fails to capture the complex spatiotemporal variations in videos; (2) A novel benchmark dataset, namely FineVidBench, for rigorously assessing Video-LLMs' performance at both the scene and fragment levels, offering a comprehensive evaluation of their capabilities. We assessed multiple models and validated the effectiveness of SF$^2$T on them. Experimental results reveal that our approach improves their ability to capture and interpret spatiotemporal details.

A System for Comprehensive Assessment of RAG Frameworks

Mattia Rengo,Senad Beadini,Domenico Alfano,Roberto Abbruzzese

Task: 提出一个名为SCARF的模块化评估框架，用于全面评估RAG系统的性能。

Motivation: 现有评估框架无法全面评估RAG系统在实际部署场景中的表现。

Details

Method: 设计了一个模块化、灵活的评估框架SCARF，支持端到端的黑盒评估方法，并集成自动化测试和性能报告功能。 Result: SCARF能够系统地对不同RAG框架进行性能比较，并支持多种部署配置和实际考量。 Conclusion: SCARF为研究者和行业专业人士提供了一个可扩展且适应性强的RAG评估解决方案。 Abstract: Retrieval Augmented Generation (RAG) has emerged as a standard paradigm for enhancing the factual accuracy and contextual relevance of Large Language Models (LLMs) by integrating retrieval mechanisms. However, existing evaluation frameworks fail to provide a holistic black-box approach to assessing RAG systems, especially in real-world deployment scenarios. To address this gap, we introduce SCARF (System for Comprehensive Assessment of RAG Frameworks), a modular and flexible evaluation framework designed to benchmark deployed RAG applications systematically. SCARF provides an end-to-end, black-box evaluation methodology, enabling a limited-effort comparison across diverse RAG frameworks. Our framework supports multiple deployment configurations and facilitates automated testing across vector databases and LLM serving strategies, producing a detailed performance report. Moreover, SCARF integrates practical considerations such as response coherence, providing a scalable and adaptable solution for researchers and industry professionals evaluating RAG applications. Using the REST APIs interface, we demonstrate how SCARF can be applied to real-world scenarios, showcasing its flexibility in assessing different RAG frameworks and configurations. SCARF is available at GitHub repository.

PIDSR:ComplementaryPolarizedImageDemosaicingandSuper-Resolution

Shuangfan Zhou,Chu Zhou,Youwei Lyu,Heng Guo,Zhanyu Ma,Boxin Shi,Imari Sato

Task: 提出一种联合框架PIDSR，直接从CPFA原始图像中获取高质量高分辨率偏振图像，并提高DoP和AoP的准确性。

Motivation: 现有偏振图像去马赛克（PID）方法无法提升分辨率，而偏振图像超分辨率（PISR）方法会保留或放大去马赛克引入的误差。

Details

Method: 提出PIDSR框架，联合进行偏振图像去马赛克和超分辨率。 Result: 在合成和真实数据上均达到最先进性能，并提升下游任务效果。 Conclusion: PIDSR能够直接从CPFA原始图像中稳健地获取高质量高分辨率偏振图像，且DoP和AoP更准确。 Abstract: Polarization cameras can capture multiple polarized images with different polarizer angles in a single shot, bringing convenience to polarization-based downstream tasks. However, their direct outputs are color-polarization filter array (CPFA) raw images, requiring demosaicing to reconstruct full-resolution, full-color polarized images; unfortunately, this necessary step introduces artifacts that make polarization-related parameters such as the degree of polarization (DoP) and angle of polarization (AoP) prone to error. Besides, limited by the hardware design, the resolution of a polarization camera is often much lower than that of a conventional RGB camera. Existing polarized image demosaicing (PID) methods are limited in that they cannot enhance resolution, while polarized image super-resolution (PISR) methods, though designed to obtain high-resolution (HR) polarized images from the demosaicing results, tend to retain or even amplify errors in the DoP and AoP introduced by demosaicing artifacts. In this paper, we propose PIDSR, a joint framework that performs complementary Polarized Image Demosaicing and Super-Resolution, showing the ability to robustly obtain high-quality HR polarized images with more accurate DoP and AoP from a CPFA raw image in a direct manner. Experiments show our PIDSR not only achieves state-of-the-art performance on both synthetic and real data, but also facilitates downstream tasks.

Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models

Hongcheng Guo,Juntao Yao,Boyang Wang,Junjia Du,Shaosheng Cao,Donglin Di,Shun Zhang,Zhoujun Li

Task: 提出一种名为C-Prune的两阶段框架，用于自适应地压缩MoE大语言模型。

Motivation: 解决MoE模型中专家冗余和层间相似性问题，以实现更高效的模型部署。

Details

Method: 通过层内专家聚类和全局聚类剪枝，利用参数相似性度量和统一重要性评分机制。 Result: C-Prune能有效减小模型规模，并在性能上优于现有MoE剪枝方法。 Conclusion: C-Prune为MoE模型的压缩提供了一种高效且性能优越的解决方案。 Abstract: Mixture-of-Experts (MoE) architectures have emerged as a promising paradigm for scaling large language models (LLMs) with sparse activation of task-specific experts. Despite their computational efficiency during inference, the massive overall parameter footprint of MoE models (e.g., GPT-4) introduces critical challenges for practical deployment. Current pruning approaches often fail to address two inherent characteristics of MoE systems: 1).intra-layer expert homogeneity where experts within the same MoE layer exhibit functional redundancy, and 2). inter-layer similarity patterns where deeper layers tend to contain progressively more homogeneous experts. To tackle these issues, we propose Cluster-driven Expert Pruning (C-Prune), a novel two-stage framework for adaptive task-specific compression of MoE LLMs. C-Prune operates through layer-wise expert clustering, which groups functionally similar experts within each MoE layer using parameter similarity metrics, followed by global cluster pruning, which eliminates redundant clusters across all layers through a unified importance scoring mechanism that accounts for cross-layer homogeneity. We validate C-Prune through extensive experiments on multiple MoE models and benchmarks. The results demonstrate that C-Prune effectively reduces model size while outperforming existing MoE pruning methods.

Exploring a Patch-Wise Approach for Privacy-Preserving Fake ID Detection

Javier Muñoz-Haro,Ruben Tolosana,Ruben Vera-Rodriguez,Aythami Morales,Julian Fierrez

Task: 提出一种基于分块的隐私保护假ID检测方法，并探索隐私与性能之间的权衡。

Motivation: 当前假ID检测领域缺乏公开的真实ID文档数据，且大多数研究依赖私有数据库，限制了研究进展。

Details

Method: 采用分块方法，结合两种匿名化级别（完全匿名和伪匿名）和不同分块大小配置，同时考虑Vision Transformers和Foundation Models等先进技术。 Result: 在DLC-2021数据库上，提出的方法在分块和ID文档级别分别实现了13.91%和0%的EER，表现出良好的泛化能力。 Conclusion: 研究不仅提出了一种有效的隐私保护假ID检测方法，还首次公开了一个包含真实和假ID文档分块的数据集，推动了该领域的发展。 Abstract: In an increasingly digitalized world, verifying the authenticity of ID documents has become a critical challenge for real-life applications such as digital banking, crypto-exchanges, renting, etc. This study focuses on the topic of fake ID detection, covering several limitations in the field. In particular, no publicly available data from real ID documents exists, and most studies rely on proprietary in-house databases that are not available due to privacy reasons. In order to shed some light on this critical challenge that makes difficult to advance in the field, we explore a trade-off between privacy (i.e., amount of sensitive data available) and performance, proposing a novel patch-wise approach for privacy-preserving fake ID detection. Our proposed approach explores how privacy can be enhanced through: i) two levels of anonymization for an ID document (i.e., fully- and pseudo-anonymized), and ii) different patch size configurations, varying the amount of sensitive data visible in the patch image. Also, state-of-the-art methods such as Vision Transformers and Foundation Models are considered in the analysis. The experimental framework shows that, on an unseen database (DLC-2021), our proposal achieves 13.91% and 0% EERs at patch and ID document level, showing a good generalization to other databases. In addition to this exploration, another key contribution of our study is the release of the first publicly available database that contains 48,400 patches from both real and fake ID documents, along with the experimental framework and models, which will be available in our GitHub.

What the HellaSwag? On the Validity of Common-Sense Reasoning Benchmarks

Pavel Chizhov,Mattia Nee,Pierre-Carl Langlais,Ivan P. Yamshchikov

Task: 分析HellaSwag基准在评估语言模型常识推理能力时的构建效度问题。

Motivation: HellaSwag作为广泛使用的常识推理评估基准存在严重问题，可能导致模型选择的不准确决策。

Details

Method: 通过多种生成语言模型的评估，揭示HellaSwag的语法错误、误导性提示等问题。 Result: 发现超过65%的模型预测在仅基于答案文本或无意义提示时保持不变，表明基准效度不足。 Conclusion: HellaSwag当前状态不适合评估常识推理，提出未来基准应满足的要求，并发布修正版GoldenSwag。 Abstract: Common-sense reasoning is a key language model capability because it encapsulates not just specific factual knowledge but rather general language and world understanding. Measuring common-sense reasoning, therefore, is crucial for language models of different sizes and applications. One of the most widely used benchmarks for evaluating such capabilities is HellaSwag; however, in this paper, we show that it has severe construct validity issues. These issues range from basic ungrammaticality and numerous typos to misleading prompts or equally correct options. Furthermore, we show that if models are evaluated only on answer texts, or with "Lorem ipsum dolor..." instead of the question, more than 65% of model predictions remain the same, and this cannot be attributed merely to contamination. Since benchmark scores are an essential part of model selection in both research and commercial applications, these validity issues can have severe consequences. In particular, knowing that taking benchmark scores at face value is ubiquitous, inadequate evaluation leads to ill-informed decisions about models. In this paper, we thoroughly investigate critical validity issues posed by HellaSwag and illustrate them with various evaluations using generative language models of different sizes. We argue that this benchmark does not accurately measure common-sense reasoning and, therefore, should not be used for evaluation in its current state. Based on the results of our study, we propose requirements that should be met by future common-sense reasoning benchmarks. In addition, we release GoldenSwag, a corrected subset of HellaSwag, which, to our belief, facilitates acceptable common-sense reasoning evaluation.

Towards Micro-Action Recognition with Limited Annotations: An Asynchronous Pseudo Labeling and Training Approach

Yan Zhang,Lechao Cheng,Yaxiong Wang,Zhun Zhong,Meng Wang

Task: 提出一种名为异步伪标签与训练（APLT）的新框架，以解决半监督微动作识别（SSMAR）中伪标签不准确导致的过拟合问题。

Motivation: 传统半监督学习方法在SSMAR中容易因伪标签不准确而过拟合，导致性能下降。

Details

Method: 通过分离伪标签生成与模型训练，引入半监督聚类方法和自适应阈值策略生成更准确的伪标签，并构建基于内存的原型分类器指导训练。 Result: 在三个MAR数据集上，APLT显著优于现有半监督学习方法，例如在MA-12数据集上仅使用50%标注数据时，准确率比FixMatch提高14.5%。 Conclusion: APLT框架通过异步伪标签与训练，有效提升了半监督微动作识别的性能。 Abstract: Micro-Action Recognition (MAR) aims to classify subtle human actions in video. However, annotating MAR datasets is particularly challenging due to the subtlety of actions. To this end, we introduce the setting of Semi-Supervised MAR (SSMAR), where only a part of samples are labeled. We first evaluate traditional Semi-Supervised Learning (SSL) methods to SSMAR and find that these methods tend to overfit on inaccurate pseudo-labels, leading to error accumulation and degraded performance. This issue primarily arises from the common practice of directly using the predictions of classifier as pseudo-labels to train the model. To solve this issue, we propose a novel framework, called Asynchronous Pseudo Labeling and Training (APLT), which explicitly separates the pseudo-labeling process from model training. Specifically, we introduce a semi-supervised clustering method during the offline pseudo-labeling phase to generate more accurate pseudo-labels. Moreover, a self-adaptive thresholding strategy is proposed to dynamically filter noisy labels of different classes. We then build a memory-based prototype classifier based on the filtered pseudo-labels, which is fixed and used to guide the subsequent model training phase. By alternating the two pseudo-labeling and model training phases in an asynchronous manner, the model can not only be learned with more accurate pseudo-labels but also avoid the overfitting issue. Experiments on three MAR datasets show that our APLT largely outperforms state-of-the-art SSL methods. For instance, APLT improves accuracy by 14.5\% over FixMatch on the MA-12 dataset when using only 50\% labeled data. Code will be publicly available.

MuSaRoNews: A Multidomain, Multimodal Satire Dataset from Romanian News Articles

Răzvan-Alexandru Smădu,Andreea Iuga,Dumitru-Clementin Cercel

Task: 构建一个多模态语料库MuSaRoNews，用于检测罗马尼亚新闻文章中的讽刺内容。

Motivation: 讽刺和假新闻虽然目的不同，但都会传播虚假信息，仅依赖文本难以检测其表面与实际含义的不一致，需要结合其他信息（如视觉）来提高检测效果。

Details

Method: 收集了117,834篇来自真实和讽刺新闻来源的公开新闻文章，构建了罗马尼亚语的首个多模态讽刺检测语料库。 Result: 实验表明，结合多模态信息能提高讽刺检测的性能。 Conclusion: 多模态方法在讽刺检测中具有优势，MuSaRoNews为相关研究提供了重要资源。 Abstract: Satire and fake news can both contribute to the spread of false information, even though both have different purposes (one if for amusement, the other is to misinform). However, it is not enough to rely purely on text to detect the incongruity between the surface meaning and the actual meaning of the news articles, and, often, other sources of information (e.g., visual) provide an important clue for satire detection. This work introduces a multimodal corpus for satire detection in Romanian news articles named MuSaRoNews. Specifically, we gathered 117,834 public news articles from real and satirical news sources, composing the first multimodal corpus for satire detection in the Romanian language. We conducted experiments and showed that the use of both modalities improves performance.

Breaking the Barriers: Video Vision Transformers for Word-Level Sign Language Recognition

Alexander Brettmann,Jakob Grävinghoff,Marlene Rüschoff,Marie Westhues

Task: 提出一种基于Video Vision Transformer（ViViT）的模型，用于动态词级美国手语（ASL）识别。

Motivation: 解决听力人群对手语不熟练导致的沟通障碍，提升自动手语识别的性能，尤其是在动态词级识别中捕捉时空依赖关系的挑战。

Details

Method: 采用Video Vision Transformer（ViViT）模型，利用自注意力机制捕捉视频序列中的全局时空依赖关系。 Result: 在WLASL100数据集上，VideoMAE模型的Top-1准确率达到75.58%，优于传统CNN模型的65.89%。 Conclusion: 基于Transformer的架构在手语识别中具有巨大潜力，能够克服沟通障碍并促进聋哑及听力障碍人群的包容性。 Abstract: Sign language is a fundamental means of communication for the deaf and hard-of-hearing (DHH) community, enabling nuanced expression through gestures, facial expressions, and body movements. Despite its critical role in facilitating interaction within the DHH population, significant barriers persist due to the limited fluency in sign language among the hearing population. Overcoming this communication gap through automatic sign language recognition (SLR) remains a challenge, particularly at a dynamic word-level, where temporal and spatial dependencies must be effectively recognized. While Convolutional Neural Networks have shown potential in SLR, they are computationally intensive and have difficulties in capturing global temporal dependencies between video sequences. To address these limitations, we propose a Video Vision Transformer (ViViT) model for word-level American Sign Language (ASL) recognition. Transformer models make use of self-attention mechanisms to effectively capture global relationships across spatial and temporal dimensions, which makes them suitable for complex gesture recognition tasks. The VideoMAE model achieves a Top-1 accuracy of 75.58% on the WLASL100 dataset, highlighting its strong performance compared to traditional CNNs with 65.89%. Our study demonstrates that transformer-based architectures have great potential to advance SLR, overcome communication barriers and promote the inclusion of DHH individuals.

MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations

Genglin Liu,Salman Rahman,Elisa Kreiss,Marzyeh Ghassemi,Saadia Gabriel

Task: 提出一个开源的社会网络模拟框架MOSAIC，用于分析用户行为和内容传播动态。

Motivation: 为了更好地理解用户如何判断在线社交内容的真实性，并研究欺骗行为的涌现。

Details

Method: 结合生成语言代理（LLM）和有向社交图，构建多样化的用户角色，进行多代理模拟。 Result: 评估了三种内容审核策略，发现它们不仅能减少虚假内容的传播，还能提高用户参与度。 Conclusion: 开源模拟软件以促进AI和社会科学的进一步研究。 Abstract: We present a novel, open-source social network simulation framework, MOSAIC, where generative language agents predict user behaviors such as liking, sharing, and flagging content. This simulation combines LLM agents with a directed social graph to analyze emergent deception behaviors and gain a better understanding of how users determine the veracity of online social content. By constructing user representations from diverse fine-grained personas, our system enables multi-agent simulations that model content dissemination and engagement dynamics at scale. Within this framework, we evaluate three different content moderation strategies with simulated misinformation dissemination, and we find that they not only mitigate the spread of non-factual content but also increase user engagement. In addition, we analyze the trajectories of popular content in our simulations, and explore whether simulation agents' articulated reasoning for their social interactions truly aligns with their collective engagement patterns. We open-source our simulation software to encourage further research within AI and social sciences.

Nonlocal Retinex-Based Variational Model and its Deep Unfolding Twin for Low-Light Image Enhancement

Daniel Torres,Joan Duran,Julia Navarro,Catalina Sbert

Task: 提出一种基于Retinex分解的变分方法，用于低光照图像增强。

Motivation: 低光照条件下捕获的图像在细节、对比度和噪声方面存在显著限制，影响图像分割和目标检测等任务。

Details

Method: 结合颜色校正预处理、非局部梯度型保真项和自动伽马校正模块，并扩展为深度展开模型。 Result: 实验结果表明，该方法在视觉和质量指标上优于多种现有技术。 Conclusion: 变分模型在不依赖学习策略的情况下，性能优于大多数深度学习方法。 Abstract: Images captured under low-light conditions present significant limitations in many applications, as poor lighting can obscure details, reduce contrast, and hide noise. Removing the illumination effects and enhancing the quality of such images is crucial for many tasks, such as image segmentation and object detection. In this paper, we propose a variational method for low-light image enhancement based on the Retinex decomposition into illumination, reflectance, and noise components. A color correction pre-processing step is applied to the low-light image, which is then used as the observed input in the decomposition. Moreover, our model integrates a novel nonlocal gradient-type fidelity term designed to preserve structural details. Additionally, we propose an automatic gamma correction module. Building on the proposed variational approach, we extend the model by introducing its deep unfolding counterpart, in which the proximal operators are replaced with learnable networks. We propose cross-attention mechanisms to capture long-range dependencies in both the nonlocal prior of the reflectance and the nonlocal gradient-based constraint. Experimental results demonstrate that both methods compare favorably with several recent and state-of-the-art techniques across different datasets. In particular, despite not relying on learning strategies, the variational model outperforms most deep learning approaches both visually and in terms of quality metrics.

The KL3M Data Project: Copyright-Clean Training Resources for Large Language Models

Michael J Bommarito II,Jillian Bommarito,Daniel Martin Katz

Task: Introduce a comprehensive training data pipeline (KL3M Data Project) to minimize legal risks related to copyright and breach of contract in large language model pre-training.

Motivation: Address the uncertainty and potential legal risks associated with pre-training data for large language models, ensuring ethical and legal compliance.

Details

Method: Develop a verified corpus of over 132 million documents from 16 sources, following strict copyright and licensing protocols, and release the entire pipeline including source code, metadata, and processed data. Result: A freely available dataset and pipeline (on S3, Hugging Face, and GitHub) that supports ethical and legal AI model development. Conclusion: The KL3M Data Project promotes a more ethical, legal, and sustainable approach to AI model development by providing a verified and transparent data pipeline. Abstract: Practically all large language models have been pre-trained on data that is subject to global uncertainty related to copyright infringement and breach of contract. This creates potential risk for users and developers due to this uncertain legal status. The KL3M Data Project directly confronts this critical issue by introducing the largest comprehensive training data pipeline that minimizes risks related to copyright or breach of contract. The foundation of this project is a corpus of over 132 million documents and trillions of tokens spanning 16 different sources that have been verified to meet the strict copyright and licensing protocol detailed herein. We are releasing the entire pipeline, including 1) the source code to acquire and process these documents, 2) the original document formats with associated provenance and metadata, 3) extracted content in a standardized format, 4) pre-tokenized representations of the documents, and 5) various mid- and post-train resources such as question-answer, summarization, conversion, drafting, classification, prediction, and conversational data. All of these resources are freely available to the public on S3, Hugging Face, and GitHub under CC-BY terms. We are committed to continuing this project in furtherance of a more ethical, legal, and sustainable approach to the development and use of AI models.

P2Object: Single Point Supervised Object Detection and Instance Segmentation

Pengfei Chen,Xuehui Yu,Xumeng Han,Kuiran Wang,Guorong Li,Lingxi Xie,Zhenjun Han,Jianbin Jiao

Task: 提出了一种基于单点监督的目标识别方法，通过生成实例级提案包和离散到连续的优化策略，提升性能。

Motivation: 单点监督目标识别的性能与全监督算法差距较大，现有方法在提案生成和优化上存在不足。

Details

Method: 提出P2BNet、P2BNet++和P2MNet，通过实例级提案包、连续采样策略和像素级感知优化目标边界。 Result: 在COCO、VOC、SBD和Cityscapes数据集上显著超越先前方法，缩小了与全监督任务的性能差距。 Conclusion: 通过离散到连续的优化和像素级感知，方法在目标识别和分割任务中表现出色，具有广泛应用潜力。 Abstract: Object recognition using single-point supervision has attracted increasing attention recently. However, the performance gap compared with fully-supervised algorithms remains large. Previous works generated class-agnostic \textbf{\textit{proposals in an image}} offline and then treated mixed candidates as a single bag, putting a huge burden on multiple instance learning (MIL). In this paper, we introduce Point-to-Box Network (P2BNet), which constructs balanced \textbf{\textit{instance-level proposal bags}} by generating proposals in an anchor-like way and refining the proposals in a coarse-to-fine paradigm. Through further research, we find that the bag of proposals, either at the image level or the instance level, is established on discrete box sampling. This leads the pseudo box estimation into a sub-optimal solution, resulting in the truncation of object boundaries or the excessive inclusion of background. Hence, we conduct a series exploration of discrete-to-continuous optimization, yielding P2BNet++ and Point-to-Mask Network (P2MNet). P2BNet++ conducts an approximately continuous proposal sampling strategy by better utilizing spatial clues. P2MNet further introduces low-level image information to assist in pixel prediction, and a boundary self-prediction is designed to relieve the limitation of the estimated boxes. Benefiting from the continuous object-aware \textbf{\textit{pixel-level perception}}, P2MNet can generate more precise bounding boxes and generalize to segmentation tasks. Our method largely surpasses the previous methods in terms of the mean average precision on COCO, VOC, SBD, and Cityscapes, demonstrating great potential to bridge the performance gap compared with fully supervised tasks.

Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs

Yichun Yin,Wenyong Huang,Kaikai Song,Yehui Tang,Xueyu Wu,Wei Guo,Peng Guo,Yaoyuan Wang,Xiaojun Meng,Yasheng Wang,Dong Li,Can Chen,Dandan Tu,Yin Li,Fisher Yu,Ruiming Tang,Yunhe Wang,Baojun Wang,Bin Wang,Bo Wang,Boxiao Liu,Changzheng Zhang,Duyu Tang,Fei Mi,Hui Jin,Jiansheng Wei,Jiarui Qin,Jinpeng Li,Jun Zhao,Liqun Deng,Lin Li,Minghui Xu,Naifu Zhang,Nianzu Zheng,Qiang Li,Rongju Ruan,Shengjun Cheng,Tianyu Guo,Wei He,Wei Li,Weiwen Liu,Wulong Liu,Xinyi Dai,Yonghan Dong,Yu Pan,Yue Li,Yufei Wang,Yujun Li,Yunsheng Ni,Zhe Liu,Zhenhe Zhang,Zhicheng Liu

Task: 训练一个具有1350亿参数的密集Transformer模块的大型语言模型Pangu Ultra。

Motivation: 尽管近年来大型语言模型在规模和能力上取得了前所未有的进展，但训练如此大规模的模型仍面临显著的优化和系统挑战。

Details

Method: 提出深度缩放三明治归一化以稳定训练过程，并在8192个Ascend NPU上进行大规模预训练和后训练优化。 Result: Pangu Ultra在多个基准测试中显著优于Llama 405B和Mistral Large 2等密集模型，并与参数更多的稀疏模型DeepSeek-R1竞争。 Conclusion: Ascend NPU能够高效训练超过1000亿参数的密集模型，Pangu Ultra及其系统将面向商业客户开放。 Abstract: We present Pangu Ultra, a Large Language Model (LLM) with 135 billion parameters and dense Transformer modules trained on Ascend Neural Processing Units (NPUs). Although the field of LLM has been witnessing unprecedented advances in pushing the scale and capability of LLM in recent years, training such a large-scale model still involves significant optimization and system challenges. To stabilize the training process, we propose depth-scaled sandwich normalization, which effectively eliminates loss spikes during the training process of deep models. We pre-train our model on 13.2 trillion diverse and high-quality tokens and further enhance its reasoning capabilities during post-training. To perform such large-scale training efficiently, we utilize 8,192 Ascend NPUs with a series of system optimizations. Evaluations on multiple diverse benchmarks indicate that Pangu Ultra significantly advances the state-of-the-art capabilities of dense LLMs such as Llama 405B and Mistral Large 2, and even achieves competitive results with DeepSeek-R1, whose sparse model structure contains much more parameters. Our exploration demonstrates that Ascend NPUs are capable of efficiently and effectively training dense models with more than 100 billion parameters. Our model and system will be available for our commercial customers.

AerialVG: A Challenging Benchmark for Aerial Visual Grounding by Exploring Positional Relations

Junli Liu,Qizhi Chen,Zhigang Wang,Yiwen Tang,Yiting Zhang,Chi Yan,Dong Wang,Xuelong Li,Bin Zhao

Task: 提出并解决从航拍视角进行视觉定位的新任务AerialVG。

Motivation: 传统视觉定位方法在航拍图像中表现不佳，因为航拍图像中的目标对象视觉相似度高且空间关系复杂，需要新的数据集和方法。

Details

Method: 提出AerialVG数据集，包含5K航拍图像和50K标注描述；设计Hierarchical Cross-Attention和Relation-Aware Grounding模块，用于目标区域聚焦和空间关系推理。 Result: 实验验证了数据集和方法的有效性，强调了空间推理在航拍视觉定位中的重要性。 Conclusion: AerialVG任务和提出的方法为航拍视觉定位提供了新的解决方案，代码和数据集将公开。 Abstract: Visual grounding (VG) aims to localize target objects in an image based on natural language descriptions. In this paper, we propose AerialVG, a new task focusing on visual grounding from aerial views. Compared to traditional VG, AerialVG poses new challenges, \emph{e.g.}, appearance-based grounding is insufficient to distinguish among multiple visually similar objects, and positional relations should be emphasized. Besides, existing VG models struggle when applied to aerial imagery, where high-resolution images cause significant difficulties. To address these challenges, we introduce the first AerialVG dataset, consisting of 5K real-world aerial images, 50K manually annotated descriptions, and 103K objects. Particularly, each annotation in AerialVG dataset contains multiple target objects annotated with relative spatial relations, requiring models to perform comprehensive spatial reasoning. Furthermore, we propose an innovative model especially for the AerialVG task, where a Hierarchical Cross-Attention is devised to focus on target regions, and a Relation-Aware Grounding module is designed to infer positional relations. Experimental results validate the effectiveness of our dataset and method, highlighting the importance of spatial reasoning in aerial visual grounding. The code and dataset will be released.

Token Level Routing Inference System for Edge Devices

Jianshu She,Wenhao Zheng,Zhengzhong Liu,Hongyi Wang,Eric Xing,Huaxiu Yao,Qirong Ho

Task: 提出一种新型协作解码推理系统，以解决大型语言模型在边缘设备上部署效率低的问题。

Motivation: 大型语言模型（LLM）推理的计算复杂度限制了其在边缘设备上的部署效率，而小型语言模型虽然速度快、资源消耗低，但响应质量较差且容易产生幻觉。

Details

Method: 通过协作解码，利用大型模型生成关键令牌，小型模型执行设备端推理，并选择性咨询云端大型模型。 Result: 系统在CommonsenseQA上实现了60%的性能提升，仅使用0.5B模型，且仅有不到7%的令牌生成需要上传到云端大型模型。 Conclusion: 协作解码是一种有效的解决方案，能够平衡模型性能与部署效率。 Abstract: The computational complexity of large language model (LLM) inference significantly constrains their deployment efficiency on edge devices. In contrast, small language models offer faster decoding and lower resource consumption but often suffer from degraded response quality and heightened susceptibility to hallucinations. To address this trade-off, collaborative decoding, in which a large model assists in generating critical tokens, has emerged as a promising solution. This paradigm leverages the strengths of both model types by enabling high-quality inference through selective intervention of the large model, while maintaining the speed and efficiency of the smaller model. In this work, we present a novel collaborative decoding inference system that allows small models to perform on-device inference while selectively consulting a cloud-based large model for critical token generation. Remarkably, the system achieves a 60% performance gain on CommonsenseQA using only a 0.5B model on an M1 MacBook, with under 7% of tokens generation uploaded to the large model in the cloud.

V2V3D: View-to-View Denoised 3D Reconstruction for Light-Field Microscopy

Jiayin Zhao,Zhenqi Fu,Tao Yu,Hui Qiao

Task: 提出一种名为V2V3D的无监督框架，用于联合优化光场显微镜（LFM）图像去噪和3D重建。

Motivation: 现有LFM重建算法对传感器噪声高度敏感或需要难以获取的真实标注数据。

Details

Method: 采用无监督的view2view框架，结合noise2noise原理进行去噪，并提出基于波光学的特征对齐技术。 Result: V2V3D在计算效率和性能上优于现有方法，并提供了一个包含LF图像和3D强度体积的数据集。 Conclusion: V2V3D为挑战性条件下的3D成像提供了有前景的解决方案。 Abstract: Light field microscopy (LFM) has gained significant attention due to its ability to capture snapshot-based, large-scale 3D fluorescence images. However, existing LFM reconstruction algorithms are highly sensitive to sensor noise or require hard-to-get ground-truth annotated data for training. To address these challenges, this paper introduces V2V3D, an unsupervised view2view-based framework that establishes a new paradigm for joint optimization of image denoising and 3D reconstruction in a unified architecture. We assume that the LF images are derived from a consistent 3D signal, with the noise in each view being independent. This enables V2V3D to incorporate the principle of noise2noise for effective denoising. To enhance the recovery of high-frequency details, we propose a novel wave-optics-based feature alignment technique, which transforms the point spread function, used for forward propagation in wave optics, into convolution kernels specifically designed for feature alignment. Moreover, we introduce an LFM dataset containing LF images and their corresponding 3D intensity volumes. Extensive experiments demonstrate that our approach achieves high computational efficiency and outperforms the other state-of-the-art methods. These advancements position V2V3D as a promising solution for 3D imaging under challenging conditions.

Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge

Riccardo Cantini,Alessio Orsino,Massimo Ruggiero,Domenico Talia

Task: 提出一个可扩展的基准框架，用于评估大语言模型（LLMs）在对抗性偏见引发中的鲁棒性。

Motivation: 大语言模型在关键社会领域的广泛应用引发了对其嵌入偏见的担忧，这些偏见可能延续刻板印象并损害公平性。

Details

Method: 采用多任务方法系统性地探测模型在不同社会文化维度上的偏见，使用LLM-as-a-Judge方法量化鲁棒性，并通过越狱技术调查安全机制的漏洞。 Result: 揭示了模型大小与安全性之间的关键权衡，并发布了CLEAR-Bias数据集以支持系统性漏洞评估。 Conclusion: 研究结果为开发更公平和鲁棒的未来语言模型提供了重要参考。 Abstract: Large Language Models (LLMs) have revolutionized artificial intelligence, driving advancements in machine translation, summarization, and conversational agents. However, their increasing integration into critical societal domains has raised concerns about embedded biases, which can perpetuate stereotypes and compromise fairness. These biases stem from various sources, including historical inequalities in training data, linguistic imbalances, and adversarial manipulation. Despite mitigation efforts, recent studies indicate that LLMs remain vulnerable to adversarial attacks designed to elicit biased responses. This work proposes a scalable benchmarking framework to evaluate LLM robustness against adversarial bias elicitation. Our methodology involves (i) systematically probing models with a multi-task approach targeting biases across various sociocultural dimensions, (ii) quantifying robustness through safety scores using an LLM-as-a-Judge approach for automated assessment of model responses, and (iii) employing jailbreak techniques to investigate vulnerabilities in safety mechanisms. Our analysis examines prevalent biases in both small and large state-of-the-art models and their impact on model safety. Additionally, we assess the safety of domain-specific models fine-tuned for critical fields, such as medicine. Finally, we release a curated dataset of bias-related prompts, CLEAR-Bias, to facilitate systematic vulnerability benchmarking. Our findings reveal critical trade-offs between model size and safety, aiding the development of fairer and more robust future language models.

SAMJAM: Zero-Shot Video Scene Graph Generation for Egocentric Kitchen Videos

Joshua Li,Fernando Jose Pena Cantu,Emily Yu,Alexander Wong,Yuchen Cui,Yuhao Chen

Task: 提出一种零样本的视频场景图生成方法SAMJAM，结合SAM2的时间跟踪和Gemini的语义理解，以解决动态厨房环境中的场景图生成问题。

Motivation: 当前视频场景图生成模型需要大量训练，且现有视觉语言模型（如Gemini）在动态环境中难以保持对象身份的稳定性。

Details

Method: 结合SAM2的时间跟踪和Gemini的语义理解，通过匹配算法将场景图中的对象与SAM2生成的掩码关联，生成时间一致的场景图。 Result: 在EPIC-KITCHENS和EPIC-KITCHENS-100数据集上，SAMJAM的均值召回率比Gemini高出8.33%。 Conclusion: SAMJAM是一种有效的零样本方法，显著提升了动态环境中的视频场景图生成性能。 Abstract: Video Scene Graph Generation (VidSGG) is an important topic in understanding dynamic kitchen environments. Current models for VidSGG require extensive training to produce scene graphs. Recently, Vision Language Models (VLM) and Vision Foundation Models (VFM) have demonstrated impressive zero-shot capabilities in a variety of tasks. However, VLMs like Gemini struggle with the dynamics for VidSGG, failing to maintain stable object identities across frames. To overcome this limitation, we propose SAMJAM, a zero-shot pipeline that combines SAM2's temporal tracking with Gemini's semantic understanding. SAM2 also improves upon Gemini's object grounding by producing more accurate bounding boxes. In our method, we first prompt Gemini to generate a frame-level scene graph. Then, we employ a matching algorithm to map each object in the scene graph with a SAM2-generated or SAM2-propagated mask, producing a temporally-consistent scene graph in dynamic environments. Finally, we repeat this process again in each of the following frames. We empirically demonstrate that SAMJAM outperforms Gemini by 8.33% in mean recall on the EPIC-KITCHENS and EPIC-KITCHENS-100 datasets.

Redefining Machine Translation on Social Network Services with Large Language Models

Hongcheng Guo,Fei Zhao,Shaosheng Cao,Xinze Lyu,Ziyan Liu,Yue Wang,Boyang Wang,Zhoujun Li,Chonggang Lu,Zhe Xu,Yao Hu

Task: 开发一个专门用于社交网络服务（SNS）翻译的72B大型语言模型（LLM）RedTrans。

Motivation: 全球化社交互动增加了对社交网络服务机器翻译的需求，但传统模型难以处理文化细微内容（如表情包、俚语和流行文化引用），且现有大型语言模型在SNS内容上的表现有限。

Details

Method: 通过三种创新方法训练RedTrans：1）双LLM反向翻译采样的监督微调；2）通过专家标注纠正错误偏好对的Rewritten Preference Optimization（RePO）算法；3）首个SNS翻译基准RedTrans-Bench。 Result: 实验表明RedTrans优于现有最先进的LLM，并已在实际生产环境中部署。 Conclusion: 领域特定适配能有效弥合通用与文化基础翻译系统之间的差距。 Abstract: The globalization of social interactions has heightened the need for machine translation (MT) on Social Network Services (SNS), yet traditional models struggle with culturally nuanced content like memes, slang, and pop culture references. While large language models (LLMs) have advanced general-purpose translation, their performance on SNS-specific content remains limited due to insufficient specialized training data and evaluation benchmarks. This paper introduces RedTrans, a 72B LLM tailored for SNS translation, trained on a novel dataset developed through three innovations: (1) Supervised Finetuning with Dual-LLM Back-Translation Sampling, an unsupervised sampling method using LLM-based back-translation to select diverse data for large-scale finetuning; (2) Rewritten Preference Optimization (RePO), an algorithm that identifies and corrects erroneous preference pairs through expert annotation, building reliable preference corpora; and (3) RedTrans-Bench, the first benchmark for SNS translation, evaluating phenomena like humor localization, emoji semantics, and meme adaptation. Experiments show RedTrans outperforms state-of-the-art LLMs. Besides, RedTrans has already been deployed in a real-world production environment, demonstrating that domain-specific adaptation, effectively bridges the gap between generic and culturally grounded translation systems.

SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement

Xiyao Wang,Zhengyuan Yang,Chao Feng,Hongjin Lu,Linjie Li,Chung-Ching Lin,Kevin Lin,Furong Huang,Lijuan Wang

Task: 提出一种基于蒙特卡洛树搜索（MCTS）的样本难度量化方法，以增强视觉推理模型的性能。

Motivation: 在少量训练样本的情况下，通过选择适当难度的样本，可以显著提升模型的推理能力。

Details

Method: 利用MCTS量化样本难度，筛选出11k样本进行强化微调（RFT），训练模型ThinkLite-VL。 Result: ThinkLite-VL在八个基准测试中平均性能提升7%，在MathVista上达到75.1%的SoTA准确率。 Conclusion: 通过MCTS筛选挑战性样本，能在少量数据下显著提升视觉推理模型的性能。 Abstract: In this paper, we present an effective method to enhance visual reasoning with significantly fewer training samples, relying purely on self-improvement with no knowledge distillation. Our key insight is that the difficulty of training data during reinforcement fine-tuning (RFT) is critical. Appropriately challenging samples can substantially boost reasoning capabilities even when the dataset is small. Despite being intuitive, the main challenge remains in accurately quantifying sample difficulty to enable effective data filtering. To this end, we propose a novel way of repurposing Monte Carlo Tree Search (MCTS) to achieve that. Starting from our curated 70k open-source training samples, we introduce an MCTS-based selection method that quantifies sample difficulty based on the number of iterations required by the VLMs to solve each problem. This explicit step-by-step reasoning in MCTS enforces the model to think longer and better identifies samples that are genuinely challenging. We filter and retain 11k samples to perform RFT on Qwen2.5-VL-7B-Instruct, resulting in our final model, ThinkLite-VL. Evaluation results on eight benchmarks show that ThinkLite-VL improves the average performance of Qwen2.5-VL-7B-Instruct by 7%, using only 11k training samples with no knowledge distillation. This significantly outperforms all existing 7B-level reasoning VLMs, and our fairly comparable baselines that use classic selection methods such as accuracy-based filtering. Notably, on MathVista, ThinkLite-VL-7B achieves the SoTA accuracy of 75.1, surpassing Qwen2.5-VL-72B, GPT-4o, and O1. Our code, data, and model are available at https://github.com/si0wang/ThinkLite-VL.

FG-RAG: Enhancing Query-Focused Summarization with Context-Aware Fine-Grained Graph RAG

Yubin Hong,Chaofan Li,Jingyi Zhang,Yingxia Shao

Task: 提出一种名为Context-Aware Fine-Grained Graph RAG (FG-RAG)的方法，以提升Query-Focused Summarization (QFS)任务的性能。

Motivation: 现有的GraphRAG方法在QFS任务中主要关注粗粒度信息摘要，缺乏对特定查询的感知，且检索内容缺乏足够的上下文信息。

Details

Method: FG-RAG通过Context-Aware Entity Expansion扩展图中实体的覆盖范围，并提供足够的上下文信息；同时采用Query-Level Fine-Grained Summarization在生成响应时融入细粒度细节。 Result: FG-RAG在QFS任务中，在全面性、多样性和增强性等多个指标上优于其他RAG系统。 Conclusion: FG-RAG通过细粒度和上下文感知的方法显著提升了QFS任务的性能。 Abstract: Retrieval-Augmented Generation (RAG) enables large language models to provide more precise and pertinent responses by incorporating external knowledge. In the Query-Focused Summarization (QFS) task, GraphRAG-based approaches have notably enhanced the comprehensiveness and diversity of generated responses. However, existing GraphRAG-based approaches predominantly focus on coarse-grained information summarization without being aware of the specific query, and the retrieved content lacks sufficient contextual information to generate comprehensive responses. To address the deficiencies of current RAG systems, we propose Context-Aware Fine-Grained Graph RAG (FG-RAG) to enhance the performance of the QFS task. FG-RAG employs Context-Aware Entity Expansion in graph retrieval to expand the coverage of retrieved entities in the graph, thus providing enough contextual information for the retrieved content. Furthermore, FG-RAG utilizes Query-Level Fine-Grained Summarization to incorporate fine-grained details during response generation, enhancing query awareness for the generated summarization. Our evaluation demonstrates that FG-RAG outperforms other RAG systems in multiple metrics of comprehensiveness, diversity, and empowerment when handling the QFS task. Our implementation is available at https://github.com/BuptWululu/FG-RAG.

Beyond the Frame: Generating 360° Panoramic Videos from Perspective Videos

Rundong Luo,Matthew Wallingford,Ali Farhadi,Noah Snavely,Wei-Chiu Ma

Task: 研究如何从标准视角视频生成全景（360度）视频。

Motivation: 360度视频能更完整地呈现动态视觉世界，但现有视频模型在生成全景视频方面仍有不足。

Details

Method: 利用在线丰富的360度视频资源，设计高质量数据过滤流程，并开发几何和运动感知操作以优化生成过程。 Result: 模型能够从真实场景的标准视频生成逼真且一致的全景视频。 Conclusion: 该方法不仅实现了高质量的全景视频生成，还展示了在视频稳定、相机视角控制和交互式视觉问答等领域的应用潜力。 Abstract: 360{\deg} videos have emerged as a promising medium to represent our dynamic visual world. Compared to the "tunnel vision" of standard cameras, their borderless field of view offers a more complete perspective of our surroundings. While existing video models excel at producing standard videos, their ability to generate full panoramic videos remains elusive. In this paper, we investigate the task of video-to-360{\deg} generation: given a perspective video as input, our goal is to generate a full panoramic video that is consistent with the original video. Unlike conventional video generation tasks, the output's field of view is significantly larger, and the model is required to have a deep understanding of both the spatial layout of the scene and the dynamics of objects to maintain spatio-temporal consistency. To address these challenges, we first leverage the abundant 360{\deg} videos available online and develop a high-quality data filtering pipeline to curate pairwise training data. We then carefully design a series of geometry- and motion-aware operations to facilitate the learning process and improve the quality of 360{\deg} video generation. Experimental results demonstrate that our model can generate realistic and coherent 360{\deg} videos from in-the-wild perspective video. In addition, we showcase its potential applications, including video stabilization, camera viewpoint control, and interactive visual question answering.

Relevance Isn't All You Need: Scaling RAG Systems With Inference-Time Compute Via Multi-Criteria Reranking

Will LeVine,Bijan Varjavand

Task: 研究检索增强生成（RAG）系统中仅优化上下文相关性对下游响应质量的影响，并提出一种多标准优化的新方法REBEL。

Motivation: 传统理论认为仅最大化上下文相关性可能导致信息瓶颈，现代LLM系统中的RAG方法也面临类似问题，需要兼顾上下文相关性和答案质量。

Details

Method: 提出REBEL方法，通过多标准优化（如Chain-of-Thought提示和多轮对话）改进RAG系统，使其在推理时间内实现更好的性能和速度权衡。 Result: 实验表明，REBEL在上下文相关性和答案质量上均优于现有RAG方法，且能随着推理时间增加而提升性能。 Conclusion: REBEL为RAG系统提供了一种新的优化路径，兼顾相关性和质量，并展示了性能与速度的可扩展性。 Abstract: Modern Large Language Model (LLM) systems typically rely on Retrieval Augmented Generation (RAG) which aims to gather context that is useful for response generation. These RAG systems typically optimize strictly towards retrieving context that is maximally relevant to the query. However, conventional theory suggests that retrieval systems which seek to maximize context relevance without any additional explicit criteria can create information bottlenecks. We reaffirm this finding in the modern age of LLM's by showing that in standard RAG pipelines, maximizing for context relevance alone can degrade downstream response quality. In response, we show evaluations of existing RAG methods which account for both context relevance and answer quality. These evaluations introduce a novel finding that existing RAG systems scale poorly with inference time compute usage when considering our combined metric. We introduce "RErank BEyond reLevance (REBEL)", which enables RAG systems to scale with inference-time compute via injection of multi-criteria optimization using Chain-of-Thought prompting (and optionally Multi-Turn dialogue). Ultimately, this enables a new performance/speed tradeoff curve, where RAG systems are able to achieve both higher relevance of retrieved contexts and superior answer quality as inference time increases. Code for the implementation of our method in llama-index can be found at the following PR: https://github.com/run-llama/llama_index/pull/17590. Code for running experiments using this llama-index implementation can be found at https://github.com/microsoft/REBEL.

MARS: a Multimodal Alignment and Ranking System for Few-Shot Segmentation

Nico Catalano,Stefano Samele,Paolo Pertino,Matteo Matteucci

Task: 提出一种名为MARS的插件式排名系统，用于改进少样本分割任务中的掩码选择方法。

Motivation: 当前少样本分割文献缺乏超越视觉相似性的掩码选择方法，导致预测结果不理想。

Details

Method: 利用多模态线索对掩码提议进行评分、过滤和合并，通过局部和全局层面的多模态评分评估提议。 Result: 在COCO-20i、Pascal-5i、LVIS-92i和FSS-1000等数据集上验证了MARS的有效性，并实现了新的最先进结果。 Conclusion: MARS能够轻松集成到多种掩码提议系统中，显著提升少样本分割任务的性能。 Abstract: Current Few Shot Segmentation literature lacks a mask selection method that goes beyond visual similarity between the query and example images, leading to suboptimal predictions. We present MARS, a plug-and-play ranking system that leverages multimodal cues to filter and merge mask proposals robustly. Starting from a set of mask predictions for a single query image, we score, filter, and merge them to improve results. Proposals are evaluated using multimodal scores computed at local and global levels. Extensive experiments on COCO-20i, Pascal-5i, LVIS-92i, and FSS-1000 demonstrate that integrating all four scoring components is crucial for robust ranking, validating our contribution. As MARS can be effortlessly integrated with various mask proposal systems, we deploy it across a wide range of top-performer methods and achieve new state-of-the-art results on multiple existing benchmarks. Code will be available upon acceptance.

OSCAR: Online Soft Compression And Reranking

Maxime Louis,Thibault Formal,Hervé Dejean,Stéphane Clinchant

Task: 提出一种名为OSCAR的查询依赖在线软压缩方法，以减少检索增强生成（RAG）管道的计算开销。

Motivation: 随着检索规模的增大，RAG管道的计算成本显著增加，需要一种高效的方法来降低开销。

Details

Method: OSCAR动态地在推理时压缩检索到的信息，避免了存储开销，并支持更高的压缩率，同时结合了重排序功能。 Result: 实验表明，OSCAR在1B到24B参数的LLMs上实现了2-5倍的推理加速，且准确性损失极小。 Conclusion: OSCAR是一种高效且性能优越的压缩方法，显著提升了RAG管道的效率。 Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external knowledge, leading to improved accuracy and relevance. However, scaling RAG pipelines remains computationally expensive as retrieval sizes grow. To address this, we introduce OSCAR, a novel query-dependent online soft compression method that reduces computational overhead while preserving performance. Unlike traditional hard compression methods, which shorten retrieved texts, or soft compression approaches, which map documents to continuous embeddings offline, OSCAR dynamically compresses retrieved information at inference time, eliminating storage overhead and enabling higher compression rates. Additionally, we extend OSCAR to simultaneously perform reranking, further optimizing the efficiency of the RAG pipeline. Our experiments demonstrate state-of-the-art performance with a 2-5x speed-up in inference and minimal to no loss in accuracy for LLMs ranging from 1B to 24B parameters. The models are available at: https://huggingface.co/collections/naver/oscar-67d446a8e3a2551f57464295.

HoloPart: Generative 3D Part Amodal Segmentation

Yunhan Yang,Yuan-Chen Guo,Yukun Huang,Zi-Xin Zou,Zhipeng Yu,Yangguang Li,Yan-Pei Cao,Xihui Liu

Task: 将3D形状分解为完整且语义有意义的部分，即使被遮挡。

Motivation: 现有3D部分分割方法仅识别可见表面，限制了其应用价值。

Details

Method: 提出一种两阶段方法，结合现有3D部分分割技术和新型扩散模型HoloPart，通过局部和全局注意力机制完成部分分割。 Result: HoloPart在ABO和PartObjaverse-Tiny数据集上显著优于现有形状补全方法。 Conclusion: HoloPart与现有分割技术结合，为3D部分全分割任务提供了新思路，适用于几何编辑、动画和材质分配等应用。 Abstract: 3D part amodal segmentation--decomposing a 3D shape into complete, semantically meaningful parts, even when occluded--is a challenging but crucial task for 3D content creation and understanding. Existing 3D part segmentation methods only identify visible surface patches, limiting their utility. Inspired by 2D amodal segmentation, we introduce this novel task to the 3D domain and propose a practical, two-stage approach, addressing the key challenges of inferring occluded 3D geometry, maintaining global shape consistency, and handling diverse shapes with limited training data. First, we leverage existing 3D part segmentation to obtain initial, incomplete part segments. Second, we introduce HoloPart, a novel diffusion-based model, to complete these segments into full 3D parts. HoloPart utilizes a specialized architecture with local attention to capture fine-grained part geometry and global shape context attention to ensure overall shape consistency. We introduce new benchmarks based on the ABO and PartObjaverse-Tiny datasets and demonstrate that HoloPart significantly outperforms state-of-the-art shape completion methods. By incorporating HoloPart with existing segmentation techniques, we achieve promising results on 3D part amodal segmentation, opening new avenues for applications in geometry editing, animation, and material assignment.

Proposed 2MW Wind Turbine for Use in the Governorate of Dhofar at the Sultanate of Oman

Osama Ahmed Marzouk,Omar Rashid Hamdan Al Badi,Maadh Hamed Salman Al Rashdi,Hamed Mohammed Eid Al Balushi

Task: 设计一种水平轴风力涡轮机（HAWT）作为阿曼Dhofar风电场项目的候选方案。

Motivation: 为GCC地区首个商业规模（50MW）风电场提供2MW的电力生成方案。

Details

Method: 研究阿曼的风图，确定最大平均风速（6m/s），应用建模方程估算功率输出，并通过MATLAB代码匹配设计变量。 Result: 设计出3叶片、直径70m、转速24rpm的转子，输出功率2.37MW，超过目标2MW。 Conclusion: 提出的设计满足目标功率需求，并考虑了变速箱和发电机的功率损耗。 Abstract: In this work, we propose a preliminary design of a horizontal-axis wind turbine (HAWT) as a candidate for the Dhofar Wind Farm project, in the southern Omani Governorate "Dhofar", at the southwest part of the Sultanate of Oman. This wind farm (under construction) is considered to be the first commercial, utility-scale (50MW) wind farm in the GCC (Gulf Cooperation Council) area. The proposed wind turbine has an expected electricity generation of 2MW. We studied the wind atlas of Oman and from which we determined the maximum possible mean wind speed in the entire Sultanate and built our design based on that reference value, which is 6m/s (21.6km/h). After this, we applied a set of modeling equations that estimate the power output from the wind turbine rotor and matched the target electric power to the design variables using a MATLAB computer code. We reached a suitable design and we present here the distribution of the blade angle (twist angle), and the power per unit span along the rotor blade. The rotor design has 3 blades with a diameter of 70m and a rotational speed of 24rpm. This rotor gives 2.37MW of output power, which exceeds the target 2MW output, allowing for about 15% of power losses in the gearbox and generator. We utilized some commercial designs of wind turbines from different international manufacturers as references for typical limits or recommended values of some design parameters.

GenEAva: Generating Cartoon Avatars with Fine-Grained Facial Expressions from Realistic Diffusion-based Faces

Hao Yu,Rupayan Mallick,Margrit Betke,Sarah Adel Bargal

Task: 提出一种名为GenEAva的框架，用于生成高质量、具有细粒度面部表情的卡通头像。

Motivation: 现有卡通头像数据集和生成方法难以呈现高度表现力的头像，且常基于真实身份，引发隐私问题。

Details

Method: 通过微调先进的文本到图像扩散模型合成细致表情，并结合风格化模型将真实面部转化为卡通头像。 Result: 生成了首个表现力丰富的卡通头像数据集GenEAva 1.0，包含13,230个头像和135种表情，且未记忆真实身份。 Conclusion: GenEAva框架和数据集为卡通头像生成提供了多样化和表现力强的基准。 Abstract: Cartoon avatars have been widely used in various applications, including social media, online tutoring, and gaming. However, existing cartoon avatar datasets and generation methods struggle to present highly expressive avatars with fine-grained facial expressions and are often inspired from real-world identities, raising privacy concerns. To address these challenges, we propose a novel framework, GenEAva, for generating high-quality cartoon avatars with fine-grained facial expressions. Our approach fine-tunes a state-of-the-art text-to-image diffusion model to synthesize highly detailed and expressive facial expressions. We then incorporate a stylization model that transforms these realistic faces into cartoon avatars while preserving both identity and expression. Leveraging this framework, we introduce the first expressive cartoon avatar dataset, GenEAva 1.0, specifically designed to capture 135 fine-grained facial expressions, featuring 13,230 expressive cartoon avatars with a balanced distribution across genders, racial groups, and age ranges. We demonstrate that our fine-tuned model generates more expressive faces than the state-of-the-art text-to-image diffusion model SDXL. We also verify that the cartoon avatars generated by our framework do not include memorized identities from fine-tuning data. The proposed framework and dataset provide a diverse and expressive benchmark for future research in cartoon avatar generation.

Holistic Capability Preservation: Towards Compact Yet Comprehensive Reasoning Models

Ling Team,Caizhi Tang,Chilin Fu,Chunwei Wu,Jia Guo,Jianwen Wang,Jingyu Hu,Liang Jiang,Meng Li,Peng Jiao,Pingping Liu,Shaomian Zheng,Shiwei Liang,Shuaicheng Li,Yalin Zhang,Yingting Wu,Yongkang Liu,Zhenyu Huang

Task: 开发一个轻量级推理模型Ring-Lite-Distill，通过高质量数据训练和高效训练范式，提升推理能力并保持参数效率。

Motivation: 通过优化训练方法和数据质量，使轻量级MoE模型Ling-Lite具备更强的推理能力，同时覆盖更全面的任务能力。

Details

Method: 采用高质量数据筛选和高效训练范式，专注于提升推理能力并保持通用能力（如指令遵循、工具使用和知识保留）。 Result: Ring-Lite-Distill的推理能力与DeepSeek-R1-Distill-Qwen-7B相当，通用能力显著超越。 Conclusion: Ring-Lite-Distill成功实现了高效轻量级推理架构，具备全面能力覆盖，适用于多样化任务。 Abstract: This technical report presents Ring-Lite-Distill, a lightweight reasoning model derived from our open-source Mixture-of-Experts (MoE) Large Language Models (LLMs) Ling-Lite. This study demonstrates that through meticulous high-quality data curation and ingenious training paradigms, the compact MoE model Ling-Lite can be further trained to achieve exceptional reasoning capabilities, while maintaining its parameter-efficient architecture with only 2.75 billion activated parameters, establishing an efficient lightweight reasoning architecture. In particular, in constructing this model, we have not merely focused on enhancing advanced reasoning capabilities, exemplified by high-difficulty mathematical problem solving, but rather aimed to develop a reasoning model with more comprehensive competency coverage. Our approach ensures coverage across reasoning tasks of varying difficulty levels while preserving generic capabilities, such as instruction following, tool use, and knowledge retention. We show that, Ring-Lite-Distill's reasoning ability reaches a level comparable to DeepSeek-R1-Distill-Qwen-7B, while its general capabilities significantly surpass those of DeepSeek-R1-Distill-Qwen-7B. The models are accessible at https://huggingface.co/inclusionAI

InteractAvatar: Modeling Hand-Face Interaction in Photorealistic Avatars with Deformable Gaussians

Kefan Chen,Sergiu Oprea,Justin Theiss,Sreyas Mohan,Srinath Sridhar,Aayush Prakash

Task: 提出InteracttAvatar模型，以高保真度捕捉动态手部与非刚性手-脸交互的光照真实外观。

Motivation: 随着数字虚拟形象的社区兴趣增长，以及表情和手势在沟通中的重要性，建模自然虚拟行为成为跨行业（如远程会议、游戏和AR/VR）的重要挑战。

Details

Method: 结合模板模型、3D高斯泼溅和动态细化模块的Dynamic Gaussian Hand模型，以及手-脸交互模块，捕捉姿态依赖的变化和细微几何与外观动态。 Result: 通过新视角合成、自我重演和跨身份重演实验，证明InteracttAvatar能从单目或多视角视频中高保真重建手部和手-脸交互，并支持新姿态动画。 Conclusion: InteracttAvatar是首个能忠实捕捉动态手部与非刚性手-脸交互光照真实外观的模型，为虚拟行为建模提供了重要突破。 Abstract: With the rising interest from the community in digital avatars coupled with the importance of expressions and gestures in communication, modeling natural avatar behavior remains an important challenge across many industries such as teleconferencing, gaming, and AR/VR. Human hands are the primary tool for interacting with the environment and essential for realistic human behavior modeling, yet existing 3D hand and head avatar models often overlook the crucial aspect of hand-body interactions, such as between hand and face. We present InteracttAvatar, the first model to faithfully capture the photorealistic appearance of dynamic hand and non-rigid hand-face interactions. Our novel Dynamic Gaussian Hand model, combining template model and 3D Gaussian Splatting as well as a dynamic refinement module, captures pose-dependent change, e.g. the fine wrinkles and complex shadows that occur during articulation. Importantly, our hand-face interaction module models the subtle geometry and appearance dynamics that underlie common gestures. Through experiments of novel view synthesis, self reenactment and cross-identity reenactment, we demonstrate that InteracttAvatar can reconstruct hand and hand-face interactions from monocular or multiview videos with high-fidelity details and be animated with novel poses.

R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents

Naman Jain,Jaskirat Singh,Manish Shetty,Liang Zheng,Koushik Sen,Ion Stoica

Task: 解决现实世界软件工程任务（如GitHub问题）的开源模型改进问题。

Motivation: 面临两个关键挑战：1）可扩展的执行环境构建；2）测试时计算的最优扩展。

Details

Method: 引入AgentGym，一个程序化构建的可执行训练环境，包含8.7K任务，采用SYNGEN（合成数据生成方法）和混合测试时扩展策略。 Result: 在SWE-Bench Verified基准测试中，32B模型的pass@1性能达到34.4%，混合方法最终达到51%，优于现有开源模型。 Conclusion: 通过AgentGym和混合测试时扩展策略，首次展示了开源模型在软件工程任务上与专有模型的竞争力。 Abstract: Improving open-source models on real-world SWE tasks (solving GITHUB issues) faces two key challenges: 1) scalable curation of execution environments to train these models, and, 2) optimal scaling of test-time compute. We introduce AgentGym, the largest procedurally-curated executable gym environment for training real-world SWE-agents, consisting of more than 8.7K tasks. AgentGym is powered by two main contributions: 1) SYNGEN: a synthetic data curation recipe that enables scalable curation of executable environments using test-generation and back-translation directly from commits, thereby reducing reliance on human-written issues or unit tests. We show that this enables more scalable training leading to pass@1 performance of 34.4% on SWE-Bench Verified benchmark with our 32B model. 2) Hybrid Test-time Scaling: we provide an in-depth analysis of two test-time scaling axes; execution-based and execution-free verifiers, demonstrating that they exhibit complementary strengths and limitations. Test-based verifiers suffer from low distinguishability, while execution-free verifiers are biased and often rely on stylistic features. Surprisingly, we find that while each approach individually saturates around 42-43%, significantly higher gains can be obtained by leveraging their complementary strengths. Overall, our approach achieves 51% on the SWE-Bench Verified benchmark, reflecting a new state-of-the-art for open-weight SWE-agents and for the first time showing competitive performance with proprietary models such as o1, o1-preview and sonnet-3.5-v2 (with tools). We will open-source our environments, models, and agent trajectories.

Scaling Laws for Native Multimodal Models Scaling Laws for Native Multimodal Models

Mustafa Shukor,Enrico Fini,Victor Guilherme Turrisi da Costa,Matthieu Cord,Joshua Susskind,Alaaeldin El-Nouby

Task: 研究原生多模态模型（NMMs）的架构设计，并比较早期融合与晚期融合架构的性能。

Motivation: 探讨晚期融合架构是否天生优于早期融合架构，以及如何通过架构设计提升多模态模型的性能。

Details

Method: 对457个不同架构和训练混合的模型进行扩展规律研究，并引入混合专家（MoEs）以学习模态特定权重。 Result: 早期融合架构在较低参数量下表现更强，训练效率更高，部署更简单；引入MoEs显著提升性能。 Conclusion: 早期融合架构在多模态模型中具有优势，结合MoEs可进一步优化性能。 Abstract: Building general-purpose models that can effectively perceive the world through multimodal signals has been a long-standing goal. Current approaches involve integrating separately pre-trained components, such as connecting vision encoders to LLMs and continuing multimodal training. While such approaches exhibit remarkable sample efficiency, it remains an open question whether such late-fusion architectures are inherently superior. In this work, we revisit the architectural design of native multimodal models (NMMs)--those trained from the ground up on all modalities--and conduct an extensive scaling laws study, spanning 457 trained models with different architectures and training mixtures. Our investigation reveals no inherent advantage to late-fusion architectures over early-fusion ones, which do not rely on image encoders. On the contrary, early-fusion exhibits stronger performance at lower parameter counts, is more efficient to train, and is easier to deploy. Motivated by the strong performance of the early-fusion architectures, we show that incorporating Mixture of Experts (MoEs) allows for models that learn modality-specific weights, significantly enhancing performance.

Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression

Hanqi Xiao,Yi-Lin Sung,Elias Stengel-Eskin,Mohit Bansal

Task: 开发一种名为TaCQ的混合精度后训练量化方法，以在低比特（2-3位）设置下保持模型性能。

Motivation: 后训练量化（PTQ）在低比特设置下会显著降低模型性能，因此需要一种能够在不增加过多内存开销的情况下保持性能的方法。

Details

Method: TaCQ通过模拟自动电路发现，将量化过程与特定权重电路（与下游任务性能相关的权重集合）直接关联，保留这些权重为16位，其他权重量化。使用梯度信息预测量化对任务性能的影响。 Result: 在多个任务（QA、数学推理、文本到SQL）和模型（Llama-3、Qwen2.5）上，TaCQ在相同校准数据和更低权重预算下优于现有方法，尤其在2-3位量化中表现突出。 Conclusion: TaCQ在低比特量化中显著提升性能，且其识别重要权重的能力不仅限于任务特定设置。 Abstract: Post-training quantization (PTQ) reduces a model's memory footprint by mapping full precision weights into low bit weights without costly retraining, but can degrade its downstream performance especially in low 2- to 3-bit settings. We develop a new mixed-precision PTQ approach, Task-Circuit Quantization (TaCQ), that draws parallels to automated circuit discovery, directly conditioning the quantization process on specific weight circuits -- which we define as sets of weights associated with downstream task performance. These weights are kept as 16-bit weights, while others are quantized, maintaining performance while only adding a marginal memory cost. Specifically, TaCQ contrasts unquantized model weights with a uniformly-quantized model to estimate the expected change in weights due to quantization and uses gradient information to predict the resulting impact on task performance, allowing us to preserve task-specific weights. We compare TaCQ-based quantization to existing mixed-precision quantization methods when conditioning both on general-purpose and task-specific data. Across QA, math reasoning, and text-to-SQL tasks for both Llama-3 and Qwen2.5, we find that TaCQ outperforms baselines using the same calibration data and a lower weight budget, achieving major improvements in the 2 and 3-bit regime. With only 3.1 bits we are able to recover 96% of Llama-3-8B-Instruct's unquantized 16-bit MMLU performance, obtaining a 5.25% absolute improvement over SPQR. We also observe consistently large gains over existing methods in the 2-bit regime, with an average gain of 14.74% over the strongest baseline, SliM-LLM. Moreover, we observe a 7.20% gain without conditioning on specific tasks, showing TaCQ's ability to identify important weights is not limited to task-conditioned settings.

Perception-R1: Pioneering Perception Policy with Reinforcement Learning

En Yu,Kangheng Lin,Liang Zhao,Jisheng Yin,Yana Wei,Yuang Peng,Haoran Wei,Jianjian Sun,Chunrui Han,Zheng Ge,Xiangyu Zhang,Daxin Jiang,Jingyu Wang,Wenbing Tao

Task: 探索基于规则的强化学习（RL）在多模态大语言模型（MLLM）后训练中对感知策略学习的影响。

Motivation: 尽管初步实验显示RL在视觉感知任务中表现不一致，但研究希望深入理解RL在视觉感知中的核心作用及其影响因素。

Details

Method: 提出Perception-R1框架，使用GRPO算法进行MLLM后训练，并分析感知复杂性和奖励设计对RL效果的影响。 Result: Perception-R1在多个任务中显著提升性能，如RefCOCO+（+4.2%）、PixMo-Count（+17.9%）和COCO2017 val（31.9% AP）。 Conclusion: 感知复杂性和奖励设计是RL在视觉感知中有效的关键因素，Perception-R1为感知策略学习提供了强基线。 Abstract: Inspired by the success of DeepSeek-R1, we explore the potential of rule-based reinforcement learning (RL) in MLLM post-training for perception policy learning. While promising, our initial experiments reveal that incorporating a thinking process through RL does not consistently lead to performance gains across all visual perception tasks. This leads us to delve into the essential role of RL in the context of visual perception. In this work, we return to the fundamentals and explore the effects of RL on different perception tasks. We observe that the perceptual complexity is a major factor in determining the effectiveness of RL. We also observe that reward design plays a crucial role in further approching the upper limit of model perception. To leverage these findings, we propose Perception-R1, a scalable RL framework using GRPO during MLLM post-training. With a standard Qwen2.5-VL-3B-Instruct, Perception-R1 achieves +4.2% on RefCOCO+, +17.9% on PixMo-Count, +4.2% on PageOCR, and notably, 31.9% AP on COCO2017 val for the first time, establishing a strong baseline for perception policy learning.

Leveraging LLMs for Multimodal Retrieval-Augmented Radiology Report Generation via Key Phrase Extraction

Kyoyun Choi,Byungmu Yoon,Soobum Kim,Jonggwon Park

Task: 提出一种基于检索增强生成的方法，用于自动生成放射学报告，以减少计算资源需求并避免幻觉问题。

Motivation: 多模态大语言模型（MLLMs）在放射学报告生成中资源消耗大，需要大量数据和计算成本。

Details

Method: 结合多模态检索和大语言模型（LLMs），通过提取关键短语、图像编码器结构搜索、文本嵌入噪声添加和对比学习等方法。 Result: 在MIMIC-CXR数据集上取得CheXbert指标的最优结果和RadGraph F1指标的竞争性表现，且无需微调LLM。 Conclusion: 该方法在多视角放射学报告生成中表现出强大的泛化能力，适合临床应用。 Abstract: Automated radiology report generation (RRG) holds potential to reduce radiologists' workload, especially as recent advancements in large language models (LLMs) enable the development of multimodal models for chest X-ray (CXR) report generation. However, multimodal LLMs (MLLMs) are resource-intensive, requiring vast datasets and substantial computational cost for training. To address these challenges, we propose a retrieval-augmented generation approach that leverages multimodal retrieval and LLMs to generate radiology reports while mitigating hallucinations and reducing computational demands. Our method uses LLMs to extract key phrases from radiology reports, effectively focusing on essential diagnostic information. Through exploring effective training strategies, including image encoder structure search, adding noise to text embeddings, and additional training objectives, we combine complementary pre-trained image encoders and adopt contrastive learning between text and semantic image embeddings. We evaluate our approach on MIMIC-CXR dataset, achieving state-of-the-art results on CheXbert metrics and competitive RadGraph F1 metric alongside MLLMs, without requiring LLM fine-tuning. Our method demonstrates robust generalization for multi-view RRG, making it suitable for comprehensive clinical applications.

BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimation

Yuanhong Yu,Xingyi He,Chen Zhao,Junhao Yu,Jiaqi Yang,Ruizhen Hu,Yujun Shen,Xing Zhu,Xiaowei Zhou,Sida Peng

Task: 提出一种基于RGB的通用方法，用于解决稀疏视角下的物体姿态估计问题。

Motivation: 现有方法在遮挡和稀疏参考视角下泛化能力有限，限制了实际应用。

Details

Method: 引入物体边界框的角点作为中间表示，通过参考点合成器估计目标视角的2D角点，并结合PnP算法建立2D-3D对应关系。 Result: 在YCB-Video和Occluded-LINEMOD数据集上的实验表明，该方法优于现有技术。 Conclusion: 提出的表示方法显著提升了物体姿态估计的泛化能力，对实际应用至关重要。 Abstract: This paper presents a generalizable RGB-based approach for object pose estimation, specifically designed to address challenges in sparse-view settings. While existing methods can estimate the poses of unseen objects, their generalization ability remains limited in scenarios involving occlusions and sparse reference views, restricting their real-world applicability. To overcome these limitations, we introduce corner points of the object bounding box as an intermediate representation of the object pose. The 3D object corners can be reliably recovered from sparse input views, while the 2D corner points in the target view are estimated through a novel reference-based point synthesizer, which works well even in scenarios involving occlusions. As object semantic points, object corners naturally establish 2D-3D correspondences for object pose estimation with a PnP algorithm. Extensive experiments on the YCB-Video and Occluded-LINEMOD datasets show that our approach outperforms state-of-the-art methods, highlighting the effectiveness of the proposed representation and significantly enhancing the generalization capabilities of object pose estimation, which is crucial for real-world applications.

RadZero: Similarity-Based Cross-Attention for Explainable Vision-Language Alignment in Radiology with Zero-Shot Multi-Task Capability

Jonggwon Park,Soobum Kim,Byungmu Yoon,Kyoyun Choi

Task: 提出RadZero，一种基于相似性的跨注意力框架，用于放射学中的视觉-语言对齐，具备零样本多任务能力。

Motivation: 现有方法难以有效利用复杂放射学报告进行学习，依赖低分辨率图像，且注意力机制的解释性有限。

Details

Method: RadZero利用大语言模型提取放射学报告中的最小语义句子，采用多正对比学习策略捕获图像与多个相关文本描述的关系，并使用预训练视觉编码器与可训练Transformer层处理高分辨率图像。 Result: 在公共胸部X光基准测试中，RadZero在零样本分类、定位和分割任务上优于现有方法，并通过跨模态相似性图分析提升了解释性。 Conclusion: RadZero在医学影像中表现出色，具备开放词汇语义分割能力，验证了其有效性。 Abstract: Recent advancements in multi-modal models have significantly improved vision-language alignment in radiology. However, existing approaches struggle to effectively utilize complex radiology reports for learning, rely on low-resolution images, and offer limited interpretability in attention mechanisms. To address these challenges, we introduce RadZero, a novel similarity-based cross-attention framework for vision-language alignment in radiology with zero-shot multi-task capability. RadZero leverages large language models to extract minimal semantic sentences from radiology reports and employs a multi-positive contrastive learning strategy to effectively capture relationships between images and multiple relevant textual descriptions. It also utilizes a pre-trained vision encoder with additional trainable Transformer layers, allowing efficient high-resolution image processing. By computing similarity between text embeddings and local image patch features, RadZero enables zero-shot inference with similarity probability for classification and pixel-level cross-modal similarity maps for grounding and segmentation. Experimental results on public chest radiograph benchmarks show that RadZero outperforms state-of-the-art methods in zero-shot classification, grounding, and segmentation. Furthermore, cross-modal similarity map analysis highlights its potential for improving explainability in vision-language alignment. Additionally, qualitative evaluation demonstrates RadZero's capability for open-vocabulary semantic segmentation, further validating its effectiveness in medical imaging.

VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning

Yukun Qi,Yiming Zhao,Yu Zeng,Xikun Bao,Wenxuan Huang,Lin Chen,Zehui Chen,Jie Zhao,Zhongang Qi,Feng Zhao

Task: 提出VCR-Bench，一个用于全面评估大型视觉语言模型（LVLMs）视频链式思维推理能力的新基准。

Motivation: 当前视频基准无法充分评估推理过程或区分感知与推理能力的缺陷，因此需要更严格的评估框架。

Details

Method: VCR-Bench包含859个视频和1,034个高质量问答对，每个问答对附带手动标注的分步链式思维推理依据，并设计七个任务维度和CoT评分。 Result: 实验显示当前LVLMs表现有限，最高模型CoT评分仅62.8%，准确率56.7%，多数模型评分低于40%。感知能力是主要瓶颈。 Conclusion: VCR-Bench可作为标准化评估框架，揭示复杂视频推理任务中的实际缺陷，并验证链式思维推理的重要性。 Abstract: The advancement of Chain-of-Thought (CoT) reasoning has significantly enhanced the capabilities of large language models (LLMs) and large vision-language models (LVLMs). However, a rigorous evaluation framework for video CoT reasoning remains absent. Current video benchmarks fail to adequately assess the reasoning process and expose whether failures stem from deficiencies in perception or reasoning capabilities. Therefore, we introduce VCR-Bench, a novel benchmark designed to comprehensively evaluate LVLMs' Video Chain-of-Thought Reasoning capabilities. VCR-Bench comprises 859 videos spanning a variety of video content and durations, along with 1,034 high-quality question-answer pairs. Each pair is manually annotated with a stepwise CoT rationale, where every step is tagged to indicate its association with the perception or reasoning capabilities. Furthermore, we design seven distinct task dimensions and propose the CoT score to assess the entire CoT process based on the stepwise tagged CoT rationals. Extensive experiments on VCR-Bench highlight substantial limitations in current LVLMs. Even the top-performing model, o1, only achieves a 62.8% CoT score and an 56.7% accuracy, while most models score below 40%. Experiments show most models score lower on perception than reasoning steps, revealing LVLMs' key bottleneck in temporal-spatial information processing for complex video reasoning. A robust positive correlation between the CoT score and accuracy confirms the validity of our evaluation framework and underscores the critical role of CoT reasoning in solving complex video reasoning tasks. We hope VCR-Bench to serve as a standardized evaluation framework and expose the actual drawbacks in complex video reasoning task.

LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document Reranking

Qi Liu,Haozhe Duan,Yiqun Chen,Quanfeng Lu,Weiwei Sun,Jiaxin Mao

Task: 提出一个统一的框架LLM4Ranking，用于利用开源或闭源API的大语言模型（LLMs）进行文档重排序。

Motivation: 近年来，利用LLMs进行文档重排序成为热门研究方向，但缺乏统一的框架支持不同排名方法和模型的应用与评估。

Details

Method: 开发了LLM4Ranking框架，提供简单可扩展的接口，支持文档重排序、评估和微调脚本。 Result: 在多个广泛使用的数据集上评估了不同模型和方法，提供了可复现的结果。 Conclusion: LLM4Ranking是一个实用且可扩展的框架，适用于LLMs在文档重排序中的研究和应用。 Abstract: Utilizing large language models (LLMs) for document reranking has been a popular and promising research direction in recent years, many studies are dedicated to improving the performance and efficiency of using LLMs for reranking. Besides, it can also be applied in many real-world applications, such as search engines or retrieval-augmented generation. In response to the growing demand for research and application in practice, we introduce a unified framework, \textbf{LLM4Ranking}, which enables users to adopt different ranking methods using open-source or closed-source API-based LLMs. Our framework provides a simple and extensible interface for document reranking with LLMs, as well as easy-to-use evaluation and fine-tuning scripts for this task. We conducted experiments based on this framework and evaluated various models and methods on several widely used datasets, providing reproducibility results on utilizing LLMs for document reranking. Our code is publicly available at https://github.com/liuqi6777/llm4ranking.

MM-IFEngine: Towards Multimodal Instruction Following

Shengyuan Ding,Shenxi Wu,Xiangyu Zhao,Yuhang Zang,Haodong Duan,Xiaoyi Dong,Pan Zhang,Yuhang Cao,Dahua Lin,Jiaqi Wang

Task: 提出MM-IFEngine，用于生成高质量图像-指令对，并构建MM-IFInstruct-23k和MM-IFDPO-23k数据集，以提升多模态大语言模型的指令跟随能力。

Motivation: 现有多模态指令跟随训练数据稀缺，基准测试简单且评估策略不精确，无法满足精确输出约束任务的需求。

Details

Method: 通过MM-IFEngine生成大规模、多样化和高质量的训练数据，并构建MM-IFEval基准测试，结合规则评估和模型评估。 Result: 在多个基准测试上（如MM-IFEval、MIA和IFEval）取得了显著提升（+10.2%、+7.6%和+12.3%）。 Conclusion: MM-IFEngine及其生成的数据集和基准测试有效提升了多模态大语言模型的指令跟随能力。 Abstract: The Instruction Following (IF) ability measures how well Multi-modal Large Language Models (MLLMs) understand exactly what users are telling them and whether they are doing it right. Existing multimodal instruction following training data is scarce, the benchmarks are simple with atomic instructions, and the evaluation strategies are imprecise for tasks demanding exact output constraints. To address this, we present MM-IFEngine, an effective pipeline to generate high-quality image-instruction pairs. Our MM-IFEngine pipeline yields large-scale, diverse, and high-quality training data MM-IFInstruct-23k, which is suitable for Supervised Fine-Tuning (SFT) and extended as MM-IFDPO-23k for Direct Preference Optimization (DPO). We further introduce MM-IFEval, a challenging and diverse multi-modal instruction-following benchmark that includes (1) both compose-level constraints for output responses and perception-level constraints tied to the input images, and (2) a comprehensive evaluation pipeline incorporating both rule-based assessment and judge model. We conduct SFT and DPO experiments and demonstrate that fine-tuning MLLMs on MM-IFInstruct-23k and MM-IFDPO-23k achieves notable gains on various IF benchmarks, such as MM-IFEval (+10.2$\%$), MIA (+7.6$\%$), and IFEval (+12.3$\%$). The full data and evaluation code will be released on https://github.com/SYuan03/MM-IFEngine.

LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation

Juzheng Zhang,Jiacheng You,Ashwinee Panda,Tom Goldstein

Task: 提出一种名为LoRI的参数高效微调方法，以减少多任务场景中的参数干扰和计算开销。

Motivation: LoRA在多任务场景中存在显著的计算开销和参数干扰问题，需要改进。

Details

Method: 冻结投影矩阵A为随机投影，并通过任务特定掩码稀疏化矩阵B，以减少可训练参数并保持任务性能。 Result: LoRI在自然语言理解、数学推理、代码生成和安全对齐任务中表现优于全微调和现有PEFT方法，且可训练参数比LoRA少95%。 Conclusion: LoRI通过减少跨任务干扰和支持持续学习，在多任务场景中表现出色。 Abstract: Low-Rank Adaptation (LoRA) has emerged as a popular parameter-efficient fine-tuning (PEFT) method for Large Language Models (LLMs), yet it still incurs notable overhead and suffers from parameter interference in multi-task scenarios. We propose LoRA with Reduced Interference (LoRI), a simple yet effective approach that freezes the projection matrices $A$ as random projections and sparsifies the matrices $B$ using task-specific masks. This design substantially reduces the number of trainable parameters while maintaining strong task performance. Moreover, LoRI minimizes cross-task interference in adapter merging by leveraging the orthogonality between adapter subspaces, and supports continual learning by using sparsity to mitigate catastrophic forgetting. Extensive experiments across natural language understanding, mathematical reasoning, code generation, and safety alignment tasks demonstrate that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference. Code is available at: https://github.com/juzhengz/LoRI

Detect Anything 3D in the Wild

Hanxue Zhang,Haoran Jiang,Qingsong Yao,Yanan Sun,Renrui Zhang,Hao Zhao,Hongyang Li,Hongzi Zhu,Zetong Yang

Task: 提出DetAny3D，一种可提示的3D检测基础模型，用于在任意相机配置下检测新颖物体。

Motivation: 现有深度学习方法在零样本泛化到新颖物体和相机配置方面表现不佳，且3D标注数据有限。

Details

Method: 利用预训练的2D基础模型知识，通过2D聚合器和3D解释器模块实现2D到3D的知识迁移。 Result: DetAny3D在新颖类别和相机配置上表现优异，并在域内数据上超越多数竞争对手。 Conclusion: DetAny3D展示了3D基础模型在现实场景中的潜力，为开放世界3D任务提供了新方向。 Abstract: Despite the success of deep learning in close-set 3D object detection, existing approaches struggle with zero-shot generalization to novel objects and camera configurations. We introduce DetAny3D, a promptable 3D detection foundation model capable of detecting any novel object under arbitrary camera configurations using only monocular inputs. Training a foundation model for 3D detection is fundamentally constrained by the limited availability of annotated 3D data, which motivates DetAny3D to leverage the rich prior knowledge embedded in extensively pre-trained 2D foundation models to compensate for this scarcity. To effectively transfer 2D knowledge to 3D, DetAny3D incorporates two core modules: the 2D Aggregator, which aligns features from different 2D foundation models, and the 3D Interpreter with Zero-Embedding Mapping, which mitigates catastrophic forgetting in 2D-to-3D knowledge transfer. Experimental results validate the strong generalization of our DetAny3D, which not only achieves state-of-the-art performance on unseen categories and novel camera configurations, but also surpasses most competitors on in-domain data.DetAny3D sheds light on the potential of the 3D foundation model for diverse applications in real-world scenarios, e.g., rare object detection in autonomous driving, and demonstrates promise for further exploration of 3D-centric tasks in open-world settings. More visualization results can be found at DetAny3D project page.

VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model

Haozhan Shen,Peng Liu,Jingcheng Li,Chunxin Fang,Yibo Ma,Jiajia Liao,Qiaoli Shen,Zilun Zhang,Kangjia Zhao,Qianqian Zhang,Ruochen Xu,Tiancheng Zhao

Task: 研究如何通过强化学习（RL）提升视觉语言模型（VLMs）的视觉推理能力。

Motivation: 观察到视觉理解任务通常具有明确的真实标注，适合规则化奖励机制，因此探索将R1风格的强化学习扩展到VLMs。

Details

Method: 开发了VLM-R1框架，利用强化学习优化VLMs在视觉语言任务中的表现。 Result: 实验表明，基于RL的模型在视觉理解任务中表现优异，且泛化能力超越监督微调（SFT）。 Conclusion: 通过分析揭示了强化学习对视觉语言模型能力的提升机制，并开源代码以推动社区发展。 Abstract: Recently DeepSeek R1 has shown that reinforcement learning (RL) can substantially improve the reasoning capabilities of Large Language Models (LLMs) through a simple yet effective design. The core of R1 lies in its rule-based reward formulation, which leverages tasks with deterministic ground-truth answers to enable precise and stable reward computation. In the visual domain, we similarly observe that a wide range of visual understanding tasks are inherently equipped with well-defined ground-truth annotations. This property makes them naturally compatible with rule-based reward mechanisms. Motivated by this observation, we investigate the extension of R1-style reinforcement learning to Vision-Language Models (VLMs), aiming to enhance their visual reasoning capabilities. To this end, we develop VLM-R1, a dedicated framework designed to harness RL for improving VLMs' performance on general vision-language tasks. Using this framework, we further explore the feasibility of applying RL to visual domain. Experimental results indicate that the RL-based model not only delivers competitive performance on visual understanding tasks but also surpasses Supervised Fine-Tuning (SFT) in generalization ability. Furthermore, we conduct comprehensive ablation studies that uncover a series of noteworthy insights, including the presence of reward hacking in object detection, the emergence of the "OD aha moment", the impact of training data quality, and the scaling behavior of RL across different model sizes. Through these analyses, we aim to deepen the understanding of how reinforcement learning enhances the capabilities of vision-language models, and we hope our findings and open-source contributions will support continued progress in the vision-language RL community. Our code and model are available at https://github.com/om-ai-lab/VLM-R1

CCMNet: Leveraging Calibrated Color Correction Matrices for Cross-Camera Color Constancy

Dongyoung Kim,Mahmoud Afifi,Dongyun Kim,Michael S. Brown,Seon Joo Kim

Task: 提出一种基于学习的方法，用于跨相机的色彩恒常性，无需重新训练即可适应新相机。

Motivation: 由于白平衡算法需要在相机特定的原始色彩空间中运行，因此必须适应不同相机，而现有方法通常需要针对新相机重新训练。

Details

Method: 利用ISP中预校准的色彩校正矩阵（CCMs）将预定义的照明颜色映射到测试相机的原始空间，并通过相机指纹嵌入（CFE）使网络适应未见过的相机。 Result: 在多个数据集和骨干网络上，该方法实现了最先进的跨相机色彩恒常性，且轻量级且仅依赖ISP中现成的数据。 Conclusion: 该方法通过利用预校准CCMs和数据增强技术，有效解决了跨相机色彩恒常性问题，具有实际应用价值。 Abstract: Computational color constancy, or white balancing, is a key module in a camera's image signal processor (ISP) that corrects color casts from scene lighting. Because this operation occurs in the camera-specific raw color space, white balance algorithms must adapt to different cameras. This paper introduces a learning-based method for cross-camera color constancy that generalizes to new cameras without retraining. Our method leverages pre-calibrated color correction matrices (CCMs) available on ISPs that map the camera's raw color space to a standard space (e.g., CIE XYZ). Our method uses these CCMs to transform predefined illumination colors (i.e., along the Planckian locus) into the test camera's raw space. The mapped illuminants are encoded into a compact camera fingerprint embedding (CFE) that enables the network to adapt to unseen cameras. To prevent overfitting due to limited cameras and CCMs during training, we introduce a data augmentation technique that interpolates between cameras and their CCMs. Experimental results across multiple datasets and backbones show that our method achieves state-of-the-art cross-camera color constancy while remaining lightweight and relying only on data readily available in camera ISPs.

CollEX -- A Multimodal Agentic RAG System Enabling Interactive Exploration of Scientific Collections

Florian Schneider,Narges Baba Ahmadi,Niloufar Baba Ahmadi,Iris Vogel,Martin Semmann,Chris Biemann

Task: 介绍并验证CollEx，一种多模态代理增强检索生成（RAG）系统，用于增强对大规模科学文献集合的交互式探索。

Motivation: 传统搜索系统在面对庞大且复杂的科学文献集合时缺乏直观性和交互性，给学习者、教育者和研究者带来障碍。

Details

Method: 利用先进的大型视觉语言模型（LVLMs）作为多模态代理，通过直观的聊天界面实现复杂交互的抽象化。 Result: CollEx显著简化了对多样化科学文献集合的访问，支持教育场景并促进跨学科连接的发现。 Conclusion: CollEx通过多模态集成和代理技术，有效提升了科学文献的探索体验，适用于教育和研究场景。 Abstract: In this paper, we introduce CollEx, an innovative multimodal agentic Retrieval-Augmented Generation (RAG) system designed to enhance interactive exploration of extensive scientific collections. Given the overwhelming volume and inherent complexity of scientific collections, conventional search systems often lack necessary intuitiveness and interactivity, presenting substantial barriers for learners, educators, and researchers. CollEx addresses these limitations by employing state-of-the-art Large Vision-Language Models (LVLMs) as multimodal agents accessible through an intuitive chat interface. By abstracting complex interactions via specialized agents equipped with advanced tools, CollEx facilitates curiosity-driven exploration, significantly simplifying access to diverse scientific collections and records therein. Our system integrates textual and visual modalities, supporting educational scenarios that are helpful for teachers, pupils, students, and researchers by fostering independent exploration as well as scientific excitement and curiosity. Furthermore, CollEx serves the research community by discovering interdisciplinary connections and complementing visual data. We illustrate the effectiveness of our system through a proof-of-concept application containing over 64,000 unique records across 32 collections from a local scientific collection from a public university.

VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning

Zhong-Yu Li,Ruoyi Du,Juncheng Yan,Le Zhuo,Zhen Li,Peng Gao,Zhanyu Ma,Ming-Ming Cheng

Task: 提出一种通用的图像生成框架VisualCloze，支持多种任务和泛化能力。

Motivation: 当前任务特定模型效率低，通用模型面临任务指令泛化、任务分布和统一架构设计的挑战。

Details

Method: 结合视觉上下文学习，使用Graph200K数据集增强任务密度和知识迁移。 Result: VisualCloze支持多任务、泛化到未见任务，并利用预训练模型的生成先验。 Conclusion: VisualCloze为通用图像生成提供了高效解决方案。 Abstract: Recent progress in diffusion models significantly advances various image generation tasks. However, the current mainstream approach remains focused on building task-specific models, which have limited efficiency when supporting a wide range of different needs. While universal models attempt to address this limitation, they face critical challenges, including generalizable task instruction, appropriate task distributions, and unified architectural design. To tackle these challenges, we propose VisualCloze, a universal image generation framework, which supports a wide range of in-domain tasks, generalization to unseen ones, unseen unification of multiple tasks, and reverse generation. Unlike existing methods that rely on language-based task instruction, leading to task ambiguity and weak generalization, we integrate visual in-context learning, allowing models to identify tasks from visual demonstrations. Meanwhile, the inherent sparsity of visual task distributions hampers the learning of transferable knowledge across tasks. To this end, we introduce Graph200K, a graph-structured dataset that establishes various interrelated tasks, enhancing task density and transferable knowledge. Furthermore, we uncover that our unified image generation formulation shared a consistent objective with image infilling, enabling us to leverage the strong generative priors of pre-trained infilling models without modifying the architectures.

Zero-Shot Cross-Domain Code Search without Fine-Tuning

Keyu Liang,Zhongxin Liu,Chao Liu,Zhiyuan Wan,David Lo,Xiaohu Yang

Task: 解决零样本跨领域代码搜索的问题，提出一种无需微调的方法。

Motivation: 预训练语言模型在跨领域场景中表现不佳，现有方法如RAPID需要高成本微调，亟需一种零样本且无需微调的方法。

Details

Method: 将查询-代码匹配分解为查询-注释匹配和代码-代码匹配，利用大语言模型生成注释和伪代码，结合三种匹配模式进行相似性评分和融合。 Result: 在三个数据集上平均MRR分别比CoCoSoDa和UniXcoder高出21.4%和24.9%，且效果与需要微调的RAPID相当或更好。 Conclusion: CodeBridge是一种高效、无需微调的零样本跨领域代码搜索方法，显著优于现有技术。 Abstract: Code search aims to retrieve semantically relevant code snippets for natural language queries. While pre-trained language models (PLMs) have shown remarkable performance in this task, they struggle in cross-domain scenarios, often requiring costly fine-tuning or facing performance drops in zero-shot settings. RAPID, which generates synthetic data for model fine-tuning, is currently the only effective method for zero-shot cross-domain code search. Despite its effectiveness, RAPID demands substantial computational resources for fine-tuning and needs to maintain specialized models for each domain, underscoring the need for a zero-shot, fine-tuning-free approach for cross-domain code search. The key to tackling zero-shot cross-domain code search lies in bridging the gaps among domains. In this work, we propose to break the query-code matching process of code search into two simpler tasks: query-comment matching and code-code matching. Our empirical study reveals the strong complementarity among the three matching schemas in zero-shot cross-domain settings, i.e., query-code, query-comment, and code-code matching. Based on the findings, we propose CodeBridge, a zero-shot, fine-tuning-free approach for cross-domain code search. Specifically, CodeBridge uses Large Language Models (LLMs) to generate comments and pseudo-code, then combines query-code, query-comment, and code-code matching via PLM-based similarity scoring and sampling-based fusion. Experimental results show that our approach outperforms the state-of-the-art PLM-based code search approaches, i.e., CoCoSoDa and UniXcoder, by an average of 21.4% and 24.9% in MRR, respectively, across three datasets. Our approach also yields results that are better than or comparable to those of the zero-shot cross-domain code search approach RAPID, which requires costly fine-tuning.

Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction

Zeren Jiang,Chuanxia Zheng,Iro Laina,Diane Larlus,Andrea Vedaldi

Task: 提出Geo4D方法，利用视频扩散模型进行单目动态场景的3D重建。

Motivation: 利用视频模型捕获的动态先验，仅需合成数据训练即可在真实数据上实现零样本泛化。

Details

Method: 预测多种互补几何模态（点、深度和射线图），并使用多模态对齐算法及滑动窗口进行融合。 Result: 在多个基准测试中显著超越现有视频深度估计方法，包括专为动态场景设计的MonST3R。 Conclusion: Geo4D能够实现长视频的稳健且准确的4D重建。 Abstract: We introduce Geo4D, a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic prior captured by such video models, Geo4D can be trained using only synthetic data while generalizing well to real data in a zero-shot manner. Geo4D predicts several complementary geometric modalities, namely point, depth, and ray maps. It uses a new multi-modal alignment algorithm to align and fuse these modalities, as well as multiple sliding windows, at inference time, thus obtaining robust and accurate 4D reconstruction of long videos. Extensive experiments across multiple benchmarks show that Geo4D significantly surpasses state-of-the-art video depth estimation methods, including recent methods such as MonST3R, which are also designed to handle dynamic scenes.

Deceptive Automated Interpretability: Language Models Coordinating to Fool Oversight Systems

Simon Lermen,Mateusz Dziemian,Natalia Pérez-Campanero Antolín

Task: 研究AI代理如何通过自动神经网络可解释性协调欺骗监督系统。

Motivation: 探讨语言模型如何生成欺骗性解释以逃避检测，揭示模型可能因害怕负面后果而发展欺骗策略。

Details

Method: 使用稀疏自编码器（SAEs）作为实验框架，测试语言模型（Llama、DeepSeek R1和Claude 3.7 Sonnet）生成欺骗性解释的能力，并采用隐写术隐藏信息。 Result: 所有测试的语言模型代理均能成功欺骗监督模型，同时保持与参考标签相当的可解释性评分。 Conclusion: 提出缓解策略，强调需要建立强大的理解和防御机制以应对欺骗行为。 Abstract: We demonstrate how AI agents can coordinate to deceive oversight systems using automated interpretability of neural networks. Using sparse autoencoders (SAEs) as our experimental framework, we show that language models (Llama, DeepSeek R1, and Claude 3.7 Sonnet) can generate deceptive explanations that evade detection. Our agents employ steganographic methods to hide information in seemingly innocent explanations, successfully fooling oversight models while achieving explanation quality comparable to reference labels. We further find that models can scheme to develop deceptive strategies when they believe the detection of harmful features might lead to negative consequences for themselves. All tested LLM agents were capable of deceiving the overseer while achieving high interpretability scores comparable to those of reference labels. We conclude by proposing mitigation strategies, emphasizing the critical need for robust understanding and defenses against deception.

GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentation

Lang Lin,Xueyang Yu,Ziqi Pang,Yu-Xiong Wang

Task: 提出一种利用多模态大语言模型（MLLMs）进行参考视频对象分割（RefVOS）的新框架。

Motivation: 现有基于MLLM的方法在全局推理（理解关键帧）和局部推理（跟踪连续帧）之间存在矛盾，且依赖外部工具。

Details

Method: 提出GLUS框架，通过稀疏的“上下文帧”提供全局信息，连续的“查询帧”进行局部跟踪，并结合预训练VOS记忆库联合训练。 Result: 在MeViS和Ref-Youtube-VOS基准测试中达到新的最优性能。 Conclusion: GLUS框架简单有效，统一了全局和局部一致性，为MLLMs在RefVOS任务中提供了新基准。 Abstract: This paper proposes a novel framework utilizing multi-modal large language models (MLLMs) for referring video object segmentation (RefVOS). Previous MLLM-based methods commonly struggle with the dilemma between "Ref" and "VOS": they either specialize in understanding a few key frames (global reasoning) or tracking objects on continuous frames (local reasoning), and rely on external VOS or frame selectors to mitigate the other end of the challenge. However, our framework GLUS shows that global and local consistency can be unified into a single video segmentation MLLM: a set of sparse "context frames" provides global information, while a stream of continuous "query frames" conducts local object tracking. This is further supported by jointly training the MLLM with a pre-trained VOS memory bank to simultaneously digest short-range and long-range temporal information. To improve the information efficiency within the limited context window of MLLMs, we introduce object contrastive learning to distinguish hard false-positive objects and a self-refined framework to identify crucial frames and perform propagation. By collectively integrating these insights, our GLUS delivers a simple yet effective baseline, achieving new state-of-the-art for MLLMs on the MeViS and Ref-Youtube-VOS benchmark. Our project page is at https://glus-video.github.io/.

Understanding Learner-LLM Chatbot Interactions and the Impact of Prompting Guidelines

Cansu Koyuturk,Emily Theophilou,Sabrina Patania,Gregor Donabauer,Andrea Martinenghi,Chiara Antico,Alessia Telari,Alessia Testa,Sathya Bursic,Franca Garzotto,Davinia Hernandez-Leo,Udo Kruschwitz,Davide Taibi,Simona Amenta,Martin Ruskov,Dimitri Ognibene

Task: 研究如何通过结构化提示指导提升用户与大型语言模型（LLM）交互的效果。

Motivation: 尽管LLM在自然语言交互中表现出色，但用户常因提示不精确而获得低效响应，现有研究揭示了这一问题的普遍性。

Details

Method: 通过教育实验比较三种提示指导方法（任务特定框架和两种基线方法），并分析642次交互数据。 Result: 研究发现结构化提示指导能显著改善用户行为、提示策略的遵循度及AI生成响应的质量。 Conclusion: 结构化提示指导对提升用户与LLM交互能力具有重要价值，为AI素养和聊天机器人设计提供了新见解。 Abstract: Large Language Models (LLMs) have transformed human-computer interaction by enabling natural language-based communication with AI-powered chatbots. These models are designed to be intuitive and user-friendly, allowing users to articulate requests with minimal effort. However, despite their accessibility, studies reveal that users often struggle with effective prompting, resulting in inefficient responses. Existing research has highlighted both the limitations of LLMs in interpreting vague or poorly structured prompts and the difficulties users face in crafting precise queries. This study investigates learner-AI interactions through an educational experiment in which participants receive structured guidance on effective prompting. We introduce and compare three types of prompting guidelines: a task-specific framework developed through a structured methodology and two baseline approaches. To assess user behavior and prompting efficacy, we analyze a dataset of 642 interactions from 107 users. Using Von NeuMidas, an extended pragmatic annotation schema for LLM interaction analysis, we categorize common prompting errors and identify recurring behavioral patterns. We then evaluate the impact of different guidelines by examining changes in user behavior, adherence to prompting strategies, and the overall quality of AI-generated responses. Our findings provide a deeper understanding of how users engage with LLMs and the role of structured prompting guidance in enhancing AI-assisted communication. By comparing different instructional frameworks, we offer insights into more effective approaches for improving user competency in AI interactions, with implications for AI literacy, chatbot usability, and the design of more responsive AI systems.

PixelFlow: Pixel-Space Generative Models with Flow

Shoufa Chen,Chongjian Ge,Shilong Zhang,Peize Sun,Ping Luo

Task: 提出PixelFlow，一种直接在原始像素空间操作的图像生成模型家族。

Motivation: 简化图像生成过程，消除对预训练变分自编码器（VAE）的需求，并使整个模型可端到端训练。

Details

Method: 通过高效的级联流建模，在像素空间中实现可负担的计算成本。 Result: 在256×256 ImageNet类条件图像生成基准上，FID达到1.98；定性文本到图像结果显示PixelFlow在图像质量、艺术性和语义控制方面表现出色。 Conclusion: PixelFlow为下一代视觉生成模型提供了新的范式，并可能激发新的研究机会。 Abstract: We present PixelFlow, a family of image generation models that operate directly in the raw pixel space, in contrast to the predominant latent-space models. This approach simplifies the image generation process by eliminating the need for a pre-trained Variational Autoencoder (VAE) and enabling the whole model end-to-end trainable. Through efficient cascade flow modeling, PixelFlow achieves affordable computation cost in pixel space. It achieves an FID of 1.98 on 256$\times$256 ImageNet class-conditional image generation benchmark. The qualitative text-to-image results demonstrate that PixelFlow excels in image quality, artistry, and semantic control. We hope this new paradigm will inspire and open up new opportunities for next-generation visual generation models. Code and models are available at https://github.com/ShoufaChen/PixelFlow.

Dual Engines of Thoughts: A Depth-Breadth Integration Framework for Open-Ended Analysis

Fei-Hsuan Yu,Yun-Cheng Chou,Teng-Ruei Chen

Task: 提出Dual Engines of Thoughts (DEoT)框架，用于全面开放性问题推理。

Motivation: 传统推理框架主要针对单一答案问题，而DEoT专门设计用于开放性问题，支持更广更深的分析探索。

Details

Method: 框架包含三个关键组件：Base Prompter（优化用户查询）、Solver Agent（任务分解、执行与验证）、Dual-Engine System（广度引擎探索多样性因素，深度引擎进行深入分析）。 Result: 实验结果显示，DEoT在复杂多面问题上的表现优于现有推理模型，胜率达77-86%。 Conclusion: DEoT在平衡广泛覆盖与深度分析方面表现出色，具有高度可定制性，适用于实际应用。 Abstract: We propose the Dual Engines of Thoughts (DEoT), an analytical framework for comprehensive open-ended reasoning. While traditional reasoning frameworks primarily focus on finding "the best answer" or "the correct answer" for single-answer problems, DEoT is specifically designed for "open-ended questions," enabling both broader and deeper analytical exploration. The framework centers on three key components: a Base Prompter for refining user queries, a Solver Agent that orchestrates task decomposition, execution, and validation, and a Dual-Engine System consisting of a Breadth Engine (to explore diverse impact factors) and a Depth Engine (to perform deep investigations). This integrated design allows DEoT to balance wide-ranging coverage with in-depth analysis, and it is highly customizable, enabling users to adjust analytical parameters and tool configurations based on specific requirements. Experimental results show that DEoT excels in addressing complex, multi-faceted questions, achieving a total win rate of 77-86% compared to existing reasoning models, thus highlighting its effectiveness in real-world applications.

Boundary representation learning via Transformer

Qiang Zou,Lizhen Zhu

Task: 提出一种名为边界表示Transformer（BRT）的新方法，将Transformer网络应用于边界表示（B-rep）模型的学习。

Motivation: 尽管Transformer在自然语言处理、计算机视觉和图形学中取得了显著成功，但其在计算机辅助设计（CAD）中处理B-rep模型的应用仍未被充分探索。

Details

Method: BRT提出了一种连续几何嵌入方法，将B-rep表面编码为Bézier三角形，并采用拓扑感知嵌入方法将这些几何嵌入组织为适合Transformer的离散标记序列。 Result: 实验表明，BRT在零件分类和特征识别任务中实现了最先进的性能。 Conclusion: BRT通过结合几何和拓扑信息，成功地将Transformer应用于B-rep模型的学习，填补了该领域的空白。 Abstract: The recent rise of generative artificial intelligence (AI), powered by Transformer networks, has achieved remarkable success in natural language processing, computer vision, and graphics. However, the application of Transformers in computer-aided design (CAD), particularly for processing boundary representation (B-rep) models, remains largely unexplored. To bridge this gap, this paper introduces Boundary Representation Transformer (BRT), a novel method adapting Transformer for B-rep learning. B-rep models pose unique challenges due to their irregular topology and continuous geometric definitions, which are fundamentally different from the structured and discrete data Transformers are designed for. To address this, BRT proposes a continuous geometric embedding method that encodes B-rep surfaces (trimmed and untrimmed) into B\'ezier triangles, preserving their shape and continuity without discretization. Additionally, BRT employs a topology-aware embedding method that organizes these geometric embeddings into a sequence of discrete tokens suitable for Transformers, capturing both geometric and topological characteristics within B-rep models. This enables the Transformer's attention mechanism to effectively learn shape patterns and contextual semantics of boundary elements in a B-rep model. Extensive experiments demonstrate that BRT achieves state-of-the-art performance in part classification and feature recognition tasks.

How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective

Qi Liu,Jiaxin Mao,Ji-Rong Wen

Task: 系统研究大型语言模型（LLM）如何通过模块化机制理解并实现相关性判断。

Motivation: 现有研究未深入探索现成LLM内部如何理解和操作相关性，因此需要揭示其机制以优化信息检索任务。

Details

Method: 采用机制解释性方法，通过激活修补技术分析不同模型组件的作用，揭示相关性判断的多阶段渐进过程。 Result: 发现LLM在早期层提取查询和文档信息，中层处理相关性信息，后期层通过特定注意力头生成所需格式的相关性判断。 Conclusion: 研究揭示了LLM相关性评估的机制，为未来利用LLM进行信息检索任务提供了重要启示。 Abstract: Recent studies have shown that large language models (LLMs) can assess relevance and support information retrieval (IR) tasks such as document ranking and relevance judgment generation. However, the internal mechanisms by which off-the-shelf LLMs understand and operationalize relevance remain largely unexplored. In this paper, we systematically investigate how different LLM modules contribute to relevance judgment through the lens of mechanistic interpretability. Using activation patching techniques, we analyze the roles of various model components and identify a multi-stage, progressive process in generating either pointwise or pairwise relevance judgment. Specifically, LLMs first extract query and document information in the early layers, then process relevance information according to instructions in the middle layers, and finally utilize specific attention heads in the later layers to generate relevance judgments in the required format. Our findings provide insights into the mechanisms underlying relevance assessment in LLMs, offering valuable implications for future research on leveraging LLMs for IR tasks.

MESA: Text-Driven Terrain Generation Using Latent Diffusion and Global Copernicus Data

Paul Borne--Pons,Mikolaj Czerkawski,Rosalie Martin,Romain Rouffet

Task: 通过训练扩散模型利用全球遥感数据生成高质量地形样本。

Motivation: 传统地形建模依赖需要大量领域专业知识和手工规则的程序化技术，缺乏灵活性和可扩展性。

Details

Method: 提出MESA方法，基于扩散模型利用全球遥感数据生成地形样本。 Result: 模型能够生成逼真且多样化的地形景观，并发布了Major TOM Core-DEM扩展数据集。 Conclusion: 数据驱动模型在遥感数据训练下可为地形建模和生成提供强大工具。 Abstract: Terrain modeling has traditionally relied on procedural techniques, which often require extensive domain expertise and handcrafted rules. In this paper, we present MESA - a novel data-centric alternative by training a diffusion model on global remote sensing data. This approach leverages large-scale geospatial information to generate high-quality terrain samples from text descriptions, showcasing a flexible and scalable solution for terrain generation. The model's capabilities are demonstrated through extensive experiments, highlighting its ability to generate realistic and diverse terrain landscapes. The dataset produced to support this work, the Major TOM Core-DEM extension dataset, is released openly as a comprehensive resource for global terrain data. The results suggest that data-driven models, trained on remote sensing data, can provide a powerful tool for realistic terrain modeling and generation.

Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory

Mirac Suzgun,Mert Yuksekgonul,Federico Bianchi,Dan Jurafsky,James Zou

Task: 提出一种名为Dynamic Cheatsheet（DC）的轻量级框架，为黑盒语言模型提供持久且动态演化的记忆能力。

Motivation: 当前语言模型在处理输入时缺乏记忆能力，无法保留和复用之前的解决方案或错误，导致效率低下。

Details

Method: 通过DC框架，模型能够在推理时存储和复用积累的策略、代码片段和问题解决经验，无需显式标签或人工反馈。 Result: 在多项任务中显著提升性能，例如Claude 3.5 Sonnet在AIME数学考试中准确率翻倍，GPT-4o在Game of 24中的成功率从10%提升至99%。 Conclusion: DC框架为语言模型提供了持久记忆能力，弥补了孤立推理与人类经验驱动学习之间的差距。 Abstract: Despite their impressive performance on complex tasks, current language models (LMs) typically operate in a vacuum: Each input query is processed separately, without retaining insights from previous attempts. Here, we present Dynamic Cheatsheet (DC), a lightweight framework that endows a black-box LM with a persistent, evolving memory. Rather than repeatedly re-discovering or re-committing the same solutions and mistakes, DC enables models to store and reuse accumulated strategies, code snippets, and general problem-solving insights at inference time. This test-time learning enhances performance substantially across a range of tasks without needing explicit ground-truth labels or human feedback. Leveraging DC, Claude 3.5 Sonnet's accuracy more than doubled on AIME math exams once it began retaining algebraic insights across questions. Similarly, GPT-4o's success rate on Game of 24 increased from 10% to 99% after the model discovered and reused a Python-based solution. In tasks prone to arithmetic mistakes, such as balancing equations, DC enabled GPT-4o and Claude to reach near-perfect accuracy by recalling previously validated code, whereas their baselines stagnated around 50%. Beyond arithmetic challenges, DC yields notable accuracy gains on knowledge-demanding tasks. Claude achieved a 9% improvement in GPQA-Diamond and an 8% boost on MMLU-Pro problems. Crucially, DC's memory is self-curated, focusing on concise, transferable snippets rather than entire transcript. Unlike finetuning or static retrieval methods, DC adapts LMs' problem-solving skills on the fly, without modifying their underlying parameters. Overall, our findings present DC as a promising approach for augmenting LMs with persistent memory, bridging the divide between isolated inference events and the cumulative, experience-driven learning characteristic of human cognition.

MoEDiff-SR: Mixture of Experts-Guided Diffusion Model for Region-Adaptive MRI Super-Resolution

Zhe Wang,Yuhua Ru,Aladine Chetouani,Fang Chen,Fabian Bauer,Liping Zhang,Didier Hans,Rachid Jennane,Mohamed Jarraya,Yung Hsin Chen

Task: 提出一种基于混合专家（MoE）引导的扩散模型MoEDiff-SR，用于区域自适应的磁共振成像（MRI）超分辨率重建。

Motivation: 低场强MRI（如3T）的空间分辨率有限，难以捕捉临床诊断和神经影像研究所需的精细解剖细节。

Details

Method: MoEDiff-SR通过Transformer提取多尺度特征，利用MoE门控网络动态选择针对不同脑部区域的扩散去噪专家，实现区域自适应超分辨率重建。 Result: 实验表明，MoEDiff-SR在图像质量指标、感知保真度和计算效率上优于现有方法，临床评估也验证了其在识别细微病理特征上的优越性。 Conclusion: MoEDiff-SR通过区域自适应去噪显著提升了MRI超分辨率性能，具有临床实用价值。 Abstract: Magnetic Resonance Imaging (MRI) at lower field strengths (e.g., 3T) suffers from limited spatial resolution, making it challenging to capture fine anatomical details essential for clinical diagnosis and neuroimaging research. To overcome this limitation, we propose MoEDiff-SR, a Mixture of Experts (MoE)-guided diffusion model for region-adaptive MRI Super-Resolution (SR). Unlike conventional diffusion-based SR models that apply a uniform denoising process across the entire image, MoEDiff-SR dynamically selects specialized denoising experts at a fine-grained token level, ensuring region-specific adaptation and enhanced SR performance. Specifically, our approach first employs a Transformer-based feature extractor to compute multi-scale patch embeddings, capturing both global structural information and local texture details. The extracted feature embeddings are then fed into an MoE gating network, which assigns adaptive weights to multiple diffusion-based denoisers, each specializing in different brain MRI characteristics, such as centrum semiovale, sulcal and gyral cortex, and grey-white matter junction. The final output is produced by aggregating the denoised results from these specialized experts according to dynamically assigned gating probabilities. Experimental results demonstrate that MoEDiff-SR outperforms existing state-of-the-art methods in terms of quantitative image quality metrics, perceptual fidelity, and computational efficiency. Difference maps from each expert further highlight their distinct specializations, confirming the effective region-specific denoising capability and the interpretability of expert contributions. Additionally, clinical evaluation validates its superior diagnostic capability in identifying subtle pathological features, emphasizing its practical relevance in clinical neuroimaging. Our code is available at https://github.com/ZWang78/MoEDiff-SR.

Perception-R1: Pioneering Perception Policy with Reinforcement Learning

En Yu,Kangheng Lin,Liang Zhao,Jisheng Yin,Yana Wei,Yuang Peng,Haoran Wei,Jianjian Sun,Chunrui Han,Zheng Ge,Xiangyu Zhang,Daxin Jiang,Jingyu Wang,Wenbing Tao

Task: 探索基于规则的强化学习（RL）在多模态大语言模型（MLLM）后训练中对感知策略学习的潜力。

Motivation: 初步实验表明，通过RL引入思考过程并未在所有视觉感知任务中带来性能提升，因此需要深入研究RL在视觉感知中的本质作用。

Details

Method: 提出Perception-R1，一个可扩展的RL框架，使用GRPO方法在MLLM后训练中优化感知任务。 Result: 在多个基准测试中显著提升性能，如RefCOCO+（+4.2%）、PixMo-Count（+17.9%）、PageOCR（+4.2%），并在COCO2017 val上首次达到31.9% AP。 Conclusion: 感知任务的复杂性及奖励设计是决定RL效果的关键因素，Perception-R1为感知策略学习建立了强基线。 Abstract: Inspired by the success of DeepSeek-R1, we explore the potential of rule-based reinforcement learning (RL) in MLLM post-training for perception policy learning. While promising, our initial experiments reveal that incorporating a thinking process through RL does not consistently lead to performance gains across all visual perception tasks. This leads us to delve into the essential role of RL in the context of visual perception. In this work, we return to the fundamentals and explore the effects of RL on different perception tasks. We observe that the perceptual complexity is a major factor in determining the effectiveness of RL. We also observe that reward design plays a crucial role in further approching the upper limit of model perception. To leverage these findings, we propose Perception-R1, a scalable RL framework using GRPO during MLLM post-training. With a standard Qwen2.5-VL-3B-Instruct, Perception-R1 achieves +4.2% on RefCOCO+, +17.9% on PixMo-Count, +4.2% on PageOCR, and notably, 31.9% AP on COCO2017 val for the first time, establishing a strong baseline for perception policy learning.

Identifying regions of interest in whole slide images of renal cell carcinoma

Mohammed Lamine Benomar,Nesma Settouti,Eric Debreuve,Xavier Descombes,Damien Ambrosetti

Task: 开发一个全自动系统，用于在肾细胞癌（RCC）的全切片图像（WSI）中检测感兴趣区域（ROIs），以减少分析时间并辅助病理学家做出更准确的诊断。

Motivation: 组织病理学图像包含大量信息，诊断过程耗时且繁琐，因此需要自动化系统来提高效率和准确性。

Details

Method: 使用基于主导旋转局部二值模式（DRLBP）的高效纹理描述符和颜色变换，结合特征提取和分类器（如SVM和基于迁移学习的深度学习方法）对图像进行分类。 Result: 在1800个肾癌图像块上，SVM分类器取得了99.17%的最高精确度，迁移学习方法（如ResNet-50）达到了98.50%的精确度。 Conclusion: 提出的方法在肾癌全切片图像中高效地识别了ROIs，为病理诊断提供了自动化支持。 Abstract: The histopathological images contain a huge amount of information, which can make diagnosis an extremely timeconsuming and tedious task. In this study, we developed a completely automated system to detect regions of interest (ROIs) in whole slide images (WSI) of renal cell carcinoma (RCC), to reduce time analysis and assist pathologists in making more accurate decisions. The proposed approach is based on an efficient texture descriptor named dominant rotated local binary pattern (DRLBP) and color transformation to reveal and exploit the immense texture variability at the microscopic high magnifications level. Thereby, the DRLBPs retain the structural information and utilize the magnitude values in a local neighborhood for more discriminative power. For the classification of the relevant ROIs, feature extraction of WSIs patches was performed on the color channels separately to form the histograms. Next, we used the most frequently occurring patterns as a feature selection step to discard non-informative features. The performances of different classifiers on a set of 1800 kidney cancer patches originating from 12 whole slide images were compared and evaluated. Furthermore, the small size of the image dataset allows to investigate deep learning approach based on transfer learning for image patches classification by using deep features and fine-tuning methods. High recognition accuracy was obtained and the classifiers are efficient, the best precision result was 99.17% achieved with SVM. Moreover, transfer learning models perform well with comparable performance, and the highest precision using ResNet-50 reached 98.50%. The proposed approach results revealed a very efficient image classification and demonstrated efficacy in identifying ROIs. This study presents an automatic system to detect regions of interest relevant to the diagnosis of kidney cancer in whole slide histopathology images.

VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning

Yukun Qi,Yiming Zhao,Yu Zeng,Xikun Bao,Wenxuan Huang,Lin Chen,Zehui Chen,Jie Zhao,Zhongang Qi,Feng Zhao

Task: 提出VCR-Bench，一个用于全面评估大型视觉语言模型（LVLMs）视频链式思维推理能力的新基准。

Motivation: 当前视频基准无法充分评估推理过程或区分感知与推理能力的缺陷，因此需要一种更严格的评估框架。

Details

Method: VCR-Bench包含859个视频和1,034个高质量问答对，每个问答对附带逐步标注的链式思维推理依据，并设计七个任务维度和CoT评分。 Result: 实验显示当前LVLMs表现有限，最佳模型CoT评分仅62.8%，准确率56.7%，大多数模型评分低于40%，且感知能力是瓶颈。 Conclusion: VCR-Bench可作为标准化评估框架，揭示复杂视频推理任务的实际缺陷，并验证链式思维推理的重要性。 Abstract: The advancement of Chain-of-Thought (CoT) reasoning has significantly enhanced the capabilities of large language models (LLMs) and large vision-language models (LVLMs). However, a rigorous evaluation framework for video CoT reasoning remains absent. Current video benchmarks fail to adequately assess the reasoning process and expose whether failures stem from deficiencies in perception or reasoning capabilities. Therefore, we introduce VCR-Bench, a novel benchmark designed to comprehensively evaluate LVLMs' Video Chain-of-Thought Reasoning capabilities. VCR-Bench comprises 859 videos spanning a variety of video content and durations, along with 1,034 high-quality question-answer pairs. Each pair is manually annotated with a stepwise CoT rationale, where every step is tagged to indicate its association with the perception or reasoning capabilities. Furthermore, we design seven distinct task dimensions and propose the CoT score to assess the entire CoT process based on the stepwise tagged CoT rationals. Extensive experiments on VCR-Bench highlight substantial limitations in current LVLMs. Even the top-performing model, o1, only achieves a 62.8% CoT score and an 56.7% accuracy, while most models score below 40%. Experiments show most models score lower on perception than reasoning steps, revealing LVLMs' key bottleneck in temporal-spatial information processing for complex video reasoning. A robust positive correlation between the CoT score and accuracy confirms the validity of our evaluation framework and underscores the critical role of CoT reasoning in solving complex video reasoning tasks. We hope VCR-Bench to serve as a standardized evaluation framework and expose the actual drawbacks in complex video reasoning task.

Synthetic CT Generation from Time-of-Flight Non-Attenutaion-Corrected PET for Whole-Body PET Attenuation Correction

Weijie Chen,James Wang,Alan McMillan

Task: 开发一种从TOF NAC PET图像生成合成CT（sCT）的深度学习方法，以改进PET/MR系统的衰减校正。

Motivation: PET/MR系统中缺乏CT图像，而CT对衰减校正是必要的，因此需要一种替代方法直接从PET图像生成sCT。

Details

Method: 使用预训练的自然图像模型进行CT重建任务，并在35对TOF NAC PET和CT数据上进行微调。 Result: 模型在体轮廓区域内实现了最低的MAE（74.49 HU）和最高的PSNR（28.66 dB），视觉评估显示对骨和软组织的重建效果提升。 Conclusion: 预训练深度学习模型在医学图像转换任务中有效，未来将探索更多架构和数据集以进一步提升性能。 Abstract: Positron Emission Tomography (PET) imaging requires accurate attenuation correction (AC) to account for photon loss due to tissue density variations. In PET/MR systems, computed tomography (CT), which offers a straightforward estimation of AC is not available. This study presents a deep learning approach to generate synthetic CT (sCT) images directly from Time-of-Flight (TOF) non-attenuation corrected (NAC) PET images, enhancing AC for PET/MR. We first evaluated models pre-trained on large-scale natural image datasets for a CT-to-CT reconstruction task, finding that the pre-trained model outperformed those trained solely on medical datasets. The pre-trained model was then fine-tuned using an institutional dataset of 35 TOF NAC PET and CT volume pairs, achieving the lowest mean absolute error (MAE) of 74.49 HU and highest peak signal-to-noise ratio (PSNR) of 28.66 dB within the body contour region. Visual assessments demonstrated improved reconstruction of both bone and soft tissue structures from TOF NAC PET images. This work highlights the effectiveness of using pre-trained deep learning models for medical image translation tasks. Future work will assess the impact of sCT on PET attenuation correction and explore additional neural network architectures and datasets to further enhance performance and practical applications in PET imaging.

Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments

Lorenz Linhardt,Tom Neuhäuser,Lenka Tětková,Oliver Eberle

Task: 评估32种公开可用的语言模型在单词三元组任务中与人类相似性判断的表征和行为对齐。

Motivation: 研究小型和中型生成语言模型在表征和行为层面的交互，以及它们在语义关联任务中的表现。

Details

Method: 通过单词三元组任务比较语言模型的表征和行为与人类相似性判断的对齐程度。 Result: 发现小型语言模型的表征可以达到人类水平的对齐，指令调优模型变体显著提高一致性，对齐模式因模型而异，行为对齐与模型大小高度相关。 Conclusion: 语言模型的表征和行为对齐在不同规模和类型的模型中表现出显著差异，为语义关联研究提供了新视角。 Abstract: Small and mid-sized generative language models have gained increasing attention. Their size and availability make them amenable to being analyzed at a behavioral as well as a representational level, allowing investigations of how these levels interact. We evaluate 32 publicly available language models for their representational and behavioral alignment with human similarity judgments on a word triplet task. This provides a novel evaluation setting to probe semantic associations in language beyond common pairwise comparisons. We find that (1) even the representations of small language models can achieve human-level alignment, (2) instruction-tuned model variants can exhibit substantially increased agreement, (3) the pattern of alignment across layers is highly model dependent, and (4) alignment based on models' behavioral responses is highly dependent on model size, matching their representational alignment only for the largest evaluated models.

Novel Pooling-based VGG-Lite for Pneumonia and Covid-19 Detection from Imbalanced Chest X-Ray Datasets

Santanu Roy,Ashvath Suresh,Palak Sahu,Tulika Rudra Gupta

Task: 提出一种基于池化的VGG-Lite模型，以解决胸部X光（CXR）数据集中类别不平衡的问题。

Motivation: 由于2020年新冠变种的出现，深度学习模型在CXR图像中自动检测肺炎成为一个重要研究领域，但标准CNN模型面临类别不平衡的挑战。

Details

Method: 提出轻量级CNN模型VGG-Lite，并结合边缘增强模块（EEM），包括负图像层和新型2Max-Min池化层。 Result: 在两个CXR数据集上，提出的框架优于预训练CNN模型和现有模型，在肺炎不平衡数据集上达到95%的准确率。 Conclusion: VGG-Lite结合EEM能有效解决类别不平衡问题，并在肺炎检测中表现优异。 Abstract: This paper proposes a novel pooling-based VGG-Lite model in order to mitigate class imbalance issues in Chest X-Ray (CXR) datasets. Automatic Pneumonia detection from CXR images by deep learning model has emerged as a prominent and dynamic area of research, since the inception of the new Covid-19 variant in 2020. However, the standard Convolutional Neural Network (CNN) models encounter challenges associated with class imbalance, a prevalent issue found in many medical datasets. The innovations introduced in the proposed model architecture include: (I) A very lightweight CNN model, `VGG-Lite', is proposed as a base model, inspired by VGG-16 and MobileNet-V2 architecture. (II) On top of this base model, we leverage an ``Edge Enhanced Module (EEM)" through a parallel branch, consisting of a ``negative image layer", and a novel custom pooling layer ``2Max-Min Pooling". This 2Max-Min Pooling layer is entirely novel in this investigation, providing more attention to edge components within pneumonia CXR images. Thus, it works as an efficient spatial attention module (SAM). We have implemented the proposed framework on two separate CXR datasets. The first dataset is obtained from a readily available source on the internet, and the second dataset is a more challenging CXR dataset, assembled by our research team from three different sources. Experimental results reveal that our proposed framework has outperformed pre-trained CNN models, and three recent trend existing models ``Vision Transformer", ``Pooling-based Vision Transformer (PiT)'' and ``PneuNet", by substantial margins on both datasets. The proposed framework VGG-Lite with EEM, has achieved a macro average of 95% accuracy, 97.1% precision, 96.1% recall, and 96.6% F1 score on the ``Pneumonia Imbalance CXR dataset", without employing any pre-processing technique.

PhaseGen: A Diffusion-Based Approach for Complex-Valued MRI Data Generation

Moritz Rempe,Fabian Hörst,Helmut Becker,Marco Schlimbach,Lukas Rotkopf,Kevin Kröninger,Jens Kleesiek

Task: 提出一种名为PhaseGen的复杂值扩散模型，用于生成基于临床常用幅度图像的合成MRI原始数据。

Motivation: 临床和现有AI方法仅关注幅度图像，忽略了相位数据的潜在价值，而相位数据对下游任务（如肿瘤分割和分类）有重要作用。

Details

Method: 使用PhaseGen模型生成合成MRI原始数据，并在k-Space中进行颅骨剥离和MRI重建任务评估。 Result: 实验表明，合成相位数据显著提高了颅骨分割的准确性（从41.1%提升至80.1%），并增强了MRI重建效果。 Conclusion: PhaseGen通过生成AI填补了基于幅度图像的数据集与复杂值MRI原始数据之间的鸿沟，为更准确的诊断任务提供了支持。 Abstract: Magnetic resonance imaging (MRI) raw data, or k-Space data, is complex-valued, containing both magnitude and phase information. However, clinical and existing Artificial Intelligence (AI)-based methods focus only on magnitude images, discarding the phase data despite its potential for downstream tasks, such as tumor segmentation and classification. In this work, we introduce $\textit{PhaseGen}$, a novel complex-valued diffusion model for generating synthetic MRI raw data conditioned on magnitude images, commonly used in clinical practice. This enables the creation of artificial complex-valued raw data, allowing pretraining for models that require k-Space information. We evaluate PhaseGen on two tasks: skull-stripping directly in k-Space and MRI reconstruction using the publicly available FastMRI dataset. Our results show that training with synthetic phase data significantly improves generalization for skull-stripping on real-world data, with an increased segmentation accuracy from $41.1\%$ to $80.1\%$, and enhances MRI reconstruction when combined with limited real-world data. This work presents a step forward in utilizing generative AI to bridge the gap between magnitude-based datasets and the complex-valued nature of MRI raw data. This approach allows researchers to leverage the vast amount of avaliable image domain data in combination with the information-rich k-Space data for more accurate and efficient diagnostic tasks. We make our code publicly $\href{https://github.com/TIO-IKIM/PhaseGen}{\text{available here}}$.

Extending Visual Dynamics for Video-to-Music Generation

Xiaohao Liu,Teng Tu,Yunshan Ma,Tat-Seng Chua

Task: 提出一种名为DyViM的新框架，用于增强视频到音乐生成中的动态建模。

Motivation: 现有方法在特定场景下表现有限或低估了视觉动态，需要解决动态复杂性和视频与音乐表示之间的时间错位问题。

Details

Method: 通过简化的运动编码器提取帧级动态特征，利用自注意力模块聚合帧内特征，并结合交叉注意力机制传递高级语义，采用退火调优策略微调音乐解码器。 Result: 实验证明DyViM在视频到音乐生成任务中优于现有最先进方法。 Conclusion: DyViM通过动态建模和时间对齐，显著提升了视频到音乐生成的质量和适应性。 Abstract: Music profoundly enhances video production by improving quality, engagement, and emotional resonance, sparking growing interest in video-to-music generation. Despite recent advances, existing approaches remain limited in specific scenarios or undervalue the visual dynamics. To address these limitations, we focus on tackling the complexity of dynamics and resolving temporal misalignment between video and music representations. To this end, we propose DyViM, a novel framework to enhance dynamics modeling for video-to-music generation. Specifically, we extract frame-wise dynamics features via a simplified motion encoder inherited from optical flow methods, followed by a self-attention module for aggregation within frames. These dynamic features are then incorporated to extend existing music tokens for temporal alignment. Additionally, high-level semantics are conveyed through a cross-attention mechanism, and an annealing tuning strategy benefits to fine-tune well-trained music decoders efficiently, therefore facilitating seamless adaptation. Extensive experiments demonstrate DyViM's superiority over state-of-the-art (SOTA) methods.

Heart Failure Prediction using Modal Decomposition and Masked Autoencoders for Scarce Echocardiography Databases

Andrés Bell-Navas,María Villalba-Orero,Enrique Lara-Pezzi,Jesús Garicano-Mena,Soledad Le Clainche

Task: 提出一种基于深度学习的实时超声心动图视频序列分析系统，用于预测心力衰竭发生时间。

Motivation: 心力衰竭对医疗行业构成巨大压力，亟需早期、快速且有效的预测系统。

Details

Method: 系统分为两阶段：第一阶段使用HODMD算法进行数据增强和特征提取，第二阶段构建并训练Vision Transformer（ViT），采用自监督学习方法。 Result: 实验结果表明HODMD算法的有效性，且所提系统优于多种ViT和CNN架构。 Conclusion: 该系统在心力衰竭时间预测任务中表现出高效性和优越性。 Abstract: Heart diseases constitute the main cause of international human defunction. According to the World Health Organization (WHO), approximately 18 million deaths happen each year due to precisely heart diseases. In particular, heart failures (HF) press the healthcare industry to develop systems for their early, rapid and effective prediction. In this work, an automatic system which analyses in real-time echocardiography video sequences is proposed for the challenging and more specific task of prediction of heart failure times. This system is based on a novel deep learning framework, and works in two stages. The first one transforms the data included in a database of echocardiography video sequences into a machine learning-compatible collection of annotated images which can be used in the training phase of any kind of machine learning-based framework, including a deep learning one. This initial stage includes the use of the Higher Order Dynamic Mode Decomposition (HODMD) algorithm for both data augmentation and feature extraction. The second stage is focused on building and training a Vision Transformer (ViT). Self-supervised learning (SSL) methods, which have been so far barely explored in the literature about heart failure prediction, are applied to effectively train the ViT from scratch, even with scarce databases of echocardiograms. The designed neural network analyses images from echocardiography sequences to estimate the time in which a heart failure will happen. The results obtained show the efficacy of the HODMD algorithm and the superiority of the proposed system with respect to several established ViT and Convolutional Neural Network (CNN) architectures.

CollEX -- A Multimodal Agentic RAG System Enabling Interactive Exploration of Scientific Collections

Florian Schneider,Narges Baba Ahmadi,Niloufar Baba Ahmadi,Iris Vogel,Martin Semmann,Chris Biemann

Task: 介绍并验证CollEx，一种多模态代理增强检索生成（RAG）系统，用于增强对大规模科学文献的交互式探索。

Motivation: 传统搜索系统在面对庞大且复杂的科学文献时缺乏直观性和交互性，给用户带来障碍。

Details

Method: 利用先进的大型视觉语言模型（LVLMs）作为多模态代理，通过直观的聊天界面实现复杂交互的抽象化。 Result: CollEx显著简化了对多样化科学文献的访问，支持教育场景并促进跨学科连接。 Conclusion: CollEx通过多模态交互和代理技术，有效提升了科学文献的探索体验。 Abstract: In this paper, we introduce CollEx, an innovative multimodal agentic Retrieval-Augmented Generation (RAG) system designed to enhance interactive exploration of extensive scientific collections. Given the overwhelming volume and inherent complexity of scientific collections, conventional search systems often lack necessary intuitiveness and interactivity, presenting substantial barriers for learners, educators, and researchers. CollEx addresses these limitations by employing state-of-the-art Large Vision-Language Models (LVLMs) as multimodal agents accessible through an intuitive chat interface. By abstracting complex interactions via specialized agents equipped with advanced tools, CollEx facilitates curiosity-driven exploration, significantly simplifying access to diverse scientific collections and records therein. Our system integrates textual and visual modalities, supporting educational scenarios that are helpful for teachers, pupils, students, and researchers by fostering independent exploration as well as scientific excitement and curiosity. Furthermore, CollEx serves the research community by discovering interdisciplinary connections and complementing visual data. We illustrate the effectiveness of our system through a proof-of-concept application containing over 64,000 unique records across 32 collections from a local scientific collection from a public university.

Localization Meets Uncertainty: Uncertainty-Aware Multi-Modal Localization

Hye-Min Won,Jieun Lee,Jiyong Oh

Task: 提出一种基于不确定性的拒绝策略，以提升复杂室内环境中机器人定位的可靠性。

Motivation: 可靠的定位对于机器人在复杂室内环境中的导航至关重要。

Details

Method: 采用基于百分位数的拒绝策略，结合RGB图像和2D LiDAR数据的多模态端到端定位方法。 Result: 实验表明，应用更严格的不确定性阈值显著降低了位置和方向误差，并有效去除了极端异常值。 Conclusion: 该方法首次定量证明了基于百分位数的不确定性拒绝在多模态端到端定位任务中的优势，提升了实际部署中的定位可靠性和准确性。 Abstract: Reliable localization is critical for robot navigation in complex indoor environments. In this paper, we propose an uncertainty-aware localization method that enhances the reliability of localization outputs without modifying the prediction model itself. This study introduces a percentile-based rejection strategy that filters out unreliable 3-DoF pose predictions based on aleatoric and epistemic uncertainties the network estimates. We apply this approach to a multi-modal end-to-end localization that fuses RGB images and 2D LiDAR data, and we evaluate it across three real-world datasets collected using a commercialized serving robot. Experimental results show that applying stricter uncertainty thresholds consistently improves pose accuracy. Specifically, the mean position error is reduced by 41.0%, 56.7%, and 69.4%, and the mean orientation error by 55.6%, 65.7%, and 73.3%, when applying 90%, 80%, and 70% thresholds, respectively. Furthermore, the rejection strategy effectively removes extreme outliers, resulting in better alignment with ground truth trajectories. To the best of our knowledge, this is the first study to quantitatively demonstrate the benefits of percentile-based uncertainty rejection in multi-modal end-to-end localization tasks. Our approach provides a practical means to enhance the reliability and accuracy of localization systems in real-world deployments.

Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation

Yanglin Huang,Kai Hu,Yuan Zhang,Zhineng Chen,Xieping Gao

Task: 提出一种名为HeteroAKD的通用知识蒸馏方法，用于解决异构架构（如CNN和Transformer）在语义分割中的知识迁移问题。

Motivation: 现有知识蒸馏方法在语义分割中主要关注同构架构，忽略了异构架构中多样化的知识，而这些知识对学生模型的精确性和全面性理解至关重要。

Details

Method: 通过将教师和学生的中间特征投影到对齐的logits空间，消除架构特定信息的影响，并引入知识混合机制（KMM）和知识评估机制（KEM）来利用异构知识。 Result: 在三个主流基准测试中，HeteroAKD在异构架构间的知识蒸馏效果优于现有方法。 Conclusion: HeteroAKD是一种有效的异构架构知识蒸馏方法，能够提升学生模型在语义分割中的性能。 Abstract: Current knowledge distillation (KD) methods for semantic segmentation focus on guiding the student to imitate the teacher's knowledge within homogeneous architectures. However, these methods overlook the diverse knowledge contained in architectures with different inductive biases, which is crucial for enabling the student to acquire a more precise and comprehensive understanding of the data during distillation. To this end, we propose for the first time a generic knowledge distillation method for semantic segmentation from a heterogeneous perspective, named HeteroAKD. Due to the substantial disparities between heterogeneous architectures, such as CNN and Transformer, directly transferring cross-architecture knowledge presents significant challenges. To eliminate the influence of architecture-specific information, the intermediate features of both the teacher and student are skillfully projected into an aligned logits space. Furthermore, to utilize diverse knowledge from heterogeneous architectures and deliver customized knowledge required by the student, a teacher-student knowledge mixing mechanism (KMM) and a teacher-student knowledge evaluation mechanism (KEM) are introduced. These mechanisms are performed by assessing the reliability and its discrepancy between heterogeneous teacher-student knowledge. Extensive experiments conducted on three main-stream benchmarks using various teacher-student pairs demonstrate that our HeteroAKD outperforms state-of-the-art KD methods in facilitating distillation between heterogeneous architectures.

Virtual-mask Informed Prior for Sparse-view Dual-Energy CT Reconstruction

Zini Chen,Yao Xiao,Junyan Zhang,Shaoyu Wang,Liu Shi,Qiegen Liu

Task: 提出一种基于双域虚拟掩码的扩散模型，用于稀疏视角双能计算机断层扫描（DECT）重建。

Motivation: 稀疏视角采样在DECT中虽能降低辐射剂量和加快成像速度，但易产生伪影；现有扩散模型多聚焦于图像域且缺乏全局约束，导致重建质量不足。

Details

Method: 设计虚拟掩码并应用于高低能数据以构建高维张量作为扩散模型的先验信息；采用双域协作策略整合小波域和投影域信息以优化全局结构和局部细节。 Result: 实验结果表明该方法在多个数据集上表现优异。 Conclusion: 提出的双域虚拟掩码扩散模型有效提升了稀疏视角DECT的重建质量。 Abstract: Sparse-view sampling in dual-energy computed tomography (DECT) significantly reduces radiation dose and increases imaging speed, yet is highly prone to artifacts. Although diffusion models have demonstrated potential in effectively handling incomplete data, most existing methods in this field focus on the image do-main and lack global constraints, which consequently leads to insufficient reconstruction quality. In this study, we propose a dual-domain virtual-mask in-formed diffusion model for sparse-view reconstruction by leveraging the high inter-channel correlation in DECT. Specifically, the study designs a virtual mask and applies it to the high-energy and low-energy data to perform perturbation operations, thus constructing high-dimensional tensors that serve as the prior information of the diffusion model. In addition, a dual-domain collaboration strategy is adopted to integrate the information of the randomly selected high-frequency components in the wavelet domain with the information in the projection domain, for the purpose of optimizing the global struc-tures and local details. Experimental results indicated that the present method exhibits excellent performance across multiple datasets.

PRAD: Periapical Radiograph Analysis Dataset and Benchmark Model Development

Zhenhuan Zhou,Yuchen Zhang,Ruihong Xu,Xuansen Zhao,Tao Li

Task: 提出一个名为PRAD-10K的数据集和PRNet深度学习网络，用于牙周根尖片（PR）的辅助分析。

Motivation: 由于牙周根尖片在牙科诊断中的广泛应用及其标注和识别的挑战，现有公开数据集稀缺，阻碍了深度学习在该领域的应用。

Details

Method: 构建包含10,000张临床PR图像的PRAD-10K数据集，并提供像素级标注和分类标签；提出PRNet网络用于PR分割任务。 Result: PRNet在PRAD-10K数据集上的表现优于现有医学图像分割模型。 Conclusion: PRAD-10K和PRNet为PR分析提供了高质量的数据集和基准模型，推动了深度学习在牙科诊断中的应用。 Abstract: Deep learning (DL), a pivotal technology in artificial intelligence, has recently gained substantial traction in the domain of dental auxiliary diagnosis. However, its application has predominantly been confined to imaging modalities such as panoramic radiographs and Cone Beam Computed Tomography, with limited focus on auxiliary analysis specifically targeting Periapical Radiographs (PR). PR are the most extensively utilized imaging modality in endodontics and periodontics due to their capability to capture detailed local lesions at a low cost. Nevertheless, challenges such as resolution limitations and artifacts complicate the annotation and recognition of PR, leading to a scarcity of publicly available, large-scale, high-quality PR analysis datasets. This scarcity has somewhat impeded the advancement of DL applications in PR analysis. In this paper, we present PRAD-10K, a dataset for PR analysis. PRAD-10K comprises 10,000 clinical periapical radiograph images, with pixel-level annotations provided by professional dentists for nine distinct anatomical structures, lesions, and artificial restorations or medical devices, We also include classification labels for images with typical conditions or lesions. Furthermore, we introduce a DL network named PRNet to establish benchmarks for PR segmentation tasks. Experimental results demonstrate that PRNet surpasses previous state-of-the-art medical image segmentation models on the PRAD-10K dataset. The codes and dataset will be made publicly available.

Focal Cortical Dysplasia Type II Detection Using Cross Modality Transfer Learning and Grad-CAM in 3D-CNNs for MRI Analysis

Lorenzo Lasagni,Antonio Ciccarone,Renzo Guerrini,Matteo Lenge,Ludovico D'incerti

Task: 研究3D卷积神经网络（3D-CNNs）在FCD检测中的应用，并探索跨模态迁移学习和可解释人工智能（XAI）技术的优势。

Motivation: FCD II型是药物难治性癫痫的主要原因，但MRI诊断困难，易误诊，因此需要更准确的检测方法。

Details

Method: 使用170名受试者（85名FCD患者和85名对照）的T1加权和FLAIR MRI扫描数据，采用ResNet架构（ResNet-18、-34和-50），结合迁移学习和Grad-CAM技术。 Result: 迁移学习显著提高了分类准确率（达80.3%）和可解释性，通过Heat-Score指标验证了模型对临床相关区域的关注。 Conclusion: 迁移学习和XAI技术对提升AI在医学诊断中的应用具有重要意义，尤其是对FCD等难以诊断的病理。 Abstract: Focal cortical dysplasia (FCD) type II is a major cause of drug-resistant epilepsy, often curable only by surgery. Despite its clinical importance, the diagnosis of FCD is very difficult in MRI because of subtle abnormalities, leading to misdiagnosis. This study investigates the use of 3D convolutional neural networks (3D-CNNs) for FCD detection, using a dataset of 170 subjects (85 FCD patients and 85 controls) composed of T1-weighted and FLAIR MRI scans. In particular, it investigates the benefits obtained from cross-modality transfer learning and explainable artificial intelligence (XAI) techniques, in particular Gradient-weighted Class Activation Mapping (Grad-CAM). ResNet architectures (ResNet-18, -34, and -50) were implemented, employing transfer learning strategies that used pre-trained weights from segmentation tasks. Results indicate that transfer learning significantly enhances classification accuracy (up to 80.3%) and interpretability, as measured by a novel Heat-Score metric, which evaluates the model's focus on clinically relevant regions. Improvements in the Heat-Score metric underscore the model's seizure zone localization capabilities, bringing AI predictions and clinical insights closer together. These results highlight the importance of transfer learning, including cross-modality, and XAI in advancing AI-based medical diagnostics, especially for difficult-to-diagnose pathologies such as FCD.

Adaptive Detection of Fast Moving Celestial Objects Using a Mixture of Experts and Physical-Inspired Neural Network

Peng Jia,Ge Li,Bafeng Cheng,Yushan Li,Rongyu Sun

Task: 提出一种新颖的算法，用于在星场中检测快速移动的天体。

Motivation: 传统的地基望远镜方法在空间望远镜的多样化观测模式下效果不佳，需要新的检测方法。

Details

Method: 通过将先进的快速移动天体检测神经网络转化为物理启发的神经网络，利用望远镜的点扩散函数和观测模式作为先验信息。 Result: 在模拟和真实观测数据中，该方法能有效检测不同观测模式下的快速移动天体。 Conclusion: 提出的算法解决了传统技术的局限性，适用于空间望远镜的多样化观测模式。 Abstract: Fast moving celestial objects are characterized by velocities across the celestial sphere that significantly differ from the motions of background stars. In observational images, these objects exhibit distinct shapes, contrasting with the typical appearances of stars. Depending on the observational method employed, these celestial entities may be designated as near-Earth objects or asteroids. Historically, fast moving celestial objects have been observed using ground-based telescopes, where the relative stability of stars and Earth facilitated effective image differencing techniques alongside traditional fast moving celestial object detection and classification algorithms. However, the growing prevalence of space-based telescopes, along with their diverse observational modes, produces images with different properties, rendering conventional methods less effective. This paper presents a novel algorithm for detecting fast moving celestial objects within star fields. Our approach enhances state-of-the-art fast moving celestial object detection neural networks by transforming them into physical-inspired neural networks. These neural networks leverage the point spread function of the telescope and the specific observational mode as prior information; they can directly identify moving fast moving celestial objects within star fields without requiring additional training, thereby addressing the limitations of traditional techniques. Additionally, all neural networks are integrated using the mixture of experts technique, forming a comprehensive fast moving celestial object detection algorithm. We have evaluated our algorithm using simulated observational data that mimics various observations carried out by space based telescope scenarios and real observation images. Results demonstrate that our method effectively detects fast moving celestial objects across different observational modes.

Revisiting Likelihood-Based Out-of-Distribution Detection by Modeling Representations

Yifan Ding,Arturas Aleksandrauskas,Amirhossein Ahmadian,Jonas Unger,Fredrik Lindsten,Gabriel Eilertsen

Task: 探讨基于似然的深度生成模型在分布外（OOD）检测中的有效性。

Motivation: 解决深度生成模型在OOD检测中表现不佳的问题，尤其是似然值对OOD数据评分高于分布内数据的现象。

Details

Method: 使用扩散模型的概率流公式作为似然估计器，并将其应用于预训练编码器的表示空间。 Result: 证明基于似然的方法在表示空间中可以达到与先进方法相当的性能。 Conclusion: 似然本身并非无效，关键在于选择合适的估计器和表示空间。 Abstract: Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning systems, particularly in safety-critical applications. Likelihood-based deep generative models have historically faced criticism for their unsatisfactory performance in OOD detection, often assigning higher likelihood to OOD data than in-distribution samples when applied to image data. In this work, we demonstrate that likelihood is not inherently flawed. Rather, several properties in the images space prohibit likelihood as a valid detection score. Given a sufficiently good likelihood estimator, specifically using the probability flow formulation of a diffusion model, we show that likelihood-based methods can still perform on par with state-of-the-art methods when applied in the representation space of pre-trained encoders. The code of our work can be found at $\href{https://github.com/limchaos/Likelihood-OOD.git}{\texttt{https://github.com/limchaos/Likelihood-OOD.git}}$.

HarmonySeg: Tubular Structure Segmentation with Deep-Shallow Feature Fusion and Growth-Suppression Balanced Loss

Yi Huang,Ke Zhang,Wei Liu,Yuanyuan Wang,Vishal M. Patel,Le Lu,Xu Han,Dakai Jin,Ke Yan

Task: 提出一种名为HarmonySeg的新框架，用于医学图像中管状结构（如血管和气道树）的精确分割。

Motivation: 管状结构在医学图像中的分割对计算机辅助诊断、放射治疗和手术规划至关重要，但算法设计面临尺寸多样、拓扑复杂和标注不完整等挑战。

Details

Method: 设计了深到浅的解码器网络，结合灵活卷积块和多尺度处理；引入血管性图作为辅助信息；提出拓扑保持损失函数以平衡管状结构的生长和抑制。 Result: 在四个公共数据集上的实验表明，模型能精确分割2D和3D管状结构，性能优于现有方法；私有数据集验证了良好的泛化能力。 Conclusion: HarmonySeg框架通过多尺度处理、辅助信息和拓扑保持损失，有效解决了管状结构分割的挑战，具有高精度和泛化能力。 Abstract: Accurate segmentation of tubular structures in medical images, such as vessels and airway trees, is crucial for computer-aided diagnosis, radiotherapy, and surgical planning. However, significant challenges exist in algorithm design when faced with diverse sizes, complex topologies, and (often) incomplete data annotation of these structures. We address these difficulties by proposing a new tubular structure segmentation framework named HarmonySeg. First, we design a deep-to-shallow decoder network featuring flexible convolution blocks with varying receptive fields, which enables the model to effectively adapt to tubular structures of different scales. Second, to highlight potential anatomical regions and improve the recall of small tubular structures, we incorporate vesselness maps as auxiliary information. These maps are aligned with image features through a shallow-and-deep fusion module, which simultaneously eliminates unreasonable candidates to maintain high precision. Finally, we introduce a topology-preserving loss function that leverages contextual and shape priors to balance the growth and suppression of tubular structures, which also allows the model to handle low-quality and incomplete annotations. Extensive quantitative experiments are conducted on four public datasets. The results show that our model can accurately segment 2D and 3D tubular structures and outperform existing state-of-the-art methods. External validation on a private dataset also demonstrates good generalizability.

The Efficacy of Semantics-Preserving Transformations in Self-Supervised Learning for Medical Ultrasound

Blake VanBerlo,Alexander Wong,Jesse Hoey,Robert Arntfield

Task: 系统研究数据增强和预处理策略在肺部超声自监督学习中的影响。

Motivation: 自然图像的数据增强方法在医学影像任务中可能不适用，需探索适合超声影像的策略。

Details

Method: 评估三种数据增强流程：基线流程、语义保留流程和蒸馏流程，并在多个分类任务中测试预训练模型。 Result: 语义保留数据增强在COVID-19分类任务中表现最佳，而裁剪方法在B线和胸腔积液分类任务中表现更好。 Conclusion: 为超声自监督学习提供了数据增强和预处理策略的实践指导。 Abstract: Data augmentation is a central component of joint embedding self-supervised learning (SSL). Approaches that work for natural images may not always be effective in medical imaging tasks. This study systematically investigated the impact of data augmentation and preprocessing strategies in SSL for lung ultrasound. Three data augmentation pipelines were assessed: (1) a baseline pipeline commonly used across imaging domains, (2) a novel semantic-preserving pipeline designed for ultrasound, and (3) a distilled set of the most effective transformations from both pipelines. Pretrained models were evaluated on multiple classification tasks: B-line detection, pleural effusion detection, and COVID-19 classification. Experiments revealed that semantics-preserving data augmentation resulted in the greatest performance for COVID-19 classification - a diagnostic task requiring global image context. Cropping-based methods yielded the greatest performance on the B-line and pleural effusion object classification tasks, which require strong local pattern recognition. Lastly, semantics-preserving ultrasound image preprocessing resulted in increased downstream performance for multiple tasks. Guidance regarding data augmentation and preprocessing strategies was synthesized for practitioners working with SSL in ultrasound.

Zero-Shot Low-dose CT Denoising via Sinogram Flicking

Yongyi Shi,Ge Wang

Task: 提出一种基于正弦图闪烁的零样本低剂量CT成像方法，解决现有方法因降采样操作导致图像分辨率下降的问题。

Motivation: 临床实践中难以获取大量成对的噪声和干净图像，现有零样本自监督方法（如ZS-N2N）依赖于单幅图像信息，但会因降采样操作降低分辨率。

Details

Method: 通过随机共轭射线匹配在正弦图域生成大量内容一致但噪声模式不同的正弦图，利用轻量级模型（基于ZS-NSN）训练网络。 Result: 仿真研究表明，该方法优于ZS-N2N等现有先进方法。 Conclusion: 提出的正弦图闪烁方法在保持图像分辨率的同时，有效解决了零样本低剂量CT成像问题。 Abstract: Many low-dose CT imaging methods rely on supervised learning, which requires a large number of paired noisy and clean images. However, obtaining paired images in clinical practice is challenging. To address this issue, zero-shot self-supervised methods train denoising networks using only the information within a single image, such as ZS-N2N. However, these methods often employ downsampling operations that degrade image resolution. Additionally, the training dataset is inherently constrained to the image itself. In this paper, we propose a zero-shot low-dose CT imaging method based on sinogram flicking, which operates within a single image but generates many copies via random conjugate ray matching. Specifically, two conjugate X-ray pencil beams measure the same path; their expected values should be identical, while their noise levels vary during measurements. By randomly swapping portions of the conjugate X-rays in the sinogram domain, we generate a large set of sinograms with consistent content but varying noise patterns. When displayed dynamically, these sinograms exhibit a flickering effect due to their identical structural content but differing noise patterns-hence the term sinogram flicking. We train the network on pairs of sinograms with the same content but different noise distributions using a lightweight model adapted from ZS-NSN. This process is repeated to obtain the final results. A simulation study demonstrates that our method outperforms state-of-the-art approaches such as ZS-N2N.