当前位置:网站首页>TTVOS:基于自适应模板注意模块和时间一致性丢失的轻量级视频对象分割(CS CV)

TTVOS:基于自适应模板注意模块和时间一致性丢失的轻量级视频对象分割(CS CV)

2020-12-07 19:23:51 凌茜

半监督视频对象分割(semi-VOS)广泛应用于许多领域。这个任务是通过一个给定的分割掩码跟踪类无关的对象。为了做到这一点,基于光流、在线学习和记忆网络的各种方法被开发出来。这些方法显示出较高的精度,但由于推理时间较慢和复杂,很难在实际应用中使用。为了解决这一问题,设计了模板匹配方法以提高处理速度,牺牲了大量的性能。我们引入了一种新的基于模板匹配方法的半监督视频对象分割模型和一种新的时间一致性损失模型,以减少与重型模型的性能差距,同时大大加快推理时间。我们的模板匹配方法分为短期匹配和长期匹配。短期匹配增强了目标物体的定位能力,而长期匹配则通过新提出的自适应模板注意模块改善了目标物体的细节和形状变化。但是,长期匹配在更新模板时,由于过去估计结果的流入,会导致错误传播。为了解决这个问题,我们也提出了时间一致性损失的概念,以更好的时间一致性相邻帧之间采用过渡矩阵的概念。我们的模型在DAVIS16基准测试中以73.8 FPS的速度获得79.5%的J&F分数。

原文题目:TTVOS: Lightweight Video Object Segmentation with Adaptive Template Attention Module and Temporal Consistency Loss

原文:Semi-supervised video object segmentation (semi-VOS) is widely used in many applications. This task is tracking class-agnostic objects by a given segmentation mask. For doing this, various approaches have been developed based on optical flow, online-learning, and memory networks. These methods show high accuracy but are hard to be utilized in real-world applications due to slow inference time and tremendous complexity. To resolve this problem, template matching methods are devised for fast processing speed, sacrificing lots of performance. We introduce a novel semi-VOS model based on a temple matching method and a novel temporal consistency loss to reduce the performance gap from heavy models while expediting inference time a lot. Our temple matching method consists of short-term and long-term matching. The short-term matching enhances target object localization, while long-term matching improves fine details and handles object shape-changing through the newly proposed adaptive template attention module. However, the long-term matching causes error-propagation due to the inflow of the past estimated results when updating the template. To mitigate this problem, we also propose a temporal consistency loss for better temporal coherence between neighboring frames by adopting the concept of a transition matrix. Our model obtains 79.5% J&F score at the speed of 73.8 FPS on the DAVIS16 benchmark.

原文作者:Hyojin Park, Ganesh Venkatesh, Nojun Kwak

原文地址:https://arxiv.org/abs/2011.04445

原创声明,本文系作者授权云+社区发表,未经许可,不得转载。

如有侵权,请联系 yunjia_community@tencent.com 删除。

版权声明
本文为[凌茜]所创,转载请带上原文链接,感谢
https://cloud.tencent.com/developer/article/1748737