Coupled Rain Streak and Background Estimation via Separable Element-Wise Attention

Yinjie Tan, Qiang Wen, Jing Qin, Jianbo Jiao, Guoqiang Han, Shengfeng He*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Single image de-raining is challenging especially in the scenarios with dense rain streaks. Existing methods resolve this problem by predicting the rain streaks of the image, which constrains the network to focus on local rain streaks features. However, dense rain streaks are visually similar to mist or fog (with large intensities), in this case, the training objective should be shifted to image recovery instead of extracting rain streaks. In this paper, we propose a coupled rain streak and background estimation network that explores the intrinsic relations between two tasks. In particular, our network produces task-dependent feature maps, each part of the features correspond to the estimation of rain streak and background. Furthermore, to inject element-wise attention to all the convolutional blocks for better understanding the rain streaks distribution, we propose a Separable Element-wise Attention mechanism. In this way, dense element-wise attention can be obtained by a sequence of channel and spatial attention modules, with negligible computation. Extensive experiments demonstrate that the proposed method outperforms state-of-the-arts on 5 existing synthesized rain datasets and the real-world scenarios, without extra multi-scale or recurrent structure.

Original languageEnglish
Article number8963735
Pages (from-to)16627-16636
Number of pages10
JournalIEEE Access
Volume8
DOIs
Publication statusPublished - 2020

Bibliographical note

Funding Information:
This work was supported in part by the National Natural Science Foundation of China under Grant 61472145, Grant 61972162, and Grant 61702194, in part by the Hong Kong Research Grants Council under Project PolyU 152035/17E, in part by the Special Fund of Science and Technology Research and Development on Application from Guangdong Province (SF-STRDA-GD) under Grant 2016B010127003, in part by the Guangzhou Key Industrial Technology Research Fund under Grant 201802010036, in part by the Guangdong Natural Science Foundation under Grant 2017A030312008, and in part by the CCF-Tencent Open Research Fund (CCF-Tencent) under Grant RAGR20190112.

Publisher Copyright:
© 2020 IEEE.

Keywords

  • Background estimation
  • de-raining
  • element-wise attention

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint

Dive into the research topics of 'Coupled Rain Streak and Background Estimation via Separable Element-Wise Attention'. Together they form a unique fingerprint.

Cite this