Correlation Matching Transformation Transformers for UHD Image Restoration
DOI:
https://doi.org/10.1609/aaai.v38i6.28341Keywords:
CV: Low Level & Physics-based Vision, CV: ApplicationsAbstract
This paper proposes UHDformer, a general Transformer for Ultra-High-Definition (UHD) image restoration. UHDformer contains two learning spaces: (a) learning in high-resolution space and (b) learning in low-resolution space. The former learns multi-level high-resolution features and fuses low-high features and reconstructs the residual images, while the latter explores more representative features learning from the high-resolution ones to facilitate better restoration. To better improve feature representation in low-resolution space, we propose to build feature transformation from the high-resolution space to the low-resolution one. To that end, we propose two new modules: Dual-path Correlation Matching Transformation module (DualCMT) and Adaptive Channel Modulator (ACM). The DualCMT selects top C/r (r is greater or equal to 1 which controls the squeezing level) correlation channels from the max-pooling/mean-pooling high-resolution features to replace low-resolution ones in Transformers, which can effectively squeeze useless content to improve the feature representation in low-resolution space to facilitate better recovery. The ACM is exploited to adaptively modulate multi-level high-resolution features, enabling to provide more useful features to low-resolution space for better learning. Experimental results show that our UHDformer reduces about ninety-seven percent model sizes compared with most state-of-the-art methods while significantly improving performance under different training sets on 3 UHD image restoration tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes will be made available at https://github.com/supersupercong/UHDformer.Downloads
Published
2024-03-24
How to Cite
Wang, C., Pan, J., Wang, W., Fu, G., Liang, S., Wang, M., Wu, X.-M., & Liu, J. (2024). Correlation Matching Transformation Transformers for UHD Image Restoration. Proceedings of the AAAI Conference on Artificial Intelligence, 38(6), 5336-5344. https://doi.org/10.1609/aaai.v38i6.28341
Issue
Section
AAAI Technical Track on Computer Vision V