Regular surface damage detection of large concrete structures is one of the important measures to ensure their stableness and reliability. Recent advancements in computer vision and deep learning have been increasingly applied to the high-precision detection of concrete surface damage. However, most damage monitoring and localization methods are based on high-resolution images taken from close range, and the images usually contain tiny areas of actual structures. This paper proposes a contrastive embedding model to detect and localize damage on a wide range of concrete images. To achieve high-definition imaging of damage to the surface of large structures, a multifocus, and high-resolution imaging system is designed and employed. Furthermore, a concrete surface damage dataset containing 1006 detection areas and 45,375 images with seven types is constructed by manual annotation based on the obtained multiscale and multiresolution images. In addition, a contrastive embedding model based on a deep neural network is then modified, trained, and tested using the constructed dataset. To the best of our knowledge, this paper is the first to jointly use contrastive embedding to deal with damage monitoring and localization. Moreover, an evaluation framework based on sliding window iteration and nonmaximum suppression is proposed to verify the robustness and accuracy of the proposed contrastive embedding model. Experiments show that the best model achieves the classification accuracy of up to 94.84% and localization of 97.57%.