Salient object detection (SOD) in remote sensing images (RSIs) is a highly practical task. However, scale variations of salient objects and the diversity of salient objects in RSIs pose challenges for detection. To address these issues, an attention-based pyramid decoder network (APDNet) is proposed for SOD in RSIs. The APDNet consists of three key components. First, a multiscale attention block is constructed to extract multiscale information and relations between salient objects, suppressing the distraction of variations of object types and scales. Second, a pyramid decoder structure is designed to take full advantage of multilevel features by gradually fusing features from two adjacent layers for result prediction. This feature fusion allows APDNet to efficiently exploit multilevel feature information, thus enabling a better feature representation. Finally, a bidirectional residual refinement module is proposed to enhance the structural integrity and boundary retention of initial saliency predictions. Extensive experiments on two public datasets demonstrate the superiority and effectiveness of the proposed APDNet against other compared state-of-the-art methods.