Detecting spreaders, or scan sources, helps intrusion detection systems (IDS) identify potential attackers. The existing work can only detect aggressive spreaders that scan a large number of distinct destinations in a short period of time. However, stealthy spreaders may perform scanning deliberately at a low rate. We observe that these spreaders can easily evade the detection because current IDS's have serious limitations. Being lightweight, the proposed scheme can detect scan sources in high speed networking while residing in SRAM. By theoretical analysis and experiments on real Internet traffic traces, we demonstrate that the proposed scheme detects stealthy spreaders successfully.