Six small molecules with three different dye units, ethyl cyanoacetate (ECA), 1,3-indandione (IN), and 3-ethylrhodanine (RH), attached to both ends of thiophene-flanked carbazole (Cz) or fluorene (Flu) cores were synthesized and used as acceptors in organic photovoltaic cells (OPVs). Their optical and electrochemical, and consequent photovoltaic parameters varied mainly according to the dye units. The OPV cells were fabricated with the configuration ITO/PEDOT:PSS/poly(3-hexylthiophene):small molecule/LiF/Al and all devices except for Cz-IN showed photovoltaic performances with power conversion efficiencies (PCEs) in the range of 1.03–3.08% and open-circuit voltages as high as 1.03 V. Among them, the ECA and RH series had relatively high open-circuit voltages values because of their high-lying lowest unoccupied molecular orbital energy levels. Devices based on Flu-IN and the RH series showed high short-circuit currents (J_SC) because of the strong molecular aggregation, as evidenced by red-shifted UV absorption from solution to film. The adequate molecular aggregation in Flu-IN and RH films enhanced electron transport between the molecules, resulting in devices with high J_SC and PCE values. This work demonstrates that the introduction of dye end units onto carbazole- and fluorene-based small molecules provides good candidates for non-fullerene acceptors.