Glacial periods were punctuated by abrupt millennial scale climate changes, such as Dansgaard Oeschger events, Boeling-Allerod and Younger Dryas. Although glacial abrupt climate changes were shown to have a strong link to the Atlantic Meridional overturning circulation (AMOC) changes and the glacial background climate, simulating the stability and millennial change of AMOC and climate with fully coupled ocean-atmosphere GCM have been challenging. Here we present many cases of millennial scale climate variability with our Atmospheric Ocean coupled GCM, MIROC4m. A series of long transient experiments (> 10, 000 years) were performed systematically with different steady glacial conditions (CO2 level, obliquity, precession, meltwater, ice sheet size), to study the dependence of the sweet spot of millennial scale variability on the background climate and summarize the results as phase diagrams. We chose the model version which we simulate LGM AMOC weaker and shallower than the AMOC under Pre-Industrial condition. A reasonable sweet-spot of oscillation exists when the Northern Hemisphere ice sheets exist even without freshwater perturbation. In the sweet spot, self-sustained oscillation with bipolar seesaw pattern and shift between interstadial and stadial occur, with interval between abrupt events ranging from 1000 years to more than 5000 years depending on the background condition, while an abrupt shift from stadial to interstadial mode occurs in about 100 years. The sweet spot exists when the CO2 level is between 260ppm and 185ppm, depending largely on the obliquity but marginally on the precession and ice sheet size. When the obliquity or the CO2 amount is large (small), the AMOC is in a strong (weak) stable mode of about 18 (10) Sv (Sverdrup). Many aspects of the sweet spot, i.e., the duration of interstadial is longer systematically when the CO2 or obliquity is larger and the relation between the duration of interglacial and Antarctica air temperature, are very much in agreement with the ice core analysis and the deep-sea sediment.