Record lows in Arctic sea ice extent have been making frequent headlines in recent years. The change in albedo when sea ice is replaced by open water introduces a nonlinearity that has sparked an ongoing debate about the stability of the Arctic sea ice cover and the possibility of Arctic “tipping points.” Previous studies identified instabilities for a shrinking ice cover in two types of idealized climate models: (i) annual-mean latitudinally varying diffusive energy balance models (EBMs) and (ii) seasonally varying single-column models (SCMs). The instabilities in these low-order models stand in contrast with results from comprehensive global climate models (GCMs), which typically do not simulate any such instability. To help bridge the gap between low-order models and GCMs, an idealized model is developed that includes both latitudinal and seasonal variations. The model reduces to a standard EBM or SCM as limiting cases in the parameter space, thus reconciling the two previous lines of research. It is found that the stability of the ice cover vastly increases with the inclusion of spatial communication via meridional heat transport or a seasonal cycle in solar forcing, being most stable when both are included. If the associated parameters are set to values that correspond to the current climate, the ice retreat is reversible and there is no instability when the climate is warmed. The two parameters have to be reduced by at least a factor of 3 for instability to occur. This implies that the sea ice cover may be substantially more stable than has been suggested in previous idealized modeling studies.