We investigate the role of a warm sea surface temperature (SST) anomaly (hot spot of typically 3 to 5 K) on the aggregation of convection using cloud-resolving simulations in a nonrotating framework. It is well known that SST gradients can spatially organize convection. Even with uniform SST, the spontaneous self-aggregation of convection is possible above a critical SST (here 295 K), arising mainly from radiative feedbacks. We investigate how a circular hot spot helps organize convection, and how self-aggregation feedbacks modulate this organization. The hot spot significantly accelerates aggregation, particularly for warmer/larger hot spots, and extends the range of SSTs for which aggregation occurs; however, at cold SST (290 K) the aggregated cluster disaggregates if we remove the hot spot. A large convective instability over the hot spot leads to stronger convection and generates a large-scale circulation which forces the subsidence drying outside the hot spot. Indeed, convection over the hot spot brings the atmosphere toward a warmer temperature. The warmer temperatures are imprinted over the whole domain by gravity waves and subsidence warming. The initial transient warming and concomitant subsidence drying suppress convection outside the hot spot, thus driving the aggregation. The hot-spot-induced large-scale circulation can enforce the aggregation even without radiative feedbacks for hot spots sufficiently large/warm. The strength of the large-scale circulation, which defines the speed of aggregation, is a function of the hot spot fractional area. At equilibrium, once the aggregation is well established, the moist convective region with upward midtropospheric motion, centered over the hot spot, has an area surprisingly independent of the hot spot size.