1. Introduction
No matter how projections of climate change might play out in terms of an increasing frequency, intensity, and duration of extreme heat episodes over the coming decades and whether human-related climate forcing has played a role in recent events (Dole et al. 2011; Coumou and Rahmstorf 2012), it is clear that extreme heat is a major public health problem (Gosling et al. 2009; Hess et al. 2012; Kovats and Hajat 2008). For example, in 2011–12 alone, extreme heat events exacted a heavy human toll in a number of regions including the United States, India, and southeastern and central Europe (Field et al. 2012), with major past events in Europe in 2003 (Robine et al. 2008), Russia in 2010 (Revich 2011), and Chicago in 1995 (Semenza et al. 1996). This has resulted in public health policy responses at a variety of levels (Koppe et al. 2004) among which have been the development of heat health warning systems (HHWS) (Hajat et al. 2010; Pascal et al. 2006) and comparison of exposure metrics (Zhang et al. 2012).
HHWS, which are often integral components of a wider heat health action plan (Matthies and Menne 2009; WHO Regional Office for Europe 2009, 2011) are designed to provide advance warning of impending periods of extreme heat that are likely to have adverse effects on health. Typically HHWS warnings are issued at the urban scale. As heat health researchers gain access to health data at the subcity scale it is becoming increasingly clear there are marked spatial variations in heat-related health outcomes across large urban areas. Those variations are most likely a result of the intraurban variability of vulnerability to heat. As heat vulnerability is largely socioeconomically determined (Kovats and Hajat 2008; Basu and Samet 2002; Basu 2009; Hajat et al. 2007), there have been attempts to develop heat vulnerability indices based on heat risk factor variables and to map these for large cities for the purpose of understanding within city variations in heat-related health outcomes. While studies outlining the development of heat vulnerability indices (HVI) are increasing (Vescovi et al. 2005; Johnson and Wilson 2009; Tomlinson et al. 2011; Gabriel and Endlicher 2011; Loughnan et al. 2012; Reid et al. 2009), there have been few attempts tat evaluating the performance of such indices.
The purpose of this paper is to test an HVI that has been developed for London, United Kingdom (Wolf and McGregor 2013). London has already experienced a number of significant heat-related health events especially in 1976 (MacFarlane 1977), 1995 (Rooney et al. 1998), 2003 (Johnson et al. 2005), 2006 (Health Protection Agency 2006), 2009 (Health Protection Agency 2010), and 2011 (Green et al. 2012). Future projections of climate, urban development, and population vulnerability indicate that heat stress will continue to be a relevant climate-related health issue in London. Therefore, the development and testing of a heat vulnerability index for London is highly pertinent in the context of climate risk management.
The approach adopted here in testing the HVI is based on assessing the hypothesis that areas in London with high heat vulnerability also have high levels of heat-related health outcomes, especially on days with anomalously high temperatures. Of interest was this question: Does the a priori estimation of heat vulnerability predict in broad terms the general level of heat-related mortality and ambulance callout (e.g. high or low) during heat wave events?
2. Heat vulnerability indices and mapping in context
While a number of heat vulnerability studies have been undertaken, with subsequent mapping of heat risk, mainly qualitative assessments have been conducted in order to evaluate the performance of heat vulnerability indices in terms of their ability to predict spatial patterns of heat-related outcomes. Exceptions are Reid et al. (2012), who validated the performance of a national heat vulnerability index using generalized estimating equation (GEE) Poisson regression, the application of the Hosmer–Lemeshow test to validate an extreme heat vulnerability index by Johnson et al. (2009), and most recently the use of goodness-of-fit statistics by Harlan et al. (2013) for validating a range of vulnerability indicators. Apart from these U.S.-based studies we are unaware of any attempts to quantitatively validate heat vulnerability indices elsewhere. Given the burgeoning field of heat risk mapping, a brief overview of its nature and development is provided below.
A heat vulnerability index provides an indication of the degree of vulnerability or potential to experience loss or suffer from a heat event. Like other vulnerability indices for floods or droughts, it is expressed on an ordinal or continuous scale of measurement. When specified for a number of spatial units making up a larger area, such as a major conurbation or region, the index can be mapped to form a heat vulnerability map. In this way heat vulnerability maps can be viewed as an input into heat risk mapping. The purpose of heat vulnerability maps is to highlight areas with elevated vulnerability so that attention and action can be focused on such areas in terms of implementation of mitigation and adaptation strategies to reduce the health effects (Wolf and McGregor 2013).
Viewed in this way and following the typology of approaches to risk mapping developed by Atkinson et al. (2012), heat vulnerability maps fall across the “strategic” and “management” functional types with their main purpose being to provide information for long-term action in a planning rather than a response context with their application in hazard management at the local to regional scale. Along these lines the HVI intends to identify particularly vulnerable people at local scales (Hinkel 2011). In comparison to hydrometeorological hazards, historically there have been few attempts to map heat risk based on an application of heat vulnerability indices. This situation is changing, however, with a number of heat vulnerability studies emerging in the literature for London (Abrahamson and Raine 2009; Mavrogianni et al. 2009; Oven et al. 2012) and elsewhere (Blättner et al. 2009; Harlan et al. 2006; Hondula et al. 2012; Harlan et al. 2013; Johnson and Wilson 2009; Reid et al. 2009; Rinner et al. 2010; Smoyer 1998; Tomlinson et al. 2011; Uejio et al. 2011; Vescovi et al. 2005; Wilhelmi 2004; Cutter et al. 2003; Chow et al. 2012; Loughnan et al. 2012). These studies add to a literature that focuses on place-based assessments of vulnerability, which have emerged since Smoyer’s (1998) analysis of spatial risk factors for mortality during heat waves in St. Louis, Missouri. Building on this, Wilhelmi et al. (2004) and Wilhelmi and Hayden (2010) emphasized the potential of geospatial technologies [including Geographic Information Systems (GIS) and remote sensing] for improving the understanding of vulnerability to urban heat with the aim to protect public health through better community-based outreach programs (Wilhelmi et al. 2004; Wilhelmi and Hayden 2010).
An increasingly common approach adopted in the development of vulnerability indices is one based on an inductive methodology (Tate 2012). Cutter et al. (2003) were perhaps the first to develop a quantitative vulnerability index using an inductive approach. This index was developed based on 42 U.S. census variables as indicators of social vulnerability. Principal component analysis (PCA) was applied to reduce these variables to 11 factors, which were added together to form the vulnerability index. Reid et al. (2009) applied this same method to assess U.S. national vulnerability to heat stress using census track data with the analysis revealing the elevated vulnerability of urban compared to nonurban areas. Uejio et al. (2011) chose an ecological study design to investigate the relative importance of heat exposure and the built environment, socioeconomic factors, and neighborhood stability for heat distress calls or heat mortality cases in Philadelphia and Phoenix (Uejio et al. 2011).
As heat has emerged as a public health problem in Canada, attention has turned to assessing heat risk from a vulnerability perspective. For example, Vescovi et al. (2005) combine social and physical factors in an attempt to assess vulnerability to heat stress in Quebec. By integrating climate variables and socioeconomic parameters in a GIS they produced maps of estimated present and future public health risk showing that the number of locations where populations will be at risk to high temperature events will increase in the future and that urban areas will be at special risk. For Toronto, Rinner et al. (2010) applied multicriteria analysis to assess spatial patterns of heat vulnerability across the 140 neighborhoods of the city, weighting and combining census data and data from a satellite thermal image to produce a composite measure of heat vulnerability and creating cluster maps (Rinner et al. 2010). For Montreal, Smargiassi et al. (2007) not only have modeled the relation between high outdoor and indoor temperature (Smargiassi et al. 2007) but also have considered air pollution (ozone), in addition to ambient temperature and socioeconomic status as factors that influence the health effects on hot summer days (Smargiassi et al. 2009). While their objective was not to build a heat vulnerability index, they validated the outcome of a model to estimate indoor temperatures (Smargiassi et al. 2007).
Although the majority of studies have assessed heat vulnerability based mainly on social factors, some emphasize the exposure element inherent in vulnerability. For example, Harlan et al. (2006) calculated an outdoor human thermal comfort index as a function of climate variables at the neighborhood level in Phoenix. The associations between outdoor human thermal comfort and other variables were tested using one-way analysis of variance (ANOVA) and Pearson correlation coefficients (Harlan et al. 2006). Similarly, Harlan et al. (2013) estimated neighborhood effects of population characteristics and the built and natural environments on deaths due to heat exposure in Maricopa County, Arizona (2000–08), using census data and remotely sensed vegetation and land surface temperature to construct and test indicators of neighborhood vulnerability, while Chow et al. (2012) combined spatially interpolated climate, normalized vegetation difference index, and U.S. Census data to construct maps of heat vulnerability (Chow et al. 2012).
Some studies approach the heat vulnerability problem in reverse. For example Hondula et al. (2012) examine linkages between spatial patterns of heat impacts on health and sociodemographic characteristics. Specifically, they examined if the mortality response for 48 ZIP code areas around Philadelphia were associated with higher risk of death during high heat stress conditions. A randomization test was used to identify mortality excess for different temperature thresholds with environmental, demographic, and social factors associated with high-risk areas subsequently identified via principal component regression (Hondula et al. 2012). Along the same lines, Johnson and Wilson (2009) examined the spatial relationships among vulnerable populations, the satellite-detected urban heat island (UHI), and heat-related mortality during an extreme heat event in Philadelphia in 1993 (Johnson and Wilson 2009).
Outside North America there have been few studies on heat vulnerability mapping. However, a number of major heat events have spurned assessments of the spatial characteristics of heat vulnerability in Australia, the United Kingdom, and Germany. For Melbourne, Australia, Loughnan et al. (2012) built a heat vulnerability index using demographic, environmental [including Moderate Resolution Imaging Spectroradiometer (MODIS) images of urban heat island], and health information at the post code level (Loughnan et al. 2012) and identified areas of high heat risk, which were subsequently qualitatively compared with health outcomes on hot and nonhot days. For the United Kingdom, Mavrogianni et al. (2009) linked urban domestic heat demand with a heat wave vulnerability index, based on the nature of London building stock, local environmental factors, and a satellite image of surface temperature, in order to examine the risk of heat death during a 2006 heat wave. This study placed emphasis on heat exposure and made a conscious effort to improve knowledge about the physical properties of individual dwellings (Mavrogianni et al. 2009). For Birmingham in central England, Tomlinson et al. (2011) used detailed household level social and economic data and one MODIS image of nocturnal surface temperature for 18 July 2006 as a heat wave example and conducted a Spearman’s rank order correlation to determine the statistical relationships between each “exposed and vulnerable” group and the urban heat island in 641 districts (Tomlinson et al. 2011). Blättner et al. (2009) mapped demographic and microclimate data and characteristics of residential building material in Kassel, Germany, to identify areas at high risk of heat-related mortality (Blättner et al. 2009).
Using an inductive approach (Cutter et al. 2003; Tate 2012), Wolf and McGregor (2013) developed and mapped a HVI for London’s 4765 census units. To achieve this they undertook a literature review and considered the nature of data available in the London Census to identify nine heat risk factors as input into the development of the London HVI. Principal component analysis was applied to reduce the dimensionality of the heat risk dataset and identify four principal components (PCs) that accounted for the majority of heat risk factor variance across the 4765 census districts. The four PC scores, for each of the 4765 census units, were weighted by the variance explained by the respective PC and aggregated to produce a combined PC score, which was treated as the heat vulnerability index value.
The 4765 HVI values were then grouped into deciles to produce 10 ordinal vulnerability classes. These were subsequently mapped, revealing quite a heterogeneous pattern of heat vulnerability across London (Fig. 1). Noteworthy features are the generally higher vulnerability in central London and in particular areas north of the Thames and the single pockets of high vulnerability throughout greater London. While the general trend of vulnerability partially reflects the spatial patterns of the input heat risk factors that make up the HVI, the finescale heterogeneity of heat vulnerability indicates rapid changes in heat risk over short distances. As testing of the London HVI is the focus of this paper, readers are referred to Wolf and McGregor (2013) for finer details on the development of the HVI, the emergent HVI patterns across greater London, and the degree to which clusters of high heat vulnerability and heat exposure (described by remotely sensed surface temperature during an extreme temperature event) co-occur [referred to as “hot hot spots” in Wolf and McGregor (2013)].
Spatial distribution of vulnerability as indicated by 10 vulnerability classes.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00014.1
3. Approach and methodology
This present paper focuses on ways to validate the performance of the London HVI and attempts to address this question: Does the a priori estimation of heat vulnerability predict in broad terms the general level of heat-related health outcome (e.g., high or low) as represented by mortality and ambulance callout during heat wave events? This section describes the data and approaches used to address this research question.
a. Data
Daily mortality and ambulance callout data were provided by the UK Office of National Statistics and the UK National Health Service for each of the 4765 census units for the periods 1990 to 2004 and 1998 to 2006, respectively. As for the HVI values, these data were grouped into deciles to form an ordinal classification of mortality and ambulance callout on non-heat-wave and heat wave days.
Quality assured daily mean, maximum, and minimum temperature data were obtained from the British Atmospheric Data Centre (BADC) for the London Weather Centre station situated in central London. These data were used to identify the subset of non-heat-wave and heat wave days for which mortality and ambulance callout were examined. Heat wave days were defined as when the daily maximum temperature exceeds the 1990–2006 90th percentile maximum temperature value for the month in which it occurs for at least two consecutive days. In total 133 heat wave days were identified for the period 1990–2006.
Because the vulnerability index is “complex” in that it is a multivariate index derived from the outcome of a data reduction technique such as PCA with a large number of input variables, it was decided that a comparison with a more parsimonious index would assist with addressing the issue of index intricacy versus simplicity. To this end a simple “elderly score” that measures the percentage of population above 65 years was introduced into the analysis. The same approach as that applied to the HVI score was used with the elderly score, such that percentage values for the 4765 census units were grouped into deciles and assigned ordinal scale values of 1 to 10, with 10 indicating those areas with the highest percentage of people 65 years and over. The determinant “elderly” is used because it was identified as one of the predominant heat risk factors in the literature (Wolf and McGregor 2013) and is one of the variables on which the HVI is based.
b. Approaches
Three approaches were adopted to address the research question.
First, skill scores were calculated for the dichotomous (yes/no) outcome that a census unit with an above-average vulnerability score would have an above average level of mortality and/or ambulance callout associated with it, either on a heat wave or non-heat-wave day. This approach was used to assess whether a priori the HVI was a good predictor of high levels of mortality and ambulance callout and is similar to that used in dichotomous weather forecast situations, for example rain/no-rain (Thornes and Stephenson 2001). A range of skill scores, namely the accuracy (range 0–1; perfect score 1), reliability of bias score (BIAS; range 0 to infinity; perfect score 1), probability of detection (POD; range 0–1; perfect score 1), false alarm ratio (FAR; range 0–1; perfect score 0), threat score (TS; range 0–1; 1 is no skill) and the Hanssen–Kuipers discriminant (HK; range −1 to 1; 0 is no skill and 1 is a perfect score) were calculated. These are based on the figures from joint frequency distribution as represented in a 2 × 2 contingency table with four possible outcomes in the form of hits, false alarms, misses, and correct negatives. The skill scores represent various ratio and difference combinations of the four outcomes. When both the vulnerability index score and observed health impact score are above average (decile category greater than 6), this is considered a “hit.” If vulnerability is above average but the impact is not, it is a “false alarm.” An estimated low vulnerability associated with a high impact indicates a “miss.” The last combination is “correct negative,” which means that an estimated below average vulnerability is confirmed by a below average health impact.
The second approach used quasi-Poisson regression, allowing for overdispersion (McCullagh and Nelder 1989) to assess whether there is a discernible change in risk of mortality and ambulance callout level associated with changing heat vulnerability level. Poisson regression is a form of regression analysis used to model count data (nonnegative integer values) and has been widely used in environmental epidemiology to assess the change in risk of a given health outcome such as mortality, with a change in an expected determinant of this (Armstrong 2006; O’Neill et al. 2005; Schwartz et al. 1996). In this study the total daily number of deaths and ambulance callouts were used as the outcome variables, assumed to have a Poisson distribution, while heat vulnerability categories were used as the explanatory variables. Dummy variables for each month of each year were included in the models in order to account for monthly variations during each summer and long-term trends in mortality and ambulance callouts. To adjust for day of the week patterns, dummy variables for the day of the week were also included. Finally, a heat wave indicator variable was entered in the models in order to assess the increase in risk of mortality and ambulance callout on the 133 heat wave days compared to 1472 non-heat-wave days in the overall period 1990 to 2006. Interaction between heat waves and the vulnerability index was also tested. Robustness of results was tested in sensitivity analyses using an alternative definition for heat waves (two or more consecutive days with mean daily temperature greater than the 95th percentile of the monthly distribution, over the study period) and also including a temperature term in the model (natural spline of the average of current and two previous days) (Gasparrini and Armstrong 2011). (Table 3 shows the results and Table 4 summarizes the comparison of performance of the different indices and health outcomes.)
Finally an independent samples test was applied to assess whether areas of high vulnerability located within the warmer sections of London’s urban heat island correspond with elevated health impacts or not. To test this, census units [referred to as “hot hot spots” in Wolf and McGregor (2013)] possessing a co-occurrence of high surface temperature, equal to or greater than 19°C, as indicated by a MODIS satellite for the early stages of the August 2003 heat wave event in London, and the highest heat vulnerability category 10 were compared with all other census units.
4. Results
The results of the contingency table analysis and associated skill scores are presented in Table 1. For each of the skill scores the best results are highlighted in bold. At first glance, accuracy looks encouraging for both mortality and ambulance callout on heat wave and all summer days. However, the accuracy figures, which tend to be around 60% or better, may be partially misleading given that this measure of skill tends to be influenced by the most common outcomes (Thornes and Stephenson 2001), which tend to be the number of hits and correct negatives. Using the Hanssen–Kuipers (HK) score to assess all aspects of the distribution of the matches, results give a value of 0.18 for mortality on heat wave days, 0.27 for mortality on all summer days, 0.40 for ambulance callouts on heat wave days, and 0.44 for ambulance callouts on all summer days. The HK score ranges from −1 to 1, with 1 being a perfect skill and 0 no skill. Accordingly, the vulnerability index has the best skill in a relative sense when tested with ambulance callouts for all summer days. Other skill scores confirm this as indicated when considering jointly the probability of detection (0.72) and the matching false alarm rate (0.28). Overall false alarm rates (probability of detection) are highest (lowest) for predictions of above average mortality. For all summer days there appears to be no tendency for the HVI to overpredict the occurrence of above average callout based on the condition of above average heat vulnerability. BIAS figures for ambulance callouts on heat wave days and mortality on all summer days also indicate good performance of the HVI for these situations. The HVI, however, has a tendency to overpredict the occurrence of above average mortality on heat wave days as indicated by a relative high bias value.
Joint distribution table and skill scores for the vulnerability index (best skill scores are in bold).
The skill scores for the elderly index as a determinant of above average level of mortality and ambulance callout are shown in Table 2. These demonstrate almost the reverse situation to that of the HVI. Overall, the best skill scores for the elderly index are attained for the prediction of above average mortality on heat wave days. This means that in contrast to the vulnerability index, the elderly index describes mortality on heat wave days better than any of the situations. The HK skill score value demonstrates this clearly and is further corroborated by the matched scores for probability of detection and false alarm rate. The threat score also indicates satisfactory performance with 56% of above average mortality occurrences correctly predicted by above average vulnerability.
Joint distribution table and skill scores for the age score index (best skill scores are in bold).
The results of the quasi-Poisson regression are presented in quantitative and qualitative forms in Tables 3 and 4, respectively. Risk is relative to vulnerability category score 1 (score 10 as a reference is not shown). There are significant differences in health outcome over all vulnerability classes when comparing heat wave and non-heat-wave days. The differences between the single vulnerability classes are also significant. No significant interaction between heat waves and vulnerability index was found, indicating that differences between classes are similar in both heat wave and non-heat-wave days. The use of an alternative heat wave definition, as well as inclusion of a temperature term in the model, did not change vulnerability index results. The Poisson regression beta coefficients, when converted into relative risks [relative risk = exp (beta)], demonstrate the percentage change of risk. Overall, the risk of death increases by 12% on heat wave days compared to non-heat-wave days adjusting for heat vulnerability. The comparison of groups of census units with the same level of vulnerability shows an increase in risk of death in summer: 9% for vulnerability class 2, 16% for class 3, 17% for class 4, 22% for class 5, 27% for class 6, 37% for class 7, 40% for class 8, 57% for class 9, and 84% for vulnerability class 10. The models estimate the increased risk in heat wave days (taking into account the population vulnerability) and also the increased risk between classes due to increased vulnerability.
Results of the quasi-Poisson regression (reference score 1).
Overview of the Poisson regression results; 0, +, and ++ indicate no, significant, and very significant increases in relative risk, respectively.
As for mortality, increases in relative risk are also found for ambulance callouts. While the overall increased risk of ambulance callouts is only 7% on heat wave days compared to non-heat-wave days (in comparison to 12% for mortality), the increase of risk when compared to vulnerability class 1 is 18% for class 2, 24% for class 3, 59% for class 4, and reaching 165% for class 10 (Table 3, lower part). This indicates that the relative risk associated with an incremental change in the vulnerability index is more sensitive for ambulance callouts compared to mortality.
For the purposes of comparison with the vulnerability index, Poisson regression was conducted for the elderly index as well (Tables 3 and 4). For mortality, the risk increases by 10% for heat wave compared to non-heat-wave days. This is slightly less than for the vulnerability index. However, as the elderly index increases, the differences become very strong. For example, from elderly class 1 to 2 the increase in risk of death is 25%; this trend continues to 40% for class 3, 50% in class 4, 63% in class 5, 73% in class 6, and 76% in class 7. The increase in risk of death for age score class 8 is even higher than for vulnerability class 10. It increases to 108% and further increases to 121% in class 9 and 178% in class 10. These results suggest that to predict the risk of mortality during summer, a simple elderly index appears to be a better indicator than the more complex vulnerability index. This, however, is not the case for ambulance callouts, as the elderly index indicates just a 3% overall risk of ambulance callout when comparing heat wave versus non-heat-wave days. Further, the risk of ambulance callout appears to be insensitive to a changing level of the elderly index as indicated by the lack of statistical significance of the relative risk when compared to the reference level for all elderly index categories. When presented graphically, the almost invariant change in risk with increasing elderly score as represented by the elderly index is clear as shown in Fig. 2, which combines all four combinations in one figure. The change in risk from class 1 to 10 for the HVI (elderly index) when tested with mortality data is indicated by thee solid black (gray) line while ambulance callouts are portrayed by the dotted black (gray) line. Figure 3 shows the change in risk compared to class 10.
Change in mortality and ambulance call-out risk (compared to vulnerability class 1) as indicated by the heat vulnerability index and the age index.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00014.1
Change in mortality and ambulance call-out risk (compared to vulnerability class 10) as indicated by the heat vulnerability index and the age index.
Citation: Weather, Climate, and Society 6, 1; 10.1175/WCAS-D-13-00014.1
According to the testing with ambulance callouts, the difference in risk increase between heat wave and non-heat-wave days is small for both indices. However, the vulnerability index is able to accurately predict where increases in ambulance callouts occur and the increase in risk is even stronger than the risk of death. The vulnerability index predicts a higher increase in risk of callouts than the elderly index and an even higher increase for risk of mortality. This suggests that the vulnerability index is adequate for predicting the increase of ambulance callouts.
Results of the comparison of mortality and ambulance callout levels, on both heat wave and all summer days, for census units with and without the co-occurrence of high surface temperatures and high heat vulnerability are presented in Table 5. Results of the independent samples’ t tests are provided in Table 6. Qualitative comparison of the statistics in Table 5 shows that for the 94 census units deemed to be hot hot spots, using the terminology of Wolf and McGregor (2013), mortality and ambulance callout is higher than all other census units regardless of the type of day, The results of the independent t test (Levene’s test for equality of variances) confirm this. In the case of equal variances not assumed, results point to significant differences in health outcomes at the 0.05 level or better for all situations. Further, the significance values for the two-tailed test (equal variances not assumed) are all <0.05 (0.000 to 0.001), signifying that the means of the two groups (i.e., hot hot spot versus all other census units) are indeed significantly different. This reveals that areas with high temperatures and high vulnerability appear to have distinct health responses.
Summary statistics for “hot hot spots” (as defined in Wolf and McGregor 2013) and other census units (all data for “hot hot spots” are in bold).
Independent samples test for hot hot spot testing. Italic font is used for equal variances assumed and bold font represents equal variances not assumed.
5. Discussion
This study has adopted three approaches to assessing the performance of a heat vulnerability index to predict the occurrence of areas of above average mortality and ambulance callout during heat wave days and all summer days in London. In doing so it adds to the somewhat meager literature on heat vulnerability index assessment and partly addresses the general call for models of socioecological systems to be challenged with observational data (Dearing et al. 2012). Implicitly, the study has tested the hypothesis that areas within London possessing above average vulnerability to heat will demonstrate above average levels of mortality and ambulance callout on heat wave days.
The first approach based on skills scores commonly applied in weather forecast verification, revealed particular aspects of HVI performance as a predictor of level of health outcome. Skills scores indicate that the performance of the HVI is largely credible from a number of verification perspectives. However, while the accuracy score indicates a generally good prediction performance, this verification statistic ignores the number of misses and false alarms, which from a heat risk management viewpoint can be costly either in terms of saving lives (misses) or investment in resources (false alarms). Of a range of skill scores, the Hanssen and Kuipers score is often considered the true skill score (Jolliffe and Stephenson 2012) as it uses all elements of the contingency table. It can also be interpreted as (accuracy for events) + (accuracy for nonevents) − 1 and is useful in assessing in the case of this study how well did the HVI separate the predicted “yes” events (hits and false alarms) from the “no” events (misses and correct negatives) (see Table 1). Although evaluation of the HK score indicates that the other skills scores perhaps flatter the performance of the HVI, its relatively low value when compared to that of others may be a result of the fact that the climatological response of health outcomes to heat wave days is considered in this study when all heat wave events are individuals. Furthermore, for rare events such as heat waves, the HK is unduly weighted toward the first term (see Table 1), making it more useful for more frequent events, unlike those considered in this study.
The second approach utilized Poisson regression to analyze the relative change in risk of mortality and ambulance callout with changing level of heat vulnerability. Although the Poisson regression corroborates the general outcome of the assessment based on skill scores, in that the HVI and simple elderly indices are able to predict health outcomes, the Poisson regression results add value to the analysis in that they demonstrate clear statistically significant differences in risk with changing level of heat vulnerability whether modeled by the HVI or the simple elderly index. Overall, the risk of deaths increases by 12% on heat wave compared to non-heat-wave summer days. The risk of calling an ambulance increases by 7% on heat wave days compared to non-heat-wave days. Assuming that ambulance callouts and hospital admissions correlate with each other, the observed increase in ambulance callouts by 7% is in line with results on hospital admission overall from the United States. Semenza et al. (1996) found an increase in hospital admission by 11% during the heat wave in Chicago in 1995. However, Kovats et al. (2004) found that during the 2003 heat wave in London, which caused an increase in mortality, there was no significant increase in hospital admission. This could be attributed to the fact that people may die before they perceive the risk they are facing (Abrahamson et al. 2009) or for some reason are not seeking help (Conti et al. 2007; Wolf et al. 2010). Of interest are the relative rates of change of risk for mortality and ambulance callout conditioned on the HVI and the elderly index. As for the skill score analysis, the Poisson regression results reveal that the vulnerability index is more sensitive to ambulance callouts than mortality; the opposite applies in the case of mortality. Possible explanations for this observation are provided below following some general comments about some of the issues related to vulnerability indices and how these may be viewed in a validation/assessment context.
The third approach simply confirms that mortality and ambulance callouts are significantly higher in highly vulnerable areas, which are in addition located within the warmer areas of the urban heat island. This is not surprising but lays the basis for exploring multiple exposures and matters in terms of social justice.
Notwithstanding the contested nature of vulnerability indices, which relate to their ability to capture with one value what is an extremely complex phenomenon (Benson 2004; Burton et al. 1993), an issue in the field of vulnerability index development is complexity versus simplicity. This is because many vulnerability indices, through overparameterization and the inclusion of unimportant factors, introduce excess complexity, not only making the index difficult to understand beyond the conceptual level but also impeding its application because of burdening data input demands and reluctant uptake by stakeholders (Saltelli and Funtowicz 2004). As a guiding principle, indices should aim to use a minimum number of dimensions of vulnerability, ensure even representation of these, and be based on methodological simplicity (Prescott-Allen 2001). While parsimony may be the preferred option for an index, such as that tested here, what is perhaps equally important is index performance. In this regard this study has shown some intriguing contrasts between the relatively complex HVI index, when compared to a single-variable elderly index, in terms of performance in predicting health outcomes.
The contingency table and the skill scores reveal that ambulance callout has greater predictability using the HVI as a predictor. In contrast, a simple elderly score describes mortality on heat wave days better than the “complex” HVI. In short, the age score is as good at predicting mortality on heat wave days as the vulnerability index is at predicting ambulance callouts on all summer days. One reason for this contrast could be the origin of the respective health outcome. The elderly index may well be a good predictor of mortality because the elderly are perhaps the most prone to the health effects of heat such that where there is a high proportion of aged people in an area the likelihood that mortality will be above the average level tends to increase. Further, because heat waves tend not to be prolonged (Rocklöv et al. 2011, 2012) and the health effects of heat are almost immediate, with most people dying during a heat wave event (Tong et al. 2012), elderly people may quickly succumb to the effects of heat and die in their place of residence without the chance of calling the emergency services. That this is plausible is supported by analyses of hospital admissions during heat wave events in London (Johnson et al. 2005; Kovats et al. 2004), with very few admissions as a result of the majority of deaths among the elderly occurring at home. This is supported further by the outcome of the quasi-Poisson regression analysis in that with increasing vulnerability class the risk of ambulance callout falls such that where vulnerability is high most of the health outcomes are in the form of deaths and not hospital admissions following ambulance callout. A further contributing reason to the superiority of the elderly index over the HVI for mortality prediction is possibly due to information on age structure for each of the census units being masked by other variables making up the HVI. This is because the principal component on which the proportion of elderly as a variable loads is the third most important of four components describing the majority of the variance of the HVI input variables. Accordingly, in calculating the final HVI score, the component on which the elderly variable loads receives less weight with this heat risk factor exerting less influence on the HVI score and thus the prediction.
The fact that age as a heat risk factor is somewhat overshadowed by other determinants of heat vulnerability in the HVI suggests that other risk factors that comprise the HVI help account for the relative superiority of the HVI compared to the simple elderly index in the case of ambulance callout on non-heat-wave days. As noted by Wolf and McGregor (2013) the variables that carry most weight in determining the HVI score, and thus a high vulnerability classification for a census unit, relate to crowded high density housing conditions and poor health and welfare dependency. These conditions are important heat risk factors and have been found on a qualitative level to be important determinants of the spatial pattern of health outcomes during heat events for a number of locations (Hondula et al. 2012; Mavrogianni et al. 2009; Loughnan et al. 2012; Smoyer 1998) and assist with explaining the utility of the HVI to predict above average ambulance callout on heat wave days. As housing conditions, poor health, and welfare dependency are also determinants of non-heat-related outcomes in the United Kingdom (Marmot 2007) the performance of the HVI for non-heat-wave days may well reflect emergency services responses to general more prevalent health issues other than heat.
6. Conclusions
Study results point to the potential of the heat vulnerability index (HVI) as an a priori indicator of where above average mortality and ambulance callout might occur, for heat wave and non-heat-wave days in London. Although the HVI is unable to provide perfect predictions, the level of skill as assessed by using categorical statistics, and the ability of the HVI to successfully describe the changing relative risk of mortality and callout with increasing levels of heat vulnerability provides some confidence for the application of the HVI in heat risk management at a number of time scales. At the time scale of heat wave events, warnings emerging from a heat wave warning system (Koppe et al. 2004; Ebi et al. 2004; Ebi and Schmier 2005) could be targeted at areas with high vulnerability. At medium to longer-term time scales local and national government agencies in possession of an understanding of the social drivers of heat vulnerability, as embodied in the variables making up the HVI, could use the mapping of high heat vulnerability areas as a focus for special heat education efforts, deciding where to prioritize adaptive and preventive actions, and application of urban climate design principles in rebuilding programs at the dwelling to neighborhood scale.
Through undertaking an evaluation of the performance of a heat vulnerability index, this study has also explored the way environmental and social processes interact, and in doing so has provided information to support decision makers in managing risk. The approach applied in predicting health outcomes at the census unit level in this study is a deterministic one. With predictions provided in this way, a decision maker concerned with heat risk management would be faced with a taking action based on whether above average mortality or callout is expected to occur or not. However, the societal response to any given extreme event such as a heat wave is uncertain because of the dynamics of climate–weather–society relationships. Accordingly, significant uncertainty would be associated with any deterministic prediction of health outcomes based on a predictor such as the HVI. Given this, the development of probabilistic predictions of health outcomes would be a sensible way forward for evaluating the relationship between the “heat vulnerability-scape,” as represented by a mapping of the HVI, and periods of extreme heat. This would facilitate an assessment of the uncertainty associated with the HVI as a predictor of health outcomes and assist with place-based decision making related to a range of preventative and adaptation actions.
An inherent danger in developing vulnerability indices is that they become little more than mathematical expressions of an eloquent conceptual model of vulnerability if not confronted with observational data and tested. This study has attempted to avoid this peril by presenting and applying three approaches to the assessment of a heat vulnerability index developed for London in the United Kingdom. Although there are a plethora of verification methods, the simple categorical statistics and associated skill scores used in this study offer effective insights into prediction skill and appear appropriate for assessing the predictability of dichotomous heat related health outcomes. That the performance of a relatively complex multivariate index and a single variable index of heat vulnerability appear to be health outcome dependent raises the question as to whether index parsimony is indeed more important than credibility in a verification and ultimately an application/decision making context.
REFERENCES
Abrahamson, V., and Raine R. , 2009: Health and social care responses to the Department of Health Heatwave Plan. J. Public Health, 31, 478–489, doi:10.1093/pubmed/fdp059.
Abrahamson, V., Wolf J. , Lorenzoni I. , Fenn B. , Kovats S. , Wilkinson P. , Adger W. N. , and Raine R. , 2009: Perceptions of heatwave risks to health: Interview-based study of older people in London and Norwich, UK. J. Public Health, 31, 119–126.
Armstrong, B., 2006: Models for the relationship between ambient temperature and daily mortality. Epidemiology, 17, 624–631.
Atkinson, P., Brubaker M. , Lewis H. G. , Bevington J. , Murdock A. , and Branson J. , 2012: State-of-the-art in risk mapping. Government Office for Science, 82 pp.
Basu, R., 2009: High ambient temperature and mortality: A review of epidemiologic studies from 2001 to 2008. Environ. Health, 8, 40, doi:10.1186/1476-069X-8-40.
Basu, R., and Samet J. M. , 2002: Relation between elevated ambient temperature and mortality: A review of the epidemiologic evidence. Epidemiol. Rev., 24, 190–202, doi:10.1093/epirev/mxf007.
Benson, C., 2004: Macro-economic concepts of vulnerability: Dynamics, complexity and public policy. Mapping Vulnerability: Disasters, Development, and People, G. Bankoff et al., Eds., Earthscan, 159–173.
Blättner, B., Heckenhahn M. , Georgy S. , Grewe H. A. , and Kupski S. , 2009: Wohngebiete mit hitzeabhängigen Gesundheitsrisiken ermitteln. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz, 53, 75–82, doi:10.1007/s00103-009-0995-1.
Burton, I., Kates R. W. , and White G. F. , 1993: The Environment as Hazard. 2nd ed. Guilford Press, 290 pp.
Chow, W. T. L., Chuang W.-C. , and Gober P. , 2012: Vulnerability to extreme heat in metropolitan Phoenix: Spatial, temporal, and demographic dimensions. Prof. Geogr., 64, 286–302, doi:10.1080/00330124.2011.600225.
Conti, S., Masocco M. , Meli P. , Minelli G. , Palummeri E. , Solimini R. , Toccaceli V. , and Vichi M. , 2007: General and specific mortality among the elderly during the 2003 heat wave in Genoa (Italy). Environ. Res., 103, 267–274, doi:10.1016/j.envres.2006.06.003.
Coumou, D., and Rahmstorf S. , 2012: A decade of weather extremes. Nat. Climate Change, 2, 491–496, doi:10.1038/nclimate1452.
Cutter, S. L., Boruff B. J. , and Shirley W. L. , 2003: Social vulnerability to environmental hazards. Soc. Sci. Quart., 84, 242–261.
Dearing, J. A., Bullock S. , Costanza R. , Dawson T. P. , Edwards M. E. , Poppy G. M. , and Smith G. M. , 2012: Navigating the perfect storm: Research strategies for social-ecological systems in a rapidly evolving world. Environ. Manage., 49, 767–775, doi:10.1007/s00267-012-9833-6.
Dole, R., and Coauthors, 2011: Was there a basis for anticipating the 2010 Russian heat wave? Geophys. Res. Lett.,38, L06702, doi:10.1029/2010GL046582.
Ebi, K. L., and Schmier J. K. , 2005: A stitch in time: Improving public health early warning systems for extreme weather events. Epidemiol. Rev., 27, 115–121, doi:10.1093/epirev/mxi006.
Ebi, K. L., Teisberg T. J. , Kalkstein L. S. , Robinson L. , and Weiher R. F. , 2004: Heat watch/warning systems save lives: Estimated costs and benefits for Philadelphia 1995–98. Bull. Amer. Meteor. Soc., 85, 1067–1073.
Field, C. B., and Coauthors, Eds., 2012: Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation. Cambridge University Press, 592 pp.
Gabriel, K. M. A., and Endlicher W. R. , 2011: Urban and rural mortality rates during heat waves in Berlin and Brandenburg, Germany. Environ. Pollut., 159, 2044–2050, doi:10.1016/j.envpol.2011.01.016.
Gasparrini, A., and Armstrong B. , 2011: The impact of heat waves on mortality. Epidemiology, 22, 68–73, doi:10.1097/EDE.0b013e3181fdcd99.
Gosling, S. N., Lowe J. A. , and McGregor G. R. , 2009: Projected impacts on heat-related mortality from changes in the mean and variability of temperature with climate change. IOP Conf. Ser.: Earth Environ. Sci.,6, 142010, doi:10.1088/1755-1307/6/14/142010.
Green, H. K., Andrews N. J. , Bickler G. , and Pebody R. G. , 2012: Rapid estimation of excess mortality: Nowcasting during the heatwave alert in England and Wales in June 2011. J. Epidemiol. Community Health, 66, 866–868, doi:10.1136/jech-2011-200962.
Hajat, S., Kovats R. S. , and Lachowycz K. , 2007: Heat-related and cold-related deaths in England and Wales: Who is at risk? Occup. Environ. Med., 64, 93–100.
Hajat, S., O’Connor M. , and Kosatsky T. , 2010: Health effects of hot weather: From awareness of risk factors to effective health protection. Lancet, 375, 856–863, doi:10.1016/S0140-6736(09)61711-6.
Harlan, S. L., Brazel A. J. , Prashad L. , Stefanov W. L. , and Larsen L. , 2006: Neighborhood microclimates and vulnerability to heat stress. Soc. Sci. Med., 63, 2847–2863.
Harlan, S. L., Declet-Barreto J. H. , Stefanov W. L. , and Petitti D. B. , 2013: Neighborhood effects on heat deaths: Social and environmental predictors of vulnerability in Maricopa County, Arizona. Environ. Health Perspect., 121, 197–204, doi:10.1289/ehp.1104625.
Health Protection Agency, 2006: Rapid evaluation of 2006 heat wave: Epidemiological aspects. 35 pp. [Available online at http://www.hpa.org.uk/web/HPAwebFile/HPAweb_C/1219130165676.]
Health Protection Agency, 2010: Chemical Hazards and Poisons Report. June 2010, issue 17. [Available online at https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/203572/HPA_Chemical_Hazards_17.pdf.]
Hess, J. J., McDowell J. Z. , and Luber G. , 2012: Integrating climate change adaptation into public health practice: Using adaptive management to increase adaptive capacity and build resilience. Environ. Health Perspect., 120, 171–179, doi:10.1289/ehp.1103515.
Hinkel, J., 2011: “Indicators of vulnerability and adaptive capacity”: Towards a clarification of the science–policy interface. Global Environ. Change, 21, 198–208, doi:10.1016/j.gloenvcha.2010.08.002.
Hondula, D. M., Davis R. E. , Leisten M. J. , Saha M. V. , Veazey L. M. , and Wegner C. R. , 2012: Fine-scale spatial variability of heat-related mortality in Philadelphia County, USA, from 1983-2008: A case-series analysis. Environ. Health, 11, 16, doi:10.1186/1476-069X-11-16.
Johnson, D. P., and Wilson J. S. , 2009: The socio-spatial dynamics of extreme urban heat events: The case of heat-related deaths in Philadelphia. Appl. Geogr., 29, 419–434.
Johnson, D. P., Wilson J. S. , and Luber G. C. , 2009: Socioeconomic indicators of heat-related health risk supplemented with remotely sensed data. Int. J. Health Geogr., 8 (57), doi:10.1186/1476-072X-8-57.
Johnson, H., Kovats S. , McGregor G. , Stedman J. , Gibbs M. , Walton H. , Cook L. , and Black E. , 2005: The impact of the 2003 heat wave on mortality and hospital admissions in England. Health Stat. Quart.,25, 6–11.
Jolliffe, I. T., and Stephenson D. B. , 2012: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. Wiley, 543 pp.
Koppe, C., Jendritzky G. , Kovats R. S. , and Menne B. , 2004: Heat-waves: Risks and Responses. World Health Organization, 125 pp. [Available online at www.euro.who.int/__data/assets/pdf_file/0008/96965/E82629.pdf.]
Kovats, R. S., and Hajat S. , 2008: Heat stress and public health: A critical review. Annu. Rev. Public Health, 29, 41–55.
Kovats, R. S., Hajat S. , and Wilkinson P. , 2004: Contrasting patterns of mortality and hospital admissions during hot weather and heat waves in Greater London, UK. Occup. Environ. Med., 61, 893–898.
Loughnan, M., Nicholls N. , and Tapper N. J. , 2012: Mapping heat health risks in urban areas. Int. J. Popul. Res., 2012, 518687, doi:10.1155/2012/518687.
MacFarlane, A., 1977: Daily deaths in Greater London. Popul. Trends, 5,20–25.
Marmot, M., 2007: Achieving health equity: From root causes to fair outcomes. Lancet, 370, 1153–1163, doi:10.1016/S0140-6736(07)61385-3.
Matthies, F., and Menne B. , 2009: Prevention and management of health hazards related to heatwaves. Int. J. Circumpolar Health, 68, 8–22.
Mavrogianni, A., Davies M. , Chalabi Z. , Wilkinson P. , Kolokotroni M. , and Milner J. , 2009: Space heating demand and heatwave vulnerability: London domestic stock. Build. Res. Inform., 37, 583–597, doi:10.1080/09613210903162597.
McCullagh, P., and Nelder J. A. , 1989: Generalized Linear Models. 2nd ed. Chapman and Hall/CRC, 532 pp.
O’Neill, M. S., Hajat S. , Zanobetti A. , Ramirez-Aguilar M. , and Schwartz J. , 2005: Impact of control for air pollution and respiratory epidemics on the estimated associations of temperature and daily mortality. Int. J. Biometeor., 50, 121–129, doi:10.1007/s00484-005-0269-z.
Oven, K. J., and Coauthors, 2012: Climate change and health and social care: Defining future hazard, vulnerability and risk for infrastructure systems supporting older people’s health care in England. Appl. Geogr., 33, 16–24, doi:10.1016/j.apgeog.2011.05.012.
Pascal, M., and Coauthors, 2006: France’s heat health watch warning system. Int. J. Biometeor., 50, 144–153.
Prescott-Allen, R., 2001: The Wellbeing of Nations: A Country-By-Country Index of Quality of Life and the Environment. Island Press, 372 pp.
Reid, C. E., O’Neill M. S. , Gronlund C. J. , Brines S. J. , Brown D. G. , Diez-Roux A. V. , and Schwartz J. , 2009: Mapping community determinants of heat vulnerability. Environ. Health Perspect., 117, 1730–1736.
Reid, C. E., and Coauthors, 2012: Evaluation of a heat vulnerability index on abnormally hot days: An environmental public health tracking study. Environ. Health Perspect., 120, 715–720, doi:10.1289/ehp.1103766.
Revich, B. A., 2011: Heat-wave, air quality and mortality in European Russia in summer 2010: Preliminary assessment. Human Ecol.,7,3–9.
Rinner, C., Patychuk D. , Bassil K. , Nasr S. , Gower S. , and Campbell M. , 2010: The role of maps in neighborhood-level heat vulnerability assessment for the city of Toronto. Cartogr. Geogr. Info. Sci., 37, 31–44, doi:10.1559/152304010790588089.
Robine, J.-M., Cheung S. L. , Le Roy S. , Van Oyen H. , Griffith C. , Michel J.-P. , and Herrmann F. R. , 2008: Death toll exceed 70,000 in Europe during the summer of 2003. C. R. Biol.,331, 171–178.
Rocklöv, J., Ebi K. , and Forsberg B. , 2011: Mortality related to temperature and persistent extreme temperatures: A study of cause-specific and age-stratified mortality. Occup. Environ. Med., 68, 531–536, doi:10.1136/oem.2010.058818.
Rocklöv, J., Barnett A. G. , and Woodward A. , 2012: On the estimation of heat-intensity and heat-duration effects in time series models of temperature-related mortality in Stockholm, Sweden. Environ. Health, 11, 23, doi:10.1186/1476-069X-11-23.
Rooney, C., McMichael A. , Kovats R. , and Coleman M. , 1998: Excess mortality in England and Wales, and in Greater London, during the 1995 heatwave. J. Epidemiol. Community Health, 52, 482–486.
Saltelli, A., and Funtowicz S. , 2004: The precautionary principle: Implications for risk management strategies. Int. J. Occup. Med. Environ. Health, 17, 47–57.
Schwartz, J., and Coauthors, 1996: Methodological issues in studies of air pollution and daily counts of deaths or hospital admissions. J. Epidemiol. Community Health, 50 (Suppl 1), S3–S11.
Semenza, J. C., Rubin C. H. , Falter K. H. , Selanikio J. D. , Flanders W. D. , Howe H. L. , and Wilhelm J. L. , 1996: Heat-related deaths during the July 1995 heat wave in Chicago. N. Engl. J. Med., 335, 84–90.
Smargiassi, A., Fournier M. , Griot C. , Baudouin Y. , and Kosatsky T. , 2007: Prediction of the indoor temperatures of an urban area with an in-time regression mapping approach. J Exposure Sci. Environ. Epidemiol.,18, 282–288, doi:10.1038/sj.jes.7500588.
Smargiassi, A., Goldberg M. S. , Plante C. , Fournier M. , Baudouin Y. , and Kosatsky T. , 2009: Variation of daily warm season mortality as a function of micro-urban heat islands. J. Epidemiol. Community Health, 63, 659–664, doi:10.1136/jech.2008.078147.
Smoyer, K., 1998: Putting risk in its place: Methodological considerations for investigating extreme event health risk. Soc. Sci. Med., 47, 1809–1824.
Tate, E., 2012: Social vulnerability indices: A comparative assessment using uncertainty and sensitivity analysis. Nat. Hazards, 63, 325–347, doi:10.1007/s11069-012-0152-2.
Thornes, J. E., and Stephenson D. B. , 2001: How to judge the quality and value of weather forecast products. Meteor. Appl., 8, 307–314.
Tomlinson, C., Chapman L. , Thornes J. , and Baker C. , 2011: Including the urban heat island in spatial heat health risk assessment strategies: A case study for Birmingham, UK. Int. J. Health Geogr., 10, 42.
Tong, S., Wang X. Y. , and Guo Y. , 2012: Assessing the short-term effects of heatwaves on mortality and morbidity in Brisbane, Australia: Comparison of case-crossover and time series analyses. PLoS ONE, 7, e37500, doi:10.1371/journal.pone.0037500.
Uejio, C. K., Wilhelmi O. V. , Golden J. S. , Mills D. M. , Gulino S. P. , and Samenow J. P. , 2011: Intra-urban societal vulnerability to extreme heat: The role of heat exposure and the built environment, socioeconomics, and neighborhood stability. Health Place, 17, 498–507, doi:10.1016/j.healthplace.2010.12.005.
Vescovi, L., Rebetez M. , and Rong F. , 2005: Assessing public health risk due to extremely high temperature events: Climate and social parameters. Climate Res., 30, 71–78.
WHO Regional Office for Europe, 2009: Improving public health responses to extreme weather/heat-waves—EuroHEAT. World Health Organization, 60 pp.
WHO Regional Office for Europe, cited2011: Public health response to heat-waves: Information package. [Available online at http://www.euro.who.int/en/what-we-do/health-topics/environment-and-health/Climate-change/publications/2011/public-health-response-to-heat-waves-information-package.]
Wilhelmi, O. V., and Hayden M. H. , 2010: Connecting people and place: A new framework for reducing urban vulnerability to extreme heat. Environ. Res. Lett., 5, 014021, doi:10.1088/1748-9326/5/1/014021.
Wilhelmi, O. V., Purvis K. , and Harriss R. , 2004: Designing a geospatial information infrastructure for mitigation of heat wave hazards in urban areas. Nat. Hazards Rev., 5, 147–158.
Wolf, J., Adger W. N. , Lorenzoni I. , Abrahamson V. , and Raine R. , 2010: Social capital, individual responses to heat waves and climate change adaptation: An empirical study of two UK cities. Global Environ. Change, 20, 44–52, doi:10.1016/j.gloenvcha.2009.09.004.
Wolf, T., and McGregor G. , 2013: The development of a heat wave vulnerability index for London, United Kingdom. Wea. Climate Extremes,1, 59–68.
Zhang, K., Rood R. B. , Michailidis G. , Oswald E. M. , Schwartz J. D. , Zanobetti A. , Ebi K. L. , and O’Neill M. S. , 2012: Comparing exposure metrics for classifying “dangerous heat” in heat wave and health warning systems. Environ. Int., 46, 23–29, doi:10.1016/j.envint.2012.05.001.
