1. Introduction

The number of human observations of weather is decreasing worldwide (Fiebrich 2009; Eden 2009; Tuomenvirta 2004), as manned weather stations are replaced by automatic weather stations. Usually, the frequency and homogeneity of automated observations is better than that of the manual ones, but some parameters can still be better observed by human eyes, such as the precise type of hydrometeors. In this paper we argue that automatic observations could be augmented with data from users of Internet social network sites. Heymann et al. (2007) defined social network sites as those that capture and display content (i.e., ranging from photos to text) generated by untrusted users. Here we have used the term “social media” in a broader sense to describe services through which users can report details of their life and experiences and browse other users’ experiences, which are often sorted by tags and themes.

People of the generation born since 1982 have grown up using computer technology (Barnes 2009). Cell phones, text messaging, and the Internet are all part of their culture. Now they are acquiring more and more devices with good quality cameras and global positioning system (GPS) abilities. They share photos and reports with friends and strangers alike. Typical messages can be divided into two categories: “this is what I saw” and “this is what happened to me.” At first glance, these data are unreliable, unorganized, and uncontrolled. But the amount of data is huge and increasing and should not be ignored, and its reliability should be assessed. From these data we might be able to obtain first-hand, widespread, sometimes even near-real-time information of things that really matter to the people at their location. Results of the first application of social media to the sciences have been published. Ginsberg et al. (2009) showed that the influenza activity estimates made by Google Flu Trends based on the search queries of ordinary users correlate with the weekly statistics compiled by the U.S. Centers for Disease Control (CDC) from doctors’ reports, and that Google Flu Trends can estimate influenza activity with a shorter reporting lag than the weekly CDC reports.

Weather is related to many incidents described in social media. Some messages tell about weather indirectly: a person who tells how he fell and crashed his bike does not see it as a report of road conditions. People, who take photos of the hailstorm that interrupted their meal at an outdoor restaurant, do not think that they are making weather reports. People tend to take and share images, when something unusual happens. A September snowstorm in Paris probably gets more attention than a snowstorm in northern Scandinavia in February. Therefore, by observing the messages in social media, we may collect information of high-impact weather phenomena that would otherwise be imperfectly known, if at all, by the meteorological community.

Social media services have different characteristics, for example, timeliness varies between types of services. At the moment, text-oriented services, such as Twitter, are more up to date, while services for publishing photos and videos (e.g., Flickr and YouTube) react more slowly to events. The former might be used to real-time monitoring (Vieweg et al. 2010), while the latter are more suitable for posthoc research of phenomena and verification of forecasts.

This study concentrates on the use of free-form descriptive texts and metadata to extract meteorological information from photos on the Flickr service. Quantitative estimates of the reliability of the metadata are presented in section 3, and Flickr photos depicting hail on the ground are used in case studies for the validation of hail detection algorithms in section 4.

2. Data source: Tagged photos in the Flickr service

Flickr is an image-sharing service established in 2004, and is at present owned by the Internet services provider Yahoo!. At the time of writing, more than one billion photos were available for viewing, 6% of these having geographical information included. Even though most of the photos were taken in North America and western Europe, the distribution of the photos is truly global (Fig. 1). Of all the existing social media services, Flickr was chosen for this study because the considerable amount of photos meant the probability of finding useful data was high; additionally the application programmer’s interface (API) allowed the data to be processed automatically. Images can have extensive metadata attached, typically at least a short description of the photo, when it was uploaded (“posted”) and information about the camera settings (see the example in the appendix). Automatically saved technical details in the exchangeable image file format (EXIF) can include the date when the photo was taken and geographical information from the GPS. In addition to these parameters, the user can manually add freely chosen “tags” that describe the photo. On one hand, this makes it easier for users to add tags and encourages their use; on the other hand, homonyms and synonyms can make their utilization later harder. For our purposes, the date, the geographical information, and the tags were the most important. The tags are usually less ambiguous than the short description.

3. Quantifying the reliability of Flickr metadata

Before photos and tags can be used, the quality of the metadata must be assessed. For example, the layman who labeled the photos and the scientist who uses them might not always have the same conception of different weather phenomena. Another important question is the reliability of temporal and spatial information: are the photos taken when and where the metadata indicate?

4. Case studies with hail detection

Hailstorms are short-lived, local phenomena, and a lack of reference surface observations is a major problem for comparisons of different hail detection algorithms (Tuovinen et al. 2009). The Finnish Meteorological Institute (FMI) uses a hail detection algorithm that identifies convective cores in the upper parts of clouds with weather radars (Saltikoff et al. 2010). The method is based on work by Waldvogel et al. (1979) and compares the altitude of echoes stronger than 45 dBZ with the altitude of the freezing level. The higher the storm echoes rise, the larger is the probability of occurrence of hail. In addition, a new method of hydrometeor classification based on dual-polarization radars is being tested in a joint project of FMI and Helsinki University. The method uses the polarimetric moments along a radial for classifying hail, snow, and other hydrometeor types (Keränen et al. 2007). Both these methods suffer from a fundamental property of radar beams: because of the curvature of the earth, even the lowest possible measurement gets higher and higher above the ground with increasing distance from the radar. Hence, the further a location is from the radar, the harder it is to be sure that the hailstones detected at higher altitude will actually reach the ground and thus provide a positive verification case for the algorithm.

For hail algorithm testing, photos from Flickr gave much needed ground level evidence. For case studies, photos without precise GPS coordinates can sometimes be located by recognizable targets (e.g., a golf course, an open-air restaurant). But for the processing of larger amounts of images, coordinates are needed. To select the two example cases used in this study, a search was made with the relevant keywords in Finnish (e.g., “raekuuro” for “hailstorm”), and the resulting 16 cases were sorted by the availability of radar data, sufficient metadata in the photos, copyright license to allow further use of the images, and, finally, the aesthetic values of the images. The use of the Finnish language by and large restricted the search results to the area covered by the FMI radar network. Two cases having different characteristics were selected: one was within the coverage of the dual-polarization radar, while the other was far from the radar.

For the study of the first case, in southern Finland, two approaches were explored: the hydrometeor classification based on dual-polarization radar (Fig. 5a) and a cross section of radar reflectivity from the volume scan of a conventional radar (Fig. 5b). Both indicate the formation of hail aloft, and indeed, the only available surface observation, a photo posted in Flickr (Fig. 5c), showed hail on the ground. The photo was taken with a cell phone with a built-in GPS, so the location is very precise. According to the time stamp, it was taken 12 min after the radar images, which explains the slight mispositioning of the convective core, moving eastbound. In this case radar information near the surface was available. But often, because of the observation geometry of radar, this is not the case.

For the study of the second case, in western Finland, a hail photograph from Flickr was compared with a cross section westward through the measurement volume of radar at Vimpeli, western Finland (Fig. 6a). The radar echoes above the golf course at Vaasa (105 km from radar) indicated hail formation with probability of 30%. At this distance, the lowest radar measurement was made 1.4 km above the ground. It would be possible that, even if the hail were correctly diagnosed, it would have melted before reaching the ground. The photo from the golf course (Fig. 6b) confirmed that the hail did reach the ground.

In these examples, only one photo was used for verification of each case, as we were confident of their spatial and temporal accuracy. Used this way, photos gave us more evidence of the existence of hail, but the quality of the radar method cannot be determined from photos alone, as it is rarely possible to use the photos to depict nonevents (i.e., to show that there was no hail).

5. Discussion and conclusions

In this article we used data that were originally not created to be used as weather observations. Of course, such data can be wildly inaccurate or misleading. Date information especially can be unreliable, as users may not bother or know how to set the time of their devices. Even the geographical information can be misleading, as the number of photos wrongly located at 0° latitude, 0° longitude show (Fig. 1). The tags themselves depend on the integrity and competence of the users. In social media, there are even “spam tags” where users try to maximize their publicity by tagging their data with popular but misleading tags (Heymann et al. 2007), but in the course of this study, we found no traces of them in Flickr. The tags used in this study were not of such a kind that they could be used to lure users to inappropriate content. Additionally, as we have shown above, the metadata for Flickr photos are, for the most part, reasonably reliable, especially those that have been created with automatic GPS devices.

Especially in projects where remote sensing data are compared with surface observations, some tolerance must be accepted. Therefore, the temporal and spatial uncertainty of Flickr data (Figs. 3 and 4) should be compared with the resolution of other mesoscale observations. When testing the hail classification algorithm of a polarimetric Weather Surveillance Radar-1988 Doppler (WSR-88D), Schuur et al. (2003) used two hail-chasing cars. They accepted cases where data were recorded within the storm’s radius of influence, which varied from 3.2 to 5 km depending on the speed of storm movement. In time, they allowed a 12-min time window. Special arrangements, such as chase cars or very dense hail pad networks, can only be used for scientific experiments with a limited time and area. When creating a systematic hail pad calibration procedure for operational hail suppression in Greece, Dalezios et al. (1991) used a “normal grid” with 30 × 30 cm pads at intervals of 4.5 km. They noted that one of the main drawbacks of a hail pad system is the lack of time resolution—they changed the pads every 10 days, and after each day when hail was known to have occurred. Compared with these tolerances, the uncertainties of Flickr images are quite tolerable. In general, the needs of a particular research project drive the needs for the accuracy of the particular methods.

In some sense, we are at the mercy of service providers and their policies. It would have been very interesting to do this experiment with the free-form short messages or “micro blogs” of Twitter, but, at the time of writing, an archive of only one week of old messages was available for developers and researchers. Use of Twitter in research would need more of a proactive approach, where researchers themselves would have to collect all messages for a certain time period and from a certain geographical area. In general, data can also disappear without warning as users lose interest in sharing their data, fill up their quotas and delete older documents, or even when network sites discontinue their services.

This new type of data exposes the researcher to issues previously unfamiliar to many people in the field of meteorology (e.g., copyright and privacy issues have to be taken into account). By default the authors own the copyright to the shared documents, and researchers cannot, for example, publish individual images without permission. Some data are shared with a more permissive license (e.g., Creative Commons; Fitzgerald et al. 2007), which can ease these issues. The images used in this paper were already available under such a permissive license. Even so, we contacted the authors, who in fact had no objections to the use of their photos and released their real names for publication. Generally, privacy issues are taken into account by Flickr letting the author decide if the photo is viewable by all or only by selected users. The same decision can be made separately for the geolocation information of the photos. However, this does not solve the problem of the privacy of the people in the photos, as it is not known whether they have given permission for their images to appear in public. The researcher thus needs to exercise some discretion when using photos showing recognizable people. This kind of research might even require approval from Human Subjects Committees or other similar regulatory bodies. However, from our experience the copyright issues are the more pressing. Creative Commons offers a variety of nuanced licenses. The attribution rules determine how the author must be acknowledged, and the other options may limit the further processing of the image.

Here we concentrated on extracting information from the metadata of Flickr photos. Extensive research has been done in extracting information from images themselves or from free-flowing text in the spheres of image processing, pattern recognition, and information retrieval (e.g., Gonzalez and Woods 2008; Manning et al. 2008; Bishop 2006); utilizing such methods would be an interesting line to pursue in future research. Similar work has been carried out in content analysis where the reliability and validity of, for example, coding human communication into discrete categories has been discussed more thoroughly than in this study (e.g., Krippendorff 2004).

Flickr photos do not arrive from its users as a batch but in a steady stream. Of the photos used in this study, around 50% were available 1 day after the photo was taken and around 75% were available after 1 week. Other popular social media communities, such as Twitter and Facebook, may be more real-time oriented. But the use of Twitter in research is hampered by the short archive, while Facebook is by its nature very concerned with privacy, and thus much less data is available to researchers than in Flickr. Here our passive, unobtrusive approach used in this study could be complemented with a more active approach in which users are encouraged to submit relevant up-to-date information to researchers or even to forecasters (e.g., Ferree et al. 2009). For submitting information and discussing results, all social media services provide tools for like-minded users to gather together, usually called “groups” or “channels.” Technically this is not difficult and weather enthusiasts like storm spotters would be surely motivated to join. However, widening the participation to the general public may prove to be difficult if the service does not reward the users somehow. For storm spotters excitement of participating in research is a reward in itself, but if this is enough for the general public remains to be seen. Our unobtrusive approach has the advantage that we reach users not directly interested in weather observations. The third, even more direct, approach is the active collection of observations by contacting the businesses and residences in the area under study using telephone interviews as in Ortega et al. (2009). In the future, this might be performed utilizing social media and the geolocation of users, but the services must mature considerably before this is technically feasible and takes into account the privacy concerns. All these three approaches have their uses, and do not exclude each other. Again, which approach is the preferred one depends on the needs of the researchers.

In our experience, Flickr data are useful for case studies when photos are available but meteorological data are not. For more quantitative studies, we would be more cautious in recommending Flickr. Sometimes the data retrieval method proved to be unreliable, sometimes metadata were lacking. But we encourage others to investigate this data source. The importance of this data will grow in coming years as users, services, and devices grow more sophisticated. If a large enough sample size can be obtained with a small enough amount of erroneous data, then Flickr or any similar data source will prove to be very useful. Especially for the study of mesoscale phenomena, such as hail and tornadoes, any regular network of weather stations is too sparse. This new data could, in part, augment regular observations. But there are also parameters, which nobody has ever even tried to observe systematically, such as how slippery pavements are, and the real challenge for future scientists is to try to imagine truly novel uses for such data.