The Agile Way of Working (AoW) was applied to three innovation projects at the Royal Netherlands Meteorological Institute.
Research and development (R&D) in the meteorological and climate sciences has until recently been organized along the preference and expertise of the individual practitioner because scientific problems could be broken down into relatively small and isolated pieces, requiring only limited manpower to work on. Nowadays, such simple work practices can no longer be sustained. For instance, output from climate scenarios or early warnings of weather- and climate-related calamities need to be communicated to large and diverse interest groups with oftentimes competing or even contradictory requirements. In brief, climate and weather R&D has evolved from a monodisciplinary exercise to a multidisciplinary enterprise. In such an enterprise, teams of scientists, software engineers, and science communicators work together with stakeholders so that products can be thoughtfully tailored for decision-makers and for the general public. This multidisciplinary work environment requires innovative ways to organize the work practices around specific topics of interest.
The requirement to match the needs of interest groups and stakeholders to the output from science poses challenges to all participants involved (Briley et al. 2015), among others the development of a common language and terminology. For example, the impact of climate change demands the development of climate services that in turn produce policy-relevant indicators (Goosen et al. 2014). Voinov and Bousquet (2010) give an overview of models that describe participation of stakeholders in collaborative decision-making. Traditional project management practices available to ensure project success are explained in that paper, although sometimes the differences between these management practices appear to be subtle. Their conclusions indicate that decisions are more effectively implemented if the stakeholder bearing the consequences is involved from the start of the project. Early stakeholder involvement is particularly valuable in climate science and adaptation problems emerging from the interpretation of climate scenarios (Berkhout et al. 2014; Hegger et al. 2012). For example, Hegger and Dieperink (2014) describe a project where scientists, policy makers, and other stakeholders jointly worked on a plan to “climate proof” a deep polder in the western part of the Netherlands. In their paper they coined the term joint knowledge production to describe the early involvement of relevant interest groups other than the scientists.
Voinov and Bousquet (2010) also provide a generic set of principles that ensure optimal project outcomes. They include a continuous awareness of and care for group dynamics, which is important in a setting involving people from diverse disciplines and backgrounds.
The purpose of this paper is to describe an innovative way of working, called the Agile Way of Working (AoW) in a Scrum setting. In the AoW/Scrum, much attention is given to the manner in which projects are completed in a joint setting. Group dynamics are considered an important factor in achieving successful project outcomes. There are important differences between Agile, or flexible project management (Wingate 2015), and traditional project management practices. In the AoW conventional project management tools are discarded and terminology and processes are different from traditional management tools. For example, there are no Gantt charts, no time sheets, and no assignment of tasks to individuals, and there is a different project management approach in scope, resources, schedule, and quality. In Agile/Scrum development resources, schedule and quality are fixed. In a close relation with the stakeholder, the scope is adjusted as the product is developed. In traditional project management the project manager can trade between the constraints. However, Scrum processes are tightly choreographed and involve careful planning.
We demonstrate the usefulness of Agile/Scrum at a meteorological R&D department of the Royal Netherlands Meteorological Institute (KNMI). KNMI is a typical example of a national meteorological institute. It houses in one location R&D in weather and climate, operations of a network of automatic weather stations, an extensive department of software engineers and highly skilled technical staff, a weather room, a satellite department, and a communications department. Intensive exchange and interaction among staff is essential for meaningful weather and climate services.
The next section gives a short introduction of AoW. The following section describes the case studies on which the AoW was practiced for a short period at KNMI. Contrary to most papers describing R&D in meteorology, the final scientific results are only briefly addressed, since the emphasis of this study is on the process of going through AoW and its evaluation and not on the final scientific result. The method and experiences are then discussed in following sections, respectively. Discussion and conclusions are dealt with in the final section.
THE AGILE WAY OF WORKING.
The AoW presented here is an adaptation of the Scrum framework. Before explaining the AoW adaptation, Agile and Scrum as an Agile method will be explained. Agile as a concept was first described in the Manifesto for Agile Software Development (Beck et al. 2001) and is based on 12 principles (Fowler and Highsmith 2001). The manifesto and principles originated from ideas and concepts arising in the 1990s as a reaction to more traditional approaches. These were regarded as bureaucratic, micromanaged, slow, and hindering creativity and effectiveness of software development. There are many Agile frameworks, but Scrum is the most commonly used. Scrum (Sutherland and Schwaber 1995) has a small set of clearly defined roles, processes, and artifacts as described in the Scrum guide (Schwaber 1997). Scrum is lightweight and simple to understand but difficult to master.
Scrum defines three roles (Schwaber 1997): 1) product owner (PO), responsible for the product vision and setting priorities; 2) Scrum master (SM), to ensure that the Scrum process is optimally executed by the team and the PO; and 3) the team (three to nine people), responsible for performing the work and ensuring its quality. The team is granted full authority to do whatever it decides is necessary to achieve the goal (Schwaber and Beedle 2002). The PO creates a prioritized list of ideas for the product, called the product backlog. Ideas from the product backlog are broken down into smaller, manageable items by the team and PO. Development of the product backlog is a continuous process. During the sprint planning, the team pulls items from the top of the product backlog and decides how and how many items can be implemented in the next sprint: this becomes the sprint backlog. Together with the PO a sprint goal is defined. During the sprint the team meets every day in the daily Scrum. Within these stand-up meetings of at most 15 min, the team focus is on assessing progress, asking three questions to each team member: What have you done, what will you do today, and what are impediments to the work to be done? The SM keeps the team focused on its goal and removes impediments preventing progress. Sprints usually take between one and four weeks. Each sprint delivers a deployable product, which is demonstrated to the stakeholders at the sprint review meeting. In addition, a retrospective on the team’s work process is held. During a retrospective, interactions and tools are evaluated and discussed per theme: How well did the Scrum preparations go (Preparations); did the team act according to the Scrum rules (Scrum Conformity); how well did the team perform (Team Performance); and how well did the Scrum work for the project at hand (Apply Agile/Scrum)? Figure 1 summarizes the Scrum process, and Table 1 summarizes the specific terminology used throughout the document.
Scrum process: roles, meetings, and artifacts.
Citation: Bulletin of the American Meteorological Society 99, 12; 10.1175/BAMS-D-17-0273.1
Terminology of AoW [abbreviated from the Scrum guide (Sutherland and Schwaber 2017) and Wikipedia]. The cartoons from Fig. 1 have been included when appropriate.
At KNMI Scrum is regularly and successfully applied in software development projects. Therefore, it was decided to use an adaptation of Scrum for the implementation of the case studies to introduce this methodology to scientists. Adapting Scrum to a research environment has been done before (Hicks and Forster 2010; Ribeiro Lima et al. 2012; Barroca et al. 2015; Marchesi et al. 2007). In our adaptation of Scrum, we apply Scrum in a “pressure cooker” time frame of just two one-week sprints using truly multidisciplinary teams of scientists and software engineers, ignoring the common experience that teams usually take more than two sprint weeks to become effective. Within the two sprints the aim is to develop a minimal viable product (MVP), ready for use by the stakeholders and suitable for further development using a product backlog.
DESCRIPTION OF CASE STUDIES.
Preliminaries.
At KNMI a small portion of time (two weeks per year per person) can be reserved for innovative projects or research ideas. Following the observations strategy for KNMI (Boers et al. 2015), we decided to translate a number of themes from the strategy into exploratory case studies serving as road maps for future investigations. A list of case studies was composed in the year prior to the sprint weeks from which three were selected. The case studies required integration of information/data from different parts of the organization.
To form the teams for the three case studies, three POs were appointed and asked to pitch the cases during a plenary session that was attended by the R&D department. After the three pitches, each department member could select the case of interest by adding his/her name to the case team. Sometimes a prospective team member was asked to switch teams because a team was lacking crucial skills. Below are short descriptions of each of the three case studies.
Drones for boundary layer applications (the drone team).
KNMI’s observation strategy calls for exploring the utility of drones in institutional R&D applications (Boers et al. 2015, their action 5). Deployment of drones could be useful for a) local thermodynamic profiling for nowcasting of extreme weather, b) siting classification of KNMI’s automatic weather stations, and c) probing the early morning boundary layer transition phase. The central aim of the sprint weeks was chosen as “flying a drone to capture the boundary layer morning transition.” The two sprint weeks were broken down into a preparation week (week 1), whereby a radiosonde was roped to a tethered balloon and its output directly relayed to the ground crew, and a final experiment week (week 2), in which a drone was flown by a commercial operator at KNMI’s research station Cabauw (Monna and Bosveld 2013). In week 1, software was constructed to provide real-time transmission of output from the research tower and the radiosonde to ground crew and weather forecasters. This simulates a nowcasting situation in which forecasters would be able to make decisions based on incoming thermodynamic profile data. During week 2, the drone was flown at Cabauw during a clear morning, capturing the boundary layer morning transition. From a technical standpoint, the Scrum was a success, as all objectives set out in the preparation weeks were accomplished. Figure 2 shows the drone flight at Cabauw.
Drone flight at the Cabauw Experimental Site of Atmospheric Research. Photo is taken from the top of the tower (210 m). A radiosonde is tethered at the end of a nylon cord. Note the curvature of the nylon cord due to the strong low-level jet in the early morning.
Citation: Bulletin of the American Meteorological Society 99, 12; 10.1175/BAMS-D-17-0273.1
Quality of manual rain gauge network (the quality team).
KNMI operates a dense network of 325 manual rain gauges. The rain gauges provide daily sums of precipitation made by volunteers. The network acts as a reference in the Netherlands, so it is necessary—in addition to the regular quality control—to monitor its long-term homogeneity with respect to other sources. In the Netherlands two other sources are 1) precipitation sums from the network of automatic weather stations (AWSs) and 2) sums derived from KNMI’s rainfall radars. For the sprint weeks the aim was to develop a tool to compare long-term monthly and annual precipitation sums from the three rainfall sources using their mutual difference series. During the sprints, several kinds of functionalities were implemented. First, the precipitation sums from the AWS network and the radar data had to be aggregated to the same accumulation interval in use by the manual gauges. Thereafter, methods to apply statistical tests to detect divergences were added.
The sprints resulted in a system with working functionalities. However, owing to a lack of time, a fully automated chain of data aggregation and inhomogeneity detection was not achieved.
A merging platform for divergent data streams (the data-wrangling team).
Thanks to cheap sensing capabilities, companies and institutions are now able to quantitatively monitor (i.e., by sensing) the performance of their business processes. In particular, they are increasingly interested in combining their sensed data with high-resolution meteorological data. Data wrangling is defined as “a process of iterative data exploration and transformation that enables analysis” (Kandel et al. 2011). To facilitate data wrangling of meteorological and nonmeteorological data, KNMI decided to develop a prototype platform that 1) enables nonmeteorological parties to make better use of the KNMI meteorological observations, 2) eases the handling of complex meteorological data formats (for users outside the meteorological domain), and 3) speeds up the data preparation process.
A two-week work breakdown was organized with the goal of wrangling a dataset in plain text [comma-separated values (CSV) formatted] containing car and bike accident data with information about location and time together, with precipitation from radar observation and temperature registered by the KNMI AWS network.
The two-week effort resulted in a web-based application able to successfully wrangle precipitation from radar in gridded format [Network Common Data Form (netCDF)], traffic accident data (CSV), and temperature data from the KNMI observation network (CSV).
APPLICATION OF THE AGILE WAY OF WORKING TO THE METEOROLOGICAL CASE STUDIES.
Preparation weeks.
Six to eight weeks prior to the sprint weeks, POs, none of whom had any experience with AoW, received a basic and general training from the experienced SMs on the principles and methodologies of AoW. The training focused on a) how to write project activities and goals in terms of story maps and user stories (see Table 1 for descriptive information of these terms), b) the Agile mentality, c) the role and tasks of the SM, and d) the role and tasks of the PO. A user story for a project is a statement of intent. It identifies a) in what capacity the user makes the statement of intent, b) what the purpose of the case study is, and c) what the activity is going to be. Table 2 shows the user story for each of the three teams. Next, for each of the three case studies, the PO, SM, and team worked together in several sessions to plan their programs of work by breaking them down into small tasks that could be executed by individual sprint team members. These sessions were essential for the PO to keep sharpening and refining the project ideas and prepare the user stories. In addition, they assisted the teams to get a clear picture of the work involved in the case studies. Thus, resources could be planned and the feasibility of the wishes of the PO could be assessed well ahead of time.
Examples of user stories used in the sprint weeks.
Each team planned its work product backlogs, expected to be feasible in two one-week sprints, according to its particular case study.
The drone team faced limitations placed on flying drones and balloons over the Netherlands in civil and military air space. A full week was deemed necessary for a grand rehearsal of the drones experiment by using a tethered balloon flown on-site at KNMI. Other tasks involved planning of software construction necessary to access the radiosonde profiles in real time, breaking down the protocol for each experiment (balloon and drone), securing manpower support for balloon and radiosonde launching, securing of an on-site radiosonde ground station, surveying the field of commercial drones operators, securing financial support for procuring their services and submitting flight plans to the selected drones operator, and air traffic control.
The quality team had identified the precipitation validation team of KNMI as the end user of their product (i.e., the stakeholder). They joined the quality team in several sessions to communicate their wishes relating to the graphical user interface (GUI) of the software and provide feedback on the proposed working prototype. At these consultations it became clear that the original ideas of the POs were too ambitious. In breaking down the project into smaller tasks, the POs tried to align the activities as much as possible with existing skills of the team members, which included reusing earlier work. Attention was also given to a common framework, such as a common scripting language.
The central task for the data-wrangling team was to construct architecture to be used as a skeleton for the wrangling platform; this included the diverse datasets to ingest, establishing approved functionalities (e.g., wrangle precipitation and temperature meteorological data only; support of just CSV format), securing user work space, and developing user authentication protocols. Story mapping [Patton and Economy (2014), see their Table 1 for descriptive information] and a synthetic biography of a factitious user of the product (Haikara 2007) were used to design the platform from the user perspective (e.g., upload data, selection of wrangling variables). User stories were refined and put on the story map to show priority of implementation for each phase.
The end result of the preparation weeks for each team was a product/sprint backlog consisting of a whiteboard with Post-its that described in detail the tasks to be executed during the first week of the sprint sessions (see Fig. 3 for one of these backlogs).
Example of a backlog in use, as developed during the preparation weeks.
Citation: Bulletin of the American Meteorological Society 99, 12; 10.1175/BAMS-D-17-0273.1
Sprint weeks.
During the sprint weeks, work centered around the Scrum board in the project room (see Fig. 4 for an example). As Agile/Scrum does not define how a Scrum board is organized, it is up to the team to set it up. The Scrum board shown in Fig. 4 was set up by one of the teams as a whiteboard with five columns: To Do, In Progress, Review, Test/Verification, and Done. The To Do column contained as Post-its the list of tasks to be performed by individual team members as derived from the sprint backlog. The In Progress column was filled by Post-its shifted from the To Do column by individual team members when they picked up their task. Once a team member deemed the task completed, the Post-its were placed onto the Review column, where they remained until the results were checked by a team member on quality of the solution. Once reviewed, the Post-it shifted over to the Test/Verification column, where it remained until the PO verified whether the completed task conformed to the required functionality in the project. After verification by the PO, the Post-it was shifted to the final column, Done.
Example of a Scrum board, as in use during a sprint week.
Citation: Bulletin of the American Meteorological Society 99, 12; 10.1175/BAMS-D-17-0273.1
Each morning (typically 0900 local time) the team would gather around the Scrum board for a daily stand-up. Thereafter (most) team members reverted to work on their own tasks. In the event of blockages, part of the team continued discussing until problems were resolved or a clear avenue toward a solution could be mapped out.
Even though each team had been allotted a work space that could seat all team members, there were some team members who would rather work from behind their own desks, arguing that they could better focus (owing to the nature of the tasks: writing, thinking) and combine the project better with other tasks in a more isolated setting.
At the end of the week, a sprint review meeting was organized during which a project demonstration was given to the management and to the other teams. The project demonstration reviewed either the construction and workings of the deployable product (at the end of the first week) or the minimum viable product (at the end of the second week). Although the format of the sprint review varied from team to team, it consisted mostly of a presentation containing a condensed flow of the work, preparation, measurements, visualizations, and conclusions.
Two of the three teams conducted the sprint review through a “Keep, Improve, Discuss” retrospective board allowing everybody to reflect on the sprint using Post-it notes to be put in these three categories. Positive remarks were designated as Keep, such as knowledge sharing and new results. Impediments caused by routine work from other projects or lack of agreement whether tasks were actually completed to the satisfaction of the PO were noted as Improve. Designated as Discuss were 1) the main complaint that a full day was “lost” to demonstration, retrospective, and planning for the next sprint and 2) the optional use for pair programming. After the sprint review meeting, at the end of week 1 the product backlog for the next week was prepared (sprint planning), and at the end of week 2 a joint retrospective was organized in which all three teams participated. The joint retrospective contained a questionnaire inquiring team members on their experience and opinions about AoW.
The team room of the drone team was often manned by just one or two team members because many tasks required team members to be elsewhere on-site, or even off-site, preparing the various stages of the balloon and drone experiments. The absence of team members was on occasion an impediment to the work of the team members left in the team room.
Work in both the quality team and the data-wrangling team proceeded more in a way resembling a traditional Agile/Scrum work flow. Team members were mostly seated in the team room. The quality team on occasion required pair programming (see Table 1), where two team members were seated together at a single workstation exchanging programming details, working toward a solution. Also, work procedures common to the experienced software engineers, such as using source code repositories, database protocols, and development tools for collaborating on software code and making data available for multiple usages and defining transferable data formats, were shared among the inexperienced data scientists.
Both teams spent a significant amount of time during these two weeks on the challenge to find common ground on the usage of tools and technical solutions and learning to make the ad hoc team work within the scope and in the interest of the project. The data-wrangling team had set up a chat channel using Slack (which is a cloud-based set of proprietary team collaboration tools and services), where links, ideas, or questions could be posted when team members were working remotely.
EVALUATION OF THE SCRUM WORK PROCESS.
Level of commitment of Scrum contributors.
The level of commitment of contributors in a Scrum project is one of the essential success factors. Even though team members committed themselves to one of the three case studies, it was difficult to secure their full commitment for the entire duration of the sprints. Many had previous engagements that could not be postponed, so some tasks needed to be transferred to others in the middle of a sprint. Also, a few persons did not participate at all during the Scrums even though they had subscribed in the preliminary phase.
For the drone team it was often unclear to team members when and where their colleagues were present. Because of the many logistical arrangements and technical preparations in the laboratory/field, large parts of the work had to be performed outside the team room. Furthermore, team members apparently felt the need to get back to their routine activities whenever tasks were completed.
A limited level of commitment sometimes posed a problem in assuring the completion of tasks on time and within required specifications. Furthermore, it was hindering team building. During retrospectives, the flawed commitment was often heard as a high-priority point of improvement.
Multidisciplinary work environment and AoW.
Forced multidisciplinary teamwork in a department more oriented to research poses a challenge. Scientists are by nature and training used to working independently. Therefore, a multidisciplinary AoW requires considerable adaptation for scientists. For some of them it may perhaps be suitable only after a transition period or after adaptation to the Scrum rules. This is an important point to reflect upon for broad (i.e., company- or institution-wide) applicability. Nevertheless, close lines of communication facilitated by a joint presence in the team room were important to exchanging ideas and taking advantage of the skills of other team members.
Each team had its own multidisciplinary dynamics, summarized as follows:
In the drone team the work and tasks were very precisely divided and could be executed independently; therefore, the interaction and the team dynamics were limited.
In the quality team, the interdisciplinary aspect of the case study, curiosity, new ideas, and discussions of the many scientists in the project were obstacles in the execution of the project. Often, discussions and points of view emerged on different approaches (e.g., database structure design) or on perfecting the software application (e.g., continuous integration tooling). The discussions helped the spread of new ideas between the scientists but posed a threat to the tight schedule of completion of the project.
In the data-wrangling team the multidisciplinary aspect of the tasks was limited, since almost all the members were software developers. However, since they came from different work groups, the exchange between team members focused on the tools both to use and reuse previous knowledge acquired in projects that the members had participated in before.
Management and communication.
The idea behind a Scrum and sprints is that a project team becomes self-directing. In other words, the PO is reduced to a one-man oversight committee, ensuring that the big-picture aims are being met, while the team members decide on the work partitioning and carry out the bulk of the work. The role of the SM is thus important to guarantee that the efforts of the team match the intentions of the PO and are channeled toward achieving an MVP. However, individual teams sometimes struggled with implementing the framework of AoW because of the inexperience of the POs with AoW and their interaction with the SMs.
For the drone team the PO took a directive stance, since the success of the experiment was dependent on the timely assistance of collaborators other than the sprint team members (i.e., commercial drone company, military air traffic control, regulatory authorities, in-house radiosonde expert, balloon launchers, and Cabauw site manager and technicians).
The quality team determined that a more directive stance of the PO would have been more useful to the success of the sprints, including handing down tips and tricks to junior researchers and guarding the scope of some of the software elements. Also, it was felt that too much time had been spent on fruitless discussions between team members.
The data-wrangling team was the most successful in following the self-organizing principle of the Scrum method. This is perhaps not surprising, as most members of this team were experienced in AoW.
Ambitions vs realism.
The extent to which project aims are being met depends to a large extent on whether the required tasks are sufficiently well thought out and timed to match the specified project ambition. The articulation of realistic project ambitions by a PO experienced in an individualistic research environment where success is measured by papers written and citations gained is difficult.
For the drone team, the aim was clear: “Fly a drone with real-time output available to ground personnel and the weather room.” Provided that adverse weather conditions would not prevent the experiment, this aim is focused enough to assign clear tasks and anticipate a satisfactory outcome. However, for the two other case studies, formulating suitable tasks that could match the articulated ambitions was more difficult.
Achieving all the goals of the quality team project in two weeks’ time was not a realistic ambition. The scope of work was too big and lacked a clear breakdown of activities and products.
For the data-wrangling team, wrangling data from different domains in different formats, resolutions, and criteria (quality) was simply not feasible in a two-week period. However, in this team the ambition was adjusted in the first week to provide a proof of concept and MVP that supported a limited set of data and formats to demonstrate the potential of a wrangling tool to speed up data preparation for analysis. The experience of team members working in a Scrum setting was instrumental in achieving the change of plans and ambitions.
Team retrospective results.
After showing the team results at the final sprint review, an all-team retrospective was conducted to capture and share important lessons learned. Results were obtained using an online voting system accessible by mobile phone. After each question, the results were shown to the participants and discussed. The results are shown in Table 3.
All-team retrospective results: general experience. Note that the percentages do not necessarily add up to 100, as team members did not answer all questions.
Questions dealt with their opinions about the achievements of the Scrums, the preparation, whether the Scrum process was used, the value of the PO and SM, whether they liked working in a multidisciplinary environment, and whether they would like to do it again.
Preparations could be improved. The preparation of the user stories and Agile training was mostly done by the POs and SMs. The team members were involved at a later stage, 50% of whom had no Agile/Scrum experience. So it was felt by the teams that improved preparation (Agile training, team story refinements) should have resulted in improved estimation of the work to be done, leading to improved priorities and an improved MVP.
Regarding Scrum Conformity, and despite the unfamiliarity of team members with Agile/Scrum, the main Scrum roles, rituals, and artifacts were mostly used. Agile principles (Fowler and Highsmith 2001) of “Focus” and “Stable teams” were experienced as important, as violations of these values were mentioned as significant impediments to success. The role of the PO was valued highly by more than 90%. Interestingly, while the role of the SM was also valued highly (>70%), there was a sizable proportion of participants (∼20%) that was not sure about the value of the SM. This could not be further investigated, as no differentiation of answers according to the teams was done.
Team Performance was regarded as high by the teams. They liked working in teams, and results obtained by the teams exceeded expectations. Note that the team consisted of volunteers, so this result was expected.
The gained experience in applying Agile/Scrum was regarded as very positive by the teams. More than 80% of participants thought the achievements of the Scrums were good to excellent. Also, 66% of participants thought that AoW made their work more effective, although 33% of participants answered only “somewhat” to this question. All participants indicated they would participate again if a new Scrum session were organized the next year.
As a final feedback, participants could leave a Post-it note on the Keep and Improve flip-over boards. The clustered results of the flip-overs are shown in Table 4. Suggestions for Improve were, among others, a) selecting team members based on their competencies and b) improving communication about an individual’s commitment and availability. Also, the loss of effective working time due to the morning stand-ups, retrospectives, evaluation time, and demonstrations (in particular at the end of a sprint on Friday) was mentioned.
All-team retrospective results: what to keep, what to improve.
DISCUSSION AND CONCLUSIONS.
As work at KNMI takes on a more multidisciplinary approach, we examined AoW as a method to facilitate collaboration and coordination among scientists, software engineers, and other relevant personnel. To this end, three case studies were defined derived from KNMI’s observations strategy. The suitability of AoW for each project was tested in two sprints of one week each, and a number of observations can be made based on the outcome of this experiment.
AoW works best if the proposed work is to be achieved in a joint environment where the success of the project is dependent upon frequent exchange and interaction between team members. This was the case for the quality team and the data-wrangling team. For the drone team, most members had to work on individually assigned tasks that needed to be accomplished independently from other tasks and team members. However, the drone team did experience much benefit from the meticulous planning of the drone-flying experiment in the weeks prior to the sprints.
While AoW is often profitable in software development, its success in an R&D environment depends on the institutional ability to fuse the typical work practice of a researcher with that of a software engineer. Many researchers work individually. In fact, they have been educated and trained to do so, and an adjustment to work in a joint setting is not always beneficial to their productivity. During the sprints, a number of researchers felt it necessary to withdraw occasionally to the confines of their office to think and resume their traditional work practice. Therefore, a relaxation of AoW to a more flexible work environment (rather than a joint setting) could be beneficial to institutions interested in engaging their personnel in AoW. Exploration of adapting Scrum or applying methods such as Kanban (Ahmad et al. 2013) may be considered.
In an R&D environment, the PO is often a researcher or research manager with little or no experience in AoW. Therefore, in an R&D AoW setting, an experienced SM is needed to channel the efforts of both the PO and the team toward an MVP. During the sprints at KNMI, the experience and flexibility of the SMs were essential in adjusting ambitions of the POs to obtain an MVP in the appointed time.
Commitment from all team members is of vital importance to the success of the project. Team members need to feel engaged in the process, and frequent absences due to prior commitments or other pressing tasks are detrimental to the other team members. Lack of commitment was identified as the largest obstacle to team building. No doubt, efforts to ensure commitment in the preparatory phase of the Scrum could be improved.
The answers to the questionnaire by the participants showed that they rated the experience and the results of the AoW highly and valued the joint teamwork toward a common goal. Nevertheless, they also felt that some aspects of the AoW, namely, the stand-ups, group demonstrations, and retrospectives, took too much time and could have a negative impact on the results of their team. However, these group processes are essential to the success of the Agile/Scrum method, and it is suspected that more exposure to Agile/Scrum would improve the understanding of the relevance of these processes to the success of projects.
The Scrums that were held at KNMI are but a relatively small effort involving a single department consisting of researchers, software engineers, and technicians (50+ people). Plans are under way to extend the AoW to projects involving more departments for tests in a more complex institutional and programmatic setting. At the moment it is clear that no single method will suit our aims. Fortunately, Agile methods can be continuously improved and adapted to diverse teams, their varying tasks, and ever-changing circumstances.
ACKNOWLEDGMENTS
The case studies would not have been possible without the dedicated assistance of a number of people. We specifically mention Marc Allaart, who was essential in orchestrating the launching of the radiosondes; Marcel Brinkenberg for facilitating the drone launch at the research station of Cabauw; Netty Huisman, Frits van de Peppel, Jelis van der Berg, Jos van Dun, René van Aken, Hans Olminkhof, Karin Tukker, and Jeroen van Zomeren for arranging computer hardware and facilitating other infrastructure; Gijsbert Boon for facilitating the all-team retrospective; Fred Bosveld for his advice on organizing measurements to probe the boundary layer morning transition; and Albert Klein Tank for his support for organizing the Scrum. Finally, we thank the team members who contributed to the sprint sessions: Jurian Baas, Else van den Besselaar, Cisco de Bruijn, Marieke Dirksen, Siebren de Haan, Henk Klein Baltink, Stephen Knoop, Susanne Kok, Michal Koutek, Hidde Leijnse, Jonas Matser, Andrej Mihajlovski, Ian van der Neut, Cor van Oort, Corné Oudshoorn, Maarten Plieger, Reinder Ronda, Andrew Stepek, Antonello Squintu, Maarten Terpstra, Rob Tjalma, Hans Verhoef, Ernst de Vreede, Lotte de Vos, Saskia Wagenaar, Wiel Wauben, and Ine Wijnant.
REFERENCES
Ahmad, M. O., J. Markkula, and M. Oivo, 2013: Kanban in software development: A systematic literature review. Proc. 39th Euromicro Conf. on Software Engineering and Advanced Applications, Santander, Spain, IEEE, 9–16.
Barroca, L, H. Sharp, D. Sala, K. J. Taylor, and P. Gregory, 2015: Bridging the gap between research and agile practice: An evolutionary model. Int. J. Syst. Assur. Eng. Manage., 9, 323–334, https://doi.org/10.1007/s13198-015-0355-5.
Beck, K., and Coauthors, 2001: Manifesto for agile software development. www.agilemanifesto.org.
Berkhout, F., B. van den Hurk, J. Bessembinder, J. de Boer, B. Bregman, and M. van Drunen, 2014: Framing climate uncertainty: Socio-economic and climate scenarios in vulnerability and adaptation assessments. Reg. Environ. Change, 14, 879–893, https://doi.org/10.1007/s10113-013-0519-2.
Boers, R., A. Klein Tank, T. Brandsma, G. Lenderink, J. F. Meirink, R. Sluijter, R. Sluiter, and W. Wauben, 2015: Observations strategy KNMI: 2015–2024. KNMI Publ. 233, 38 pp., http://bibliotheek.knmi.nl/knmipubmetnummer/knmipub233.pdf.
Briley, L., D. Brown, and S. E. Kalafatis, 2015: Overcoming barriers during the co-production of climate information for decision-making. Climate Risk Manage ., 9, 41–49, https://doi.org/10.1016/j.crm.2015.04.004.
Fowler, M., and J. Highsmith, 2001: The Agile manifesto. Software Dev ., 9, 28–32.
Goosen, H., and Coauthors, 2014: Climate adaptation services for the Netherlands: An operational approach to support spatial adaptation planning. Reg. Environ. Change, 14, 1035–1048, https://doi.org/10.1007/s10113-013-0513-8.
Haikara, J., 2007: Usability in Agile software development: Extending the interaction design process with personas approach. Agile Processes in Software Engineering and Extreme Programming, G. Concas et al., Eds., Springer, 153–156.
Hegger, D., and C. Dieperink, 2014: Toward successful joint knowledge production for climate change adaptation: Lessons from six regional projects in the Netherlands. Ecol. Soc., 19, 34, https://doi.org/10.5751/ES-06453-190234.
Hegger, D., M. Lamers, A. Van Zeijl-Rozema, and C. Dieperink, 2012: Conceptualising joint knowledge production in regional climate change adaptation projects: Success conditions and levers for action. Environ. Sci. Policy, 18, 52–65, https://doi.org/10.1016/j.envsci.2012.01.002.
Hicks, M., and J. S. Forster, 2010: Adapting Scrum to managing a research group. University of Maryland Dept. of Computer Science Tech. Rep. CS-TR-4966, 9 pp., www.cs.umd.edu/∼mwh/papers/score.pdf.
Kandel, S., and Coauthors, 2011: Research directions in data wrangling: Visualizations and transformations for usable and credible data. Inf. Visualization, 10, 271–288, https://doi.org/10.1177/1473871611415994.
Marchesi, M., K. Mannaro, S. Uras, and M. Locci, 2007: Distributed Scrum in research project management. Agile Processes in Software Engineering and Extreme Programming, G. Concas et al., Eds., Springer, 240–244.
Monna, W., and F. Bosveld, 2013: In higher spheres: 40 years of observations at the Cabauw site. KNMI Publ. 232, 56 pp., http://bibliotheek.knmi.nl/knmipubmetnummer/knmipub232.pdf.
Patton, J., and P. Economy, 2014: User Story Mapping: Discover the Whole Story, Build the Right Product. O’Reilly Media, Inc., 324 pp.
Ribeiro Lima, I., T. de Castro Freire, and H. Costa, 2012: Adapting and using Scrum in a software research and development laboratory. Revista de Sistemas de Informação da FSMA, No. 9, 16–23, www.fsma.edu.br/si/edicao9/FSMA_SI_2012_1_Principal_2_en.pdf.
Schwaber, K., 1997: SCRUM development process. Business Object Design and Implementation, J. Sutherland et al., Eds., Springer, 117–134.
Schwaber, K., and M. Beedle, 2002: Agile Software Development with Scrum. Prentice Hall, 158 pp.
Sutherland, J., and K. Schwaber, 1995: Business Object Design and Implementation. Springer-Verlag, 167 pp.
Sutherland, J., and K. Schwaber, 2017: The Scrum guide: The definitive guide to Scrum; the rules of the game. www.scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf.
Voinov, A., and F. Bousquet, 2010: Modelling with stakeholders. Environ. Modell. Software, 25, 1268–1281, https://doi.org/10.1016/j.envsoft.2010.03.007.
Wingate, L., 2015: Project Management for Research and Development. CRC Press, 517 pp.
