CHAPTER 2 International Data Collection Daniel Zell CONTENTS 2.1 Introduction 2-1 2.2 Data Sources 2-2 2.3 Case Studies 2-3 2.3.1 Latin America 2-3 2.3.2 Central Asia 2-4 2.3.3 South Asia 2-4 2.4 General Process for Data Collection 2-5 2.5 Internet Sources 2-5 In developing countries, dams, irrigation schemes, watershed management plans, water and wastewater systems, and flood mitigation works have grown in both number and complexity. Because these data intensive approaches, such as river basin planning, are being embraced worldwide and funded by multilateral and bilateral organizations, the need for data has increased. China, for example, has undertaken massive water resources projects on a scale never seen before, and vast irrigation rehabilitation projects are underway in Afghanistan. In many Latin American countries, water management projects are a top priority. With the demand for such types of projects comes the need for data, information, knowledge management, and, in particular, people who fall under the broad category of water resources engineers. This need is made sharper given the imminent retirement of the seasoned professionals of the post-WWII generation who have spent their lives in water resources, leaving a younger, less experienced cohort of engineers to tackle the future. 2.1 INTRODUCTION When, as often happens, engineers and other technical professionals in the water resources field are asked to render a technical judgment, they usually need a large set of data to analyze the issue. In the United States, general information such as precipitation, topography, stream- flow, and other related data is usually readily available from standard sources: previous studies are usually on-hand with the implementing agency, e.g., state government, military, and the private sector. But some- times, data are harder to find, less reliable, maybe even lost. This latter case is the normal starting point for water resources projects in less developed countries. The engineer will have to invest a good deal of time, effort, 2-1 q 2006 by Taylor & Francis Group, LLC and sometimes money to get the sort of data that are, usually, freely available in developed countries. This chapter gives engineers an idea of what to expect, some approaches to gathering data, and international internet data sources. There has been and always will be a need for rapid and complete data collection for water resources projects in the developing world, a process made even more challenging since data are fragmented among various government and private organizations. Due to both varied organizational arrangements and cultural factors, procedures for data collection vary from region to region and country to country, and it is not possible within the confines of this chapter to give a comprehensive step-by- step method for accomplishing it. Here, we present a brief picture of the challenges of data collection through some selected examples, an outline of a brief process, and a listing of sources of data for international water resources projects. This chapter, then, will be most useful for those with little experience in water resources projects in developing nations. Now, before getting into the details, a few definitions and explanations. 1. Less developed countries is a term of conveni- ence, generally meaning any country except the United States, the former Soviet Union, Canada, Western Europe, Australia, New Zealand, and Japan. But even though such a simple dichotomy does not really exist—other countries undoubtedly deserve to be considered developed and may indeed boast superior data repositories—this term, because of its wide- spread acceptance, will be used throughout this chapter. 2. Engineer is defined here as a person responsible for technical aspects of a water-related project. Although the engineer is normally an outsider to the country or region under study and perhaps initially unfamiliar with its cultural norms and practices, he or she engineer understands not only the data collection need but also its intended application and eventual output. In some cases engineers will be scientists, economists, even policymakers. The term, again, is chosen out of convenience. 3. To make the chapter as useful as possible to the widest audience, it generally refers to data in the generic sense, rather than, for example, average precipitation and uses illustrative examples rather than an analytical examination. Without digressing too much into the world of development economics, we note that the engineer needs to understand the stage or level of development. In general, there is a positive correlation between wealth, or Gross Domestic Product per capita, and the centraliza- tion and quality of water resources data. In Afghanistan, one of the poorest countries in the world, for example, where data sources are widely scattered, the results are incomplete, with much historical data, and studies lost in the warfare over the past few decades. Climate also plays a factor, as natural disasters, humidity, and even rodents often destroy hard copies in countries with limited computerized archives. The overall lesson is that even in the relatively more developed countries of Central and South America, the engineer will need to search for secondary sources of data outside of the government, such as previous consultants’ reports, private water companies’ data, and others (see the list of websites at the end of this chapter). An outside engineer’s local contacts will be essential in setting appointments, identifying possible resources, and even collecting data themselves. But—a word of caution—it is unlikely that this task can be accomplished without the involvement of the engineer. If you are not willing to travel to the country of assignment, then you should probably find somebody else to do the job. Both the credibility and realism of the outputs will suffer from a lack of field presence. Besides, as an outsider, typically seen as immune to local politics and prejudices, the engineer who is willing to travel and spend enough time in country can surmount obstacles that an indigenous assistant could not. 2.2 DATA SOURCES For general purposes, let us consider a simple dichotomy in terms of data availability: † Initial Conditions (given data), and † Needs (data objectives). Initial data may be given by your client or employer or may be found through some cursory searching of the internet resources listed in this chapter. Data needs or objectives depend on the particular assignment and will likely evolve throughout the problem solving process. In any case, it is well worth taking the time to explore most, if not all, of the websites listed at the end of the chapter with a high-speed internet connection. You will spend much less time than trying to find data from alternate sources, such as government departments, previous and/or current projects, and well-known experts in the field. And the websites listed are generally regarded as reliable. But website searches are not the only route. I recommend a variety of methods, especially if the THE WATER ENCYCLOPEDIA: HYDROLOGIC DATA AND INTERNET RESOURCES2-2 q 2006 by Taylor & Francis Group, LLC country and region are unfamiliar to the engineer. Before arriving in a country, establish primary contacts and introductions through the employer or client. Email, however convenient, is no substitute for phone calls in building relationships. These contacts may, if sufficiently motivated, be in a much better position to collect the required data. Unfortunately, due primarily to the long and frequently inconclusive history of most local water resources projects with international engineers, it will be a difficult task convincing someone unknown to you, that your cause is worthwhile. This is yet another reason why you should use your in-country network, even if it is only your employer or client, as a foundation for collecting data. The social ties and relationships that locals have established are a resource you should not ignore. A frequent issue when working in an unfamiliar region and culture is payment. How much do things cost? From CAD and GIS operators, to internet usage, to maps, most, if not all, goods and services have a value. In my experience, the issue is not whether or not to pay, but rather, “what is the correct price?” This is not to say that you can not get data for free; indeed, that is how I got most of mine. The engineer will have to rely on his or her in-country network and awareness of local social norms to determine whether or not a payment is required. In general, if work is required beyond normal hours or duties, then a payment is more likely to be required. Another thing to be prepared for is equipment and procedures that do not work. Copiers, for example, are usually broken because of a lack of basic supplies and/or maintenance. Or it may not exist or be available to anyone except the senior staff. Regulations may not let you take records from the office for off-site duplication. Unless you have a penchant for data entry, this is another case where it may be appropriate to pay the correct fee for the data to be digitally entered. I suggest offering an electronic copy of the records by email to both the department and data entry person. By making them part of the process, rather than a cog in a wheel, you may use these people as resources for a long time to come, as I did with a local hydrologist in Afghanistan months after my return. (It did, however, take much longer than expected and required several phone calls and visits by contacts in Afghanistan.) 2.3 CASE STUDIES The following case studies will give the engineer a general perspective of the operational aspects of data collection in a few developing nations. Let me note at once, however, the enormous importance of learning about and paying attention to the unique cultures, conditions, and particularly the values of each nation or region. Ignoring them will imperil both the engineer and the project itself. 2.3.1 Latin America Overall, although collecting water resources data in Latin America is like working in developing countries worldwide,* the relatively higher income in Latin America can make the work easier, as education level generally correlates with income. Because hydropower is a highly developed sector in Latin America, its organizations often have the best data. However, the data are often considered proprietary and, being tightly controlled, take concerted and time-consuming effort by the engineer and local counterparts to get it. As in other regions, engineers should rely on their counterparts, local staff, or client by insisting that reasonable efforts be expended to gather data in advance. Due to the culture of Latin America, counterparts can be relied on quite heavily, and will likely be highly competent. In this manner, the engineer’s time can be best and efficiently used to address the remaining obstacles, like analyzing the data and writing reports. Still, the engineer should be prepared to complete an assignment with much less data in both quantity and quality than is the norm in developed countries. When no data are found for a specific basin, data for a similar region—in combination with field interviews, maps, and surveys—is used to build a model. The uncertainties resulting from using such data should be plainly and simply stated in the engineer’s work. According to Dr Molina, the process of cleaning data—searching for errors—although time consuming and tedious, has repeatedly proved itself to be worth the trouble. Common errors include converting units, converting gauge readings to flow, and a myriad of other possibilities. These errors, if unchecked and corrected, will skew the results of an analysis. On the other hand, careful advance planning can minimize delays, for example, by contracting a local firm or consultants to carefully screen the data for anomalies and enter it into a useful format. * According to a June 25, 2004, interview with Dr Medardo Molina, a Peruvian-born water resources expert who has been active in water resources since 1965 as an international consultant and professor. Dr Molina has published numerous papers in both Spanish and English. INTERNATIONAL DATA COLLECTION 2-3 q 2006 by Taylor & Francis Group, LLC 2.3.2 Central Asia In Afghanistan, collecting water resources data were quite complicated, based on my personal experiences in 2003. What little historical data existed was fragmen- ted and incomplete. Thirty years of instability destroyed critical records and the hydrological network. The language barrier, with data sometimes recorded in Dari, Pashto, or Russian, further complicated data collection. The assigned task—to construct a national water balance and determine water availability for rehabilita- ting irrigation—necessitated casting a wide net to collect all sources of data. Before visiting Afghanistan, my thorough search of libraries and databases yielded only few results, but one of them, a previous water balance study, was quite useful. Although I made contacts in advance, scheduling appointments from outside of Afghanistan did not work. I now understand that the local custom is to pay a brief introductory visit, without an appointment. Then, a later appointment can be scheduled where the useful work will be accomplished. This introductory visit may seem full of pleasantries and even a bit useless. To the novice it may seem not along the critical path. Quite to the contrary, the visits are used to evaluate the engineer, to understand what the engineer is doing, and to understand if the engineer is worthy of help. Besides, since data are rarely immedi- ately available, the first meeting gives the local source time to prepare. The team conducted initial and follow-up visits to the following organizations: Ministries: Irrigation and Water Resources and Environment, Rehabilitation and Rural Development, Agriculture and Animal Husban- dry, and the Central Statistics Office United Nations: Food and Agriculture Organi- zation and World Food Program, in particular the Vulnerability Analysis Mapping Unit for socio-economic data, and the Development Program Others: ICARDA (International Center for Agri- cultural Research in the Dry Areas), the U.S. Agency for International Development, and various private consultants, firms, and non- government organizations From these and subsequent visits over a 12-month period, the consultant team gathered the hydrological station data. We hired a local professional consultant to both construct and digitize a map of the monitoring stations. More importantly, the team was able to persuade a wide range of policy makers and water resources professionals of the report’s thoroughness and usefulness by conducting follow-up visits to the data repositories, courtesy calls to government officials to present the report and findings, and formal presentations. The United Nations’ organizations provided the most accurate satellite and agricultural data, and previous consultants’ reports were essential in constructing the overall water balance. Although we still had to do careful fact checking based on the newly available data, the efforts of independent and knowledgeable local staff helped us get data not only more rapidly but also with a higher degree of reliability than would have otherwise been possible. It is important to note that the local staff’s efforts had to be supplemented with continual phone calls and emails by the consultants as well as periodic visits to Afghanistan. As outsiders, foreign consultants can often bypass the social norms and traditions that hamper local staff. At the same time, continued communication and visits let the team more completely understand the extent of the problem or issues to be addressed and more importantly, convey the engineers’ recommended solution to the client as it evolved. This served two purposes: 1. By avoiding the “parachute in” approach where a consultant works rapidly and alone, often behind a closed door, the team won a sense of buy-in and respect from the client, which ensured that the client would actually understand the results of the work. 2. By working hand-in-hand with a client who lacked basic institutional capacity, the con- sultant team was able to increase the client’s capacity to use the team’s results. With so many people interested in the report, the team decided to circulate a draft copy. The resulting informal peer review process increased the credibility of the findings, and although it led to a series of revisions, in the end, it was the only way to accomplish the task. 2.3.3 South Asia In contrast to the situation in Afghanistan, data availability was relatively high during a 2004 assign- ment to the Indian State of Orissa. At the state water resources agency, I saw many rooms with neatly bound papers stacked literally from floor to ceiling. More remarkably, when motivated by the department head, almost everything we asked for was quickly found. And because the information requested was relatively recent, THE WATER ENCYCLOPEDIA: HYDROLOGIC DATA AND INTERNET RESOURCES2-4 q 2006 by Taylor & Francis Group, LLC we did not encounter the issue of deteriorating paper records in a humid environment without climate control. The main problems were poor database design and data entry, and resistance to improving quality control standards. To make sure that our primary data on average daily flow from reservoirs were accurate, my team went to one of the more remote locations to look at primary records. This entailed a journey across the small rice paddies that characterize the regions, a courtesy call to the local supervisor, and even more driving to where the road literally ended at the reservoir. The water resources department had told the reservoir about our trip, and the records were ready. However, local customs dictated a long, slow, and quite good meal as well as an exchanging of pleasantries before the ledger was produced. Then, we asked the data recorders how they recorded the gate height and translated that measurement into a daily flow. Spot checking a few calculations for accuracy, we found no mistakes. A universal issue when dealing with government agencies is territorialism. This issue nearly derailed the entire project in India. An agency insisted that a task was within their realm of authority but showed neither the will nor the capacity to perform the work as required. Rather the agency did the bare minimum and strongly resisted any efforts at improvement. The lesson the engineer should take from this is that relying on someone or some agency out of your control is risky. In hindsight, the solution would have been to accept the agency as a partner, then help them, as cooperatively and diligently as possible, to complete the project. 2.4 GENERAL PROCESS FOR DATA COLLECTION The following process serves as a guideline for engineers unfamiliar with international water resources engineer- ing. It is highly likely that additional steps will be needed in any actual project depending on the requirements and country. 1. Learn key features of the country’s culture norms and values by both talking to someone from the country and reading appropriate articles and even the literature. 2. Identify the problem. This step consists of more than simply reading the terms of reference. The engineer should plan on conducting one-to-one discussions with senior and mid-level staff at the local or client agency. 3. Identify data requirements. 4. Identify local staff requirements such as cartographers, hydrologists, statisticians, field surveyors, etc. 5. Contact the client or local agency. 5.1. Contact made well in advance of the initial visit so that the staff are aware of the planned work and can plan accordingly. 5.2. Contact at regular intervals, at least weekly. Contact by phone and email is recommended. 6. Make initial visit. Prearranged visits with organizations should be conducted. Social outings, while exhaust- ing, may prove useful for particularly difficult data collection situations. 7. Collect the data. 8. Review data collected. Determine what steps need to be taken to ensure the data are both accurate and in a usable format. 9. Conduct field validation if required. 10. Make more visits. These visits will supplement initial data gatherings and present initial findings. 11. Perform analysis. 12. Prepare and circulate draft report. 13. Incorporate comments as appropriate. 14. Prepare report. 2.5 INTERNET SOURCES The following addresses give access to a lot of data. The descriptions of the 2006 websites come directly from the web pages themselves. AQUASTAT is the Food and Agricultural Organization’s (FAO) global information system of water and agriculture developed by the Land and Water Development Division of FAO. AQUASTAT is FAO’s global information system of water and agriculture developed by the Land and Water Development Division of FAO. The objective of AQUASTAT is to provide users with comprehensive information on the state of agricultural water management across the world, with emphasis on developing countries and countries in transition (www.fao.org/WAICENT/ FAOINFO/AGRICULT/AGL/AGLW/aquastat/main/ index.stm). The following is an excerpt from the AQUASTAT country profile of Afghanistan INTERNATIONAL DATA COLLECTION 2-5 q 2006 by Taylor & Francis Group, LLC The tables above are an example of the output produced by FAO STAT, in this case rural and urban populations in African Developing Nations from 1993 to 2003. The output is also available in .csv format. GRDC (Global Runoff Data Center) The GRDC makes the unique offer to the international research and science community of easy and universal access to river flow information on a global scale. On request, data Production Land use and irrigation Fertilizer and pesticides Trade Forest products Agricultural machinery Food balance sheets Fishery products Food aid shipments Producer prices Population Exports by destination Forestry trade flow Food quality control faostat.fao.org/default.jsp?languageZEN. Year 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 Population-estimates rural (1000) Africa developing 428,145 435,774 443,382 450,948 458,471 465,941 473,370 480,738 488,033 495,224 502,287 AGG_COUNTRIES 428,145 435,774 443,382 450,948 458,471 465,941 473,370 480,738 488,033 495,224 502,287 Population-estimates urban (1000) Africa developing 205,568 214,269 223,156 232,235 241,525 251,050 260,846 270,932 281,347 292,111 303,245 AGG_COUNTRIES 205,568 214,269 223,156 232,235 241,525 251,050 260,846 270,932 281,347 292,111 303,245 Water Resources Based on the hydrographic systems, the country can be divided into four zones: † the northern basin (24% of the territory) with the Amu Darya and its tributaries (14%), which drain toward the Aral sea, and the rivers of northern Afghanistan (10%), which disappear within the country before joining the Amu Darya; † the western region (12%) consisting of the Hari Rud river basin (6%) and the Murgab river basin (6%), both rivers disappearing in Turkmenistan; † the south-western basin (52%) with the Helmand river flowing toward the Sistan swamps, located on the border of Iran and Afghanistan. In 1972, a document was signed between Afghanistan and Iran to allocate a discharge of 26 m/s of Helmand river water to Iran all year round; † the eastern Kabul basin (12%), which is the only river system having an outlet to the sea, joining the Indus at Attock in Pakistan. Internal renewable water resources are estimated at 55 km 3 yr K1 . The Kunar river, which originates in Pakistan, crosses the border with an average annual flow of 10 km 3 and joins the Kabul river at Jalalabad about 180 km further downstream. The Kabul river flows again into Pakistan 80 km further downstream. Total water withdrawal was estimated at 26.11 km 3 in 1987, of which 99% for agricultural purposes. Recently, there has been a large development of groundwater use in some provinces. In 1986, there were two dams higher than 15 m. The installed capacity of the hydroelectric plants was 281 MW in 1992, which is about 70% of total installed capacity. There is considerable potential for the generation of hydropower, by both large dams and microhydropower stations. FAOSTAT is an on-line and multilingual database currently containing over 3 million time-series records covering international statistics in the following areas: THE WATER ENCYCLOPEDIA: HYDROLOGIC DATA AND INTERNET RESOURCES2-6 q 2006 by Taylor & Francis Group, LLC products are developed and specialized databases are assembled for projects on both regional and global scale. The GRDC serves as a communication platform between institutions, advisors, and scientists and also transfers information about other relevant databases with a hydrological content such as the Global Precipitation Climatology Centre (GPCC) and the Programme Office of the Global Environment Monitoring System—Water (GWPO) of UNEP. It also maintains close ties to the UNESCO Water project “Flow Regimes from Inter- national Experimental and Network Data” (FRIEND). The GRDC contains long-term mean annual freshwater surface water fluxes into the world oceans. Estimates are based on 251 discharge stations of major rivers. Data available include mean, minimum, maximum monthly discharges, and time series of mean, minimum, maximum annual discharge for 3035 stations. UNH-GRDC Global Composite Runoff Fields combines observed river discharge information with a climate-driven Water Balance Model (http:// grdc.bafg.de). The following is a sample output: INTERNATIONAL DATA COLLECTION 2-7 q 2006 by Taylor & Francis Group, LLC CGIAR is a global link to research on agriculture, hunger, poverty, and the environment. CGIAR (Consulta- tive Group on International Agricultural Research) is a good starting point for international water issues related to agriculture (www.cigar.org). To access any of CGIARs publications published by the 15 research centers, go to the CGIAR Library Gateway (www.cgiar.org/publications/library/index. html). CGIAR Reefbase—ReefBase is the world’s premier online information system on coral reefs, and provides information services to coral reef professionals involved in management, research, monitoring, conservation, and education. Its goal is to facilitate sustainable manage- ment of coral reefs and related coastal/marine environ- ments, in order to benefit poor people in developing countries whose livelihoods depend on these natural resources. ReefBase’s Online Geographic Information System (GIS) allows you to display coral reef related data and information on interactive maps. You can zoom, search, and query datalayers, and save or bookmark the map (www.reefbase.org). ICARDA (International Center for Research in the Dry Areas) is one of the research centers of CGIAR, whose mission is to improve and integrate the manage- ment of soil, water, nutrients, plants, and animals in ways that optimize sustainable agricultural production. There are many relevant articles, publications, and datasets on on-farm water use and water efficiency (www.icarda. cgiar.org). IWMI’s (International Water Management Institute) on-line publication section contains several thousand pages of peer reviewed research on water management. All research outputs and publications produced by IWMI are international public goods, freely available to partners in developing countries and to members of the international development, academic and research com- munities (www.iwmi.cgiar.org/pubs/mindex.htm). The Remote Sensing and GIS Unit (RS GIS Unit) of the International Water Management Institute (IWMI), is a centralized facility for all spatial data- related activities of IWMI at the headquarters in Sri Lanka and Regional Offices located in different parts of the world. Currently, the RS GIS Unit holds over 1 terabyte of data. Although the emphasis is on IWMI benchmark river basins, large volumes of data are also available at National, Regional, and Global levels. These data are catalogued, streamlined, and released to the public through the IWMI Data Storehouse Pathway (DSP). Much of the river basin and other datasets are composed as single mega files of hundreds or sometimes thousands of bands consisting of continuous streams of 8-day or monthly time series data in several wavebands and/or indices. Large volumes of multitemporal data from multiple satellite sensors are used in several IWMI research projects. These projects include: (a) Global Irrigated Area Mapping (GIAM) at global to local scales, (b) the Wetland project in the Limpopo river basin of four Southern African Nations, (c) the Krishna river basin project in India, (d) the Indo-Gangetic river basin project in India and Pakistan, (e) the Drought Assessment and Mitigation project in Afghanistan, Pakistan, and parts of India, and (f) the biodiversity project in Sri Lanka. More information on the RS GIS Unit and its activities can be found in several areas of this web site (www.iwmidsp.org/iwmi/info/centerprofile.asp). Hydrological Processes is a relevant international journal with abstracts freely accessible on the web. Of particular interest are the past articles on mathematical and methodological aspects of hydrological processes and modeling. Accessed through Wiley Intersciences (www. interscience.wiley.com). Although traditionally thought of as a repository of United States data sets only, the United States Geological Survey (USGS) has a notably thorough database on selected countries that will likely grow over time. In particular, datasets for Jordan and Israel can be found through their project websites (www.watercare.org and www.exact-me.org). Also on USGS, a general homepage with linkages to Ukraine, United Arab Emirates, Bangladesh, and Cyprus (international.usgs.gov/disciplines/water.htm). Winrock Water is both a discussion forum and data clearinghouse that includes an annotated bibliography. Winrock water has selected leading reference materials, research and discussions of major issues in the water resources field through the internet. The links section is of particular interest to those in data collection (www. winrockwater.org). Sakia.org is an information and communication service in the area of “land and water”. Sakia.org hosts several services such as the email discussion list IRRIGATION-L the WWW Virtual Library Irrigation & Hydrology (content filling stage), the WWW Database on Irrigation & Hydrology Software—IRRISOFT (under revision), the e-Journal of Land and Water, an open access and peer reviewed international scientific journal for research and developments and the Journal of Applied Irrigation Science. Sakia.org is fostering the open and free access to knowledge in support of the “land and water” community (www.sakia.org). THE WATER ENCYCLOPEDIA: HYDROLOGIC DATA AND INTERNET RESOURCES2-8 q 2006 by Taylor & Francis Group, LLC . CHAPTER 2 International Data Collection Daniel Zell CONTENTS 2. 1 Introduction 2- 1 2. 2 Data Sources 2- 2 2. 3 Case Studies 2- 3 2. 3.1 Latin America 2- 3 2. 3 .2 Central Asia 2- 4 2. 3.3 South Asia 2- 4 2. 4. 27 0,9 32 281,347 29 2,111 303 ,24 5 AGG_COUNTRIES 20 5,568 21 4 ,26 9 22 3,156 23 2 ,23 5 24 1, 525 25 1,050 26 0,846 27 0,9 32 281,347 29 2,111 303 ,24 5 Water Resources Based on the hydrographic systems, the country. 443,3 82 450,948 458,471 465,941 473,370 480,738 488,033 495 ,22 4 5 02, 287 Population-estimates urban (1000) Africa developing 20 5,568 21 4 ,26 9 22 3,156 23 2 ,23 5 24 1, 525 25 1,050 26 0,846 27 0,9 32 281,347 29 2,111