README FILE FOR AUSTRALIAN WATER AVAILABILITY PROJECT (AWAP) MAPS AND DATA CONTENTS -------- 1) DATA CUSTODIAN 2) ACCESS AND CONDITIONS OF USE 3) CITATION 4) DISCLAIMER, COPYRIGHT AND LICENCE 5) DATA SOURCE 6) DATA INFORMATION 7) DIRECTORY, FILENAME AND DATA FORMATS 8) GEOGRAPHICAL COVERAGE 9) DATA QUALITY 10) VERSION NOMENCLATURE 11) VERSION RELEASE NOTES 12) UPDATES 13) REFERENCES 14) CONTACTS 1) DATA CUSTODIAN -------------------------------------------------------- CSIRO Marine and Atmospheric Research is the custodian of this dataset. This dataset has been developed by the Australian Water Availability Project (AWAP) Team, CSIRO Marine and Atmospheric Research (CMAR). The original meteorological data were supplied by the Bureau of Meteorology Australia (BoM). Project management was undertaken by the Bureau of Rural Sciences (BRS) in the Department of Agriculture, Fisheries and Forestry (DAFF). Between 2008-2012 funding for CSIRO AWAP science and data projects was provided by the South Eastern Australian Climate Initiative (SEACI), a collaboration between the Murray-Darling Basin Authority, Victorian Department of Sustainability and Environment, Department of Climate Change, Managing Climate Variability Program, CSIRO, and the Bureau of Meteorology. 2) ACCESS AND CONDITIONS OF USE -------------------------------------------------------- These data are available from: http://www.csiro.au/awap Please note the citation and legal information in Sections 3 and 4 below. In order to help us keep track of who is using the data, please do not pass the data on to a third party. Instead, refer them to the web site. Currently AWAP model results (excluding BoM meteorology) are a research level product and are available for collaborative use with CSIRO AWAP team members. Where AWAP model data are to be used for significant scientific outputs (such as publications) we will wish to discuss co-authorship. In all cases, we ask that you send us citations and copies of publications arising from work that use these data to (Peter.Briggs@csiro.au). These data form ongoing research. The project partners have agreed to release the maps publicly for informational purposes. The data files may be released for research purposes. 3) CITATION -------------------------------------------------------- When referring to this dataset in publications, please cite both: Raupach MR, PR Briggs, V Haverd, EA King, M Paget, CM Trudinger (2009), Australian Water Availability Project (AWAP): CSIRO Marine and Atmospheric Research Component: Final Report for Phase 3. CAWCR Technical Report No. 013. 67 pp. Raupach MR, PR Briggs, V Haverd, EA King, M Paget, CM Trudinger (2012), Australian Water Availability Project. CSIRO Marine and Atmospheric Research, Canberra, Australia. . Date of access. 4) DISCLAIMER, COPYRIGHT AND LICENCE -------------------------------------------------------- Use of these data is subject to the Legal Notice and Disclaimer at http://www.csiro.au/org/LegalNoticeAndDisclaimer.html These data are Copyright, CSIRO, 2012. These data are made available under the conditions of the Creative Commons Attribution-Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/ 5) DATA SOURCE -------------------------------------------------------- Data are produced by the WaterDyn model, which has been developed by CSIRO (Raupach et al. 2009, 2008) to model the terrestrial water balance across continental Australia. Key inputs and constraints on the model are the meteorology (solar radiation, precipitation, minimum and maximum daily temperatures) and continental parameter maps (e.g. albedo, soil characteristics, seasonality of vegetation greenness). The meteorological fields are generated by the BoM. They generate both daily products (near-real time) and reprocessed products (including quality control and incorporating data that arrive too late for inclusion in the near-real time product). The WaterDyn model is critically dependent on the timely arrival and accuracy of the meteorological data. Operational outputs (weekly and monthly near-real time data) will be delayed if the daily meteorological fields are not available. The parameter maps are static for each WaterDyn model version but will be updated if new information becomes available. Updates of the parameter maps will always invoke an update of the model version. To date there have been no changes to the parameter maps since the original 2007 release. In the event that the WaterDyn model is modified the affected datasets will be reprocessed with the latest version of the model. The available datasets are generally a century-plus series from the beginning of 1900 to the end of the latest year of the reprocessed meteorological data (monthly and annual data), and an operational series from the beginning of the last complete year of reprocessed meteorological data to the present (weekly and monthly data). 6) DATA INFORMATION -------------------------------------------------------- Data are available for a variety of terrestrial applications. Products include: Description Variable Units (maps) Units (data files) Note ----------- -------- ----------- ------------------- ---- Incident solar radiation SolarMJ MJ/m^2/d MJ/m^2/d Daily maximum temperature TempMax degC degC Daily minimum temperature TempMin degC degC Precipitation FWPrec mm/d m/d Relative Soil Moisture (Upper Layer) WRel1 Fraction (0-1) Fraction (0-1) Relative Soil Moisture (Upper Layer) at end of aggregation period WRel1End Fraction (0-1) Fraction (0-1) 1 Relative Soil Moisture (Lower Layer) WRel2 Fraction (0-1) Fraction (0-1) Relative Soil Moisture (Lower Layer) at end of aggregation period WRel2End Fraction (0-1) Fraction (0-1) 1 Total Evaporation (Soil+Vegetation) FWE mm/d m/d Total Transpiration FWTra mm/d m/d Soil Evaporation FWSoil mm/d m/d Potential Evaporation FWPT mm/d m/d Local Discharge (Runoff+Drainage) FWDis mm/d m/d Surface Runoff FWRun mm/d m/d Open Water Evaporation ('pan' equiv) FWWater mm/d m/d 2 Deep Drainage FWLch2 mm/d m/d Daily Sensible Heat Flux PhiH W/m^2 W/m^2 Daily Latent Heat Flux PhiE W/m^2 W/m^2 Note that sometimes the website maps are shown with different units than are used in the data files. WRel1End and WRel2End are not displayed at the website but are available from the ftp site. Notes: 1. WRel1 and WRel2 give average values for the specified aggregation period (weekly, monthly, annual, etc.). WRel1End and WRel2End give values for WRel1 and WRel2 at the end of the aggregation period, which is more useful for water balance calculations. 2. The inclusion of open-water evaporation (FWWater) post-dates the AWAP Phase III final report and so is documented here: FWWater is Penman evaporation with assumed aerodynamic conductance of 0.01 m/s and saturation deficit estimated as: (Saturation Vapour Pressure at Tmax) - (Saturation Vapour Pressure at Tmin) It is a little larger than Priestley-Taylor evaporation. For each pixel on the continent, open-water evaporation is an estimate of the evaporation that would occur from a small open water body if there happened to be one at that pixel. "Small" implies that the evaporation doesn't modify the meteorology through evaporative cooling etc. FWWater can be considered a model of pan evaporation. 6a) METEOROLOGICAL DATA -------------------------------------------------------- The meteorological data are provided by the BoM as daily gridded fields. Rainfall data are available for 1900 to present, temperatures from 1911 to present and solar irradiance from 1990 to present. The rainfall and temperature data come from the BoM's network of rain gauges and weather stations. The solar irradiance data are obtained using satellite imagery from geostationary meteorological satellites. Starting with the January 2011 release, AWAP historical runs use the BoM Version 3 Recalibrated Rainfall product. This product is generated by rescaling daily rainfall at the end of each month so that the sum of daily rainfalls matches the subsequent monthly reanalyses. The discrepancy between the sums of original daily rainfalls and the end-of-month reanalyses is due to differences in the interpolation methods applied to daily and monthly rainfall because of their different spatial structures. Interpolation failures in the original daily data occur in sparsely gauged areas of the continent such as the Central and Western Deserts. These are removed in the recalibrated rainfall product and replaced with missing data values. For AWAP modelling these areas are filled (at CSIRO) with the daily average calculated from the corresponding monthly reanalyses. Weekly AWAP operational runs continue to use unrecalibrated original rainfall, which is available in near real time. These are updated using recalibrated rainfall where noted. For historic runs from 1900 to present, missing solar irradiance data in the BoM weather archive, including the period 1900 to 1989 and any instrument- related pixel losses after that, are replaced with a monthly radiation climatology from the period 1990 to 2011. Missing temperatures for 1900 to 1910 are replaced with a monthly temperature climatology from the nearest contemporary 30-year period, 1911 to 1940. Key references for Bureau of Meteorology AWAP data are Jones et al. (2007; 2009) and Grant et al. (2008) and the information at http://www.bom.gov.au/jsp/awap/. 6b) PARAMETER DATA -------------------------------------------------------- Data and maps of the following spatially explicit parameters are available at the AWAP website www.csiro.au/awap. Albedo ------ Mean annual shortwave albedo (surface reflectivity) is used in AWAP as part of the calculation of available energy during daylight hours. The data were supplied by Ian Grant at the Bureau of Meteorology, and are the average of monthly data for the period 1994-1999 derived from AVHRR. The original data were supplied at 0.25 deg resolution and were resampled to 0.05 deg for AWAP modelling. The other important surface radiative property, emissivity, is set to a uniform value of 1. All-sky shortwave albedo is an average of direct and diffuse albedos appropriate for clear skies, where the direct albedo is an average appropriate for all solar zenith angles over a month. The differences between clear sky and diffuse or "white-sky" albedos (also calculated) are almost always less than 0.01 (Ian Grant, Bureau of Meteorology, pers. comm., 17 Sept 2004). The data are currently unpublished, but were created using standard methods. The atmospheric correction was done as described in Dilley et al. (2000) with the BRDF dynamically modelled using a method based on that described by Schaaf et al. (2002) for application to MODIS (Grant, pers. comm., 22 June 2005). Soil Properties --------------- Spatially explicit soil properties for two soil layers are defined by maps obtained from interpretations of the Digital Atlas of Australian Soils. The (rasterised) Atlas assigns a dominant soil type to each 0.05 deg grid cell across the continent, from over 700 possible soil types. Pedotransfer functions then assign physical soil properties in upper and lower layers (A and B horizons) to each soil type. The properties used for AWAP are saturated volumetric water content, soil layer thickness. Saturated hydraulic conductivities are set to uniform values for the two layers. Key references for these data are McKenzie and Hook (1992), McKenzie et al. (2000) and Northcote et al. (1960-1968). The Digital Atlas of Australian Soils is available from http://data.daff.gov.au/anrdl/metadata_files/pa_daaslr9abd_00111a01.xml Vegetation Greenness from SeaWiFs FAPAR --------------------------------------- FAPAR from SeaWiFS is a derived product available globally at ~0.04 deg spatial and monthly time resolution (Gobron et al. 2002), continuously from September 1997 to June 2006. For AWAP the data were resampled to 0.05 deg resolution and the data from complete years only (1998-2005) were averaged to create a monthly climatology from which a climatology of vegetation cover fraction was created for use here. Spatial Mask of Australia ------------------------- The WaterDyn model requires a full complement of parameters and meteorology, and cannot model land areas with missing or unspecified data. Features that are not modelled include lakes, salt lakes and salt pans (e.g. South Australia and coastal NW Western Australia, which are not specified in the soil parameter data. Model results should be plotted in conjunction with a standard coastline definition, such as Geoscience Australia's Geodata Topo 10M (http://www.ga.gov.au/meta/ANZCW0703005262.html). 6c) PERCENTILE RANKS -------------------------------------------------------- Two types of data are produced for each variable: absolute data and percentile rank data. Percentile ranks are designed to make dynamical variability clearer and show the rank of the data within the cumulative probability distribution for that month (weekly and monthly outputs), or for all years (annual outputs), over a standard climatological period, usually 1961 to 1990. Cumulative probability distributions and percentile ranks are calculated separately at each 0.05 deg grid cell. Because of their high dependence on solar radiation, percentile ranks for SolarMJ and FWPT are based on a 1990-2011 climatology, i.e. during the period of satellite operation. 7) DIRECTORY, FILENAME AND DATA FORMATS ------------------------------------------------------- The maps are publicly available at http://www.csiro.au/awap. The data files are available to authorised users only. The data files are distributed as zip files each containing the model outputs in physical units and percentile ranks for one quantity for one year. Zip files have the generic filename, yyyymmdd_YYYYMMDD.var.vv.fmt.zip where, yyyymmdd First date of the period (inclusive), usually yyyy0101. YYYYMMDD Last date of the period (inclusive), usually YYYY1231. var Variable name, between 4 and 7 characters (e.g. PhiE, WRel2, TempMax etc.). vv Version number, generally '00'. fmt File format, either flt (ESRI Float) or nc (NetCDF). There are two zip files for each quantity/year, one containing the data in ESRI Float and the other in NetCDF format. Within each flt.zip file, the data are given in ESRI binary raster format (for ArcGIS import) including two files, an IEEE floating-point file (.flt) with an accompanying ASCII header file (.hdr). The header file contains the information required to read the float file. An example header file is: ncols 813 # Number of columns nrows 670 # Number of rows xllcorner 112.9250 # Longitude of lower-left corner yllcorner -43.57500 # Latitude of lower-left corner cellsize 5.0000001E-02 # Cell or grid size (degrees) nodata_value -9999.000 # Null value byteorder lsbfirst # Byte order The generic filename for float and header files is, [pcr_]xxx_var_yyyymmdd.(flt,hdr) where, pcr_ Optional prefix that denotes percentile rank data. xxx Time period of the file. This can be 'ann' = annual 'mth' = monthly 'run' = particular to the type of WaterDyn model run, usually weekly or monthly. var Variable name. yyyymmdd Final date of the time period (inclusive). The same file naming conventions are used for the NetCDF files, except the suffix is .nc rather than .flt, and there is no header file. All the metadata is stored in the NetCDF file as part of the data dimension and coordinate variables, or else as attributes of the main data variable. Each NetCDF file has two global attributes: 1 :history which provides some key background information and pointers to further information (such as this Readme file). 2 :version which stores the data versioning information (see Sec. 10 below). 8) GEOGRAPHICAL COVERAGE ------------------------------------------------------- Data are unprojected, in geographic decimal degrees, referenced to GDA94 (equivalent to WGS84 for all practical purposes). The data have a spatial resolution of 0.05 degrees latitude and longitude (~5 km x 5km). Whole-degrees cross cell centres. The grid dimensions are given in the header files that accompany the float-format data files. The map images (*.gif) are generated from the float-format files but are not geo-referenced. The exact spatial coverage across the continent is determined by the model's mask, discussed above. 9) DATA QUALITY ------------------------------------------------------- The accuracy of the model results is subject to the limitations imposed by the assumptions and parameterisations inherent in the model, and the limitations of the parameter data and meteorology, in particular the sparseness of the sampling networks involved in their creation. In addition, suitable data for model comparison are available at only a limited number of sites. For a discussion of model testing and accuracy, and full details of the WaterDyn model, refer to the report Raupach et al. 2009. 10) VERSION NOMENCLATURE ------------------------------------------------------------------ The version of the WaterDyn model and the run number are the major versioning component of these data. The relevant model version can always be found in the top bar of the data series webpage. Run numbers are given for historical runs, but not specified for operational runs (which are identified by their date range). If you are accessing the data files then the model version and run number, when specified, will be included in the source directory name. At present the model version is not included in the data file, zip file nor map file names. The WaterDyn model version number, as given in the top bar of the data series webpage, is given as, WaterDynXXy where, XX Incremental version number. y Mode of operation where, 'M' = Forward model without model-data fusion 'P' = Model-data fusion using PEST 'K' = Model-data fusion using Kalman Filter 'G' = Model-data fusion using Genetic Algorithm. The run number, when specified, is usually of the form, RunXXz where, XX WaterDyn model version number (without the mode identifier). z Denotes a unique run of the model, usually incremental from 'a'. As mentioned in section 7, the zip files do have a version number but this usually relates to the operational processing of the weekly and monthly data. 11) VERSION RELEASE NOTES ------------------------------------------------------------------ Historical Run Updates May 2014 - WaterDyn 26M - Run 26j Historical Re-run and update of the historical series from 1900 to end of February 2014 with replaced or updated meteorology including: new recalibrated daily rainfall with monthly fill (2011-2014/02); daily minimum and maximum temperature (2011- 2014/02); and a complete redownload of solar radiation (1990-2014/02) with BoM bias recorrections applied from November 2010 and incorporated in the 1990-2011 monthly climatology used to create the synthetic daily solar radiation series for 1900-1989. Model Version: AWAP WaterDyn 26M (unchanged) Meteorology: BoM AWAP meteorology for Run 26j was downloaded on the following dates. Date Range-> 1900-1910 1911-1949 1950-1989 1990-2008 2009 2010 2011-2014/02 Solar Radiation Daily Synthetic Synthetic Synthetic Mar-2014 Mar-2014 Mar-2014 Mar-2014 Recal Rainfall Daily Mar-2010 Mar-2010 Mar-2010 Mar-2010 Nov-2010 Feb-2012 Mar-2014 Rainfall Monthly Mar-2010 Mar-2010 Mar-2010 Mar-2010 Nov-2010 Feb-2012 Mar-2014 Min/Max Temp Daily Synthetic Jun-2009 Jun-2009 Jun-2009 Nov-2010 Feb-2012 Mar-2014 - Rainfall, Temperature - Replacement of 2011 to ensure capture of data with full quality-control cycle applied (6 months rainfall, 3 months temperature). - Update to end of 2014/02 - Solar radiation - Replacement of entire solar radiation series 1990-2014/02 to ensure capture of bias-recorrected data (2010/11 to 2014/02) and any potentially undocumented improvements. Further information about the bias recorrection and its impact on AWAP soil moisture is available from www.csiro.au/awap. April 2012 - WaterDyn 26M - Run 26g Historical Summary: Re-run and update of the historical series from 1900 to end of 2011 with updated meteorology, including complete new and improved solar radiation (hence the complete rerun from 1900). Model Version: AWAP WaterDyn 26M (unchanged) Meteorology: BoM AWAP meteorology for Run 26g was downloaded on the following dates. Date Range-> 1900-1910 1911-1949 1950-1989 1990-2008 2009 2010 2011 Solar Radiation Daily Synthetic Synthetic Synthetic Feb-2012 Feb-2012 Feb-2012 Feb-2012 Recal Rainfall Daily Mar-2010 Mar-2010 Mar-2010 Mar-2010 Nov-2010 Feb-2012 Feb-2012 Rainfall Monthly Mar-2010 Mar-2010 Mar-2010 Mar-2010 Nov-2010 Feb-2012 Feb-2012 Min/Max Temp Daily Synthetic Jun-2009 Jun-2009 Jun-2009 Nov-2010 Feb-2012 Feb-2012 - Solar Radiation - BoM improvements - Buffer now applied to coastal zones and inland water bodies, eliminating reliance on Silo ground-based solar radiation for some gap-filling at continental margins - CSIRO changes - Update monthly climatology for gap-filling and synthetic series, now includes data from 1990-2011. January 2011 - WaterDyn 26M - Run 26c Historical Summary: Run 26c is significantly different from the previous Run 25a (and its interim update). These differences are entirely due to improvements in the forcing meteorology supplied by the Bureau of Meteorology, including the use of recalibrated rainfall for historical modelling. Erroneous percentile ranks for FWPT and SolarMJ, due to a bug in the post-processing software, have been corrected. WaterDyn26M (the model) is the same as WaterDyn25M except for the addition of new output products and minor fixes and efficiency improvements that have been tested and found to have no impact on the results. Input parameters are also unchanged. Model Version: AWAP WaterDyn 26M (changes from 25M, Run 25a): - Write new outputs WRel1End, WRel2End which are WRel1 and WRel2 at end of monthly averaging periods. WRel1 and WRel2 remain time-averaged over mth, ann, or run. - Calculate open-water evaporation FWWater [m/day] using Penman-Monteith equation, with open-water aerodynamic conductance GaeroW [m/s] set as a parameter. - Nullify pointers explicitly to fix memory leak. Meteorology: BoM AWAP meteorology for Run 26c was downloaded on the following dates Date Range-> 1900-1910 1911-1949 1950-1989 1990-2008 2009 2010-2011/2 Solar Radiation Daily Synthetic Synthetic Synthetic Jun-2009 Mar-2010 Mar-2011 Recal Rainfall Daily Mar-2010 Mar-2010 Mar-2010 Mar-2010 Nov-2010 Mar-2011 Rainfall Monthly Mar-2010 Mar-2010 Mar-2010 Mar-2010 Nov-2010 Mar-2011 Min/Max Temp Daily Synthetic Jun-2009 Jun-2009 Jun-2009 Nov-2010 Mar-2011 - Solar Radiation - Update monthly climatology for gap-filling and synthetic series, now includes 2008 data (1990-2008). - Rainfall - Rainfall replaced with recalibrated daily data, desert gaps filled with disaggregated monthly data. - Minimum and Maximum Temperature - Update b0905 (1911-2008) - Update b1011 (2009) - Monthly climatology for synthetic series updated from June 2009 data. Outputs - Write new outputs WRel1End, WRel2End, FWWater as noted above. - Corrected post-processing software error generating erroneous percentile ranks for FWPT and SolarMJ. Ancilliary Data - Data and maps of the spatial parameters used by AWAP WaterDyn are now available at the AWAP website. 12) UPDATES ------------------------------------------------------------------ Updates to the dataset will occur under two circumstances: a new WaterDyn model or set of parameters; arrival of a reprocessed set of meteorological data. In either case, once the WaterDyn model is deemed stable and the meteorological data have been verified, new century-long and operational (near-real time) datasets will be produced. Formal announcement of updated datasets will be via the website and by email to the project partners. 13) REFERENCES ------------------------------------------------------------------ Raupach MR, PR Briggs, V Haverd, EA King, M Paget, CM Trudinger (2009), Australian Water Availability Project (AWAP): CSIRO Marine and Atmospheric Research Component: Final Report for Phase 3. CAWCR Technical Report No. 013. 67 pp. Raupach MR, PR Briggs, V Haverd, EA King, M Paget, CM Trudinger (2008), Australian Water Availability Project. CSIRO Marine and Atmospheric Research, Canberra, Australia. . Date of access. Dilley AC, Edwards M, O'Brien DM, and Mitchell RM. 2000. Operational AVHRR processing modules: atmospheric correction, cloud masking and BRDF compensation. Aspendale: CSIRO Division of Atmospheric Research. (CSIRO Atmospheric Research internal paper; 14). 24 p. Gobron N, Pinty B, Mélin F, Taberner M, Verstraete MM (2002), Sea Wide Field-of- View Sensor (SeaWiFS) - an optimized FAPAR algorithm - theoretical basis document. JRC Publication No. EUR 20148 EN, Institute for Environment and Sustainability, Ispra, Italy, 20 pp. Grant I, D Jones, W Wang, R Fawcett, D Barratt (2008), Meteorological and remotely sensed datasets for hydrological modelling: A contribution to the Australian Water Availability Project. Proceedings of the Catchment-scale Hydrological Modelling & Data Assimilation (CAHMDA-3) International Workshop on Hydrological Prediction: Modelling, Observation and Data Assimilation, Melbourne, January 9-11 2008. Jones DA, Wang W, Fawcett R (2007), Climate data for the Australian Water Availability Project, Final Milestone Report. October 2007, 36pp. Jones DA, Wang W, Fawcett R (2009), High-quality spatial climate data-sets for Australia. Australian Meteorological and Oceanographic Journal 58:233-248 McKenzie NJ, Hook J (1992), Interpretations of the Atlas of Australian Soils. Technical Report 94/1992, CSIRO Division of Soils, Canberra. McKenzie NJ, Jacquier DW, Ashton LJ, Cresswell HP (2000), Estimation of soil properties using the Atlas of Australian Soils. CSIRO Land and Water Technical Report 11/00, CSIRO Land and Water, Canberra, Australia, 24 pp. Northcote KH, with Beckmann GG, Bettenay E, Churchward HM, Van Dijk DC, Dimmock GM, Hubble GD, Isbell RF, McArthur WM, Murtha GG, Nicolls KD, Paton TR, Thompson CH, Webb AA and Wright MJ (1960-1968), Atlas of Australian Soils, Sheets 1 to 10, With explanatory data. CSIRO Australia and Melbourne University Press, Melbourne. Schaaf CB, Gao F, Strahler AH, Lucht W, Li XW, Tsang T, Strugnell NC, Zhang XY, Jin YF, Muller JP, Lewis P, Barnsley M, Hobson P, Disney M, Roberts G, Dunderdale M, Doll C, d'Entremont RP, Hu BX, Liang SL, Privette JL, Roy D (2002) First operational BRDF, albedo nadir reflectance products from MODIS. Remote Sensing of Environment 83:135-148. 14) CONTACTS ---------------------------------------------------------------------------- May 2014 Dr Michael Raupach, Climate Change Institute, ANU michael.raupach@anu.edu.au Mr Peter Briggs, CSIRO Ocean Atmosphere Flagship peter.briggs@csiro.au