=============================================================================== NCEP Real-time Marine Observations README: 1 April 1997 =============================================================================== 1. Introduction Monthly and weekly files of surface marine data gathered from the Global Telecommunications System (GTS) by NOAA's National Centers for Environmental Prediction (NCEP), and provided courtesy of Richard W. Reynolds at NCEP, become available at NCEP two days after the data month or week. These are basic observational data (individual marine reports from ships, buoys, etc.) in a simple ASCII format containing selected weather elements. The data are largely unedited, although one major change in processing became effective with March 1997 data (see section 4). Data for the immediately preceding month and week are available at NCEP; monthly files back to January 1991 are available from NOAA's Environmental Research Laboratories (ERL) Climate Diagnostics Center (CDC). The data are also used at CDC to construct monthly summaries for 2-degree latitude x 2-degree longitude boxes. For additional information about these summary products, please refer to http://www.cdc.noaa.gov/cgi-bin/CDCData.pl. Copies of the monthly data, and the 2-degree summaries, become available at CDC about 2-5 days following each data month. Section 2 gives detailed information about ftp access to the data at NCEP or CDC, and Section 3 provides the format description. 2. Availability via anonymous ftp Although the basic data are in a simple ASCII format (section 3), the files are stored and transferred via ftp in compressed form. Thus, depending on the computer type to which you are transferring the data, binary mode should generally be used for ftp transfers. The data at NCEP are compressed using the "gzip" utility; to ensure more widespread availability, the data at CDC have been recompressed using "Z" compression. (To uncompress "filename.gz" on a Unix system, type "gunzip filename.gz"; or to uncompress "filename.Z", type "uncompress filename.Z".) a) To obtain data for the immediately preceding month or week: ftp nic.fb4.noaa.gov (user: anonymous; password: your e-mail address) cd /pub/ocean/clim1/gts The monthly file is "gtsobs.gz" and the weekly file (i.e., Sunday through Saturday) is "gtswkly.gz". These gzip'd files are overwritten on the second day of each month and week, respectively. Contact: Diane Stokes W/NP24, Room 807 4700 Silver Hill Road Stop 9910 Washington, DC 20233-9910 (301) 763-8000 x7581 wd01dm@sgi26.wwb.noaa.gov b) Data back to January 1991 (monthly file "nqYYMM.Z", where YY and MM are the year and month, e.g., "nq9601.Z"): ftp ftp.cdc.noaa.gov (user: anonymous; password: your e-mail address) cd Public/coads/ncep_obs Contact: Scott Woodruff NOAA/ERL (R/E/CD) 325 Broadway Boulder, CO 80303-3328 (303) 497-6747 sdw@cdc.noaa.gov 3. NCEP GTS surface marine data ASCII format This is a simple ASCII format with each record 49 characters in length (Table 1). The records are generally sorted by date, but not necessarily by hour (some variations may exist due to the way the data are gathered). Note that present weather, complete cloud elements, and various other minor fields have been omitted from the format. Table 1. The NCEP GTS surface marine data ASCII format. Report type, BUFR file type, and the wind speed indicator are defined in Tables 2-4. NOTE: Distinguishing platform type according to report type or BUFR file type is not always reliable. The most frequent discrepancies involve moored and drifting buoys, for which it is best to examine the third digit in the 5-digit (numeric) buoy ID: a value less than 5 is a moored buoy; a value greater than or equal to 5 is a drifting buoy. ------------------------------------------------------------------------------- Column Format Field from to =============================================================================== 1 2 I2 Year 3 4 I2 Month 5 6 I2 Day 7 10 F4.2 Hour 11 15 F5.2 Latitude (degrees) -- negative = southern hemisphere 16 20 F5.2 Longitude (degrees) -- 0 to 360 West 21 22 I2 or 2I1 Report type; or BUFR file type and wind speed indicator* 23 28 A6 Station ID/ship call sign 29 32 F4.1 Sea level press minus 900.0 (mb) missing=9999 33 35 I3 Wind direction (degs) missing=999 36 38 I3 Wind speed (knots) missing=999 39 42 F4.1 Air temp (deg c) missing=9999 43 45 F3.1 Dew point depression (deg c) missing=999 46 46 A1 Cloud cover** missing=9 47 49 F3.1 Sea surface temp (deg c) missing=999 ------------------------------------------------------------------------------- * Prior to March 1997, positions 21-22 contain Office Note 124 report type (Table 2); starting during March 1997 (see section 4), position 21 contains BUFR file type (Table 3), and position 22 contains the wind speed indicator (Table 4). Presently, all wind speed indicators are missing=9 (planned for future addition to the BUFR data). ** Prior to 4 August 1991 this contains low cloud amount; starting then it contains total cloud amount. ---------- Table 2. Office Note 124 report types. Until March 1997 (see section 4), positions 21-22 contain report type minus 500 (this field may be unreliable as an indicator of platform type; see Table 1). ------------------------------------------------------------------------------- Code Type of Report =============================================================================== ---------Ocean Station------- 521 Fixed (OWS) 522 Moving ship with name 523 Moving ship without name ---------Marine reporting station (MARS)--------- 531 Fixed (Stationary) 532 Moving ---------Quality control data--------- 551 By latitude-longitude* ---------Buoy--------- 561 Moored 562 Drifting ------------------------------------------------------------------------------- * This is listed as a valid "report type" in NCEP documentation, but no reports of this type have ever been encountered. ---------- Table 3. BUFR file type. Starting in March 1997 (see section 4), position 21 contains the type of BUFR file from which the report was obtained (this field may be unreliable as an indicator of platform type; see Table 1). ------------------------------------------------------------------------------- Code BUFR file type =============================================================================== 1 ship 2 drifting buoy 3 moored buoy 4 Coastal-Marine Automated Network (C-MAN) ------------------------------------------------------------------------------- Table 4. The wind speed indicator (starting in March 1997), showing the units in which and the method by which wind speed was originally recorded (WMO code 1855). All winds are stored in this format in knots. ------------------------------------------------------------------------------- Code =============================================================================== 0 meter per second, estimated 1 meter per second, measured (obtained from anemometer) 3 knot, estimated 4 knot, measured (obtained from anemometer) 9 missing ------------------------------------------------------------------------------- 4. Technical notes a) Original weekly files that existed in some of the earlier years have been consolidated into single monthly files for retention at CDC. b) The original files for January-12 February 1991 were supplemented with some ATLAS buoy data that may be unavailable in the regular NCEP archives for that time period. c) During ftp, a small amount of data was truncated from the original January 1991 file "nq9101.Z," which was also used for construction of the existing 2-degree summary files. Missing data, except possibly the supplementary ATLAS data noted above, have been recovered in the file "nq9101_complete.Z." Similarly, days 1-8 were inadvertently omitted, and days 9-16 duplicated, from the original June 1991 file "nq9106.Z," also used for construction of the existing 2-degree summary files. Missing data have been recovered in the file "nq9106_complete.Z." d) Starting 1 March 1997, data are available processed by NCEP into the BUFR format (file "nq9103_bufr.Z"). In addition, overlapping data were processed into NCEP's previous Office Note (ON) 124 format until 31 March ("nq9703.Z"). Due to concerns about temperature biases in initially available BUFR data, the ON file was used to construct the 2-degree summaries for March 1997. e) In March 1997 we became aware that ON data from moored buoy 52312 reported close to 180 longitude were sporadically impacted by a change in the latitude sign, such that latitudes ~2S were changed to ~2N. Data for March 1997 were manually edited (only in the "nq9703.Z" file; not in the original "gtsobs.gz" file) by changing positive latitudes to negative with the same magnitude. No previous data were corrected, and the scope of the problem is not fully known. File "nq9703_52312" contains all reports from 52312 (edited plus unedited). f) "Dup/merge" software is used at NCEP in processing the BUFR data to combine partially complete duplicate reports. Duplicate reports were not merged in the ON data; only the most recent report was kept. g) Delayed data that are not retained in these files, and marine reports containing the full suite of observed weather elements, eventually become available in the full NCEP GTS archives stored at the National Climatic Data Center (NCDC) in Asheville, and at the National Center for Atmospheric Research (NCAR) in Boulder. The full marine reports, plus a variety of delayed data, are merged into the Comprehensive Ocean-Atmosphere Data Set (COADS). For more information about COADS including current data availability, please see our COADS homepage (http://www.cdc.noaa.gov/coads/).