James Clark Ross Multimet Data Quality Control Report
Jennifer Lovell and Shawn R. Smith
World Ocean Circulation Experiment
WOCE Surface Meteorological Data Analysis Center
Center for Ocean Atmospheric Prediction Studies
Florida State University
August 6, 2001
Report WOCEMET 01-12
Version 1.0
INTRODUCTION:
This report summarizes the quality of surface meteorological data collected by the research vessel James Clark Ross (identifier: ZDLP) during two cruises completed in 1996, 1997, and 1998. The data were provided to the Florida State University Data Assembly Center (DAC) in multimet electronic format by D. M. Gould (BODC) and were converted to standard DAC netCDF format. The data arrived from the British Oceanographic Data Center (BODC) already quality controlled and included the BODCís own unique set of flags (e.g. G-good data, B-bad data, I-interpolated value which is assumed to be good, S-suspect data, N-null or absent value). Upon arrival, these flags were converted to WOCEMETís quality control guidelines (e.g. Z-good data, J-bad data, R-interpolated value which is assumed to be good, Q-suspect data from previous quality control, Z-null or absent value (good data)). The data were then processed using an automated screening program, which added quality control flags to the data, highlighting potential problems. Finally, the Data Quality Evaluator (DQE) reviewed the data and current flags (both DAC and BODC), whereby flags were added, removed, or modified according to the judgment of the DQE and other DAC personnel. Details of the quality control procedures can be found in Smith et al. (1994). The data quality control report summarizes the flags for the James Clark Ross meteorological data, including those added by the BODC, the WOCEMET preprocessor, and the DQE.
DATA VARIABLES:
The James Clark Ross data are expected to include observations averaged once every minute on these cruises. Values for the following variables were collected:
Time Latitude Longitude Platform Heading Platform Course Platform Speed Platform Relative Wind Direction* Platform Relative Wind Speed* Earth Relative Wind Direction* Earth Relative Wind Speed* Sea Temperature** Atmospheric Pressure Air Temperature Downwelling Short Wave Radiation Photosynthetically Available Radiation |
(TIME) (LAT) (LON) (PL_HD) (PL_CRS) (PL_SPD) *(PL_WDIR) *(PL_WSPD) *(DIR) *(SPD) **(TS) (P) (T) (RAD) (RAD2) |
*Earth relative wind direction (DIR) and earth relative wind speed (SPD) were not included in the public release of the data for the SR_01_/13 cruise. The calculated earth relative winds (true winds) were assessed many E-flags. E-flags show that the data failed the resultant wind recomputation check. A failed wind test occurs when the recomputed wind direction is less than 20 degrees and the recomputed wind speed is less than 2.5 ms-1 (Smith). After careful inspection of both variables, it was determined by WOCEMET to discard DIR and SPD, as no scientific reason was found for the discontinuities.
Platform relative wind direction (PL_WDIR) and platform relative wind speed (PL_WSPD) were also omitted from the publice release of the data for the SR_01_/13 cruise. True wind direction and speed are calculated from the platform heading (PL_HD), platform course (PL_CRS), platform speed over ground (PL_SPD), and the platform relative winds (PL_WDIR and PL_WSPD). When the true wind direction and speed can not be calculated within the guidelines above from the above variables, the DAC typically finds problems in one or more of these five necessary parameters. It was determined by WOCEMET that the PL_HD, PL_CRS, and PL_SPD were recorded correctly, but the PL_WDIR and PL_WSPD contained an unknown problem, perhaps the wind vane was rotated 180 degrees. Insufficient metadata and contact with the data provider could not resolve the problem on the SR_01_/13 cruise, so WOCEMET decided to discard PL_WDIR and PL_WSPD. These variables were quality controlled and the information is available, but not included with this report.
**Sea temperature (TS) was not a recorded variable during the 97/98 James Clark Ross SR_01_/13 cruise.
1996 FLAG SUMMARY
Statistical Information:
Details of the 1996 cruise are listed in Table 1 and include the cruise dates, number of records, number of values, number of flags, and total percentage of data flagged. A total of 476,070 values were evaluated with 14,729 flags added by both the preprocessor and the DQE resulting in a total of 3.09% of the values being flagged.
Table 1: Statistical Cruise Information
Cruise Identifier |
Cruise Dates |
Number of Records |
Number of Values |
Number of Flags |
Percent Flagged |
SR_01_/11 |
11/10/96 - 12/05/96 |
31,738 |
476,070 |
14,729 |
3.09 |
Summary:
The overall 1996 multimet data from the James Clark Ross proves to be of good quality with 3.09% of the reported values flagged for potential problems. Note: R-flags are not necessarily bad data, but simply interpolated data. Therefore R-flags were not included in the overall data quality. The distribution of flags for each variable are detailed in Table 2.
Table 2: Number of Flags and Percentage Flagged for Each Variable
Variable |
B |
E |
G |
K |
Q |
R |
S |
Total Number of Flags |
Percentage of Variable Flagged |
TIME LAT LON PL_HD PL_CRS PL_SPD PL_WDIR PL_WSPD DIR SPD TS P T RAD RAD2 |
6 |
4 |
1,357
6 |
1,822 300
2,201 2,479 |
3,528 2 2 152 131 1 5 973
1,718 |
9 9 |
24 |
0 9 9 3,534 2 2 152 131 5 1,851 2,630 0 1,724 2,201 2,479 |
0.00 0.03 0.03 11.13 0.01 0.01 0.48 0.41 0.02 5.83 8.29 0.00 5.43 6.93 7.81 |
Total Number of Flags |
6 |
4 |
1,363 |
6,802 |
6,512 |
18 |
24 |
14,729 |
|
Percent of All Values Flagged |
0.00* |
0.00* |
0.29 |
1.43 |
1.37 |
0.00* |
0.01 |
3.09 |
*Percentages< 0.01
B-flags
Platform heading (PL_HD) received six B-flags on the SR_01_/11 cruise for recorded values below zero. All flagged values recorded were -10.
E-flags
Earth relative wind direction (DIR) received four E-flags on data that failed the wind recomputation test.
G-flags:
Note: During the SR_01_/11 cruise, the ship traversed south of 40-60 degrees South Latitude. In this region of the globe, little information is known about the climatology, as the data is sparse. Consequently, the G-flagged data values may be realistic, though extreme observations.
Sea temperature (TS) had 1,357 G-flags over the SR_01_/11 cruise. Overall, the flagged sea temperatures were approximately six to seven degrees Celsius greater; therefore, the DQE feels these values are realistic, though extreme, sea temperatures.
Temperature (T) was assessed six G-flags for values that were approximately seven to nine degrees greater than the climatological value.
The G-flags were left in place to emphasize values that are greater than four standard deviations from the climatological mean (da Silva et al. 1994).
K-flags:
The K-flag represents suspect data and should be used with caution. Throughout the SR_01_/11 cruise, numerous data were assessed the K-flag. The most significant use of the K-flag was to reveal signatures of ship motion in the variables. Variables such as earth relative wind speed (SPD), atmospheric radiation (RAD), and photosynthetically available radiation (RAD2) showed stair steps in the data. These stair steps are related to a change in platform course (PL_CRS), heading (PL_HD), and/or platform speed (PL_SPD) and should not exist in earth relative data. Subsequently, the data were flagged as suspect.
The earth relative wind speed (SPD) had stair steps occurring throughout the data set. The cause was likely due to flow distortion. Flow distortion is the disturbance of airflow from other objects or instruments upstream from the anemometer. The significance of the stair stepping varied throughout the data set; therefore, the earth relative winds should be used with caution.
Sea temperature (TS) received 301 K-flags on 12/01/96. All meteorological data were missing from 11/24/96 through 11/30/96, when TS resumed operation it recorded values that were extremely highl, on the order of 16 degrees Celsius near the Antarctic Peninsula. Sea temperatures near 60 degrees South latitude should not be this high thus, the data were flagged as suspect. Extremely high sea temperatures following missing data may be the result of a possible instrument malfunction.
During the SR_01_/11 cruise, atmospheric radiation (RAD) and photosynthetically available radiation (RAD2) were assessed a total of 4,680 K-flags. When ship's heading ranged from 140 to 180 degrees, the radiation values would decrease. This change in radiation was due to a potential shadowing problem associated with the ship's position. The time of the year, November and December, along with the ship's position at sea, 60 degrees South latitude near Antarctica, reveals the potential for a shadowing problem. Note: The DQE id noy certain exactly where on the ship the radiation sensors are located. Knowing the exact location is essential to verify a shadowing problem. Occasionally, the radiation values would increase with a change in the ship's heading, but a likely cause is unknown.
Q-flags:
Data from the SR_01_/11 that were deemed suspect by the BODC were assessed Q-flags by WOCEMET.
R-flags:
Several R-flags were assessed on the SR_01_/11 cruise to latitude (LAT) and longitude (LON). Interpolated values are interpolated by the data provider, BODC, and are assumed to be good data.
Spikes:
The BODC evaluated several spikes. Additional spikes were identified during visual inspection by the DQE and they were assigned the S-flag. These spikes are a relatively common occurrence with automated data, caused by various factors (e.g. electrical interference, ship movement, etc.).
97/98 FLAG SUMMARY
Statistical Information:
Details of the 97/98 cruise are listed in Table 1 and include the cruise dates, number of records, number of values, number of flags, and total percentage of data flagged. A total of 269,000 values were evaluated with 6,108 flags added by both the preprocessor and the DQE resulting in a total of 2.27% of the values being flagged.
Table 1: Statistical Cruise Information
Cruise Identifier |
Cruise Dates |
Number of Records |
Number of Values |
Number of Flags |
Percent Flagged |
SR_01_/13 |
12/18/97 - 01/05/98 |
26,900 |
269,000 |
6,108 |
2.27 |
Summary:
The overall 97/98 multimet data from the James Clark Ross proves to be of excellent quality with 2.27% of the reported values flagged for potential problems. Note: R-flags are not necessarily bad data, but simply interpolated data. Therefore R-flags were not included in the overall data quality. The distribution of flags for each variable are detailed in Table 2.
Table 2: Number of Flags and Percentage Flagged for Each Variable
Variable |
B |
K |
Q |
R |
Total Number of Flags |
Percentage of Variable Flagged |
TIME LAT LON PL_HD PL_CRS PL_SPD P T RAD RAD2 |
6 |
334 1,710 1,433 2,590 |
15 |
10 10 |
10 10 6
334 1,725 1,433 2,590 |
0.00 0.04 0.04 0.02 0.00 0.00 1.24 6.41 5.33 9.63 |
Total Number of Flags |
6 |
6,067 |
16 |
20 |
6,108 |
|
Percent of All Values Flagged |
0.00* |
1.87 |
0.00* |
0.01 |
2.27 |
*Percentages< 0.01
B-flags:
Platform heading (PL_HD) received six B-flags on the SR_01_/13 cruise for recorded values below zero. All flagged values recorded were -10.
K-flags:
The K-flag represents suspect data and should be used with caution. Throughout the SR_01_/13 cruise, numerous data were assessed the K-flag. The most significant use of the K-flag was to reveal signatures of ship motion in the variables. Variables such as earth relative wind speed (SPD), atmospheric radiation (RAD), and photosynthetically available radiation (RAD2) showed stair steps in the data. These stair steps are related to a change in platform course (PL_CRS), heading (PL_HD), and/or platform speed (PL_SPD) and should not exist in earth relative data. Subsequently, the data were flagged as suspect.
Stair steps in the pressure (P) data were a result of a change in either forward speed or direction. These stair steps were associated with approximately a 1/4 millibar (mb) increase or decrease in pressure relative to both the forward speed and direction change of the ship.
Temperature received 1,710 K-flags due to the radiational heating of the ship. When the platform relative wind speed was low, ~4 ms-1 or less, significant increases of approximately one or two degrees Celsius in temperature were occurring during daylight hours. During this time, the increases in temperature were flagged as cautionary.
During the SR_01_/13 cruise, atmospheric radiation (RAD) and photosynthetically available radiation (RAD2) were assessed a total of 4,023 K-flags. When ship's heading ranged from 140 to 180 degrees, the radiation values would decrease. This change in radiation was due to a potential shadowing problem associated with the ship's position. The time of the year, December and January, along with the ship's position at sea, 60 degrees South latitude near Antarctica, reveals the potential for a shadowing problem. Note: The DQE is not certain exactly where on the ship the radiation sensors are located. Knowing the exact location is essential to verify a shadowing problem. Occasionally, the radiation values would increase with a change in the ship's heading, but a likely cause is unknown.
Q-flags:
Data from the SR_01_/13 that were deemed suspect by the BODC were assessed Q-flags by WOCEMET.
R-flags:
Several R-flags were assessed on the SR_01_/13 cruise to latitude (LAT) and longitude (LON). Interpolated values are interpolated by the data provider, BODC, and are assumed to be good data.
FINAL DISCUSSIONS:
Special attention should be made to variables affected by Q and R flags, as WOCEMETsí DQE did not assign these flags and therefore, did not thoroughly discuss in this document the reasons for their use.
On the SR_01_/11 cruise, all meteorological data were missing from 11/24/96 through 11/30/96.
REFERENCES:
Smith, S.R., C. Harvey, and D.M. Legler, 1994: Handbook of Quality Control Procedures and Methods for Surface Meteorology Data. Report No. 141/96, Report MET 96-1, Center for Ocean-Atmospheric Prediction Studies Florida State University, Tallahassee FL 32306-2840
da Silva, A.M., C.C. Young and S. Levitus, 1994: Atlas of Surface Marine Data 1994, Volume 1: Algorithms and Procedures. NOAA Atlas Series.