Knorr IMET Data Quality Control Report

Cruises: P__06E/00
P__06C/00
P__06W/00
P__14C/00
P__31_/01
P__17A/00
P__16A/00
P__17E/00
P__19S/00
P__19C/00
AR_11_/11
AR_15_/12
A__15_/00

Daniel M. Gilmore and Shawn Smith



World Ocean Circulation Experiment(WOCE)

Surface Meteorological Data Assembly Center

Center for Ocean-Atmospheric Prediction Studies

The Florida State University



April 25, 2002


Report WOCEMET 96-7


Version 7.0



 



 

Addendum:

On January 15, 2002, WOCEMET found a discrepancy in the pressure (P) flags on cruises PR_32_/01, P__17A/00, P__17E/00, P__06W/00, and P__19C/00. The pressure data contained large, negative values with (Z-good data) flags. WOCEMET's standard for any negative pressure value (unrealistic in all cases) is to flag the value with a bounds (B) flag. Pressure flags were changed from Z flags to B flags for sixteen values on these 5 cruises. On one instance (P__06W/00; 07/14/92), a pressure value was recorded at 0.00 millibars, the flag on this erroneous data value was changed from Z to J (erroneous value).

Members of the WOCE Hydrographic Project Office (WHPO) and WOCEMET met at the 13th Data Products Committee (DPC) meeting in College Station, TX to discuss reconciliation of the WOCE cruise line designators. This was done in anticipation of the future release of version 3 of the WOCE cruise line designations.

On December 21, 2000 WOCEMET removed the WOCE designation for cruise AR_11_/11. The quality control information for this data has been left in this report for the user, but please note that the line previously known as AR_11_/11, is NOT a WOCE cruise line.

On June 19, 2001 it was brought to WOCEMET's attention that the dates table (Table 1) and statistics table (Table 2) in this report were not accurate. The tables have been updated below, Table 1.1 and Table 2.1 are shown with correct information. Note: True wind direction and true wind speed are NOT included in any of the statistics for the new tables (Table 1.1 and Table 2.1).

On September 10, 2001 WOCEMET updated the cruise track code P__31_/01 to PR_32_/01.

On April 25, 2002, WOCEMET discovered seven extreme values during the P__17A/00, 10/22/92 cruise that had not previously been flagged as out of range (B-flags) for earth relative wind speed (SPD). The wind speeds are in excess of 88 m/s and are now flagged with B-flags. The corresponding earth relative wind directions for the same times have been re-flagged with K-flags to show caution to the user. These new flags are not a part of the statistical information in this report.


 



 


Table 1.1: Dates and flags added for each cruise
CTCDatesNumber of RecordsNumber of ValuesNumber of FlagsPercentage Flagged
P__06E/0005/02/92-05/26/923382840593622815856.21
P__06C/0005/30/92-07/06/9254720656640478417.29
P__06W/0007/13/92-07/30/9225920311040121513.91
P__14C/0009/01/92-09/15/9221527258324113524.39
P__31_/0009/16/92-09/27/92171532058362461311.96
P__17A/0010/06/92-11/02/92391324695846502013.85
P__16A/0011/03/92-11/26/92335294023484447211.05
P__17E/00
P__19S/00
12/04/92-01/22/9364021768252624818.13
P__19C/0002/22/93-03/17/9333655403860205195.08
AR_15/10*03/18/93-04/13/9338036456432129652.84
A__15_/0004/02/94-04/13/94159631915566790335.45

*The DAC determined that the AR__15/10 line was never completed. The data listed from 3/18/93 to 4/13/93 does not contain variables, DIR or SPD and are considered to be the continuation of the P__19C/00 cruise. These remaining days were a transit cruise to Jacksonville, Florida.


 



 

Table 2.1: Percentage of Flags Assigned by Flag Type and Variable

VariableBFGIJKLSTotal Number of FlagsPercentage of data Flagged
TIME00.00
LAT213253854552035624414.90
LON211354917552075729215.18
PL_CRS153996175401414.31
PL_SPD227352074866185323114.10
DIR00.00
SPD00.00
P1178173711550459340679.02
T57309151207255855615.51
T2583613151425988015.86
TS1510260906160041229213424.41
RH12771528228070.74
PRECIP79362221252104092.76
RAD10641957916629211884131.48
Total:1229734245195224182643259721110859597475
Percentage of Flags Used2.710.094.310.00*5.840.210.00*0.0213.19

*:
Percentages< 0.01
B:
Data point out of bounds
F: Unreal platform movement
G: Data point >4 standard deviations from climatological mean
I: Interesting data point
J: Erroneous data point
K: Caution/Suspect Data
L: Platform position over land
S: Spike in data

Note: Causes of these flags are not discussed in the addendum, but explanations can be found later in this paper.


 



 

Introduction:
The data referenced in this report were collected from the research vessel Knorr (call sign: KCEJ; data provider: Woods Hole Oceanographic Institute/B. Walden) IMET automated data collection system from 13 different cruises. All data were received in electronic format and converted to the FSU standard format. They were then preprocessed using an automated data checking program. Next a visual inspection was completed by a Data Quality Evaluator who reviewed, modified and added appropriate quality control (QC) flags to the data. Details of the WOCE QC can be found in Smith et al (1996). The data quality control report summarizes the flags for the Knorr data, including those added by both the preprocessor and the analyst.


 



 

Statistical Information:
The data from the Knorr were expected to include observations every minute from each of 13 cruises. The cruise track code (CTC), the start and end dates, the number of records, number of observations, and the number of flags for each cruise are given in table 1. Time (TIME), latitude (LAT), longitude (LON), atmospheric pressure(P), air temperature(T), humidity temperature(T2), sea temperature(TS), relative humidity(RH), precipitation (PRECIP), and atmospheric radiation (RAD) were quality controlled. A total of 3,621,450 values were checked with 513,824 flags added resulting in 14.19 percent of the data being flagged. The distribution of flags, including the percentages flagged for each variable by type is given in table 2.


 



 

Table 1: Dates and flags added for each cruise

CTCDatesNumber of RecordsNumber of ValuesNumber of FlagsPercentage Flagged
P__06E/0005/02/92-05/26/923382854124824492145.25
P__06C/0005/30/92-07/06/925472087552016735419.11
P__06W/0007/13/92-07/30/92259204147205244812.65
P__14C/0009/01/92-09/15/92215273444325152314.96
P__31_/0009/16/92-09/27/92171532744486680124.34
P__17A/0010/06/92-11/02/923913262611214237622.74
P__16A/0011/03/92-11/26/92333975343526787112.70
P__17E/00
P__19S/00
12/04/92-01/22/9362581100129615968615.95
P__19C/0002/22/93-03/17/933365553848015513928.81
AR_15/10*04/02/93-05/03/934369969918420425629.21
AR_11_/1106/13/93-06/30/93258344133444423710.70
AR_15_/1204/02/94-04/13/941596325540810510341.15
                  A__15_/00


 

 


Summary:
After completing this report, the DAC determined that the AR__15/10 line was never completed. The data listed from 4/2/93 to 4/13/93 are the continuation of the P__19C/00 cruise. The remaining days were a transit cruise to Jacksonville, Florida. As stated in Smith et al (1996), whenever a vessel reports only ship winds to the data assembly center(DAC) as well as the other 3 necessary values--platform heading, platform speed over ground, and platform course over ground, the DAC computes true winds using the method described above and places the true wind values in the WOCE data files.

Table 2: Percentage of Flags Assigned by Flag Type and Variable


VariableBFGIJKLSTotal Number of FlagsPercentage of data Flagged
TIME00.00
LAT222854143552075663313.80
LON220955202552145768014.05
PL_CRS00.00
PL_SPD200.00
DIR00.00
SPD00.00
P1158173811552559340788.30
T58455152461296096014.85
T2596013151426112014.89
TS1510260920160041339215922.45
RH12771528228070.68
PRECIP79362221268104252.54
RAD12564057916629213806233.64
Total:1419004437197634181601048855110766513824
Percentage of Flags Used3.920.125.460.004.420.240.000.0214.19

B:
Data point out of bounds
F: Unreal platform movement
G: Data point >4 standard deviations from climatological mean
I: Interesting data point
J: Erroneous data point
K: Caution/Suspect Data
L: Platform position over land
S: Spike in data


The Knorr IMET data contained the needed parameters, but 89% of the wind compass data(a proxy for heading) was 0.0 degrees. This is unrealistic for a ship spending 3 months at sea cruise, so we consider this data to be erroneous. With erroneous data, true wind speed and direction cannot be calculated by the DAC. Thus, the following parameters have been omitted from the version 100 data files and this summary: wind compass, wind vane, platform relative wind direction, platform relative wind speed. True wind direction and true wind speed are in the final version as missing.

Despite the high percentage of flags, these data are in moderately good shape. One major problem with these data is that at random intervals, the values for LAT, LON, PL_CRS, and PL_SPD are 0.0 for extended periods of time. For cruises located in the Pacific Ocean, a 0.0 latitude, 0.0 longitude position is not
possible, and holding a course at 0 at a speed of 0 is unlikely, so these data have been flagged as "J", erroneous data.

Other issues include the 197,634 "G" flags the prescreener added to the data. These were applied to P, T, T2, TS, and RH for values that were significantly below or above the climatological mean. These flags were left by the analyst as an indication of statistically extreme values.

In addition, the prescreener added 141,900 "B" flags to the data. RAD was assigned 125,640 "B" flags. Due primarily to a likely calibration problem with the radiation sensor. At night the pyranometer routinely recorded values less than 0.0 W/m2, which is the lower bound for solar radiation data. TS was also flagged with "B" flags 15,102 times due to sea temperature values that were below 0.0oC. These values occur around the coast of Antarctica where, due to salinity features of the ocean, the sea temperature can fall below the freezing point. P was flagged 1,158 times due to very low pressures that occured in this same region. The "B" flag is applied when the atmospheric pressure falls below 950mb. This in not an uncommon occurrence near 60o S. However, all "B" flags were left by the analyst to highlight these low pressure events.

The 6,629 "K" flags were added to RAD in response to a specific pattern in the data. The RAD for one day would show a normal dirunal cycle, with about 14 hours of sunlight. The next day, the cycle would show about 6-7 hours of sunlight, with radiation readings at or within 1 W/m2 of 0.0 W/m2 for the remaining time. The obvious conclusion is that the sensor is malfunctioning. There is no corroborating evidence that this is the case, plus the sensor works well the rest of the time, so this data cannot be marked with "M" or "J" flags.

Of the 2,221 "K" flags added to PRECIP, 1440 are a result of precipitation data on 02/23/93 that shows the syphon emptying at 20mm (normally it empties at 50mm). The emptying is not complete, however, as the level within the syphon does not go to 0. Instead, it goes to about 4mm and then shows a noise range as wide as 8mm. The other flags were added on 04/07/94 where the level of the rain in the syphon is once again questionable.

Spikes were applied most often to PRECIP. These are mostly the result of data readings going to 0 for 1 data point, then returning to the previous pattern, but they can also be caused by noise that ranged too far from the normal noise pattern. Spikes are also prevalent in LAT and LON. These are from ship positions that are reported as 0o lat-0o lon. Only the points that deviate from the pattern are flagged with "S". The rest are left as they were prescreened, with the "F", platform movement unrealistic flag. The spikes in the rest of the variables are not a result of any pattern or problem in the data. Rather they are spikes that are common to any electronically recorded data set.

The only significant flags left to discuss are the "I" flags. These were added to T or T2 anytime a drastic temperature change occurred.


 



 

Final Note:
These data are in fairly good condition. Providing that the user employs the flag information, he should experience no difficulty in utilizing this data.


 



 

References:
Smith, S.R., C. Harvey, and D.M. Legler, 1996: Handbook of Quality Control Procedures and        Methods for Surface Meteorology Data. WOCE Report No. 141/96, Report WOCEMET        96-1, Center for Ocean Atmospheric Prediction Studies, Florida State University,        Tallahassee, FL 32310.