Thompson Data Quality Control Report
Cruises: P__14N/01
P__14N/02
P__10_/00
P__31_/02
ISS02_/02
ISS02_/03
ISS02_/06
ISS02_/08
ISS02_/10
IR_03_/02
Daniel M. Gilmore and Shawn Smith
World Oceanic Circulation Experiment(WOCE)
Surface Meteorological Data Assembly Center
Center for Ocean Atmospheric Prediction Studies
The Florida State University
August 19, 1996
Report WOCEMET 96-8
Version 1.0
Introduction:
The data referenced in this report were collected from the research vessel Thompson (call sign: WSRY, data provider: Neil Bogue, School of Oceanography, University of Washington) IMET automated data collection system from each of 10 seperate cruises (2 of which overlap) for WOCE. All data were recieved in electronic format. The data were converted to the FSU standard format and then pre-processed using an automated data checking program. Next a visual inspection was completed by a data quality evaluator (DQE) who reviewed, modified, and added appropriate quality control(QC) flags to the data. Details of the WOCE QC procedures can be found in Smith et al. (1996). The data quality control report summarizes the flags for the Thompson IMET data, including flags added by both the pre-processor and the analyst.
Statistical Information:
The data from the Thompson were expected to include observations every minute from 9 cruises. The start and end dates, the number of observations, and the number and percentage of non-Z flags for each cruise is given in table 1. Time (TIME), latitude (LAT), longitude (LON), platform heading (PL_HD), platform speed over water (PL_SPD), platform course (PL_CRS), platform speed over ground (PL_SPD2), earth relative wind direction (DIR), earth relative wind speed (SPD), atmospheric pressure (P), air temperature (T), sea temperature (TS), relative humidity (RH), atmospheric radiation (RAD), rainrate in millimeters/minute (RRATE), and rainrate in millimeters/hour (RRATE2) were quality controlled. A total of 5,935,690 values were checked with 108,481 flags added resulting in 1.83 percent of the values being flagged. The distribution, including percentages flagged for each variable sorted by type, is detailed in table 2.
Table 1: List of dates and number of records and flags for each of the cruises
| CTC | Dates | Number of Records | Number of Values | Number of Flags | Percent Flagged |
| P__14N/01 | 07/06/93-08/12/93 | 52,673 | 895,441 | 36,896 | 4.02 |
| P__14N/02 | 08/13/93-08/31/93 | 25,531 | 434,027 | 4,567 | 1.05 |
| P__10_/00 | 10/05/93-11/09/93 | 51,462 | 874,854 | 13,868 | 1.59 |
| P__31_/02 | 01/25/94-02/18/94 | 34,349 | 583,933 | 874 | 0.14 |
| ISS02_/02 | 01/08/95-02/03/95 | 31,628 | 537,676 | 73 | 0.01 |
ISS02_/03
IR_03_/02 | 03/13/95-04/09/95 | 38,920 | 661,640 | 2,555 | 0.39 |
| ISS02_/06 | 05/03/95-05/19/95 | 23,621 | 401,557 | 12,694 | 3.16 |
| ISS02_/08 | 07/17/95-08/15/95 | 40,840 | 694,280 | 16,891 | 2.43 |
| ISS02_/10 | 08/17/95-09/15/95 | 40,349 | 685,933 | 17,066 | 2.49
|
Table 2: Number of Flags and Percentage Flagged for Each Variable
Variable |
B |
F |
G |
H | I | J | K | L | M | S | T | Total | Percentage of Total Data Flagged |
| TIME | | | | | | | | | | | 49 | 49 | 0.01 |
| LAT | | 274 | | | | | | 133 | | | | 407 | 0.11 |
| LON | | 274 | | | | | | 133 | | | | 407 | 0.11 |
| PL_HD | 1 | | | 4 | | 101 | 82 | | | | | 187 | 0.05 |
| PL_SPD | | | | | | 102 | 91 | | | 3 | | 196 | 0.05 |
| PL_CRS | | | | 4 | | 101 | 83 | | | | | 188 | 0.05 |
| PL_SPD2 | 22 | | | | | 1326 | | | | 1 | | 1349 | 0.37 |
| DIR | | | | 4 | | 1885 | 14919 | | | 431 | | 17239 | 4.67 |
| SPD | | | | 4 | 5 | 2652 | 50774 | | | 26 | | 53461 | 14.48 |
| P | | | 4424 | | 1 | 595 | 14411 | | 1215 | | | 20646 | 5.59 |
| T | | | 32 | | 9 | 624 | 1003 | | 1215 | 12 | | 2895 | 0.78 |
| TS | | | 68 | | 9 | | 5847 | | | 18 | | 5940 | 1.61 |
| RH | | | 438 | 11 | | 557 | 1295 | | 1214 | | | 3505 | 0.95 |
| RAD | 4 | | | | | 481 | 292 | | 1215 | | | 1992 | 0.54 |
| RRATE | | | | | | | | | | | | 0 | 0.00 |
| RRATE2 | | | | | | | | | | 7 | | 7 | 0.00 |
| Total: | 27 | 548 | 4962 | 19 | 18 | 8424 | 88797 | 266 | 4859 | 498 | 49 | 36954 | 1.73 |
| Percentage of data flagged | 0.00 | 0.01 | 0.07 | 0.00 | 0.00 | 0.12 | 1.27 | 0.00 | 0.07 | 0.01 | 0.00 | 1.73 | |
B: Data Point out of Bounds
F: Unreal Ship movement
G: Data point >4 standard deviations from climatological mean
H: Discontinuity in data
I: Interesting data point
J: Erroneous data point
K: Caution/Suspect Data
L: Ship position over land
M: Mechanical Instrument Failure
S: Spike in data
T: Time not Sequential
Summary of Flags:
A. Significant problems
Normally for IMET vessels, the data assembly center(DAC) receives only winds relative to the ship along with the necessary navigation values--platform heading or wind compass, platform speed over ground and platform course over the ground. From these, the DAC computes true winds using the method described in smith et al. (1996).
For the Thompson, PL_HD, PL_SPD2, PL_CRS, PL_WDIR, and PL_WSPD were used to calculate all DIR and SPD values. The results were less than ideal, however. As is illustrated in example 1, both DIR, at 4.04% of the data, and SPD, at 13.75% of the data, retained a signal of the ship's movement. This is the reason for the high percentage of "K" flags added to DIR and SPD. Since our calculation method has been verified, other speculations for this result are left to the user.
(Example 1)
There were not any other significant problems with this data set. Of the 14,412 "K" flags added to pressure(P), all are a result of a signature of the ship's movement within the pressure observations, especially during the P__14N/01 cruise. We are assuming that the barometer was placed in a location such that when the ship was moving, an additional pressure was exerted by the oncoming air on the barometer. This caused the barometer to read higher when the ship was moving with a signature difference between 0.1mb and 2.0mb. The problem, however, seems to have been remedied before the P__14N/02 cruise as it only rarely reappears during any following cruises. No confirmation of instrument malfunction was available to date.
During preprocessing, 4,424 "G" flags were added to atmospheric pressure(P). These are due to the extreme low pressure events that occurs during times when the Thompson is near the Antarctic coast. For pressure to vary more than 4 standard deviations from climatology during one of these events is not uncommon. However, since the flags are more descriptive than cautionary, the analyst left the flags to indicate these low pressure events.
Atmospheric pressure, air temperature, relative humidity, and atmospheric radiation all have 1,215 "M" flags because the sensors recorded all observations of these variables as the same for 22 hours on 07/07/93 - 07/08/93. A similar occurance happened on 07/17/95 except that the sensors didn't go dead for an extended period of time. Rather they would transmit 1 value for many continuous observations, then transmit another value for more continuous observations with steps of 0.5mb - 1mb each time. This could indicatae on-board calibration, but no confirmation was available. These values, all 481 of them on that day for P,T,RH, and RAD, were flagged with a "J". Erroneous data, "J", flags were also added to PL_HD, PL_SPD, PL_CRS, and PL_SPD2 mainly due to the same value being transmitted for an extended period of time.
B.) Other Cautionary Flags:
- DIR had 431 "S" flags added and SPD had 26 "S". These spikes were mostly a result of the wind vane or anemometer recording inacurate measurements when the ship changed heading.
- TS had 5,847 "K" flags added because the sea temperature was exactly the same from 07/22/95 - 07/25/95.
- TS also had 68 "G" flags added due to low sea temperatures.
- RH had 1,295 "K" flags added because of readings that were at 100% for extended periods of several minutes or more.
- RH also had 438 "G" flags added by the preprocessor when RH was far below climatology.
- T had 1003 "K" flags added due mainly to 2 problems:
- On 07/25/95 and 08/28/95, the air temperature plateaus and remains constant for several minutes. This should not happen with data being recorded to the 0.01 degC.
- Then, on 08/14/95, the temperature, in about a 3 hour period, goes through several cycles of smooth increases and decreases between 30 and 35 degC.
- RAD had 292 "K" flags added because of plateauing patterns similar to that of T.
- Nine "I" flags were added to each T and TS due to rapid temperature changes.
For example, on 08/02/93 2 "I" flags are added to T because the air temperature drops 4 degC in 10 minutes. In addition to these, there were a relatively insignificant number of "B", "F", "G", "H", "L" and "S" flags that one would expect with any large data set.
Several corrections were made to the data prior to conversion to NetCDF
format. These changes are outlined in Appendix A.
Final Note:
These data are in very good condition and, with the exception of the problems noted above, the analyst foresees no difficulty in using this data.
References:
Smith, S.R., C. Harvey, and D.M. Legler, 1996: Handbook of Quality Control Procedures and Methods for Surface Meteorology Data. WOCE Report No. 141/96, Report WOCEMET 96-1, Center for Ocean Atmospheric Prediction Studies, Florida State University, Tallahassee, FL 32310.
Appendix A
An outline of all changes made to these data before conversion to NetCDF format
Type of Error
(Correction Made) | Dates & Times Occurred
(Dates are DD/MM/YY format) |
Missing Latitude Value
(Line was deleted) | 29/03/95 @ 14:22 |
Additional Characters
(Characters were deleted) | 01/02/95 @ 17:31
14/05/95 @ 03:55 |
Latitude and Longitude in incorrect format
(Format of lat and lon was adjusted) | 18/09/94 @ All Times
19/09/94 @ All Times
21/09/94 @ All Times
22/09/94 @ All Times
24/09/94 @ 0:00-12:34 |
Data Not in Correct Format for Read
(Additional comma delimiters added for missing rain data) | 11/10/94 @ 0:00-04:52 |
Data Not in Correct Format for Read
(Additional comma delimiters added for missing wind data) | 11/10/94 @ All Times
12/10/94 @ 0-14:28 |
Values input into the data in incorrect type.
(Values were changed from integer to real by adding a .0 to the end.) | 15/01/95 @ 09:11
15/01/95 @ 09:39
17/01/95 @ 17:58
18/01/95 @ 08:08
18/01/95 @ 12:19
18/01/95 @ 18:50
19/01/95 @ 13:53
21/01/95 @ 08:40
23/01/95 @ 05:07
27/01/95 @ 00:02
27/01/95 @ 05:14
27/01/95 @ 10:32
27/01/95 @ 12:14
27/01/95 @ 15:34
31/01/95 @ 16:50
01/02/95 @ 19:23
01/02/95 @ 19:47
01/02/95 @ 22:09
03/02/95 @ 01:31
03/02/95 @ 01:58
21/02/95 @ 00:31
24/02/95 @ 02:34
24/02/95 @ 02:39
24/02/95 @ 03:19
21/03/95 @ 00:00
23/03/95 @ 13:38
26/03/95 @ 20:11
30/03/95 @ 18:38
02/04/95 @ 11:09
02/04/95 @ 12:05
08/05/95 @ 02:14
14/05/95 @ 13:01
14/05/95 @ 13:09
14/05/95 @ 22:17
26/06/95 @ 02:09
26/06/95 @ 08:17
26/06/95 @ 09:11
30/06/95 @ 10:59
01/07/95 @ 00:40
02/07/95 @ 17:59
03/07/95 @ 12:24
08/07/95 @ 15:34
08/07/95 @ 17:19
08/07/95 @ 17:20
08/07/95 @ 18:45
08/07/95 @ 23:45
09/07/95 @ 04:50
09/07/95 @ 05:37
09/07/95 @ 19:08
20/07/95 @ 10:43
20/07/95 @ 22:17
28/07/95 @ 02:15
30/07/95 @ 06:28
30/07/95 @ 15:15
01/08/95 @ 06:59
01/08/95 @ 09:06
01/08/95 @ 12:01
01/08/95 @ 14:40
02/08/95 @ 23:30
04/08/95 @ 01:30
04/08/95 @ 05:04
04/08/95 @ 10:13
04/08/95 @ 17:17
04/08/95 @ 20:48
05/08/95 @ 17:03
06/08/95 @ 14:16
06/08/95 @ 14:56
06/08/95 @ 16:59
07/08/95 @ 05:59
08/08/95 @ 06:11
08/08/95 @ 23:58
10/08/95 @ 07:34
10/08/95 @ 10:53
11/08/95 @ 21:29
13/08/95 @ 08:47
13/08/95 @ 10:50
13/08/95 @ 17:01
20/08/95 @ 02:31
30/08/95 @ 18:57
30/08/95 @ 22:49
31/08/95 @ 12:12
31/08/95 @ 16:50
02/09/95 @ 17:24
09/09/95 @ 14:07
10/09/95 @ 05:52
14/09/95 @ 22:08
05/10/95 @ 13:49
07/10/95 @ 23:00
08/10/95 @ 04:18
08/10/95 @ 18:55
16/10/95 @ 03:51 |