Ronald Brown IMET Data Quality Control Report: 2002

Jeremy J. Rolph and Shawn R. Smith Research Vessel Surface Meteorological Data Center Center for Ocean Atmospheric Prediction Studies Florida State University November 2003 Report 03-03 Version 1.0
1.0 Introduction

This report summarizes the quality of the surface meteorological data collected by the research vessel Ronald Brown (identifier: WTEC) IMET system during the 11 cruises beginning 15 January 2002 and endng 23 August 2002. The data were provided to the Florida State University - Research Vessel Surface Meteorlogical Data Center in ASCII format by John Shannahoff and converted to standard RVSMDC netCDF format. The data were preprocessed using an automated screening program, which adds quality control flags to the data, highlighting potential problems. Finally, the Data Quality Evaluator (DQE) reviewed the data and current flags, whereby flags were added, removed or modified according to the judgement of the DQE and other RVSMDC personnel. Details of the quality control procedures can be found in Smith et al. (1996). The data quality control report summarizes the flags for the Ronald Brown IMET surface meteorological data, including those added by both the preprocessor and the DQE.


2.0 Statistical Information

The Ronald Brown IMET data were received as one minute averages. Observations for the following variables were provided:


Time(TIME)
Latitude(lat)
Longitude(lon)
Platform Heading(PL_HD)
Plaltform Course(PL_CRS)
Platform Speed Over Ground(PL_SPD)
IMET Platform-Relative Wind Direction (14.12 m)(PL_WDIR)
IMET Platform-Relative Wind Speed (14.12 m)(PL_WSPD)
Earth-Relative Wind Direction (14.12 m)(DIR)
Earth-Relative Wind Speed (14.12 m)(SPD)
Platform-Relative Wind Direction (25.5 m)(PL_WDIR2)*
Platform-Relative Wind Speed (25.5 m)(PL_WSPD2)*
Earth-Relative Wind Direction (25.5 m)(DIR2)*
Earth-Relative Wind Speed (25.5 m)(SPD2)*
Atmospheric Pressure(15.16 m)(P)
Air Temperature(12.98 m)(T)
Sea Temperature (5.6 m)(TS)
Relative Humidity (12.98 m)(RH)
Long-wave Atmospheric Radiation (10.01 m)(RAD)**

* -Provided for all cruises with the exception of 02-G

**-Provided for cruises 02-H thru 02-K


3.0 2002 QC Results

A total of 2,285,161 values were evaluated with 116,771 flags added by the preprocessor and the DQE resulting in 5.11 % of the values being flagged for the 2002 cruises of the Ronald Brown. A breakdown of each 2002 cruise is provided in Table 1.

Table 1: Statistical Cruise Information

Cruise Identifier*Cruise DatesNumber of RecordsNumber of valuesNumber of FlagsPercent Flagged
02-A1/15 - 1/194,66884,0247670.91
02-B1/21 - 1/276,019108,3421,0210.94
02-C2/01 - 2/2431,827572,88623,6464.13
02-D3/03 - 3/076,074109,3325,3614.90
02-E3/10 - 3/1811,308203,54412,8396.31
02-F3/21 - 3/2285515,9302631.65
02-G3/25 - 3/262,08029,1202,3928.21
02-H6/18 - 6/2915,985303,71518,5376.10
02-I7/12 - 7/2619,562371,67826,2457.06
02-J7/29 - 8/1117,751337,26917,9495.32
02-K8/17 - 8/237,859149,3217,7025.16
*-assigned by RVSMDC to ease identification


3.1 Qualtiy Control Information

The quality of the 2002 IMET data from the research vessel Ronald Brown ranged from excellent to poor depending on the cruise and the variable. Table 2 details the distribution of the flags among the variables. A discussion of the flagged and removed variables follows.


Table 2: Number of Flags and Pecentage Flagged for Each Variable
VariableBEGHJKNSTotal Number of FlagsPercentage of Variables Flagged
TIME        00.00
LAT      51 510.04
LON      51 510.04
PL_HD       5945940.48
PL_CRS       9479470.76
PL_SPD       2252250.18
PL_WDIR   2   9169180.74
PL_WSPD   2 46 5185660.46
DIR 1,712 21,14016,694 13719,68515.87
SPD 1,275521,14222,565 17525,16420.29
PL_WDIR2   4 2 1,1291,1350.92
PL_WSPD2   2 35 6717080.57
DIR2 1,467 4 20,109 17621,75617.54
SPD2 14954 27,616 17127,94522.53
P   2 748 57550.61
T  162 7,143 67,1675.78
TS8  2 1,672 4702,1521.74
RH21 742 6,562 1226,7815.47
RAD**     164 71710.28
Total Number of Flags294,603100302,282103,3561026,269116,771
Percentage of All Values Flagged0.00*0.200.00*0.00*0.094.520.00*0.27

* -Percentages < 0.01

**-Only in the last four cruises (02-H thru 02-K)


3.1.1 Deleted Data

The DQE determined that there were no data in need of deletion.


3.1.2 Missing Data

Most of the cruises from 2002 experienced some missing data. Cruises that had missing data include: 02-B, C, D, E, H, I, J, and K. For the most part, the missing data is just a random minute or two undetectable when viewing the plots of the data. The only way to identify that there is missing data is to zoom in on the area or look at the raw data. This type of situation occurred during the 02-C, D, E, H, I, J, and K cruises.

There were also periods in which all of the variables' data went missing. This occurred during the 02-B cruise twice, the 02-H cruise twice, and finally the 02-K cruise. During the 02-B cruise, the data was missing in all of the variables on 21 January from 18:14 - 18:58 Z and again from 1:50 Z on 22 January until 21:44 Z the next day. On the 02-H cruise, the data was missing on 28 June from 0:56 - 1:46 Z and then the next day, 29 June, from 11:05 Z thru 12:00Z. There was data missing for the 02-K cruise from 13:53 - 19:52 Z on 19 August.

Other times, the missing data was isolated to one or two variables. This scenario happened during the 02-C, I, J, and K cruises. On the 02-C cruise the variable in which the data was missing was the platform-relative wind speed on 3 February. The platform-relative wind speed was also missing on the 02-I cruise, along with the platform-relative wind direction on 13 July from 5:45 - 12:00 Z. The first anemometer lost its data again on 19 July for 5 hours from about 6:30 Z until around 11:30 Z. The first anemometer's data was also missing on the 02-J cruise on 9 August from 12:46 - 18:06 Z. Finally, the 02-K cruise also had anemometer one's data missing on 20 August from 20:56 - 22:56 Z. No explanations for the data gaps were available to the DQE.

Note: for those periods in which the platform-relative winds' data are missing, the subsequent, calculated true winds are K-flagged due to their uncertainty.


3.2.0 Variable Flagging


3.2.1 Stair Stepping

Stair stepping of the navigaton data is an inhernt property of these variables due to the motion of the vessel. Stair stepping of the meteorological variables in response to the change in the vessel's platform-relative speed or platform-relative direction is often an indicator of questionable meteorological data values. Meteorological data readings in the absence of flow distortion (See section 3.2.2), should not relfect the motion of the vessel, and therefore, such values received the cautionary K-flag.

The true winds of the Ronald Brown often stair step and thus received the K-flag at these times. Some steps were not flagged because there were no visible links in other variables to the steps. Examples of steps in the true winds are noticeable in each cruise. For example, 3 February, during the 02-C cruise, both the true wind speed and direction step with the vessel's heading and course and were K-flagged appropriately. Another example of can be found during the 02-E cruise on 11 March in both the true wind variables in both anemometers.

Steps may be found in the temperature and relative humidity data related to the platform-relative speed. This occurred during all but the 02-F cruise. This may be related to deck heating and ventilation issues discussed in section 3.2.4.

Another variable that experienced stepping with the vessel's motion is the pressure. This occurred in the 02-G, H, and K cruises. This happened during the 02-G cruise on 25 March when it stepped with the platform-relative winds and navigation variables. During the 02-H cruise, it occurred on 22 June. Finally, on the 02-K cruise, it took place on 22 June, again stepping with the platform-relative winds and navigation data.

Still more variables to experience stepping were the radiation and sea temperature. For the radiation, this occurred in the 02-J cruise and took place on 4 August. The radiation steps match the changes in the navigaton data and may be due to shadows from the vessel affecting the sensor. The sea Temperature stepped during two cruises, the 02-I and J. This occurred on 21 and 22 July during the 02-I cruise, stepping with the navigation data, and 7 August during the 02-J cruise, stepping with the temperature data. This may be due to the change of location as the vessel may have traveled through a current.


3.2.2 Flow Distortion

Flow distortion was suspected to be a problem on the 2002 cruises of the Ronald Brown. Some flow distortion is inevitable. Flow distortion is the result of the wind flowing over and around the cargo and structure of a vessel relative to the location of the instruments. Since the cargo varies from cruise to cruise, it is very difficult to identify with only one anemometer as in cruise 02-G. The other cruises had identifiable flow distortion issues distinguishable by the differences in platform-relative wind speeds and directions between the two different anemometers and, also, the differences between the calculated true wind speeds and directions of the anemometers. Flow distortion occurred during the 02-A cruise on 15, 17, and 19 January. For the 02-B cruise, it took place on 23 January. The 02-C cruise had 5 occurrences of the problem: 1, 6, 7, 18, and 19 February. 02-D had noticeable flow distortion on 3 and 6 March and during the 02-E cruise it was on 11, 12, and 18 March. For the 02-F cruise, it took place on 21 March. There were many times during the 02-H cruise that flow distortion was apparent: 21-24 June and again on the 27th. There were 3 days in which the 02-I had recognizable problems with flow distortion, 17 July and 25-26 July. For example, on 26 July, between 10:35 Z and 10:56 Z, the difference in speed between the two anemometers is roughly 3 m/s. The 02-J cruise had this happen on 2 August and again on the 10th. Lastly, the 02-K cruise had these problems on 17, 20, and 22 August.


3.2.3 Winds

The quality of the wind data for this set varies from cruise to cruise. The first two cruises are of good quality with the exception of the second true wind direction on the 02-A cruise. The rest of the cruises' wind data are rarely under 10% flagged for both anemometers, and are usually 15 - 30% flagged. Most of the flagging of the true winds is due to stair stepping of the winds with the motion of the vessel. Some of the flags are the result of flow distortion. Almost each day in every cruise has flags on the true winds due to stair stepping. The use of the true winds should be done with caution, particularly on the latter cruises: 02-C, both bow and mast anemometers; 02-D, with the mast's anemometer; 02-E, both; 02-G, the bow; 02-H, both the bow and mast; 02-I, both the bow and mast; 02-J, both the bow and mast; and 02-K, also with the bow and mast anemometers.


3.2.4 Ventilation

An insufficiently ventilated thermometer can experience steep rises in the temperature in a relatively short period of time when the platform-relative wind speed is low or the flow over the instrument is blocked. A ventilation problem was apparent for both the temperature and the relative humidity on nine of the cruises with the exception of the 02-F and G cruises. The main pattern used to identify this problem is a relative temperature maximum during a period of a platform-relative wind speed minimum. During these situations, the relative humidity falls. The relative humidity is not always affected during these periods as it has its own sensor and reacts differently since it has a different time constant than the thermometer. Ventilation problems are most pronounced when the atmospheric radiation is at or near the daily maximum.

Notable ventilation issues are apparent on the following cruises: 02-A, B, C, D, E, H, I, J, and K. For the 02-A cruise, the day with the ventilation problem is 18 January. Ventilation problems during the 02-B cruise were experienced on 26 January. Several days of the 02-C cruise had visible ventilation problems, including: 2, 3, 11, 17, and 20 February. For the 02-D cruise, it experienced the problem on 7 March. The 02-E cruise had this occur on 10, 12, and 15-17 March. Four days of the 02-H cruise were affected by the ventilation problem and include: 20, 23, 24, and 26 June with the 24th having exceptional problems. The days during the 02-I cruise to have ventilation problems were 18-20 July and 23-25 July. 02-J only had the 2nd and 10th of August with the problem while the 02-K cruise had this happen on 20, 22, and 23 of August. For example, on 24 June there were several occasions in which the tmperature increased a half a degree Celsius in a minute or two while the radiation value was falling and the platform-relative wind speed was at a local minimum.


3.2.5 Navigation Data

The navigation data for the vessel experienced few problems for the available data. There were times, 12 February for example, where the course was very noisy. This leads to possible issues when determining the true winds as the course is an important variable in determining the true winds. Otherwise, the vessel did not have any notable navigation problems.


3.2.6 Port Data

It is possible, while the vessel is in port, for some of the meteorological variables to experience data values quite different than those over the open ocean, especially the sea temperature. When the values recorded in port are relatively out of trend with of the data recorded at sea, the values may be representative of real conditions in the port and thus are not K-flagged. The 02-C cruise had this occur with the sea temperature while it was in port on 1 February. The sea temperature dropped 2 degrees C as it left the port and thus the latitude and longitude received the newly implemented N-flag while the vessel was in port to indicate the possibility of unlikely values.


3.2.7 B Flag

The B flag is assigned to values falling outside of a realistic, acceptable range by the preprocessor. In some cases, the bounds flag highlights extreme natural events. For the 2002 cruises, 33 B-flags were assigned by the preprocessor and after visual QC, 29 B-flags remained. Only two of the cruises had B-flags assigned, the 02-C and the 02-K. They were assigned to the relative humidity during the 02-C cruise on 13 February as there were values recorded slightly over 100%. This is technically possible during superstauration, such as in a very dense fog, but this is a rare occurrence and these values should be used with caution. During the 02-K cruise, the B-flags were assigned to the sea temperature on 18 August. This occurred just off the coast of Savannah, GA, as the sea temperature was just over 30 degrees Celsius. This is feasible as the Gulf Stream, a warm water current, runs northward in this general vicinity, and at the time it is August, one of the hotter months of the year.


3.2.8 G Flag

There were G-flags assigned by the preprocessor to values greater than four standard deviations from the climatological mean (da Silva et al.,1994). The flagged values were just greater than the limit and may represent extreme, realistic values. There were G-flags assigned to the following 2002 cruises: B(2), C(8), I(74), and J(16) for a total of 100. Both of the G-flags were assigned to the true wind speed, one for each anemometer, during the 02-B cruise on 26 January. Both anemometers recorded wind speeds of 18 m/s, a realistic value just over the limit of four standard deviations and represent realistic extremes in the true winds. The G-flags for the 02-C cruise were distributed among the true wind speed for anemometer one (4), the true wind speed from anemometer two(3), and the relative humidity (1). For the first anemometer, one flag was on 8 February when the wind was 15 m/s, a high, yet realistic value. The other three flags occurred on 17 February when the wind was around 17 m/s, again, an extreme, but realistic value. The second anemometer was flagged on the 5th and 16th of February when the winds were at 14 m/s and 16 m/s respectively. These too, are just over the limit of the four standard deviation limit and are realistic. Finally, the lone G-flag found in the relative humidity data was on 23 February when the relative humidity fell to near 45% as the vessel was just off the coast of Venezuela. This is an extreme, although realistic, for the waters near the eqatorial Atlantic. The 02-I had G-flags in the relative humidity (73) and true wind speed from the second anemometer (1). The second true wind speed was 19 m/s when flagged and is an extreme realistic value for off of the East Coast of the Untited States. The relative humidity was flagged as the vessel was off the coast of Maine and was less than 50%, again representing a realistic value. Lastly, the 02-J cruise had G-flags on the air temperature data when the temperature was between 27 and 28 degrees Celcius just northwest of Cape Cod on 30 July. This is realistic for summertime in New England.


3.2.9 H Flags

The H flag is used to identify discontinuities, large, sudden shifts in the data time series. These occur for several reasons, such as electrical interference, although a return of the values to their original trend may not take place. There were a total of 32 discontinuity flags among the cruises. They occurred on the following 2002 cruises: C, E, H, and J. All of the meteorological variables except the sea temperature on 17 March experienced a discontinuity at the same time without a return to their former trend. This was most likely due to electrical interference as all of the variables jump at the same time. On 19 June, the second anemometer's true wind direction was discontinuous as it suddenly changed from 100 to 70 degrees in one minute and was completely out of the trend of the rest of the data. The other meteorological variables continue their previous trend at the time of the discontinuity in the mast true wind directrion. During the 02-J cruise, the platform-relative wind direction from the second anemometer and true wind speed, also from the mast anemometer, were discontinuous at the same time on 7 August while the sea temperature had the problem on the 9th. The speed jumped from 3 m/s to 8 m/s in one minute and was not experienced by the other anemometer as the platform-relative wind direction changed from 20 to 60 degrees again in one minute, again out of the trend of the other data from the time. The sea temperature from 9 August fell from 29.9 to 25.2 degrees Celsius in just one minute. This is very unlikely in a natural setting as the cooling of water that much in such a short period of time would require a great amount of energy due to the large heat capacity of water.


3.2.10 Data Spikes

Isolated spikes occurred in most of the variables of the data set. Spikes are a common occurrence with automated data and can be caused by various factors (e.g. electrical intereference, ship movement, etc.). These points were assigned the S-flag. All of the variables of 2002 had some spikes with the exception of time, latitude, and longitude.

Acceleration spikes are often in the data due to the movement of the ship and therefore the instrument. They are often found as the vessel is changing speed and/or direction (Smith et al., 1999). They are visible as spikes where the time series levels off, yielding continued accelerating motion, i.e. turning, speeding up or slowing down. The main variables with acceleration spikes are the platform-speed, platform heading, platform course, and the platform-relative winds. The true winds do exhibit acceleration spikes to a lesser degree. The spikes are propagated into the true winds since they are calculated from the navigation data and platform-relative winds, which often have acceleration spikes.


3.2.11 Sea Temperature Problems

There were few problems in the sea temperature data. It did have times where it was highly variable (18 July) although the vessel could have been traveling through thermal currents. Other minor issues experienced by the sea temperature were the difference between the port and open ocean data covered by the N-flag, and the odd stepping of data with the platform-relative winds.


3.3.0 Final Comments


3.3.1 Winds and Overall Quality

The majority of the flagging of the 2002 cruises of the Ronald Brown was due to the significant amount of stair stepping by the meteorological data. Even though all of the meteorological data experienced stair stepping, overall, they proved to be of good to excellent quality. The winds were of particularly poor quality with over 15% of each true wind variable being flagged. Most of the flags were the result of stair stepping of the data with the motion of the vessel and flow distortion. The remaining meteorological variables and navigational variables were of much better quality and averaged only about 1 % of the data being flagged although the temperature and relative humidity were on the order of 5.6 %.


3.3.2 Insufficient Data

In parts of each of the cruises, the DQE would like to note that some of the data may have been left unflagged because of insufficient meteorological backing due to the lack of other data. In some cases, there was not enough evidence to say whether certain questionable data should have been flagged. It is very possible that some of the data left unflagged on these cruises are questionable and should be used with caution.


References:

da Silva, A.M., C.C. Young and S. Levitus, 1994: Atlas of Surface Marine Data 1994, Volume 1: Algorithms and Procedures. NOAA Atlas Series.

Smith, S.R., C. Harvey, and D.M. Legler, 1996: Handbook of Quality Control Procedures and Methods for Surface Meteorology Data. WOCE Report No. 141/96 , Report WOCEMET 96-1, Center for Ocean-Atmospheric Prediction Studies, Florida State University, Tallahassee FL 32306-2840.

Smith, S.R., M.A. Bourassa, and R.J. Sharp, 1999: Establishing More Truth in True Winds. J. Atmos. Oceanic Technol., 16, 939-952.