HadIOD SST reference data (SIRDS)

Announcements:

06/09/24 - SIRDS GTMBA2 files have now been added and the original SIRDS GTMBA files have been retired (stopped March 2024). See below for more information.

Introduction

The Sea Surface Temperature (SST) reference data extracted from HadIOD.1.2.0.0 (hereafter ‘reference data’) were initially created for the ESA SST CCI II project (European Space Agency Sea Surface Temperature Climate Change Initiative Phase 2, https://climate.esa.int/en/projects/sea-surface-temperature) for use in validating satellite SST data products. The reference data coverage was then extended for C3S (European Union Copernicus Climate Change Service, https://climate.copernicus.eu/, C3S_312a Lot 3) and later the UK Marine and Climate Advisory Service (MCAS). In September 2024, an updated set of GTMBA data were added for use in the ESA SST CCI project Phase 4. The data are officially referred to as the SIRDS (SST CCI Independent Reference Data Set) reference data.

SST reference data were initially made available to project users but have subsequently been released to users outside the project on request. These webpages provide information for those interested in this dataset. Please contact us if you are interested in using these data.

File description

Reference data files are in NetCDF format, the current version is v1.0. Data cover 1978-present with new data added with approximately a few months delay (a small amount of GTMBA2 data is also included for Nov-Dec 1977). Each NetCDF file contains monthly SST data from a particular platform type. Platforms selected for the reference dataset are: drifters, voluntary observing ships (ship), Global Tropical Moored Buoy Array (GTMBA), mooring data (excluding GTMBA), bottles, Conductivity-Temperature-Depth casts (CTDs), Mechanical BathyThermographs (MBTs), eXpendable BathyThermographs (XBTs), Argo floats and animal data (these are data from instrumented marine animals, e.g. elephant seals).

Files are named SSTCCI2_refdata_PLAT-TYPE_YYYYMM.nc, where YYYY=year, MM=month and PLAT-TYPE=platform type, e.g.:

SSTCCI2_refdata_animal_201701.nc.gz

SSTCCI2_refdata_argo_201701.nc.gz

SSTCCI2_refdata_argosurf_201701.nc.gz

SSTCCI2_refdata_bottle_199001.nc.gz

SSTCCI2_refdata_ctd_201701.nc.gz

SSTCCI2_refdata_drifter_201701.nc.gz

SSTCCI2_refdata_drifter_cmems_201701.nc.gz

SSTCCI2_refdata_gtmba_201701.nc.gz

SSTCCI2_refdata_gtmba2_201701.nc.gz

SSTCCI2_refdata_mbt_199001.nc.gz

SSTCCI2_refdata_mooring_201701.nc.gz

SSTCCI2_refdata_ship_201701.nc.gz

SSTCCI2_refdata_xbt_201701.nc.gz

Each file contains the observed data along with HadIOD metadata including bias corrections, measurement uncertainties and quality control decisions for each observation. The files contain several different bias correction and uncertainty variables, it is not possible to specify values for all of these variables for every observation and unspecified values are set to missing data. At a minimum we try to provide a quality flag and some estimate of uncertainty in each measurement.

For detailed information see 'File contents' below.

Note that files may not be available for every platform type each month because of changes in the usage of different platform types over time. Plots of the number of observations in the files per month for each platform type are shown in 'Figures' below.

User guide and supplementary files

Information about the reference data files is provided on these webpages. We would also strongly recommend reading the product user guide available for the HadIOD files which is of relevance here, in particular Section 2 which gives useful general information about HadIOD data and, importantly, describes the HadIOD observation error model and how to work with the bias corrections and uncertainties provided by HadIOD. Because the names used for the bias correction and uncertainty variables in the reference data files differ from the variable names used in the product user guide, the 'File Contents' section below includes a table showing which variable names are equivalent.

The bias corrections used in the reference data files are the same as those used for the HadIOD 'main' files. The HadIOD dataset also includes supplementary files that provide additional sets of bias corrections to help users explore the bias correction uncertainty. The component of uncertainty explored by these corrections is increasingly important on longer space and time scales (e.g. if concerned with decadal trends). The use of these additional corrections is encouraged and is described in the product user guide. These supplementary files could also be used with the reference data files and are linkable via the OB_ID variable.

Please contact us if you need help.

Data sources

Observations from several different sources have been ingested into the HadIOD.1.2.0.0 database. The following sources are used in the reference data files:

ICOADS.2.5.1 - [Woodruff et al., 2011]. The primary source of surface ocean measurements in HadIOD. These data are included from January 1850 to March 2022 and provide sea surface temperature measurements (not salinity) to HadIOD.

Acknowledgement: Research Data Archive/Computational and Information Systems Laboratory/National Center for Atmospheric Research/University Corporation for Atmospheric Research, et al. 1984, updated monthly. International Comprehensive Ocean-Atmosphere Data Set (ICOADS) Release 2.5, Individual Observations. Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory. https://doi.org/10.5065/D6H70CSV. Accessed 2011 onwards.
ICOADS.3.0.2 - https://icoads.noaa.gov/. Used in HadIOD as a substitute for ICOADS.2.5.1 from April 2022 onwards (the last monthly update of ICOADS.2.5.1 was expected in March 2022 [pers. comms. ICOADS team]). ICOADS.3.0.2 (2015-present) extends the coverage of ICOADS.3.0 (1662-2014) [Freeman et al., 2017]. The ICOADS.3.0.2 near real time observations are based on blended marine observations in the TAC and BUFR formats from NOAA's National Centers for Environmental Information (NCEI) GTS collections. The ICOADS.2.5.1 near real time observations are based on GTS observations from NOAA’s National Centers for Environmental Prediction (NCEP). Users should be vigilant for any impacts caused by the transition from ICOADS.2.5.1 to ICOADS.3.0.2 e.g. we note an increase in the number of ship observations across the transition.

ICOADS.3.0.2 data covering January 2015-March 2022 was later added into HadIOD, but only for use in the gtmba2 reference data files. This was to benefit from the inclusion of BUFR data in ICOADS.3.0.2 when creating an updated second version of the gtmba reference data. This was done to meet a specific project requirement and does not constitute a new version of HadIOD.

Acknowledgement: Research Data Archive/Computational and Information Systems Laboratory/National Center for Atmospheric Research/University Corporation for Atmospheric Research, Physical Sciences Laboratory/Earth System Research Laboratory/OAR/NOAA/U.S. Department of Commerce, Cooperative Institute for Research in Environmental Sciences/University of Colorado, National Oceanography Centre/University of Southampton, Met Office/Ministry of Defence/United Kingdom, Deutscher Wetterdienst (German Meteorological Service)/Germany, Department of Atmospheric Science/University of Washington, Center for Ocean-Atmospheric Prediction Studies/Florida State University, and National Centers for Environmental Information/NESDIS/NOAA/U.S. Department of Commerce. 2016, updated monthly. International Comprehensive Ocean-Atmosphere Data Set (ICOADS) Release 3, Individual Observations. Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory. https://doi.org/10.5065/D6ZS2TR3. Accessed 2015 onwards.
EN.4.2.0 / EN.4.2.1 / EN.4.2.2 - [Good et al., 2013]. EN4 is the primary source of ocean profile data (and profile quality flags) in HadIOD. These data are included from 1900-present and provide temperature and salinity measurements on depth (also potential temperature) to HadIOD. From November 2017 the switch was made to using EN4.2.1 as EN4.2.0 was retired. From the perspective of HadIOD this switchover should be seamless with no discernible impact on the data. From March 2021 the switch was made to using EN.4.2.2 ahead of EN.4.2.1 being retired. This switch is also expected to be relatively seamless though users should be aware there may be a small impact on the corrections provided in the XBT reference data files.

Acknowledgement: EN.4.2.[0,1,2] data were obtained from https://www.metoffice.gov.uk/hadobs/en4/ and are © Crown Copyright, Met Office, 2021, provided under a Non-Commercial Government Licence http://www.nationalarchives.gov.uk/doc/non-commercial-government-licence/version/2/.

EN4 acknowledges the datasets contributing to its own construction (for full details see the EN4 webpages) and which HadIOD benefits from. These are:

World Ocean Database 2013 [Boyer, T. P., et al. (2013), World Ocean Database 2013, in NOAA Atlas NESDIS 72, edited by S. Levitus and A. Mishonov (technical editor), 209 pp.; http://www.nodc.noaa.gov/OC5/WOD13/]. Note that EN.4.2.2 uses data from World Ocean Database 2018 (WOD18) but this WOD18 data is not included in HadIOD.1.2.0.0 because of the use of EN.4.2.2 from March 2021 onwards only.
Global Temperature and Salinity Profile Project [U.S. National Oceanographic Data Center: Global Temperature–Salinity Profile Programme. June 2006. U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Oceanographic Data Center, Silver Spring, Maryland, 20910, https://www.ncei.noaa.gov/products/global-temperature-and-salinity-profile-programme].
Argo [Argo data were collected and made freely available by the International Argo Project and the national initiatives that contribute to it (https://argo.ucsd.edu, https://www.ocean-ops.org). The Argo Program is part of the Global Ocean Observing System].
Arctic Synoptic Basin-wide Observations project [Takamasa Tsubouchi at National Oceanography Centre, Southampton, U.K.].

NOAA PMEL GTMBA - http://www.pmel.noaa.gov/tao/. Used to provide high temporal sampling resolution SST data from the tropical moored buoy arrays to HadIOD. A download of this archive was made on 21/09/16 providing data from the 1980s to this date for use in the gtmba reference data files.

On 03/07/24 a second download was made for use in the gtmba2 reference data files. For this download all available high resolution SST data were downloaded (beginning November 1977) regardless of sampling period (10-minutely to 8-hourly), with the aim of boosting coverage, particularly prior to the early-1990s. This was done to meet a specific project requirement to create an updated second version of the gtmba reference data and does not constitute a new version of HadIOD.
E.U. Copernicus Marine Environmental Monitoring Service (CMEMS) drifting buoys - product INSITU_GLO_NRT_OBSERVATIONS_013_030 (Global Ocean In Situ Near Real Time Observations), http://marine.copernicus.eu/. Daily updates of global NRT observations from the main global networks from 2010-present provided by Coriolis (France). CMEMS drifting buoy sea surface temperature data are included in the HadIOD database from February 2015 to present.

Acknowledgement: HadIOD is generated using E.U. Copernicus Marine Service Information.
Met Office MetDB - in November 2016, surface buoy data (drifting and moored) were absent from the ICOADS.2.5.1 Real Time updates. GTS buoy data from the Met Office Meteorological Database system were used in HadIOD as a substitute.

Terms & Conditions / Citing the data

The SST reference data were originally extracted from HadIOD.1.2.0.0 for the ESA SST CCI II project and later C3S (C3S_312a Lot 3), the UK Marine and Climate Advisory Service (MCAS) and the ESA SST CCI project Phase 4. They are officially referred to as the SIRDS (SST CCI Independent Reference Data Set) reference data. If publishing work using SIRDS reference data, please cite the data as follows:

'The observations used are taken from the SIRDS (SST CCI Independent Reference Data Set) reference data originally created for the European Space Agency (ESA) SST CCI project [https://climate.esa.int/en/projects/sea-surface-temperature] and later extended for the European Union Copernicus Climate Change Service [https://www.copernicus.eu/en, C3S_312a Lot 3], the UK Marine and Climate Advisory Service (MCAS) and the ESA SST CCI project Phase 4. The SIRDS was extracted from HadIOD [Atkinson et al., 2014] version 1.2.0.0, see https://www.metoffice.gov.uk/hadobs/hadiod/sirds.html'

Atkinson, C. P., N. A. Rayner, J. J. Kennedy and S. A. Good, 2014. An Integrated Database of Ocean Temperature and Salinity Observations, Journal of Geophysical Research: Oceans, 119, 7139-7163, doi:10.1002/2014JC010053

If applicable, for traceability, please describe which HadIOD bias corrections were used. This information can be found in the file global attribute ‘source’. This is particularly important if any additional corrections from supplementary files were used. See also 'how to cite the data' in the HadIOD product user guide for more information and references for bias corrections. For example:

'Bias corrections provided by HadIOD using the Gouretski and Reseghetti [2010] MBT/XBT correction scheme and ship ‘Mean Ensemble Member’ correction scheme were applied. An alternate set of XBT bias corrections were also applied provided by HadIOD using the Cowley et al. [2013] correction scheme.'

HadIOD is subject to Crown copyright protection and is provided under the Open Government License v3.

Users must also acknowledge the following:

'HadIOD SIRDS data are available from the Met Office and are © British Crown Copyright, Met Office, [year of first publication], provided under an Open Government License, http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/'

Contact us

Please contact us if you need any assistance.

File contents

File contents are summarised in the tables below.

Some platform type specific file information:

GTMBA reference data are a blend of (mainly) delayed mode SST data taken from NOAA PMEL supplemented by quality controlled ICOADS GTS data. ICOADS data are often sub-sampled to hourly whereas NOAA PMEL data are provided at their native sampling frequency (mainly 10-minutely and hourly) which may benefit satellite validation activities. NOAA PMEL data were originally downloaded on 21/09/16 to create the gtmba reference data. A second download of the NOAA PMEL data was made on 03/07/24 to create the gtmba2 reference data. The ICOADS contribution to the GTMBA datasets increases in the years leading up to the download date and provides all the GTMBA data after the download date (see 'Figures').

For the second download of NOAA PMEL data used in the gtmba2 files, all available high resolution SST data were downloaded (beginning November 1977) regardless of sampling period (10-minutely to 8-hourly), with the aim of boosting coverage, particularly prior to the early-1990s. In addition, in the gtmba2 reference data: (1) ICOADS.3.0.2 data were used in place of ICOADS.2.5.1 from 2015-onwards to potentially benefit from the inclusion of GTS BUFR data (the orginal gtmba files used ICOADS.2.5.1 up to March 2022); (2) Some other minor improvements and bug fixes were implemented in the code used to generate the data.

The gtmba2 reference data are now the supported version, the original gtmba reference data are now retired (stopped March 2024).

CMEMS-Coriolis drifter reference data are provided separately from February 2015-present in files named with PLAT-TYPE=’drifter_cmems’ (February 2015 is the first month these data are available in HadIOD.1.2.0.0). From June 2016-onwards the number of CMEMS-Coriolis drifter observations is much larger than for ICOADS.2.5.1, reaching approximately three times as many observations by December 2017 (see 'Figures'). From June 2016-onwards it is recommended that CMEMS-Coriolis drifter data be considered the ‘definitive’ source of drifting buoy reference data instead of ICOADS. The two drifting buoy data sources are provided with some overlap in time so that users can compare the two data sources if required. This issue has been resolved in ICOADS.3.0.2 (used by HadIOD from April 2022 onwards, see 'Figures') but for continuity users may choose to keep using CMEMS as the source of drifting buoy reference data. Note that (i) the CMEMS-Coriolis drifter data use the latest (improved) version of the basic Met Office Hadley Centre QC, whilst the ICOADS data use the previous version (see ‘QC Schemes’ below); (ii) the CMEMS-Coriolis data are made available at higher precision than ICOADS (ICOADS only provide longitude/latitude to 0.01 degrees and temperature to 0.1 degrees).

For observations made by sub-surface profiling platforms (bottle, CTD, MBT, XBT, Argo and animal), the shallowest temperature observation passing quality control in the depth range 4-6m is provided in the files. This depth range was selected as the shallowest range common to all profiling platform types; XBTs do not reach thermal equilibrium with seawater until around 4 dbar depth whilst Argo float CTD sensors stop pumping as they approach the surface to avoid sensor degradation. For observations from surface-only platforms/instruments (drifter, ship, GTMBA, mooring) all available SST data are included in the files (whether or not they pass QC).

In addition to the Argo data described above, a further set of ‘near-surface’ Argo reference data files are extracted. These contain near-surface data from Solo2 and S2A type Argo floats. These floats have been modified to continue pumped primary sampling up to 1 dbar, providing measurements of vertical temperature structure close to the surface. The near-surface Argo files include data for all depths ranging from 0-10m for profiles that sample shallower than 4m. Observations from the same profile can be identified using the file PROF_ID variable.

For MBTs and XBTs, only those observations whose metadata allow a correction to be applied have been included in the reference data files.

From December 2007 onwards, for security reasons most ship callsigns in ICOADS.2.5.1 are masked using the generic ID ‘MASKSTID’. This mainly affects ICOADS.2.5.1 Real Time monthly updates (2008-onwards) which are based on GTS data. During HadIOD production instances of ‘MASKSTID’ are unmasked where possible using data kept by the Met Office, however these unmasked IDs cannot be released externally. For the ship reference data, IDs of form ‘MKnnnnnnn’ (where nnnnnnn is a 7-digit zero-padded integer) have been mapped to these unmasked IDs for release. These are stored in the file PLAT_ID variable and can be used to track observations from the same ship. Note that some of these IDs may map to generic IDs used by multiple ships, e.g. ‘SHIP’. In particular ‘MK0000016’ and ‘MK0000172’ cannot be used for tracking as they correspond to the IDs ‘MASKSTID’ and ‘SHIP’ respectively. This ID scheme is only applied to ship reference data from December 2007 onwards, prior to this original ICOADS.2.5.1 callsigns are provided in the PLAT_ID variable. From April 2022 onwards HadIOD uses ICOADS.3.0.2 which does not mask ship callsigns, however the decision was made to continue providing HadIOD masked IDs to facilitate tracking across the transition from ICOADS.2.5.1 to ICOADS.3.0.2.

Depths for drifter, ship, gtmba and other mooring observations are inferred (i.e. an estimated depth is assigned by HadIOD because none is available from the data source).

Reference Data files

All file variables are 1D arrays with dimension N_OBS.

NetCDF variable	Description
Dimensions
N_OBS	Number of observations in the file.
STRINGID	Maximum length in characters of the PLAT_ID used to identify a platform.
Variables
OB_ID	HadIOD database identifier for an observation. This is an integer that uniquely identifies an observation within its year and month. When combined with year and month the identifier is unique within the version of HadIOD used (e.g. HadIOD.1.2.0.0).
COLLECTION	Code to identify the source of the data in HadIOD.
SUBCOL1	Code to identify sub-collections within collections.
SUBCOL2	Code to identify sub-collections within collections.
PROF_ID	HadIOD database identifier for a profile. This is an integer that uniquely identifies a profile within its year and month. All observations from the same vertical profile receive the same profile ID (observations from surface-only platforms are treated as a profile with a single observation). When combined with year and month the identifier is unique within the version of HadIOD used (e.g. HadIOD.1.2.0.0)
PLAT_ID	Identifier for the platform making the observations.
LONGITUDE	Longitude of observation.
LATITUDE	Latitude of observation.
YEAR	Year of observation.
MONTH	Month of observation.
DAY	Day of observation.
HOUR	Hour of observation.
MINUTE	Minute of observation.
SECOND	Second of observation.
DEPTH	Depth of an observation below the surface (metres).
DEPTH_CORR	Depth bias correction. Added to depth to help remove biases in reported depth.
SST	Observed sea surface temperature (DegC).
SST_TYPE_CORR	Temperature correction for observations made by the reference data platform type (e.g. ship, XBT). Added to observed temperature to help remove biases in the observed quantity.
SST_TYPE_CORR_UNC	The uncertainty in the value of SST_TYPE_CORR (std. dev.). Once the correction is applied, this describes the spread of a probability density function from which the (unknown) residual bias is drawn. The error (residual bias) associated with this uncertainty can be correlated with errors in other observations of the same platform type.
SST_PLAT_CORR	Temperature correction for observations made by an individual platform (e.g. an individual ship or float), see PLAT_ID. Added to observed temperature to help remove biases in the observed quantity.
SST_PLAT_CORR_UNC	The uncertainty in the value of SST_PLAT_CORR (std. dev.). Once the correction is applied, this describes the spread of a probability density function from which the (unknown) residual bias is drawn. The error (residual bias) associated with this uncertainty can be correlated with errors in other observations from the same individual platform.
SST_RAND_UNC	The random measurement error uncertainty (std. dev.) associated with the observed temperature. The error associated with this uncertainty is not correlated with errors in other observations.
SST_COMB_UNC	The uncertainty obtained by combining all other temperature uncertainty terms in quadrature (std. dev.). Valid when corrections have been applied.
QC1	Observation quality flag 1.
QC2	Observation quality flag 2.
Global Attributes
description	Description of file contents including reference data version number.
source	Gives more details about the source of the data in the file, e.g. HadIOD version number and which bias correction schemes were used.
usage	Gives any file-specific rules that must be followed when using the data (e.g. how bias corrections should be applied).
history	States when the file was created.

Variables explained

Further useful information about some of the variables is given below.

'COLLECTION', 'SUBCOL1' and 'SUBCOL2' variable values

The content of the SUBCOL1 and SUBCOL2 variables varies dependent on the value of the COLLECTION variable. This is detailed in the table below.

Source	COLLECTION	SUBCOL1	SUBCOL2
ICOADS.2.5.1, ICOADS.3.0.2	1, 9	ICOADS Deck ID*	ICOADS Source ID*
EN.4.2.0, EN.4.2.1, EN.4.2.2	2, 4, 8	1=WOD13, 2=Argo GDAC, 3=GTSPP**	N/A
MetDB	3	N/A	N/A
PMEL GTMBA (2016 download), PMEL GTMBA (2024 download)	5, 10	1=TAO/TRITON, 2=PIRATA, 3=RAMA	PMEL GTMBA SST source code***
CMEMS-Coriolis	7	N/A	N/A

*See Table AII and AIII in [Woodruff et al., 2011] and R3.0.2-imma1.pdf

**EN4 blends observations from multiple sources (see 'Data Sources').

***PMEL SST source codes:

0 - No sensor, no data
1 - Real time (telemetered mode)
2 - Derived from real time
3 - Temporally interpolated from real time
4 - Source code inactive at present
5 - Recovered from instrument RAM (delayed mode)
6 - Derived from RAM
7 - Temporally interpolated from RAM

QC Schemes

The content of the quality control variables QC1 and QC2 varies dependent on file platform type. This is detailed in the table below. It is recommended that users omit observations that fail QC. In the case of PMEL GTMBA data, users should filter based on which QC values seem most appropriate (see footnotes below).

Type	QC1	QC2
drifter, ship	Basic Met Office Hadley Centre QC*: pass=0, fail=1	SST tracking QC**: pass=0, fail=1
CMEMS-Coriolis drifter	Improved Met Office Hadley Centre QC***: pass=0, fail=1	N/A
mooring	Basic Met Office Hadley Centre QC*: pass=0, fail=1	N/A
CTD, XBT, MBT, bottle, animal Argo, Argo-surface	Basic EN4 QC****: pass=0, fail>=1	N/A
GTMBA^* (COLLECTION=1, 3 or 9)	Basic Met Office Hadley Centre QC*: pass=0, fail=1	N/A
GTMBA^* (COLLECTION=5 or 10)	PMEL SST flag^**	PMEL position flag^***

*The Met Office Hadley Centre QC suite used for the ICOADS observations comprises basic sanity checks, position track check, climatology check and buddy check. If any of the individual checks fail, the QC flag is set to fail.

**The SST track check used for ICOADS.2.5.1 is an updated version of the QC checks described in Atkinson et al. [2013]. SSTs from individual drifters and ships are tracked against a satellite-based reference and persistently poor quality observations are flagged. Position reports are used to flag buoys that have run aground or been picked-up. The coverage of the check is 1986-2015.

***An updated version of the Met Office Hadley Centre QC suite (MDS4) became available for the addition of CMEMS observations into HadIOD. This performs a similar suite of checks to the previous suite (MDS3) used for the ICOADS observations(*) but with improvements to the performance of checks such as the track, climatology and buddy checks. A repeated value check was also added.

****Profile observations have been QC'd by the EN4 system which has many individual checks (see EN4 webpages). Observations are flagged as failing if they have failed any of the EN4 checks. For profile reference data types, only observations passing QC are included in the files, with the exception of Argo-surface reference data which includes observations both passing and failing QC.

^*GTMBA2 files contain a blend of data from ICOADS (COLLECTION ID=1 or 9) and PMEL (COLLECTION ID=10). [The original GTMBA files contained a blend of data from ICOADS (COLLECTION ID=1 or 9) and PMEL (COLLECTION ID=5), in November 2016 MetDB data (COLLECTION ID=3) were used as a substitute for ICOADS.2.5.1 data)].

^**PMEL SST flags:

0 - datum missing
1 - highest quality; pre/post-deployment calibrations agree to within sensor specifications. In most cases only pre-deployment calibrations have been applied
2 - default quality; default value for sensors presently deployed and for sensors which were either not recovered, not calibratable when recovered, or for which pre-deployment calibrations have been determined to be invalid. In most cases, only pre-deployment calibrations have been applied.
3 - adjusted data; pre/post calibrations differ, or original data do not agree with other data sources (e.g., other in situ data or climatology), or original data are noisy. Data have been adjusted in an attempt to reduce the error.
4 - lower quality; pre/post calibrations differ, or data do not agree with other data sources (e.g. other in situ data or climatology), or data are noisy. Data could not be confidently adjusted to correct for error.
5 - sensor or tube failed

^***PMEL position flags:

0 - datum missing (notional position added based on site metadata)
1 - highest quality; Pre/post-deployment calibrations agree to within sensor specifications. In most cases only pre-deployment calibrations have been applied.
2 - default quality; default value for sensors presently deployed and for sensors which were either not recovered, not calibratable when recovered, or for which pre-deployment calibrations have been determined to be invalid. In most cases, only pre-deployment calibrations have been applied.
3 - adjusted data; Pre/post calibrations differ, or original data do not agree with other data sources (e.g., other in situ data or climatology), or original data are noisy. Data have been adjusted in an attempt to reduce the error.
4 - lower quality; Pre/post calibrations differ, or data do not agree with other data sources (e.g., other in situ data or climatology), or data are noisy. Data could not be confidently adjusted to correct for error.
5 - sensor or tube failed
7 - datum missing (median reported position added) [gtmba2 reference data only]
8 - has moved from deployed location
9 - daily drift speed exceeds .1 kts (0.05 m/s)
>=10 - missing value(s) in-filled by linear interpolation of neighbouring values (for gaps up to and including 10-days in gtmba reference data and 20-days in gtmba2 reference data); flag value is created by taking the lowest quality indicator of the neighbouring flag values used for interpolation and adding 10 (e.g. if the two neighbouring flag values are 2 [default quality] and 9 [drift speed exceeds .1 kts] then the flag for interpolated values will be 9 + 10 = 19.

Bias correction and uncertainty variables

Bias corrections should be added to the corresponding observations, e.g.:

SST_CORRECTED = SST + SST_TYPE_CORR + SST_PLAT_CORR
DEPTH_CORRECTED = DEPTH + DEPTH_CORR

Further rules on applying the correctons may be given in a file's 'usage' global attribute. Do read this. For example, the 'usage' may state that "DEPTH_CORR and SST_TYPE_CORR must be applied together (and not independently)" i.e. if applying SST_TYPE_CORR to SST, then DEPTH_CORR must also be applied to DEPTH (this might be because, for example, SST corrections are made relative to some reference level).

The reference data files use the same HadIOD observation error model as the HadIOD files. The HadIOD product user guide explains the HadIOD error model and how to work with the bias corrections and uncertainties provided by HadIOD. Reading Section 2 of the user guide is strongly recommended. Because the names used for the bias correction and uncertainty variables in the reference data files differ from the variable names used in the product user guide, the table below lists which variable names are equivalent.

Reference data variable	HadIOD file variable
SST_TYPE_CORR	temp_type_corr
SST_TYPE_CORR_UNC	temp_type_corr_unc
SST_PLAT_CORR	temp_plat_corr
SST_PLAT_CORR_UNC	temp_plat_corr_unc
SST_RAND_UNC	temp_rand_unc
SST_COMB_UNC	temp_comb_unc

Bias corrections and their uncertainties are not provided for all platform types (see the product user guide). For example, SST_TYPE_CORR corrections are only provided for ship, XBT and MBT instrument types and SST_TYPE_CORR_UNC is only provided for ships. At a minimum we try to provide some estimate of uncertainty in each measurement.

Figures

Monthly counts of the number of SST observations in the reference data files. Period shown is January 1977-March 2024. (Top) number of observations from surface-based platforms (these are not filtered for observations passing QC); for GTMBA and GTMBA2, PMEL-only ('_pmel') and PMEL+ICOADS ('_all') are shown; for drifting buoys, ‘drifter’ and ‘drifter_cmems’ refer to ICOADS and CMEMS-Coriolis data sources respectively (note their divergence around mid-2016 - from 2018 onwards the patchy 'drifter' coverage is due to an internal change in HadIOD's processing and from April 2022 the number increases significantly with the switch to ICOADS.3.0.2). (Bottom) number of profiles from sub-surface profiling platforms (the shallowest observation passing QC in the depth range 4-6m for each profile is included in the file i.e. one observation per profile, except surface Argo where all observations in the depth range 0-10m are included). Note the difference in scale between the two plots. The drop in the number of moorings (excluding GTMBA and GTMBA2) observations in November 2016 is due to a change of data source (from ICOADS.2.5.1 to MetDB).

Commercial and media enquiries

You can access the Met Office Customer Centre, any time of the day or night by phone, fax or e-mail. Trained staff will help you find the information or products that are right for you.
Contact the Met Office Customer Centre