Comments on version 3.0 toward version 3.1, from May 2014
Thierry, 26/05/2014 13:45
Please find the current draft version of Argo user’s manual version 3.1 on:
This document was updated from the current version 3.03, the decisions from Liverpool Argo data management meeting and a series of emails listed in “comments on version 3.0 toward version 3.1”.
Some of us already started to produce 3.1 sample files.
Ann, Sylvie, we now need a green light to accept this files on Argo GDACs.
Annie, 27/05/2014 00:12
Hi Thierry. Thank you for updating this increasingly complex document.
I have 3 comments for this draft.
1. On p64, I think the entire section 2.6.1 (Pressure axis management in core-Argo and b-Argo files) can be improved. This is the most important section for the coordination between the core- files and the b- files, and therefore needs to be absolute clear. I am attaching here my attempt to rewrite this section, with the title changed to "Management of vertical pressure levels in core-Argo and b-Argo profile files", and have included a hypothetical example. Please take a look at the attached doc.
I hope it helps.
TC: thank you Annie, you text is clear and does not conflict with the existing text. The example you provide is simple and useful. So I replace the existing chapter §2.6.1 with your text.
2. Again on p64, I suggest you change the title for section 2.6.2 to "Management of multi-dimensional spectral parameters", or something similar.
The current title "Management of array values" is not very apt.
TC: ok for a more explicit title: change of title“Management of multi-dimensional parameters”. I preferred to remove the term “spectral” from your title suggestion, a multi-dimensional parameter may not be spectral.
3. Please update Reference Table 11 to include two new rtqc tests for near-surface data. Their descriptions have already been recorded in the qc manual. They now need to be included in Table 11 and be given unique binary IDs. They are:
Test 21. Near-surface unpumped CTD salinity test.
Test 22. Near-surface mixed air/water test.
TC: yes, these tests were missing. They are now added in reference table 11.
John, 31/05/2014 02:21
Hi Thierry, A typo in manual for JULD_DEEP_PARK_START which has the long_name="Deep Descent end date of the cycle". This is an exact copy of the JULD_DEEP_DESCENT_END. Thus the JULD_DEEP_PARK_START should be different.
Comments on version 3.0 toward version 3.1, April-May 2014
Thierry, 18/04/2014 19:04
Dear Esmee and All,
Jean-Philippe and I copied the update of metadata section in the attached version of the “Argo user’s manual”.
We think that this version contains a lot of changes compared to the existing official version 3.0.3 from August 28th 2013.
Instead of using an intermediate version “3.0.4”, we should use a major version “3.1” for this manual.
To differentiate the existing Argo 3.0 NetCDF files and the files built from this new manual, we should also probably use “3.1” for profile, trajectory and metadata files.
This is not an obligation for the format checker; it can combine user manual version (3.1) and file format (3.0), but I think it would be easier to adopt “3.1” for file format versions.
The word version of the manual is available to all from :
Ann, Claudia, can you look at page 75 §3.3 “Reference table 3: parameter code table”
The question to include or not DOXY as core-Argo parameter is still open.
Maybe shall we be able next week to propose this manual as the official version?
Mike, 21/04/2014 20:27
Hi Thierry:
Please change the ftp://usgodae1.fnmoc.navy.mil/pub/outgoing/argo to ftp://usgodae.org/pub/outgoing/argo in the manual Section 4.
TC : done
Ann, 22/04/2014 04:35
I totally agree that all of these changes (and their implementation) require a new version number and 3.1 makes sense.
Regarding DOXY as a core variable, it doesn't really make sense to split it from the other bio-variables. Having said that, I don’t know if Claudia will be able to split her files and that is the big question. However – she can still provide the older version v3.0 files (which can include DOXY) if she wishes until she can get the new code broken up into B and C files. The time frame, however, should be limited.
The problem with Claudia's solution of providing DOXY in the core files without the raw data for the parameter is that it really messes up anyone who needs to do DMQC for oxygen (or any other bio variable). I think if she's providing DOXY in the version 3.0 files, then she needs to include the raw parameters, though that is very unsatisfactory. I realise she's not funded to do bio data but if she's funded to do oxygen, this is just another aspect of that data management (and will set her up for other variables down the track perhaps).
I would also suggest even if she has other bio variables she should provide only what she can. It really is a simple measure to split them into a 'duplicate' file with just the DOXY variables and raw pressure. Where it gets REALLY complicated is when you try to add FLBB data, nitrate data, EM data… Then it's a nightmare and DOES require significant programming changes so I've even sort of ignored them for now…
I'll take a closer look at thereat of the manual but since I've just finished coding profile and metadata files, I don't expect many issues.
TC : no immediate impact on the manual
John, 25/04/2014 21:42
I have two comments on the temporal resolution attributes added and not added in manual 3.1.
First, Why not add JULD and JULD_LOCATION resolution attributes in the profile file, if we are requiring it in the traj file?
TC: OK to be homogeneous between prof and trajont hat topic, added in the manual
§2.2.4 : add “JULD:resolution = X;” and “JULD_LOCATION:resolution = X;”
Second, In the traj file the JULD and JULD_ADJUSTED variables are likely to have variable resolution depending on the measurement code.
At least I know that is the case with the SOLOII. Thus I'm assuming the same procedure is followed whereby no resolution attribute is required under the variable, but then a global comment is added. This should be added to the manual, so that users know it is not only PRES. I also think that TEMP and PSAL and all other variables in the traj file could vary well have different resolutions, so phrasing allowing multiple variables to have this handling should be added.
TC: this information should be reported in the global attribute “comment_on_resolution” described on §2.3.1 page 30.
Claudia, 28/04/2014 21:10
we will not have the ability to distribute raw oxygen data in format 3.x.
TC: Ann answered below
Ann, 29/04/2014 02:01? Answer to Claudia
Actually, I just checked version 3.01 of the manual and there isn't a mention of not providing raw parameters for derived data so it would be acceptable (and that's what we're doing right now in version 3.03 files and no one has screamed about it yet). That restriction only came in with version 3.1 I would guess. So providing raw values is perfectly fine. If you want an example of one of our version 3.03 files, just let me know. I also have examples of version 3.1 C and B files…
TC: no impact on manual
Thierry, 30/04/2014 12:35
I agree with you, this parameter should be renamed « PHASE_DELAY_DOXY” (16 letters).
I think that the unit should be “microsecond” instead of “micro seconds” (the singular is used in Argo units, no plural).
Its valid_min is probably 0. Do we have a valid_maxvalue ?
Ann :This looks sensible to me, Thierry. I've never been comfortable with the phase_doxy name since it's not obvious how it differs from the other phases. This makes it very clear.
TC: done
Jean-Philippe, 30/04/2014 14:12
Exactly
BPHASE_DOXY and DPHASE_DOXY are reported by Aanderaa 3830 optodes
TPHASE_DOXY and C1/C2PHASE_DOXY are reported by Aanderaa 4330 and 4330F optodes
PHASE_DOXY is reported by SBE63
Jean-Philippe, 30/04/2014 16:11
J'ai recopié le contenu de la nouvelle ref table 3 pour le décodeur Remocean (cf. fichier joint si tu en as besoin pour les gliders).
Voici les remarques notées que j'ai corrigées pour le décodeur et qu'il faudrait reprendre (ou pas) dans le manuel et surtout dans le fichier argo-parameters-list-core-and-b.xlsx
- AUM 3.1: colonnelong_name: pour homogènéiserremplacer "by the oxygen sensor" par "by oxygen sensor"
- AUM 3.1: colonne long_name: remplacer "x nanometers" par "<x> nanometers" (pour faire comprendre que <x> sera remplacé dans le décodeur par la meta-donnée ad hoc (comme pour les <SENSOR> des labels config BIO)
- AUM 3.1: colonne long_name: remplacer double espace par un seul (pour BETA_BACKSCATTERING, UP_RADIANCE et DOWN_PAR)
- AUM 3.1: "ABSORBANCE_COR_NITRATE" au lieu de "ABSORBANCE _COR_NITRATE" (un espace dans le nom)
- AUM 3.1: retour charriot dans long name de MOLAR_DOXY
2 autres remarques sur le manuel 3.1:
- remplacer partout DOWNWELLING_IRRADIANCE_RAW par RAW_DOWNWELLING_IRRADIANCE qui est le nom dans la liste Excel
- remplacer DOWNWELLING_PAR par DOWN_PAR (incohérence entre liste Excel et manuel, je pense que l'Excel fait foi car c'est là dessus que nous avions travaillé en dernier avec Catherine) => est-ce la seule incohérence entre la liste Excel et le manuel?
TC: J’ai remplacé DOWNWELLING_IRRADIANCE_RAW par RAW_DOWNWELLING_IRRADIANCE.
Par contre, DOWNWELLING_PAR est la version longue de DOWN_PAR
Claudia, 30/04/2014 18:23
Idea:
is it possible to add to the header or foter something that shows if one is
in the meta, traj, prof or tech section of the manual?
TC: sorry, I did not find a way to add a chapter name in the footer. Does anyone know if that is possible? The PDF version has signs listing the chapters that are displayed on the left of the document.
The following is following the links in the update section:
======
§1.2 : use DOI for data and document citation
TC: I do not see any problem there (?)
======
§1.6 : update cycle definition
A cycle is defined as a series of actions made by a float and includes either a descending profile or anascending profile (or, rarely, both); it may also include immersion drift or surface drift.
--> A cycle is defined as a series of actions made by a float and includes either one or more descendingprofiles and/or one or more ascending profiles. Typically it also includes phases of subsurface andsurface drift.
TC: OK, that definition is better; there may be more than one ascending or descending profile.
======
§2.2.4 §2.3.4 §2.4.4 :FLOAT_SERIAL_NO dimension extended to 32 instead of 16
TC: corrected
======
§2.2.4, §2.3.4, §2.4.4 : add a “conventions” attribute to PLATFORM_TYPE
TC: OK, done
§2.4.4 : add a “conventions” attribute to PLATFORM_FAMILY and PLATFORM_MAKER
TC: OK, done
§2.4.7.1 : add a “conventions” attribute to SENSOR_MAKER and SENSOR_MODEL
TC: OK, done
While realizing that this document needs regular updating I'm somewhat concerned about relying on linksto point users to crucial information. Not sure what the best way to do this might be. Maybe make itavailable via the ADMT web site (instead of google docs or link from there to google docs - I lookedat the ADMT documentation site and did not see a link to the document)?
TC: we do not have a better solution now
Another problem is: the document loads awfully slow (e.g. PLATFORM_TYPE/KEY)
Also, a subset of the information seems to be in Table 23 of the user manual.
Maybe we can have pdf's instead of "goolge docs" (one for each sheet in the doc)
and link to them from the ADMT documentation page.
(I'm still waiting for PLATFORM_TYPE/KEY after typing all this.)
======
§2.3.5 : add a “resolution attribute” to JULD and JULD_ADJUSTED
TC: done
§2.3.6: add a “conventions” attribute to each cycle timing
TC: done
in general:
1) what should a DAC do if a resolution for a variable is not available?
TC: in chapter 2.2.5.1, a sentence says “don’t put a resolution attribute”. For me, it implies that “resolution” attribute should not exist if the resolution is not known. I added the sentence on §2.2.5.1 : “The resolution of a parameter is reported in “resolution” attribute. If the resolution is unknown, the attribute “resolution” is not reported.”
2) for derived quantities (like DOXY or PSAL when derived from CNDC) the
resoultion may depend on who one asks what it is.
3) 1 second resolution in terms of JULD is 1.157407407407407e-05 days if
one uses double precision. What should the resolution be set to?
1e-05 days?
4) what if the resolution depends on the profile number in a given cycle?
TC: see §2.2.5.1
======
§2.3.5.1 How to report unusual Pressure resolutions in the
N_MEASUREMENT variable group of the TRAJ file
======
§2.4 : general revision of metadata format; §2.4.6 adding launch
configuration parameters; §2.4.7 discriminate float sensor and
float parameter information
user_manual_version Eponymous
---> not sure Eponymous is the right word here (people I asked were baffled,
and the definitions I found online did not seem to fit)
TC: a proper definition replaces “eponymous”: “The version number of the user manual”
N_SENSOR N_SENSOR = <int value>; Number of sensors mounted on the float and used
to measure the parameters.
TC: OK
N_LAUNCH_CONFIG_PARAM N_LAUNCH_CONFIG_PARAM=<int value>; Number of pre-deploymentor launch configuration parameters.
TC: OK
LAUNCH_CONFIG_PARAMETER_NAME(N_LAUNCH_CONFIG_PARAM, STRING128)
LAUNCH_CONFIG_PARAMETER_VALUE(N_LAUNCH_CONFIG_PARAM)
----> I see a problem with this, because some floats of the park and profile type
have a dual launch configuration (e.g., profile depth alternating in some way between
2000 and 1000 dbar). So this is by definition a launch configuration, but it can not
be stored in LAUNCH_CONFIG_PARAMETER_....
TC: these are 2 conffigurations (see Esmee’s message below).
----> Unless we find a reasonable solution for this, we need to rethink the approach
of having LAUNCH_CONFIG_PARAMETER_.... as well as the CONFIG_PARAMETER_....
that come with CONFIG_MISSION_NUMBER and CONFIG_MISSION_COMMENT.
TC: I am not sure to understand the issue
======
§2.4.5 : add STARTUP_DATE and STARTUP_DATE_QC
======
§2.6 : B-Argo profile additional format features
§2.6.1 : manage pressure axis between core-argo and b-argo files
DOWNWELLING_IRRADIANCE_RAW:wave_length_nanometer = "115 132 149 166 183 200 217 234 251 268 285 302
319 336 353 370 387 404 421 438 455 472 489 506 523 540 557 574 591 608 625 642 659 676 693 710 727 744 761 778 795" ;
----> could there be a way to store this as a vector rather than a string in
thenc file header part? That would make it more accesible to users.
TC: maybe, I think that this exists for QC flags, should be tested.
N_VALUES41
----> N_VALUES (throughout)
======
§3.22, §3.23, §3.24, §3.25, §3.26, §3.27 : new reference tables
§ : separation of core-Argo and B-Argo parameters in reference table 3
Is most of this content from the docs under
Can't check because still waiting for it to load. See comment on this above.
TC: no, the parameters list is in Argo data-management web site :
======
§4.1.2 : separation of core-Argo data files and B-Argo data files
"4.1.2.2 B-Argo individual merged profile file
To facilitate the use of B-Argo data, the GDAC merges each B-Argo file with its corresponding
core-Argo data file.
The merged file contains the core-Argo and B-Argo parameters listed on reference table 3. The
intermediate parameters are ignored by the merged files."
----> will this allow DACs to only submit merged files? If so: How would that impact
users?
TC: no, DACs will provide core-Argo and B-Argo files, merged files will be provided by GDAC only.
regarding the B-Argo traj file: Is this intended to only contain bio-PARAM
with CYCLE_NUMBER, JULD and MEASUREMENT_CODE, so that it can be linked to
the core traj file?
TC: yes
======
§7 : add a glossary
TC: done
======
§3.1 : add the B-Argo profile and B-Argo trajectory data types in reference table 1.
TC: done
======
§3.3 : revision of valid_min and valid max on TEMP, PSAL, DOXY to be compliant
with Argo quality control manual.
Do we really want to force PSAL 2 to 41 instead of the old 0 to 42?
TC: yes
======
§2.2.3, §2.3.3: add a FILE_TYPE variable to discriminate C, B or M files.
Do we really need this one? The file name already gives it away.
TC: no, we do not need it. This information can come from the variable “data_type”. The reference table 1 contains 4 new data types : B-Argo profile, B-Argo trajectoryB-Argo profile
B-Argo trajectory, Argo profile merged, Argo trajectory merged
======
§2.2.6: list parameter names in the PARAMETER variables even if
no calibration is performed yet.
======
Page 1 : add a DOI to the manual
TC: done
======
Page 2 : add a “How to cite this document” chapter
======
§6.5: add a chapter on Argo DOIs
TC: done
Claudia, 30/04/2014 20:23
in addition to what I sent before I thought we may be able to consolidate the two tables in sections 2.4.7.1 and 2.4.7.2 (see attached word document).
TC: probably not necessary (see Esmee’s message below)
To make this work we would need to introduce N_PARAM2, which would be the maximum number of parameters an individual sensor on a float can measure.
For example, if a float has an oxygen sensor that reports a phase and a temperature then N_PARAM2=2 (or larger depending on other sensors the float may have).
Name / Definition / CommentSENSOR / char SENSOR(N_SENSOR, STRING32);
SENSOR:long_name = "Name of the sensor mounted on the float";
SENSOR:conventions = "Argo reference table 25";
SENSOR:_FillValue = " "; / Names of the sensors mounted on the float
Example: CTD_PRES, CTD_TEMP, CTD_CNDC, OXYGEN_OPTODE.
See Argo reference table 25.
Regular updates are made to an online version of this table available at:
SENSOR_MAKER / char SENSOR_MAKER(N_SENSOR, STRING256);
SENSOR_MAKER:long_name = "Name of the sensor manufacturer";
SENSOR_MAKER:conventions = "Argo reference table 26";
SENSOR_MAKER:_FillValue = " "; / Name of the manufacturer of the sensor.
Example : DRUCK, SBE, AANDERAA.
See Argo reference table 26.
Regular updates are made to an online version of this table available at:
SENSOR_MODEL / char SENSOR_MODEL(N_SENSOR, STRING256);
SENSOR_MODEL:long_name = "Type of sensor";
SENSOR_MODEL:conventions = "Argo reference table 27";
SENSOR_MODEL:_FillValue = " "; / Model of sensor.
Example: DRUCK, SBE41CP, AANDERAA_OPTODE_3930.
This field is now standardised.
See Argo reference table 27.
Regular updates are made to an online version of this table available at:
SENSOR_SERIAL_NO / char SENSOR_SERIAL_NO(N_SENSOR, STRING16);
SENSOR_SERIAL_NO:long_name = "Serial number of the sensor";
SENSOR_SERIAL_NO:_FillValue = " "; / Serial number of the sensor.
Example : 2646 036 073
SENSOR_PARAMETER / char SENSOR_PARAMETER(N_SENSOR,N_PARAM2,STRING64);
:long_name = "Name of parameter computed from float measurements”;
:conventions = "Argo reference table 3";
:_FillValue = " "; / Names of the parameters measured by float sensors or derived from float measurements.
The parameter names are listed in reference table 3.
Examples :
TEMP, PSAL, CNDC
TEMP : temperature in Celsius
PSAL : practical salinity in psu
CNDC : conductvity in mhos/m
SENSOR_PARAMETER_UNITS / char SENSOR_PARAMETER_UNITS(N_SENSOR,N_PARAM, STRING16);
:long_name = "Units of accuracy and resolution of the parameter";
:_FillValue = " "; / Units of accuracy and resolution of the parameter.
Example : psu
SENSOR_PARAMETER_ACCURACY / char SENSOR_PARAMETERACCURACY(N_SENSOR,N_PARAM2,STRING32);
:long_name = "Accuracy of the parameter";
PARAMETER_ACCURACY:_FillValue = " "; / Accuracy of the parameter.
Example: "8 micromole/l or 5%"
SENSOR_PARAMETER_RESOLUTION / char SENSOR_PARAMETER_RESOLUTION(N_SENSOR,N_PARAM2,STRING32);
:long_name = "Resolution of the parameter";
:_FillValue =" "; / Resolution of the parameter returned by the sensor (note that this is not necessarily equivalent to the resolution of the parameter returned by the float through telemetry).
Example : 0.001 micromole/l
Esmee, 01/05/2014 05:03
Some feedback on Claudia's suggested changes to the metadata manual..
I agree that "Eponymous" as a definition for the user_manual_version could be simplified. I suggest instead: "The version number of the user manual".
TC: OK, done
The conventions attributes for PLATFORM_TYPE, PLATFORM_FAMILY, PLATFORM_MAKER, SENSOR_MAKER and SENSOR_MODEL have already been added by Thierry in this latest version 3.1 (30/11/2013).
The reference tables for the PLATFORM and SENSOR variables will need to be constantly updated as new types are added.
We are still working on the SENSOR and SENSOR_MODEL tables to include all of the new Bio sensor types and making sure these are named in a consistent and sensible way.
Mathieu is maintaining these tables using google docs - I agree that it would increase accessibility to also link to these via the ADMT website. We don't want to maintain two duplicate sets though, so suggest adding the link to the google docs for now.
TC: ok, no change in the manual