Comments on Version 3.0 Toward Version 3.1, from June 2014

Comments on Version 3.0 Toward Version 3.1, from June 2014

Comments on version 3.0 toward version 3.1, from June 2014

John, 03/06/2014 00:16

Here are a couple more, Thierry.... issues with the V3.1 Traj documentation.

REPRESENTATIVE_PARK_PRESSURE:long_name ...change drift phase to park phase for consistency

TC: ok done

REPRESENTATIVE_PARK_PRESSURE_STATUS is lacking a long_name...is this on purpose? In my code I've had

REPRESENTATIVE_PARK_PRESSURE_STATUS:long_name = "Status of Representative Parking Pressure" ; Feel free to use it if you like.

TC: most of the *_STATUS variables did not have a long_name attribute. I added it to all of them.

JULD_ASCENT_END:long_name...remove the word "float's" to be consistent with JULD_ASCENT_END. Plus special characters are hard to deal with.

TC: ok done

CYCLE_NUMBER and CYCLE_NUMBER_ADJUSTED. I don't know what the decision was, back a year ago when you and Michael discussed this...but in some locations there is 0..N and in others there is 0...N (two versus three dots). I would change to three in CYCLE_NUBMER and CYCLE_NUMBER_ADJUSTED

TC: ok done, I replaced all "0..N" with "0...N"

Justin, 02/06/2014 13:45

Hello Justin,

Yes, the Argo user’s manual authors list is completely out of date.

I did not want to revisit it while no one complained.

How can we revisit it :

1. Keep everyone and add the persons who sent a feedback for version 3.1

2. Keep everyone and add the persons who sent a feedback for version 3.1 plus the DAC contacts

3. Keep only the persons who sent a feedback for version 3.1

4. Keep only the persons who sent a feedback for version 3.1 plus the DAC contacts

I do not have a preference; I do not think that having a long list is a problem.

Thierry

PS : Today’s list

Authors: Thierry Carval / Ifremer, Bob Keeley / MEDS, Yasushi Takatsuki / JAMSTEC, Takashi Yoshida / JMA, Stephen Loch / BODC, Claudia Schmid / AOML, Roger Goldsmith / WHOI, Annie Wong / UW, Rebecca McCreadie / BODC, Ann Thresher / CSIRO, Anh Tran / MEDS

De : Buck, Justin J.H. [mailto:

Envoyé : lundi 2 juin 2014 16:34

À : Thierry Carval

Objet : FW: [argo-dm] Argo user's manual version 3.1, draft from May 21st 2014

HI Thierry,

See Jon's comments below, would it be better to have an author list more like the QC manual with a few primary authors plus a general 'members of the ADMT' author?

Justin

From: Turton, Jon [mailto:

Sent: 02 June 2014 12:37

To: Buck, Justin J.H.

Subject: FW: [argo-dm] Argo user's manual version 3.1, draft from May 21st 2014

Justin,

I would suggest (to Thierry) that the author list for the revised manual is changed to reflect those who have provided input. It makes no sense to list Rebecca McCreadie, who left BODC many years ago, as an author. You will have had much more input to it.

Jon

Mike, 17/06/2014

I have been testing using some 3.1 files provided by John Gilson. He has had some feedback for the manual that has been forwarded on to Thierry in a separate message. In addition to those suggested changes, the following needs changed in the manual as well. That is, change the fillValue from

'99999.' to '99999.f' for CONFIG_PARAMETER_VALUE and LAUNCH_CONFIG_PARAMETER_VALUE.

TC: OK done

Jean-Philippe, 23/06/2014 09:46

J'ai codé le format 3.1 à partir du dernier AUM trouvé (daté du 3/06/2014).

Cependant je me suis rendu compte que les remarques de mon mail du 03/06/2014 18:54 ("Re: [argo-dm] Argo user's manual version 3.1, draft from May 21st 2014") n'y avaient pas été prises en compte.

De plus j'ai relevé un certain nombre d'anomalies (ou améliorations) supplémentaires.

Voici donc ce que j'ai mis dans la version du décodeur livrée ce jour:

1- ajout d'un long_name à JULD_PARK_START_STATUS

netcdf.putAtt(fCdf, juldParkStartStatusVarId, 'long_name', 'Status of drift start date of the cycle');

TC: added

2- ajout d'un long_name à JULD_LAST_MESSAGE_STATUS

netcdf.putAtt(fCdf, juldLastMessageStatusVarId, 'long_name', 'Status of date of latest float message received');

TC: added

3- correction du long_name de JULD_LAST_LOCATION_STATUS

netcdf.putAtt(fCdf, juldLastLocationStatusVarId, 'long_name', 'Status of date of latest location');

TC: done

4- correction du long_name de JULD_TRANSMISSION_END_STATUS

netcdf.putAtt(fCdf, juldTransmissionEndStatusVarId, 'long_name', 'Status of transmission end date');

TC: done

5- modification du long_name de CYCLE_NUMBER_INDEX_ADJUSTED

comme CYCLE_NUMBER_INDEX et CYCLE_NUMBER_INDEX_ADJUSTED on actuellement le même long_name dans le document (ce qui n'est pas terrible), j'ai modifié le long_name de CYCLE_NUMBER_INDEX_ADJUSTED

netcdf.putAtt(fCdf, cycleNumberIndexAdjustedVarId, 'long_name', 'Adjusted cycle number that corresponds to the current index');

TC: ok

6- as part(s) of day

dans la convention pour les jours Julien, on trouve dans le manuel:

'Relative julian days with decimal part (as parts of day)'

et

'Relative julian days with decimal part (as part of day)'

comme je ne sais pas lequel choisir je l'ai codé tel quel mais je pense qu'il faut demander à un anglais lequel est le bon et uniformiser le document en conséquence.

TC: I replaced all 'Relative julian days with decimal part (as part of day)' with '... (as parts of day)'

7- DATA_TYPE sur 32 caractères

avec les nouveaux items de la ref table 1, il faut étendre les DATA_TYPE à 32 caractères (pour les b files)

actuellement c'est 16 pour tous et 32 pour TECH

je n'ai pas changé cela, j'ai juste porté à 32 celui de ¨b PROF (pas utile) et b TRAJ (utile)

cela permettra de rajouter une note commune aux b files

ainsi, si cela te convient, il faut ajouter un point 2.6.6 pour dire que DATA_TYPE est sur 32 caractères pour les b et merged files.

TC: ok, B-Argo and Merged files DATA_TYPE dimension are extended from 16 to 32 characters.

8- primary/secondary controller board

même si j'ai codé le contenu du document je pense:

- qu'il peut être plus explicite de mettre 'Type of primary controller board' dans le long name de CONTROLLER_BOARD_TYPE_PRIMARY

- que le long_name de CONTROLLER_BOARD_TYPE_SECONDARY est incorrect et qu'il faut mettre 'Type of secondary controller board'

TC: ok

j'aimerai également savoir ce qu'ils entendent pas contrôleur primaire et secondaire.

Dans les Remoceans nous avons 2 cartes: une qui contrôle le vecteur (vector board), l'autre les capteurs (sensor board).

Faut-il les mettre dans primary et secondary respectivement ou mettre la vector board dans primary et demander la création d'une variable sensor board ?

TC: on va utiliser primary et secondary comme "vector board" et "sensor board"

9- d(D)ate and t(Time)ime

dans le long_name de JULD_QC(N_PROF) on a 'Quality on Date and Time' dans tous les autres (JULD_QC(N_MEASUREMENT) par exemple) on a 'Quality on date and time' je propose donc de mettre également 'Quality on date and time' dans le long name de JULD_QC(N_PROF)

TC: OK, corrected

10- 1..N

comme le dit l'historique les 0..N ont été passés en 0...N

pour les 1...N c'est la même chose sauf dans le conventions de CONFIG_MISSION_NUMBER(N_PROF) qui contient encore un 1..N

il faut donc mettre '1...N, 1 : first complete mission' dans la convention de CONFIG_MISSION_NUMBER(N_PROF)

TC: ok, done

11- corriger la conventions de PARAMETER_SENSOR

il faut mettre

netcdf.putAtt(fCdf, parameterSensorVarId, 'conventions', 'Argo reference table 25');

TC: ok

12- ajouter conventions aux DATA_TYPE

j'ai ajouté un attribut 'conventions' à toutes les variables DATA_TYPE

netcdf.putAtt(fCdf, dataTypeVarId, 'conventions', 'Argo reference table 1');

TC: ok done

Lorsque tu auras pris en compte ces modifications (et peut-être certaines autres encore dans les tuyaux), peux-tu me transmettre le AUM correspondant, je l'utiliserai pour écrire les fichiers de règles du checker JAVA (ce qui nous permettra de vérifier que nous sommes bien en phase entre les contenus des NetCDF et la description du format).

Comments on version 3.0 toward version 3.1, from May 2014

Thierry, 26/05/2014 13:45

Please find the current draft version of Argo user’s manual version 3.1 on:

This document was updated from the current version 3.03, the decisions from Liverpool Argo data management meeting and a series of emails listed in “comments on version 3.0 toward version 3.1”.

Some of us already started to produce 3.1 sample files.

Ann, Sylvie, we now need a green light to accept this files on Argo GDACs.

Annie, 27/05/2014 00:12

Hi Thierry. Thank you for updating this increasingly complex document.

I have 3 comments for this draft.

1. On p64, I think the entire section 2.6.1 (Pressure axis management in core-Argo and b-Argo files) can be improved. This is the most important section for the coordination between the core- files and the b- files, and therefore needs to be absolute clear. I am attaching here my attempt to rewrite this section, with the title changed to "Management of vertical pressure levels in core-Argo and b-Argo profile files", and have included a hypothetical example. Please take a look at the attached doc.

I hope it helps.

TC: thank you Annie, you text is clear and does not conflict with the existing text. The example you provide is simple and useful. So I replace the existing chapter §2.6.1 with your text.

2. Again on p64, I suggest you change the title for section 2.6.2 to "Management of multi-dimensional spectral parameters", or something similar.

The current title "Management of array values" is not very apt.

TC: ok for a more explicit title: change of title “Management of multi-dimensional parameters”. I preferred to remove the term “spectral” from your title suggestion, a multi-dimensional parameter may not be spectral.

3. Please update Reference Table 11 to include two new rtqc tests for near-surface data. Their descriptions have already been recorded in the qc manual. They now need to be included in Table 11 and be given unique binary IDs. They are:

Test 21. Near-surface unpumped CTD salinity test.

Test 22. Near-surface mixed air/water test.

TC: yes, these tests were missing. They are now added in reference table 11.

John, 31/05/2014 02:21

Hi Thierry, A typo in manual for JULD_DEEP_PARK_START which has the long_name="Deep Descent end date of the cycle". This is an exact copy of the JULD_DEEP_DESCENT_END. Thus the JULD_DEEP_PARK_START should be different.

Comments on version 3.0 toward version 3.1, April-May 2014

Thierry, 18/04/2014 19:04

Dear Esmee and All,

Jean-Philippe and I copied the update of metadata section in the attached version of the “Argo user’s manual”.

We think that this version contains a lot of changes compared to the existing official version 3.0.3 from August 28th 2013.

Instead of using an intermediate version “3.0.4”, we should use a major version “3.1” for this manual.

To differentiate the existing Argo 3.0 NetCDF files and the files built from this new manual, we should also probably use “3.1” for profile, trajectory and metadata files.

This is not an obligation for the format checker; it can combine user manual version (3.1) and file format (3.0), but I think it would be easier to adopt “3.1” for file format versions.

The word version of the manual is available to all from :

Ann, Claudia, can you look at page 75 §3.3 “Reference table 3: parameter code table”

The question to include or not DOXY as core-Argo parameter is still open.

Maybe shall we be able next week to propose this manual as the official version?

Mike, 21/04/2014 20:27

Hi Thierry:

Please change the ftp://usgodae1.fnmoc.navy.mil/pub/outgoing/argo to ftp://usgodae.org/pub/outgoing/argo in the manual Section 4.

TC : done

Ann, 22/04/2014 04:35

I totally agree that all of these changes (and their implementation) require a new version number and 3.1 makes sense.

Regarding DOXY as a core variable, it doesn't really make sense to split it from the other bio-variables. Having said that, I don’t know if Claudia will be able to split her files and that is the big question. However – she can still provide the older version v3.0 files (which can include DOXY) if she wishes until she can get the new code broken up into B and C files. The time frame, however, should be limited.

The problem with Claudia's solution of providing DOXY in the core files without the raw data for the parameter is that it really messes up anyone who needs to do DMQC for oxygen (or any other bio variable). I think if she's providing DOXY in the version 3.0 files, then she needs to include the raw parameters, though that is very unsatisfactory. I realise she's not funded to do bio data but if she's funded to do oxygen, this is just another aspect of that data management (and will set her up for other variables down the track perhaps).

I would also suggest even if she has other bio variables she should provide only what she can. It really is a simple measure to split them into a 'duplicate' file with just the DOXY variables and raw pressure. Where it gets REALLY complicated is when you try to add FLBB data, nitrate data, EM data… Then it's a nightmare and DOES require significant programming changes so I've even sort of ignored them for now…

I'll take a closer look at thereat of the manual but since I've just finished coding profile and metadata files, I don't expect many issues.

TC : no immediate impact on the manual

John, 25/04/2014 21:42

I have two comments on the temporal resolution attributes added and not added in manual 3.1.

First, Why not add JULD and JULD_LOCATION resolution attributes in the profile file, if we are requiring it in the traj file?

TC: OK to be homogeneous between prof and traj ont hat topic, added in the manual

§2.2.4 : add “JULD:resolution = X;” and “JULD_LOCATION:resolution = X;”

Second, In the traj file the JULD and JULD_ADJUSTED variables are likely to have variable resolution depending on the measurement code.

At least I know that is the case with the SOLOII. Thus I'm assuming the same procedure is followed whereby no resolution attribute is required under the variable, but then a global comment is added. This should be added to the manual, so that users know it is not only PRES. I also think that TEMP and PSAL and all other variables in the traj file could vary well have different resolutions, so phrasing allowing multiple variables to have this handling should be added.

TC: this information should be reported in the global attribute “comment_on_resolution” described on §2.3.1 page 30.

Claudia, 28/04/2014 21:10

we will not have the ability to distribute raw oxygen data in format 3.x.

TC: Ann answered below

Ann, 29/04/2014 02:01? Answer to Claudia

Actually, I just checked version 3.01 of the manual and there isn't a mention of not providing raw parameters for derived data so it would be acceptable (and that's what we're doing right now in version 3.03 files and no one has screamed about it yet). That restriction only came in with version 3.1 I would guess. So providing raw values is perfectly fine. If you want an example of one of our version 3.03 files, just let me know. I also have examples of version 3.1 C and B files…

TC: no impact on manual

Thierry, 30/04/2014 12:35

I agree with you, this parameter should be renamed « PHASE_DELAY_DOXY” (16 letters).

I think that the unit should be “microsecond” instead of “micro seconds” (the singular is used in Argo units, no plural).

Its valid_min is probably 0. Do we have a valid_max value ?

Ann : This looks sensible to me, Thierry. I've never been comfortable with the phase_doxy name since it's not obvious how it differs from the other phases. This makes it very clear.

TC : done

Jean-Philippe, 30/04/2014 14:12

Exactly
BPHASE_DOXY and DPHASE_DOXY are reported by Aanderaa 3830 optodes
TPHASE_DOXY and C1/C2PHASE_DOXY are reported by Aanderaa 4330 and 4330F optodes
PHASE_DOXY is reported by SBE63

Jean-Philippe, 30/04/2014 16:11

J'ai recopié le contenu de la nouvelle ref table 3 pour le décodeur Remocean (cf. fichier joint si tu en as besoin pour les gliders).

Voici les remarques notées que j'ai corrigées pour le décodeur et qu'il faudrait reprendre (ou pas) dans le manuel et surtout dans le fichier argo-parameters-list-core-and-b.xlsx

- AUM 3.1: colonne long_name: pour homogènéiser remplacer "by the oxygen sensor" par "by oxygen sensor"

- AUM 3.1: colonne long_name: remplacer "x nanometers" par "<x> nanometers" (pour faire comprendre que <x> sera remplacé dans le décodeur par la meta-donnée ad hoc (comme pour les <SENSOR> des labels config BIO)

- AUM 3.1: colonne long_name: remplacer double espace par un seul (pour BETA_BACKSCATTERING, UP_RADIANCE et DOWN_PAR)

- AUM 3.1: "ABSORBANCE_COR_NITRATE" au lieu de "ABSORBANCE _COR_NITRATE" (un espace dans le nom)

- AUM 3.1: retour charriot dans long name de MOLAR_DOXY

2 autres remarques sur le manuel 3.1:

- remplacer partout DOWNWELLING_IRRADIANCE_RAW par RAW_DOWNWELLING_IRRADIANCE qui est le nom dans la liste Excel

- remplacer DOWNWELLING_PAR par DOWN_PAR (incohérence entre liste Excel et manuel, je pense que l'Excel fait foi car c'est là dessus que nous avions travaillé en dernier avec Catherine) => est-ce la seule incohérence entre la liste Excel et le manuel?

TC : J’ai remplacé DOWNWELLING_IRRADIANCE_RAW par RAW_DOWNWELLING_IRRADIANCE.

Par contre, DOWNWELLING_PAR est la version longue de DOWN_PAR

Claudia, 30/04/2014 18 :23

Idea:

is it possible to add to the header or foter something that shows if one is

in the meta, traj, prof or tech section of the manual?

TC: sorry, I did not find a way to add a chapter name in the footer. Does anyone know if that is possible? The PDF version has signs listing the chapters that are displayed on the left of the document.

The following is following the links in the update section:

======

§1.2 : use DOI for data and document citation

TC: I do not see any problem there (?)

======

§1.6 : update cycle definition

A cycle is defined as a series of actions made by a float and includes either a descending profile or an ascending profile (or, rarely, both); it may also include immersion drift or surface drift.

--> A cycle is defined as a series of actions made by a float and includes either one or more descending profiles and/or one or more ascending profiles. Typically it also includes phases of subsurface and surface drift.

TC: OK, that definition is better; there may be more than one ascending or descending profile.

======

§2.2.4 §2.3.4 §2.4.4 :FLOAT_SERIAL_NO dimension extended to 32 instead of 16

TC: corrected

======

§2.2.4, §2.3.4, §2.4.4 : add a “conventions” attribute to PLATFORM_TYPE

TC: OK, done

§2.4.4 : add a “conventions” attribute to PLATFORM_FAMILY and PLATFORM_MAKER

TC: OK, done

§2.4.7.1 : add a “conventions” attribute to SENSOR_MAKER and SENSOR_MODEL

TC: OK, done

While realizing that this document needs regular updating I'm somewhat concerned about relying on links to point users to crucial information. Not sure what the best way to do this might be. Maybe make it available via the ADMT web site (instead of google docs or link from there to google docs - I looked at the ADMT documentation site and did not see a link to the document)?

TC: we do not have a better solution now

Another problem is: the document loads awfully slow (e.g. PLATFORM_TYPE/KEY)

Also, a subset of the information seems to be in Table 23 of the user manual.

Maybe we can have pdf's instead of "goolge docs" (one for each sheet in the doc)

and link to them from the ADMT documentation page.

(I'm still waiting for PLATFORM_TYPE/KEY after typing all this.)

======

§2.3.5 : add a “resolution attribute” to JULD and JULD_ADJUSTED

TC: done

§2.3.6: add a “conventions” attribute to each cycle timing

TC: done

in general:

1) what should a DAC do if a resolution for a variable is not available?

TC: in chapter 2.2.5.1, a sentence says “don’t put a resolution attribute”. For me, it implies that “resolution” attribute should not exist if the resolution is not known. I added the sentence on §2.2.5.1 : “The resolution of a parameter is reported in “resolution” attribute. If the resolution is unknown, the attribute “resolution” is not reported.”

2) for derived quantities (like DOXY or PSAL when derived from CNDC) the

resoultion may depend on who one asks what it is.