WORLD AREA FORECAST SYSTEM (WAFS)
workshop on new gridded WAFS forecasts for icing,
turbulence and cumulonimbus clouds
ICAO EUR/NAT Regional Office, Paris, France, 14 to 15 September 2009
Summary of Discussions
- Introduction
1.1.For a number of years, the World Area Forecast System (WAFS) has provided global gridded forecasts of elements such as wind direction, wind speed, temperature and humidity. The ICAO World Area Forecast System Operations Group has requested that the World Area Forecast Centre (WAFC) Provider States (United Kingdom and United States) develop the additional capability to provide global gridded forecasts of icing, turbulence and cumulonimbus clouds.
1.2.With a view to providing information on the new gridded products and facilitating their implementation, Conclusion 4/24 of the fourth meeting of the ICAO WAFS Operations Group meeting (held February 2008) invited the WAFC Provider States, in coordination with ICAO and the World Meteorological Organisation, to convene a workshop on the use and visualization of gridded WAFS forecasts of cumulonimbus clouds, icing and turbulence.
1.3.A two-day WAFS workshop was held at the ICAO EUR/NAT regional office, Paris, 14 to 15September 2009. The Summary of Discussions from the workshop is presented below. The workshop was attended by 46 experts representing 13 States, 4 international organisations (the International Air Transport Association (IATA), the International Federation of Airline Pilot’s Associations (IFALPA), the International Civil Aviation Organisation (ICAO) and the World Meteorological Organization (WMO)) and 6 industry representatives (Air France, COROBOR, IBL Software Engineering, IRAM, Meteo-France International and NetSys International).
1.4.Appendix Aincludes the list of attendees.
1.5.The objectives of the WAFS workshop were as follows:
- To provide information on the new products, their specifications and expected accuracy; and
- To provide an opportunity for flight planning companies and workstation suppliers andsoftware developers to familiarize themselves with and discuss issues related to the automated products.
DAY 1: MONDAY 14 SEPTEMBER 2009
- Background to the requirements
2.1.Dr Olli Turpeinen, Chief MET/AIM Section, ICAO, and Secretary of the WAFSOPSG, presented an overview of the decisions that had led to the development of the gridded icing, turbulence and cumulonimbus (CB) forecasts. Orderly development of the new gridded forecasts had taken place with consensus reached within the global community through the WAFSOPSG.
2.2.The WAFSOPG has recognised that the accuracy and compatibility of the new gridded forecasts needs to be equal to or better than the traditional (human-generated) significant weather (SIGWX) forecasts provided today by the two WAFCs (London and Washington). The visualisation of the gridded forecasts from the WAFCs needs to be simple and similar to that of the existing SIGWX – i.e. provide the same ease of use. There is expected to be two types of WAFS output – a complete GRIB2 dataset (for sophisticated flight planning systems) as well as products in chart form for flight documentation (for pilot’s situational awareness at the flight briefing stage).
2.3.Olli briefed the participants at the meeting about new changes in meteorological services that relate to the NEXTGEN and SESAR projects within the US and Europerespectively, whichwill result in major changes to the way air traffic is managed. In fact, the WAFS programme has been forward looking, where the global gridded fields for icing, turbulence and CB (in addition to the wind, temperature and humidity data) can be considered as the first step towards the 4-Dimensional Weather Data Cube concept. The requirement for a single authoritative source may, in future years, lead to a consolidation in the activities of the WAFCs
- Scientific background and content of the new gridded products
3.1.Bob Lunnon, UK Met Office (WAFC London), summarised the content of the new gridded products for icing, turbulence and CB, and gave an overview of the collaborative scientific efforts that have been undertaken by both WAFCs during their development. All of the data has been provided in GRIB1 code-form for almost 3 years for user trial and evaluation. The transition to GRIB2 code-form will afford higher temporal and spatial resolutions compared to GRIB1.
3.2.Hong Kong Observatoryexpressed user concerns over the use of the term CAT ‘potential’ as opposed to CAT ‘probability’. Bob responded by stating that recent changes to the CAT algorithm have meant that the field could be considered a ‘probability of CAT’. WAFC London is well aware that the CAT algorithm still over predicts CAT associated with mountain waves (MTW), and the intent is to address this during the transition to GRIB2.
3.3.WMO expressed that the three factors that contribute to CAT (namely vertical wind shear, MTW and convection) have a tendency to interact with each other. Bob agreed that these factors cannot be treated in isolation. Both centres are using the Ellrod algorithm to predict shear induced CAT. New, more sophisticated, algorithms can amplify errors in non-linear, differentiated, quantities, and so a certain amount of caution is required.
3.4.IFALPA noted that pilots experience icing in temperatures ranging from PS10 to MS40, whereas the WAFCs only appear to be considering the zero to MS20 range. Bob recognised that icing can occur at those quite warm and very cold temperatures; but that NWP prediction models have errors. These errors are minimised if we (numerically) concentratedattention between zero and MS20, otherwise there is a tendency for a high false alarm rate (which operators may not appreciate). A follow-on question to IFALPA clarified that this pilot was describing Total Air Temperature (TAT) which incorporates compression and is different from Static Air Temperature (SAT) which is what is given in the NWP models. SAT is the temperature of the outside air. TAT is the temperature an airplane "feels". The faster an airplane goes the more friction between the air and the airplane. Friction creates heat. TAT will always be warmer than SAT. Thus the concern for icing at PS10.
3.5.NetSys commented that by obtaining the WAFS gridded data from the two WAFCs, there may be an opportunity for workstation vendors to use the ‘worse case scenario’ from the two datasets, especially if one is more pessimistic than the other. Bob replied by saying that the users will notice subtle differences from both WAFCs data, and that verification statistics will offer users the ability to assess each models characteristics and make informed decisions.
3.6.NetSys further questioned whether the GRIB2 dataset for CAT will have probability and severity. Bob replied by outlining that the WAFC models don’t independently predict probability and severity, although maximum probability may imply severity. Hong Kong Observatory again questioned what was being provided – probability or potential, and what thresholds could/should be applied. Also, that significant events needs to be considered in the verification, rather than trace thresholds which appeared to be presented at the 3rd WAFS Science Coordination Meeting in April 2009. Bob agreed that there was certainly more discussion required with respect to verification.
3.7.IATA expressed that airlines and pilots are concerned with icing occurrence during holding patterns/phases of flight. Whilst it was noted that the typical holding pattern altitudes (levels) would be covered by the icing grids, WMO stressed that the WAFS forecasts are for flight planning purposes – not for the 15-30 minutes duration in a holding pattern, which requires much higher (finer) resolution data (both in terms of space and time). It would be inappropriate to use the WAFS forecasts for this. Bob agreed with WMO remarks, but also noted that aircraft have to have contingent fuel onboard in the event of holding in an icing layer.
3.8.Hong Kong Observatory questioned whether False Alarm Ratio had been considered in the verification in addition to, or as an alternative to, False Alarm Rate. Bob replied that False Alarm Ratio had not been employed to date as a determiner of the level of skill. Instead, the WAFCs have tended to use hit rate vs. false alarm rate.
3.9.WMO drew attention to the cost avoidance of CAT, and that the cost only arises from more severe CAT encounters. Interpolation by time and space may work for CAT, but may not work with convective parameter scheme, say. To interpolate between values could be problematical. IATA outlined that a 15 hour flight, say, will not tend to result in turbulence avoidance towards the end of the flight since the operators are aware that the accuracy of the forecast decreases with time. Bob agreed that it was important to get operators involved in the discussion.
- Verification of CB forecasts
4.1.Bob Lunnon provided an overview of how the verification of CB clouds forecasts (in the gridded and manual SIGWX forecasts) had been conducted using data from a lightning detection system. The Sferics system does not have a choice/range of threshold, merely ‘yes/no’ lightning strike. It is feasible that a different scheme will result in different verification results.
4.2.The results of recent analyses conducted during November 2008, January 2009 and May-July 2009 had identified seasonal variation and latitudinal variations. The results were very sensitive to the exact details of the verification scheme employed. WAFC Washington noted that there had been no changes to the CB algorithm by either WAFC in the period November 2008 to July 2009 to ensure that the statistics could be assess consistently.
4.3.WMO noted that thresholds for convective rain are used by the model to determine areas of CB. This was fine. However, Sferics locations do have variability, and therefore a satellite derived rainfall rate with which to verify the data against may be more suitable. Bob agreed that there are lots of other tools that could be used, including satellite derived rainfall rates.
4.4.Hong Kong Observatorywondered whether the WAFCs were considering towering cumulus (TCU) in the hazards to aviation. While Bob indicated that TCU could be considered, it needs to be understood that there were no defined requirements to provide this information in the CB forecast. Annex 3 only makes references for the WAFC to provide CB clouds. This was further reaffirmed by ICAO remarks that CB had been the only user requirement to arise from previous WAFSOPSG.
- Verification of CAT
5.1.Phil Gill, UK Met Office (WAFC London), presented an objective verification of GRIB and SIGWX CAT forecasts, where a global verification had been conducted in the period November 2008 to May 2009. The intention was to demonstrate the quality of the new gridded forecasts, and to verify the forecasts against GADS (global aircraft data set) observations.
5.2.The verification results appeared to show that the (WAFC London) automated CAT field showed greater skill that the manual (WAFC London) generated CAT forecasts. However, it was important to stress that the sample size needed to be large enough to provide adequate verification. The longer range CAT forecasts appeared to show a good degree of consistency, with little or no drop off in accuracy with lead time.
5.3.Good skill was evident in the 50-90N latitude band for both WAFCs automated products; reasonable skill was evident in the 20-50N latitude band for both WAFCs automated products. For the tropics and southern hemisphere it was difficult to draw any meaningful conclusions, principally due to the lack of sufficient observations with which to verify against.
5.4.Phil continued by outlining that the conversion of CAT gridded forecasts into automated SIGWX ‘objects’ appeared to offer promising results from initial studies by the UK Met Office. This may be something for the group to consider for future SIGWX forecasts in chart form.
5.5.Canada commented that they were surprised that there appeared little drop off in the accuracy of the CAT forecasts through time. Phil agreed that he would have expected to see some drop off; however, this may become more apparent when the new threshold (of DEVG>=4.5m/s) is used for verification.
5.6.Hong Kong Observatory remarked that converting the data into objects will have some impact on performance, and that there is a need for calibration. Phil agreed and noted that much more work was necessary to determine what the user requirement was, and which was best (grids vs. objects). IBL pointed out that the current SIGWX uses objects (in BUFR). What, therefore, will the future automated product be – e.g. 3D data or objects? Bob (Lunnon) replied that the WAFCs don’t have a definitive answer yet. The working assumption is gridded data, but this may change based on the discussion with users. IBL outlined that from the perspective of a visualisation vendor, the gridded data means that descriptions have to be determined (defined) for each field, whereas objects can have the attributes pre-assigned. NetSys agreed that it was a matter of convenience; but that the GRIB2 data could be much more powerful (visually) provided that explicit instructions on visualisation were produced. The group agreed that this cannot be done in isolation, and it must be conducted in a structured manner with all interested parties – providers through to users.
5.7.ICAO outlined that no formal decision on the replacement for manually generated high-level SIGWX has been taken by the WAFSOPSG thus far, and that users must be convinced that the new products are operationally acceptable.
- Verification of WAFS global gridded icing products
6.1.Jennifer Mahoney, NOAA (WAFC Washington), outlined the motivation behind the development of global icing fields for aviation, and that it was attributable to fact that icing is major factor in the flight planning process in terms of fuel loading. The goal of the verification was to evaluate the quality of the WAFCs global icing across a range of domains (global and regional).
6.2.As a representative dataset with which to verify the icing data against, PIREPs were not considered suitable due to their sparseness and poor sampling across all levels. Instead, the NOAA team used an icing verification proxy based on satellite derived products.
6.3.Common findings in the study, conducted November 2008 to January 2009, were that the WAFC London gridded icing had much greater spatial extent than that of WAFC Washington, and that the WAFC London product also gave higher ‘potentials’ of icing. In terms of lead time, the two WAFCs products did not seem to show much change as the lead time increased (i.e. little drop off).
6.4.Overall, the WAFCs icing forecasts tended to provide about the same forecast efficiency, but the WAFC London product provided lower operational risk.
6.5.In verifying the manually produced icing forecasts within SIGWX, the automated (gridded) forecasts from both WAFCs exceeded those prepared manually by the forecasters.
6.6.Hong Kong Observatory noted that different thresholds had been used (trace, moderate, severe), and that the WAFC London data had significantly more ‘yes’ forecasts than the WAFC Washington data. Hong Kong Observatory questioned whether the trace icing was trying to forecast those areas with no icing, since the WAFC Washington tables suggested that they forecast more ‘no’ icing events. Sean Madine, NOAA, agreed with the remark, and commented that the choice of trace in the CLIP threshold had been ‘tuned’ so that it led to a more meaningful comparison. Sean also noted that with respect to bias, the WAFC London data showed greater icing extent in the horizontal and vertical (i.e. greater volume), thus larger areas, and thus larger ‘potential’.
6.7.Germany commented that some slides had shown maximum icing potential, others maximum icing intensity. Sean responded that there are two icing attributes – mean and maximum icing in a layer. Intensity could be considered synonymous with ‘potential’ (with values 0 to +1). Bob Lunnon commented that when considering icing ‘potential’ (range 0 to +1), values in the order of >=0.5 has been shown to reflect severe icing when based on validation with PIREPs.
6.8.Germany questioned whether high potential can imply high severity. Sean commented that, in his opinion, there is a correlation between a potential and intensity.
- Observations on WAFS gridded forecasts
7.1.CM Shun, Hong Kong Observatory, presented a comparison of the trial gridded forecasts available from both WAFCs, and noted that there were a number of compatibility and quality issues that still need to be addressed.
7.2.In respect of compatibility between the two WAFCs, CM noted that the WAFC London CB field was more extensive in the horizontal and vertical compared with the WAFC Washington CB field. With regards to mean in-cloud turbulence, the WAFCs produced data in similar ranges. For maximum icing potential, there appeared significant differences in spatial coverage in the tropics and at high latitudes. With respect to maximum CAT, the WAFC London field appeared less extensive generally.