Collaborative Session Notes2017 NOAA EDM Workshop
Session 6B:
Ship Automatic Identification System (AIS) for everyone
Notes:
These notes are to augment the slides presented.
6B.1Helping the Coastal Management Community Access AISDaniel R Martin
-Supports MarineCadastre
-Back in 2007 worked with RI on AIS information for state waters for the coastal zone program
-Initial requirements looked at cover from river basins up through continental
-Sample rate: 1-min sample rate solved most of the CZM community needs
-Create a data model for AIS use (original FGDB model)
-Broadcast scheme, joined with vessel info, joined with voyage info
-Processing of AIS data broken into UTM zones for easier processing
-Can get 2009-2014 data now
-Created an AIS Data Handler for ArcGIS.
-Now deprecated, but available if needed
-Useful manual
-Have a TrackBuilder Toolbox now still supported - a python package - sets variables for track creation. Stop time/distance filters/etc..
-Just held interagency working group, led by Jorge@CG
-Questions: marinetraffic data - how does it compare?
-OCM fills long-term data niche
-Is CG’s NAIS Class A and/or Class B? Think that 2015/16 includes ClassB
-
6B.2Using AIS to determine Hydrographic Survey PrioritiesLucy Hick
Specific use case - OCS “hydrographic health”
Hydrographic health model is a new way of looking at the state of hydro surveys
-Looking at the full EEZ, not just local/regional
-AIS is a key input
- AVIS database of vessil characteristics from CG
-Had some funding from the UK HO to fund Axiom to help with the Big Data processing of the unique counts of vessels in a 500m cell
-Removed the “ferry problem” from the equation to remove duplicate vessel records
-Used 2015 data, broken by several categores (tanker/cargo/other/etc)
-3ksqmi survey area includes all survey vessels - NOAA and contract
-
6B.3Developing Scalable Data Management Solutions for Large Scale AIS data and Beyond Rob Bochenek
-Apache Spark was very successful in breaking up the AIS data into chunks that had fantastic performance for the cluster processing of the OCS (Lucy Hick’s presentation) data
-Spark is written in Scala, so Scala can be a lot faster
-Spark is written to auto deal with failures of tasks
-Testing Alluxio - a virtual distributed file system to help with the cluster processing
-Questions
-Multiple recievers for a ship ping were filtered out
-Wondering if NAIS produces any filtered info
-Are intermediate datasets available? No
-Potential for a web-based tool for end-user to for cliip-n-ship based on user needs? Lucy would like that / gauging interest
-Can this be run in the cloud? Yes - AWS may cost $40/hour, but transfer speeds end up being an issue, so local IO just ends up being better
-