2015 ASEE Northeast Section Conference

2015 ASEE Northeast Section Conference

Weather Data Analysis using Hadoop to Mitigate Event Planning Disasters

Khaled Almgren, Saud Alshahrani, Jeongkyu Lee
Department of Computer Science, University of Bridgeport, CT

Big data is data with enormous size that is very difficult to process. However, once it's processed and analyzed, we can get great knowledge out of it. Hadoop allow us to process big data by using Hadoop Distributed File System (HDFS) and MapReduce. HDFS is used to manage the files and break the data into blocks, and distribute it across clusters of machines. MapReduce will distribute tasks that perform map and reduce operations across multiple nodes.

In the united states, there are many events that occur all around the year in different cities. These events could take place outdoor or indoor. Organizations who host their events outside events such as car shows, concerts, bazaars, festivals, camping, etc, suffers a lot from frequent weather changes which happens a lot due to global warming. They need to plan and choose the date for their event months in advance so they can market for it.

This research presents the design and implementation of weather data analysis using Hadoop distributed system, which can be used for planning outdoor events. The proposed event planning system decides how many appropriate days for outdoor events and activities per month for a different attractive cities based on the analysis of historical weather data. All collected data are stored at HDFS, i.e., Hadoop Distributed File System, and then they are processed and analyzed by using MapReduce programming. As results, we can discover useful information about event planning, such as locations (city), time and statistical data