TRAP@NCI

An investigation of the processes, technologies and potential issues involved in the implementation of an end to end data analysis solution.

Brierton, David (2014) An investigation of the processes, technologies and potential issues involved in the implementation of an end to end data analysis solution. Diploma thesis, Dublin, National College of Ireland.

[img]
Preview
PDF (Diploma)
Download (882kB) | Preview

Abstract

The benefits which can be obtained in modern day businesses through successful and accurate interrogation of their data cannot be underestimated. Data analytics is the term used to describe theories and technologies which transform data into meaningful business insights. The use of data analytics and business intelligence methodologies allows organisations to gain strategic advantages, improve processes and assess performance etc.

The purpose of this paper is to investigate approaches to end to end analysis development, particularly those relating to data transformation, storage and visualisation. The datasets utilised were sourced from the OpenFlight web page, and contain data pertaining to worldwide airports, airlines and flight routes. Reference data tables were constructed as appropriate to allow drilldown analysis into the data, for example continent demographics.

Following on from research into various data analysis methodologies and associated programming languages, the platforms to deliver a data mart solution were chosen. Python was used as the platform to import, integrate, cleanse and store data extracts. The transformed data was then stored in a MySQL data mart, using the structures outlined in the project data model.

The R programming language and associated packages were used as a data analysis and visualisation interface to the data model. The final end products are an integrated data mart and information dashboard file, supported and produced by two primary programming scripts – data processing and visualisation.

Item Type: Thesis (Diploma)
Subjects: Q Science > QA Mathematics > Electronic computers. Computer science
T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science
Divisions: School of Computing > Higher Diploma in Science in Data Analytics
Depositing User: CAOIMHE NI MHAICIN
Date Deposited: 15 Dec 2014 11:24
Last Modified: 15 Dec 2014 11:24
URI: http://trap.ncirl.ie/id/eprint/1864

Actions (login required)

View Item View Item