A SPARK PROJECT!

We have this project and this data

What we need is:

1. Use Spark ML library instead of SKilearn we want to change SKilearn to Spark ML

2. Add Graphs to show the result

3. Add pictures to show the data if that possible

4. Write comments in each line to understand what you have done

5. Make the code simple and use DataFrame and RDD

The most important is to make it without SKilearn what we need is Spark ML

explain the data first through questions like what is the general sentiment of each airways company and the average confidence or what is the most airways being used and is there a correlation between the sentiment of the company and the number of tweets you have about it? By coding, if that possible

the whole thing is just to modified this project to be Spark ML instead of Skilearn and add Graphs and pictures and explain the data

tiedonlouhinta Python Spark

Projektin tunnus: #16780811

Tietoa projektista