PySpark program to generate large dataset with 100 000 columns and 50 million rows

$10-30 USD

Käynnissä

Julkaistu

yli vuosi sitten

$10-30 USD

Maksettu toimituksen yhteydessä

I need a PySpark program to generate a large dataset with 100 000 columns and 50 million rows. I should be able to set the number of dimension columns (i.e. columns with non-numeric values such as Country, State, Suburb, Product etc). The rest of the columns must be all numerically populated with random floating-point numbers. The program's output needs to save the data to a single parquet or CSV file. I need to be able to set up the dimension values with CSV tables in the format below. Dimension name: Country File name: [login to view URL] File contents: 1,United States 2,United Arab Emerates 3,Saudi Arabia Random numbers must be picked from the file above to populate the dimensions.

PySpark

Spark

Apache Spark

Projektin tunnus (ID): 34964959

Tietoa projektista

3 ehdotukset

Etäprojekti

Aktiivinen 2 vuotta sitten

Haluatko ansaita rahaa?

Sähköpostiosoite

Freelancerin tarjouskilpailun edut

Aseta budjettisi ja aikataulu

Saa maksu työstäsi

Kuvaile ehdotustasi

Rekisteröinti ja töihin tarjoaminen on ilmaista

Myönnetty käyttäjälle:

@bizzarriandrea81

Hi, I'm a senior data scientist and Python developer with more than 8 years experience. As a data scientist, I collaborated with several machine learning engineers and data engineers to design, develop, and evaluate some ML models on large datasets. I have hands-on experience on big data analysis and data visualization. I also have strong cooperation and communication skills. PERSONAL SKILLS 1. Data Science with Python and R 2. Machine Learning and Statistical Analysis with Python and R 3. Data Visualization through Tableau and Grafana 4. Data Analysis with Python(Numpy, Pandas) 5. DBMS(MySQL, Postgresql, MongoDB, Hive, Cassandra) 6. Deep Learning & Computer Vision(OpenCV, Keras, Tensorflow, Pytorch) 7. Natural Language Processing 8. Big data analysis with PySpark "Satisfy the client with my ability and passion" This is my slogan here. I hope you will be interested in me. Thanks.

$30 USD 2 päivässä

5,0

(1 arvostelu)

1,8

3 freelancerit tarjoavat keskimäärin $27 USD tätä projektia

@Valuesolutions

Hello, I hope this finds you well. I have just seen your project requiring; Spark PySpark Apache Spark I believe that my 10-year experience in this field is what you need right away. Avoid the headache of looking further. Let's save time and focus on the real task. My proposed timelines and budget are just placeholder and an open for negotiation to increase or decrease as per the full requirements at hand. Allow me to prove how better my review can be. Smile all the way! Click the message button, so initiate the conversation. Regards, Fridah

$20 USD 7 päivässä

0,0

(0 arvostelua)

0,0

@albanusklo

Dear Client, Greetings and hoping you are doing well, i welcome you to my profile where quality and client satisfaction is the Priority. CERTIFIED EXPERT IN LISTED PROJECT SKILLS & REQUIREMENTS, After KEENLY reading your description and being in possession of all CLEARLY STATED REQUIRED SKILLS (Apache Spark, Spark and PySpark) as this is my area of PROFESSIONAL SPECIALISATION having the above QUALIFICATIONS, i hereby humbly request you to consider my bid for QUALITY and PROFESSIONAL work. ###// No timeline & Quality Excuses// Unlimited Revisions// 100% SATISFACTION GUARANTEED### Message Me We Discuss More About The Project. incase the project is deleted please hire me directly using the following link; https://www.freelancer.com/u/topgradeclubltd THANK-YOU & WELCOME.

$30 USD 1 päivässä