Find Jobs
Hire Freelancers

Data Processing Scripts, PHP, Node.js, MySQL etc

£20-250 GBP

Suoritettu
Julkaistu noin 6 vuotta sitten

£20-250 GBP

Maksettu toimituksen yhteydessä
I'll provide more info on successful shortlisting of Freelancers (see criteria below) but essentially what I need you to do is: 1. Create a new Database using the Data Schema I will provide (2 tables - products table with 42 columns/data points, deals table with 63 columns/data points approximately) on my server. The tables are linked by the Product ID field (relationship). 2. Create scripts (PHP or otherwise) that can do the following, in this order: - Download a compressed file from the Retailers / URLs I provide (there are approximately 15-16 Retailers with each one having its own URL) - Unzip the file which will contain a large CSV file (one file could have around 1 million rows of data). - Extract the data in the CSV file and import it into the Database you created in the first action, separating Products and Deals from the CSV using the following rules: 1. For the Products import, the script must check first if the Product ID exists already, if so, the import must skip importing that particular product. 2. For the Deals import, it must first clear the existing deals in the Database from that URL / Retailer which can be done by using the retailer ID. - Sends an email to me with a volume of the products and the deals imported or an email with the error if the import fails. More requirements: - There must be a script per URL so I have control of which scripts I run (if I want to just import the data from one URL manually I can do) - A Cron job will need to be created so I can configure when the scripts run automatically on my server each day - I need a very simple front end HTML page that requires a login to access with a summary of the imports (I will provide flat HTML files for you to use, you'll just need to integrate it with your scripts) and the ability to trigger the scripts. - The server I have will have (at best) 2GB RAM and 2 processors. The scripts need to run optimally and complete quickly (I will allow a maximum of 1 hour for the largest script / import to run), I cannot approve this project as complete unless this happens so please think about how you plan to build this with the potential size of the CSV files.. - Each CSV file will be structured in exactly the same way, so the scripts only need to be created once as they can be applied to all the CSV files that will be downloaded from each Retailer / URL - There are other options when it comes to the files to be downloaded, if you think this would be better File format and compression options available of the files to be downloaded: - XML (no DTD) - XML (DTD 1.5) - XML (DTD 1.4) - CSV Compression: - ZIP - GZIP
Projektin tunnus (ID): 16083400

Tietoa projektista

12 ehdotukset
Etäprojekti
Aktiivinen 6 vuotta sitten

Haluatko ansaita rahaa?

Freelancerin tarjouskilpailun edut

Aseta budjettisi ja aikataulu
Saa maksu työstäsi
Kuvaile ehdotustasi
Rekisteröinti ja töihin tarjoaminen on ilmaista
Myönnetty käyttäjälle:
Käyttäjän avatar
I have carefully reviewed the existing code and identified two specific bugs which were causing the CSV included in the dropbox archive to fail the import process. I have further already identified which of the criteria listed below are covered by the existing code and which are still missing or require modification. I can turn the project around quickly and will offer reasonable follow up support. Establishing a connection with me will also be a good long term freelancer/business relationship for future projects. I am very knowledgeable in php, python, mysql, and linux shell scripting.
£175 GBP 6 päivässä
5,0 (48 arvostelua)
6,2
6,2
12 freelancers are bidding on average £154 GBP for this job
Käyttäjän avatar
Hi There, I am good in PHP, MySql and Node. I have done several projects in these areas successfully. I have done importing data from external sources and load it in to db and sending email alerts. I can do your project successfully Thanks
£210 GBP 7 päivässä
4,9 (1182 arvostelua)
9,0
9,0
Käyttäjän avatar
How are you? I am interested in your project. I think I can do your project. I have good skills in your project area. Thanks.
£150 GBP 3 päivässä
5,0 (124 arvostelua)
8,3
8,3
Käyttäjän avatar
Hi. I propose to implement the solution in Perl. I can create either a MySQL or PostgreSQL database. Perl's DBI module can handle both. Downloading the remote CSV files (compressed or uncompressed) should be straightforward using Perl's LWP framework. Prevention of duplicate records in the product table could be accomplished by setting the product_id column as the PRIMARY KEY. I think the URL could be passed to the script as a parameter - obviating the need for separate scripts. (This should be possible whether the script is invoked from a CRON job, directly on the command-line - or from an admin web panel.) I can test and develop on my own server initially, but SSH access to your server at some point would be useful (if not necessary). If it makes thing easier, I can create a Git repository on Bitbucket - or my own server - to allow you to keep track of the source code easily.
£125 GBP 5 päivässä
5,0 (6 arvostelua)
5,9
5,9
Käyttäjän avatar
Project Description I'll provide more info on successful shortlisting of Freelancers (see criteria below) but essentially what I need you to do is:
£100 GBP 7 päivässä
4,9 (87 arvostelua)
5,6
5,6
Käyttäjän avatar
A proposal has not yet been provided
£190 GBP 6 päivässä
5,0 (11 arvostelua)
5,1
5,1
Käyttäjän avatar
A proposal has not yet been provided
£100 GBP 3 päivässä
4,9 (100 arvostelua)
4,5
4,5
Käyttäjän avatar
Hello Sir, I have a very good experience in working on this technology. I have 5+ years of experience working in SQL and related database technologies. I understand you want to create a database and then import the data from a url to this database. I can certainly help you to fulfill your requirement. Looking forward to work with you and discuss more on this in detail.
£150 GBP 3 päivässä
4,8 (9 arvostelua)
3,9
3,9

Tietoja asiakkaasta

Maan UNITED KINGDOM lippu
Barnsley, United Kingdom
5,0
50
Maksutapa vahvistettu
Liittynyt maalisk. 1, 2015

Asiakkaan vahvistus

Kiitos! Olemme lähettäneet sinulle sähköpostitse linkin, jolla voit lunastaa ilmaisen krediittisi.
Jotain meni pieleen lähetettäessä sähköpostiasi. Yritä uudelleen.
Rekisteröitynyttä käyttäjää Ilmoitettua työtä yhteensä
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Ladataan esikatselua
Lupa myönnetty Geolocation.
Kirjautumisistuntosi on vanhentunut ja sinut on kirjattu ulos. Kirjaudu uudelleen sisään.