Script to analyze huge set of data


I have a 15 GB excel sheets . many sheets in one folder . all sheets have the same format

I need to have desktop script to convert all this files into 1 sql server database . The script MUST remove duplicates

the main problem here is the size of data make the computer very slow then hang and get "out of memory" exception

i tried to do it myself but i always get these problems

i prefer the script to be written using c#

every month i have new set of data so i will use this script on first day every month

so i want the script to be ready before 1 may 2017

please write in your proposal how will you avoid low speed and "out of memory" exception

i will send link for sample excel file to check on chat

budget : Up to 500$

accuracy is important

Thank You

Taidot: C# -ohjelmointi

Näytä lisää: mplementing fisher LDA method using MATLAB. - Write functions that take a data set and compute the optimal projection v, sql script data set, set joomla script ready cron, want write adult content freelance, access sql data set, want write articles paid, purchasing gpt script ready use, datepicker asp net data set, php test script speed, script speed test, want write whole letter french translation, script unique low bid auction, economic stat tools data set, need script ready, php script speed testen, test php script speed, want write patent application, download php script speed test site, fetch data excel sheet perl script, free php script speed test, poker data set, web script speed test, fast typing speed low accuracy, set start page script, php script speed test

Tietoa työnantajasta:
( 4 arvostelua ) Cairo, Egypt

Projektin tunnus: #13789249

Myönnetty käyttäjälle:


Hi, I have seen your project details. I will be very glad to do this job. I guarantee deadline commitment. I hope to discuss more details with you soon...

$25 USD / tunti
(1 arvostelu)

64 freelanceria on tarjonnut keskimäärin 21 $/tunti tähän työhön


HI , i have done similar work of combing various CSV and Excel Files and make it into 1 file. Basically we select / break the file into picese taking 100 k every time and transfer etc. Thank you

$21 USD / tunti
(106 arvostelua)

Hi Ahmed Sobhhy, Thank you to share your requirement with us. we understand situation you have faced as we also encountered the same many times due to large “SQL Script File”. Out of our experienced with it, we cou Lisää

$15 USD / tunti
(8 arvostelua)

Hi i have read your project requirement and i can solve your problem. The memory issue can be handled by processing 1 file at a time in more programmatically optimized way. I am well experienced in this. Please cont Lisää

$22 USD / tunti
(30 arvostelua)

Hi , Kindly refer below details . Sr. Software Professional having 7.5+ years experience. ****************************************************** ****************************************************** Technol Lisää

$27 USD / tunti
(17 arvostelua)

Hi... Unfotunately Im not going to delve out information regarding how Im going to do it. Reason.. then why would you need me to do it if I tell you how it should be done. :) Anyways... Duplication: The duplicati Lisää

$20 USD / tunti
(52 arvostelua)

I have created a c# script before for a customer that load csv files into a sql server database, the script I created works perfectly and it was loading about 1.7GB of files in less than 15 minutes without hanging. The Lisää

$20 USD / tunti
(18 arvostelua)

Hi I have been in this industry for 5 years and such jobs are my daily practice.I can assure you that if you work with me once, you will always work with me for these kind of projects. I will do it with in your bu Lisää

$15 USD / tunti
(22 arvostelua)

Hello, I'm a developer with over 5 years of experience. I work for Pratt and Whitney Canada where i'm responsible for process automation with tools like Access and Excel. I have extensive knowledge regarding C#, VBA Lisää

$21 USD / tunti
(4 arvostelua)

I will write a program which will take all files one by one and insert into database. I am having more than 8 years of experience in IT industry.I have worked on .net(,,asp,,mvc) with sql serv Lisää

$22 USD / tunti
(15 arvostelua)

Hi, I am software developer with 6 years of experience. I have worked on excel load in C# before, not excel big as this much. I tried to find probable solution online. I found that it may be due to OLEDB connection. Lisää

$22 USD / tunti
(8 arvostelua)

Hello Sir/Madam, I am having around 13 years of experience in mobile & .NET technologies(C#/VB). I am expert in MVC, Cordova, Xamarin, Android, IPhone, Web API,, WPF, AngularJs, TypeScript, ExtJS, YU Lisää

$27 USD / tunti
(5 arvostelua)

hello. 15 GB is size of 1 file ? or all files combined ? please reply. I can write a c# app to handle them. thanks

$20 USD / tunti
(19 arvostelua)

Hello! Want some code? :) My name is Yaroslav and I'm a Full-stack C# developer from Ukraine! Currently I'm looking for the long time (full time) job, because I have done my previous project. Little boring but IMP Lisää

$20 USD / tunti
(13 arvostelua)

Hello Sir, I have read your requirement and understood that you like to hire C# developer to Script to analyze huge set of data. I’m pleased to inform you that we have 6+ years of experienced developer team and w Lisää

$15 USD / tunti
(3 arvostelua)

I want to discuss this project with you further, let me know the best suitable time for you to schedule the meeting, Feel free to message me at any time, i used to be online 14 hrs in a day on this website so probably Lisää

$25 USD / tunti
(4 arvostelua)

To avoid out of memory - you need to not store data in data structures but process them as a stream row by row. If you need to remove duplicates, hash function on the row would do its job (just need to store hashes in Lisää

$30 USD / tunti
(8 arvostelua)

I can create desktop c# application for this task. when you select a folder then read excel data and import to sql server db.I can complete this task within 28apr2017 I have more than 14 years experience in C#, MVC Lisää

$15 USD / tunti
(6 arvostelua)

Good Day Sir, We read your description We have lot of experience in this filed we can do that also if you want We are ready to start work from today I am waiting your response thank you Sir.

$22 USD / tunti
(6 arvostelua)

Dear Sir/Madam, We can start immediately RK Websoft Technologies is a software designing and development Team. We are having very experience, enthusiastic and talented team with more than 5 years of experience on .N Lisää

$22 USD / tunti
(7 arvostelua)

Krishantha (Software Engineer) Windows base software develop using C#, vb .net MS SQL Crystal report. I have more than 5 year experience as software engineer. Graduated in university of Moratuwa,Sri lanka- Bachelor of Lisää

$22 USD / tunti
(1 arvostelu)