Peruttu

Script to analyze huge set of data

Hi,

I have a 15 GB excel sheets . many sheets in one folder . all sheets have the same format

I need to have desktop script to convert all this files into 1 sql server database . The script MUST remove duplicates

the main problem here is the size of data make the computer very slow then hang and get "out of memory" exception

i tried to do it myself but i always get these problems

i prefer the script to be written using c#

every month i have new set of data so i will use this script on first day every month

so i want the script to be ready before 1 may 2017

please write in your proposal how will you avoid low speed and "out of memory" exception

i will send link for sample excel file to check on chat

budget : Up to 500$

accuracy is important

Thank You

Taidot: C# -ohjelmointi

Näytä lisää: sql script data set, access sql data set, want write articles paid, php test script speed, script speed test, need script ready, php script speed testen, test php script speed, want write patent application, poker data set, web script speed test, set start page script, php script speed test

About the Employer:
( 4 reviews ) Cairo, Egypt

Projektin tunnus: #13789249

Myönnetty käyttäjälle:

elsayedhafez

Hi, I have seen your project details. I will be very glad to do this job. I guarantee deadline commitment. I hope to discuss more details with you soon...

$25 USD / tunti
(1 arvostelu)
3.7

65 freelanceria on tarjonnut keskimäärin 21 $/tunti tähän työhön

ehsankayani

HI , i have done similar work of combing various CSV and Excel Files and make it into 1 file. Basically we select / break the file into picese taking 100 k every time and transfer etc. Thank you

$21 USD / tunti
(105 arvostelua)
7.5
winmaclin

Hi Ahmed Sobhhy, Thank you to share your requirement with us. we understand situation you have faced as we also encountered the same many times due to large “SQL Script File”. Out of our experienced with it, we cou Lisää

$15 USD / tunti
(8 arvostelua)
6.2
mahmoudgamal008

I have created a c# script before for a customer that load csv files into a sql server database, the script I created works perfectly and it was loading about 1.7GB of files in less than 15 minutes without hanging. The Lisää

$20 USD / tunti
(18 arvostelua)
5.3
mauricemarinus

Hi... Unfotunately Im not going to delve out information regarding how Im going to do it. Reason.. then why would you need me to do it if I tell you how it should be done. :) Anyways... Duplication: The duplicati Lisää

$20 USD / tunti
(47 arvostelua)
5.2
ljnath

Hi i have read your project requirement and i can solve your problem. The memory issue can be handled by processing 1 file at a time in more programmatically optimized way. I am well experienced in this. Please cont Lisää

$22 USD / tunti
(27 arvostelua)
5.0
lukaszoz

Hello, I'm a developer with over 5 years of experience. I work for Pratt and Whitney Canada where i'm responsible for process automation with tools like Access and Excel. I have extensive knowledge regarding C#, VBA Lisää

$21 USD / tunti
(4 arvostelua)
4.7
shiva1888

Hi I have been in this industry for 5 years and such jobs are my daily practice.I can assure you that if you work with me once, you will always work with me for these kind of projects. I will do it with in your bu Lisää

$15 USD / tunti
(22 arvostelua)
4.7
jagdishbhatt38

I will write a program which will take all files one by one and insert into database. I am having more than 8 years of experience in IT industry.I have worked on .net(c#.net,vb.net,asp,asp.net,mvc) with sql serv Lisää

$22 USD / tunti
(15 arvostelua)
4.6
vanitaarora

Hi, I am software developer with 6 years of experience. I have worked on excel load in C# before, not excel big as this much. I tried to find probable solution online. I found that it may be due to OLEDB connection. Lisää

$22 USD / tunti
(8 arvostelua)
4.3
lalitatmt

Hello Sir/Madam, I am having around 13 years of experience in mobile & .NET technologies(C#/VB). I am expert in asp.net MVC, Cordova, Xamarin, Android, IPhone, Web API, asp.net, WPF, AngularJs, TypeScript, ExtJS, YU Lisää

$27 USD / tunti
(5 arvostelua)
4.1
khp31

Hi , Kindly refer below details . Sr. Software Professional having 7.5+ years experience. ****************************************************** ****************************************************** Technol Lisää

$27 USD / tunti
(13 arvostelua)
4.1
Kroshtandt

Hello! Want some code? :) My name is Yaroslav and I'm a Full-stack C# developer from Ukraine! Currently I'm looking for the long time (full time) job, because I have done my previous project. Little boring but IMP Lisää

$20 USD / tunti
(13 arvostelua)
4.2
OrangeTechsol

Hello Sir, I have read your requirement and understood that you like to hire C# developer to Script to analyze huge set of data. I’m pleased to inform you that we have 6+ years of experienced developer team and w Lisää

$15 USD / tunti
(3 arvostelua)
3.7
expertatwork

hello. 15 GB is size of 1 file ? or all files combined ? please reply. I can write a c# app to handle them. thanks

$20 USD / tunti
(18 arvostelua)
3.8
prashushinde9

I want to discuss this project with you further, let me know the best suitable time for you to schedule the meeting, Feel free to message me at any time, i used to be online 14 hrs in a day on this website so probably Lisää

$25 USD / tunti
(4 arvostelua)
3.8
mlasak

To avoid out of memory - you need to not store data in data structures but process them as a stream row by row. If you need to remove duplicates, hash function on the row would do its job (just need to store hashes in Lisää

$30 USD / tunti
(8 arvostelua)
3.7
Creative818

Hello, we worked before together , remember me ? i know these problems you are speaking about i can avoid low speed using threading techniques , i will split up the data into small sized groups so memory wil Lisää

$20 USD / tunti
(5 arvostelua)
3.2
Programmer59

Good Day Sir, We read your description We have lot of experience in this filed we can do that also if you want We are ready to start work from today I am waiting your response thank you Sir.

$22 USD / tunti
(6 arvostelua)
3.3
mahmoud9

Hello Sir , Before to tell you more about me , I have successfully delivered +270 projects for clients across +50 countries in domains like healthcare, e-commerce, social network, chat app, food delivery app Lisää

$15 USD / tunti
(7 arvostelua)
2.9
rkwst

Dear Sir/Madam, We can start immediately RK Websoft Technologies is a software designing and development Team. We are having very experience, enthusiastic and talented team with more than 5 years of experience on .N Lisää

$22 USD / tunti
(7 arvostelua)
3.0