We have a text file of government data that is 653.9 MB uncompressed it is the US DOT Federal Motor Carrier Safety Administration census file. The file contains tabular data in a plain text file each record contains approximately 170 fields and there are 1,497,640 lines or records in the file. Additionally the file is laid out like a CSV file but instead of using CSV delimited values it uses a tilde (~) delimited value.
Our project requires a programmer to write a runtime application that can take this file and parse out each record and each field and then submit the fields into a MySQL database. The app can interface with the SQL server or export the data to CSV or SQL dump files as long as the files are not too large to be imported and the data ends up in a SQL database.
Additional terms of the project.
We require a debugged compiled runtime version of the software that does what's described above and will run in Mac OS X Mavericks 10.10. If you prefer to deliver a Java, C++ Terminal File or Obj C Coacoa based interface we as long as it will run on Mac OS in JRE or Native Environment Coacoa or Terminal that's up to you.
We also require non-compete/NDA/IP rights to the work project.
We also require delivery of source code, documentation for any APIs or libraries used to create the project.
We will provide a copy of the data file via a link from our google drive.
For your review here are a few commands showing the first 10 lines of the file and a word count of the file.
FYI when we try to import the data with Excel using the data import utility it crashes Excel -- it's a big file.
Here is a link to a technical description of the fields contained in the data file:
[login to view URL]
Here is an overview of the datafile:
[login to view URL]
Here is a brief 2 line sample and word count:
A~C ~1018~VERNON PUNT~PUNT TRUCKING~~US~08~26285 389TH AVENUE~STICKNEY~003~SD~57375-6235~~6057324422~~6057324677~US~26285 389TH AVENUE~STICKNEY~003~SD~57375-6235~~46~F~MC~157777~~~~~A~~A~~~~~~1~~~~~~X~X~~~X~~~~~X~X~~~~~~~~~~~~~~~~N~0~6~11~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~6~0~6~C~0~4~~0~0~~0~4~4~~~~~~550000~500000~~~~~~~COMPASS0000001224~109~44~~2012~19740601~20131029~~~~~LS056SD~~20130827~1~~N~~VERNON PUNT~ROGER PUNT
A~C ~1034~JAMIE ADAMS~ADAM'S AUTO BODY~~US~08~515 NORTH 1400 EAST~ST GEORGE~053~UT~84770~~8016733800~~~US~515 NORTH 1400 EAST~ST GEORGE~053~UT~84770~~49~B~~~~~~~B~~A~~~~~~1~~~~X~~~~~~~~~~~~~~~~~~~~~~~~~~~~N~1~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~0~1~0~1~A~1~0~1~0~0~0~0~1~0~S~9095~19870519~~~0~5000~C~19871015~~~~~LS75WC1~109~33~~~19740601~20020128~~~~~LS056SD~~~3~~~~~
jamesmbp:~ james$ wc -l /Users/james/Desktop/[login to view URL]
1497640 /Users/james/Desktop/[login to view URL]
18 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön
I like jobs with huge data and will be happy to help you. I prefer to deliver the Java application. I know many languages, but Java is simpler for me. I have Mac OS X for debugging.
I have Java and SQL programming experience (more than 3 years), I'm a bachelor degree student and I can deliver a good source code with clear documentation.