working with large CSV and txt files.. searching for a value from CSV file in the txt file, if exists printing the line.
as of now, my code works fine but takes long time. need a code that can quickly search the values of CSV in txt file and display them.
CSV file around 6kb but txt files are around 15,000 kb.
I can't be sharing the input files but if you can help me with the code/idea to quickly get this done, that's enough. it's just sharing the idea or way to do than actually working on your own.
Ready Faster Tool , I have similar Project done where searching for a value from Excel file in the CSV file, if exists printing the line.
So i can send you my Ready Application as Trial , so that you can test performance .. then we can talk further if any improvement ...
Hi there,
Yes! I'll do that. Let me write a fast solution script (in Python). You probably need to implement mult-threaded solution. It speed ups the process incredibly. I by myself use it to speed up the processes. I helped a client to speed up his process from 3mins to 3secs, using multi-threaded setup.
Send me few sample input files, to get me started.
Best regards,
Ishaq
Hi there, I can help you with some idea's if you can describe or send the sample lines stored in txt file can decide something based on the nature of data you are working thanks.
Hello,
I can help with the code. You can try "bisect" module. Multithreading or multiprocessing might work. And always faster to use build in functions and one line for loops. "itertools" module also might help. But i think the best way is to use hash table search (google also uses that), you can try to make "dict" and search with "set" function. Best
Hi,
My name is Milos from Serbia who is specialized in Web Dev.
I am familiar with Python and I have experience with similar projects.
I will offer the best quality for your project based on your feedback.
Thanks.
Hi,
I am an experienced and good knowledge in c#.NET. I have developed these kind applications using .NET.
Your Requirement is
Searching for a value from CSV file in the txt file, if exists printing the line.
I can handle very well.
Give me a chance i will do my best.
Thanks
Nagaraj
I have been working at Microsoft Research Asia for 4+ years.
My last project was about an optimized search among GB-level text files(which is the metadata of a face reco training set).
I have done it pretty fast, and willing to provide you with ideas and possible approaches.
And, yeah, that was also done with python.
Hello Sir/Ma'am,
I am professional in this field and I can do this task easily. I am passionate about my work and I am focusing on quality 100% customer satisfaction. I’m a permanent freelancer on Freelancer.com. And I have done lot of MS office work and I can complete this work in very short time with 99.9% accuracy. I guaranteed you will be glad to see my amazing work. Waiting for your reply thank you.
Regards,
Aryansh Gupta
Hi there, hope you are having a good day. I am a professional software developer with more than 4 years of experience. I love to optimize code to make it run smoothly and efficiently. I will be glad to help you reduce the time taken for searching.
Please take a visit to my profile. Looking forward to hearing from you soon. Thanks in advance.
Hi,
Can the above project be done in r? If so I can help you write a be better code that avoids loop and saves on time. Let me know if you are interested.
Hi, I am 3 years experienced in python and data engineering. I can help you with that. Some hints might be - you need to improve your search function and get better way to read that big text file. let's chat more over chat.
