Experienced Data Engineer for Automated Data Collection & Processing

$15-25 USD / hour

Käynnissä

Julkaistu

2 kuukautta sitten

$15-25 USD / hour

[I WILL ONLY CONSIDER YOUR PROPOSAL IF YOU DESCRIBE RELEVANT PAST WORK/PROJECTS IN YOUR PROPOSAL. PLEASE READ THE ENTIRE DESCRIPTION BELOW TO GET A BETTER UNDERSTANDING OF WHAT IS NEEDED. THANK YOU] I am looking for a Data Engineer who can help automate the collection and processing of data, utilizing web scraping technology, APIs, and python libraries like beautiful soup for processing. For this project, the main tasks include: - Designing and implementing effective scraping systems to collect and process data - Integrating to our current system Skills and Experience Ideal candidates for this project should have: - Proven experience in data engineering, API, ETL, and scraping projects. - Proficiency in Python Libraries such as BeautifulSoup or Scrapy. - Demonstrable past work related to data science or data engineering Here are some additional details to help you understand the specific project requirements: Background on What My Clients Need | Information to Help Salespeople: I help sales teams perform better by providing them information about their prospects. My clients (companies) are looking for information that can help their salespeople personalize their outreach to prospective customers. As an example, let’s say I work for Microsoft. The Microsoft sales team will have challenges selling to ABC Manufacturing if they don’t know about the business priorities of ABC Manufacturing. How I Serve my Clients Today | Public APIs and Keyword Matching: If my client Microsoft wants to know the business priorities of ABC Manufacturing, I might use an API to gather the most recent news about ABC Manufacturing given a set of keywords (for example “productivity upgrades”). For example, I might find out in the news that ABC Manufacturing just mentioned in the local news that they are making “productivity upgrades.” I would then send an alert to the salesperson with the keyword + link to the article, so that they can use this as a reason to reach out to ABC Manufacturing. Where I Need Your Help to Serve My Clients | Discover New Information and Process It I need other public APIs to help these salespeople get information on their target accounts. One example would be upcoming government legislation. I would need your help finding the appropriate API to get upcoming legislation. I would also need your help structuring the data in a way that makes it very relevant and easy for sellers to use.

Projektin tunnus (ID): 37835994

Tietoa projektista

68 ehdotukset

Etäprojekti

Aktiivinen kuukausi sitten

Haluatko ansaita rahaa?

Sähköpostiosoite

Freelancerin tarjouskilpailun edut

Aseta budjettisi ja aikataulu

Saa maksu työstäsi

Kuvaile ehdotustasi

Rekisteröinti ja töihin tarjoaminen on ilmaista

Myönnetty käyttäjälle:

@Asadjutt111

Drawing on my expertise in web scraping và Data Engineering, I am the perfect match for your data automation task aimed at enhancing your sales team's performance. My proficiency with python libraries such as BeautifulSoup and prior experience in ETL processes and API integration is exactly what you need to automate data collection and processing efficiently. I have a decade-long track record of delivering quality work, and this precise project aligns with my past data engineering works that involved data scraping, analysis, and integration to enhance decision-making. Your goal of empowering salespeople with personalized insights couldn't resonate more with me! It corresponds well with my passion for unraveling hidden stories in raw information, transforming them into meaningful sales strategies. From utilizing public APIs to collecting target account-specific information like government legislation or business priorities, I will structure the obtained data effectively to make it actionable. By providing your team with timely, relevant, and organized data as per your clients' needs, we can ensure an edge for Microsoft or any other firm we collaborate with.

$15 USD 40 päivässä

0,0

(0 arvostelua)

0,0

68 freelancerit tarjoavat keskimäärin $22 USD/tunti tätä projektia

@MashoodurRehman1

I am a Data Engineer with extensive experience in automating data collection and processing. I have a proven track record in implementing effective scraping systems and integrating them into existing systems. My expertise in Python libraries, such as BeautifulSoup and Scrapy, enables me to deliver efficient solutions. Additionally, I have demonstrable past work in data science and data engineering, making me an ideal candidate for this project.

$25 USD 40 päivässä

5,0

(174 arvostelua)

7,8

@uumarkhalid31

Hello! Experienced Data Engineer adept at automating data collection and processing through advanced web scraping techniques, APIs, and Python libraries. Proven expertise in designing efficient scraping systems and seamless integration into existing frameworks. Ready to elevate your data capabilities for enhanced insights and efficiency. Thank you-

$20 USD 40 päivässä

4,8

(282 arvostelua)

7,3

@schoudhary1553

As an accomplished full stack developer, I have spent seven years honing my skills in web scraping, web automation, and designing useful scripts - all of which are essential for this project. In my career, I have successfully built web scraping systems using technologies you mentioned - like beautiful soup and Scrapy to collect and format data for clients such as Metlife GOSC, DXC Technologies, and Elite Services. This is the same level of proficiency and data science experience that I can provide your clients at Microsoft. My experience extends beyond just data collection; a crucial aspect of this project will be integrating the newly-gathered data into your current system seamlessly. This too, I have diverse experience with. I'm proficient in database (SQL, MySQL) programming that will ensure structured presentation of the information gathered from different APIs to make it relevant and easy-to-use for your sales team. In conclusion, what makes me the best candidate for this project is not just my various skills in Data Mining, Web Scraping, Python etc but also my commitment to quick delivery without compromising on quality. I emphasize on building good and long-term relationships with my employers which will ensure support beyond the scope of the project. Let's discuss cómo we go about serving your clients with better personalization through an efficient, automated data collection and processing solution.

$20 USD 40 päivässä

4,9

(143 arvostelua)

7,4

@LiveExperts

Hi there,I'm biddin on your project "Experienced Data Engineer for Automated Data Collection & Processing" I have read your project description and i'm an expert in Python and machine learning therefore i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the project. Thanks. .. .

$50 USD 67 päivässä

4,9

(29 arvostelua)

7,0

@leadconcept

Greetings Jonathan, I have read initial job details and as far as structuring the data is concerned, that's not an issue but for API, can you share what kind of API you want us to find? For quick response & one-on-one communication, you may click on the chat button, as I am online most of the time. Look forward to hearing from you thru' chat. Regards, Yasir LEADconcept

$20 USD 40 päivässä

5,0

(4 arvostelua)

6,7

@meladbadder

Hi I can automate the collection and processing of data, utilizing web scraping technology, APIs, and python libraries like beautiful soup for processing i created a lot of scrapers / systems to collect data using api for example system collect data from amazon, ecommerce websites, real estate I worked on a lot of similar projects I wait your reply i can start work now Thanks

$15 USD 50 päivässä

5,0

(55 arvostelua)

6,7

@ahmadayaz

With a strong background in Data Engineering and automation, I have a proven track record in web scraping and API integrations. I specialize in designing and implementing scraping systems using Python libraries like BeautifulSoup and Selenium to automate data collection tasks effectively. I have also worked on API integrations and ETL processes to optimize data flows for various purposes. I am confident in my ability to enhance your sales team's performance through data automation tools. To proceed, I would need more details about the APIs you intend to use and your data structuring preferences. Understanding these specifics will help tailor the solution to your exact needs. The project timeline could involve 2-3 weeks for scoping requirements and designing scraping systems, followed by 4-6 weeks for implementation and integration, depending on complexity. For compensation, I propose a structured payment plan based on project milestones. I am open to discussing any budget constraints or preferences you may have to ensure a mutually beneficial agreement. I am excited to delve deeper into your project specifics and work together to exceed your expectations. Let's start a detailed discussion to refine the project scope and begin this transformative project.

$22,50 USD 5 päivässä

5,0

(5 arvostelua)

6,1

@ZohaibRoy

✅ Data Automation Engineer - Streamlining Your Data Processes! ⭐⭐⭐⭐⭐ Hi there, I hope you're doing well. I've reviewed your project requirements and understand you're looking for a Data Engineer to automate data collection and processing using web scraping technology, APIs, and Python libraries like BeautifulSoup. ➡️ Why Me? With extensive experience in data engineering, API integration, and web scraping projects, I assure you efficient and effective solutions tailored to your needs. Let's discuss your specific requirements in detail and explore how I can add value to your project. ➡️ Let's have a quick chat to delve into your project details. I'll showcase relevant past projects where I've successfully implemented similar solutions, demonstrating my expertise and proficiency. ➡️ Skills & Experience: ✅ Data Engineering ✅ API Integration ✅ ETL (Extract, Transform, Load) ✅ Web Scraping ✅ Python Libraries (BeautifulSoup, Scrapy) ✅ Data Science ✅ Data Processing ✅ Data Analysis ✅ Automation ✅ Scripting Looking forward to discussing this with you further! Best Regards, Zohaib

$20 USD 40 päivässä

5,0

(35 arvostelua)

6,0

@avinashdubeyg209

Hi there, I am an ML engineer. I can start right away and deliver within the deadline. So, Let’s have a quick conversation. I can be more specific once we get all the requirements and information required to execute the project. Thank you!!

$20 USD 40 päivässä

5,0

(45 arvostelua)

5,3

@tangramua

Dear Jonathan Y., We carefully studied the description of your project and we can confirm that we understand your needs and are also interested in your project. Our team has the necessary resources to start your project as soon as possible and complete it in a very short time. We are 25 years in this business and our technical specialists have strong experience in Python, Web Scraping, Data Mining, Data Science, Data Engineer and other technologies relevant to your project. Please, review our profile https://www.freelancer.com/u/tangramua where you can find detailed information about our company, our portfolio, and the client's recent reviews. Please contact us via Freelancer Chat to discuss your project in details or let's appoint time for Zoom meeting to meet and discuss your project in details. Best regards, Sales department Tangram Canada Inc.

$25 USD 5 päivässä

3,9

(37 arvostelua)

7,1

@hafeelmo

Hello. How are you? I just carefully read this posting. The role is an ideal match for my skills and experience. I have rich experience in Python, GCE, Google Big Query, Cloud composer, Apache Airflow, Data Warehousing, ETL, Data pipelines, Data Analysis, Database Architecture, Data integration, R, SQL, MATLAB and so on. I led the user-level data sourcing for financial payout processing for one of my clients. The project included migrating an existing sourcing infra from an API-based workflow to a data export-based workflow using Google Cloud Dataflow and Apache Beam for data processing. Also I designed and rolled out the complete Edge layer infrastructure and API Gateways using Google Cloud Endpoints, Envoy, and Google ESP to a complex web app consisting of 80+ microservices and server-less applications. Also I developed for a project involving machine learning using TensorFlow to predict the outcome of payment retries and plan retry patterns leading to a decrease in payment failures. The project involved understanding the payment processing domain, designing warehouses for efficiently analyzing logs at scale using Big Query, and training models to serve the back ends. I love working with teams to conceive and deploy novel data science solutions. I can help you to complete your project as a experienced data engineer. I would love to build an awesome working relationship with you using my expertise. Best Regards

$20 USD 40 päivässä

5,0

(8 arvostelua)

5,3

@devendrathakur12

Hi, How are you? Very happy to bid on your project because my skills fit your project. I have 10 years of experience in Python, specializing in data engineering, API integration, and web scraping projects. I am very familiar with BeautifulSoup and Scrapy, with a proven track record in automating data collection and processing. I have successfully completed similar projects, enabling clients to personalize outreach through targeted data insights and alerts. If you award me, the project will be done perfectly. I will do my best to provide the results you are looking for. If you send the message, we can discuss the project more. Thanks.

$25 USD 40 päivässä

5,0

(17 arvostelua)

5,4

@rashidamjad

Hi there, I'm thrilled to apply for your Experienced Data Engineer for Automated Data Collection & Processing project. With 4-5 years of experience in Web Scraping, Data Science, Data Engineer, Data Mining and Python, I'm confident in my ability to bring valuable insights and expertise to your initiative. Please note that the initial bid is an estimate, and the final quote will be provided after a thorough discussion of the project requirements or upon reviewing any detailed documentation you can share. Could you please share any available detailed documentation? I'm also open to further discussions to explore specific aspects of the project. Feel free to check out my profile, showcasing my portfolio, past jobs, and client reviews. It reflects the quality and professionalism I bring to every project. My goal is to provide a competitive budget without compromising on quality. Thanks for considering my proposal. I'm eager to collaborate and contribute to your project's success. Let me know if you need any more information. Best regards, Rashid Amjad

$35 USD 31 päivässä

5,0

(9 arvostelua)

5,0

@datacode0023

Hi, I have +7 years of experience dealing with machine learning algorithms and worked on multiple projects in this field, Please contact me to discuss more. Have a nice day

$20 USD 40 päivässä

5,0

(13 arvostelua)

5,1

@vorasiddh4it

I have read the project requirements for Data Scraping with Beautifulsoup. Also, if you want to see my past work related to Web Crawler and Scaping, then I will show you. We have 11+ years of experience in software development. We have developed more than 400 projects and research papers in the fields of Data Analysis, machine learning, artificial intelligence, , network, and SEO-based web and mobile apps. We have successfully completed the projects of phishing detection, spam mail filter, shortest path, HMM, encryption decryption, face detection, UML diagram, OCR, big data, data mining, data analysis, statistics, trading, text, natural language processing (NLP), image, multiclass classification using WEKA, Azure ML, Tensorflow, R programming, OpenCV, Matlab, Hadoop, artificial intelligence program using PROLOG, robotics software, TCP-UDP networking project, cloud computing, etc. View my last projects based on web scraping, crawling, elastic search, data mining, machine learning, artificial intelligence, Python, and Java, and I can complete your project perfectly. https://www.freelancer.com/u/vorasiddh4it/portfolio/keyword-search-engine-website-crawler-nlpnlg-locationwise-4832431 Note: The project has QA, testing, and comments in the code, so it's easy to understand the flow of the project.

$30 USD 40 päivässä

4,9

(12 arvostelua)

5,0

@nlivenvw

Hi, I am writing to express my interest in the position of Experienced Data Engineer for Automated Data Collection & Processing at your esteemed organization. With a proven track record in data engineering, a keen eye for automation, and a commitment to optimizing data workflows, I am eager to contribute my skills to enhance the efficiency and accuracy of your data processes. Key Competencies: Data Engineering Expertise: Extensive experience in designing and implementing data engineering solutions. Proficient in creating robust and scalable data pipelines for efficient data processing. Automated Data Collection: Demonstrated ability to design and implement automated systems for data collection. Experience in leveraging tools and frameworks to streamline the data acquisition process. Data Processing Optimization: Proven track record of optimizing data processing workflows for speed and efficiency. Familiarity with best practices in data engineering to ensure the highest data quality. Collaborative Problem-Solving: Adept at collaborating with cross-functional teams to identify and address data-related challenges. Effective communication skills to translate business requirements into data solutions. Sincerely,

$20 USD 40 päivässä

4,7

(2 arvostelua)

4,9

@maksabakanovich

Hello dear client, I am a a senior developer. I have good experience of more than 10 years in this field and have successfully done some projects similar to yours. I fully understood your requirements, you will get a satisfied result. I'm ready to start right away. We can discuss more details via private chat. Best regards.

$20 USD 40 päivässä

5,0

(3 arvostelua)

4,1

@jinufeb14

As an experienced data engineer, I bring the technical prowess and the business acumen necessary for your project. Having successfully designed and implemented numerous scraping systems, collected and processed vast amounts of data, integrated disparate systems, and leveraged APIs to derive actionable insights, I am confident that I can meet your exact requirements. I am well-versed in Python libraries such as BeautifulSoup and Scrapy, tools vital for web scraping and processing large datasets. I have ample experience in ETL processes and building efficient workflows that maximize the relevance and utility of collected data. My skills align perfectly with your requirement to find specific APIs for your sales team to access information about target accounts including upcoming government legislation. Maintaining a high degree of accuracy is paramount when handling data; it's a skill that I'm proud to possess. My consistent success in providing virtual assistance speaks not only to my technical expertise but also to my commitment towards producing top-tier results. With me on your team, you can trust that data collection and processing will be timely, reliable, and accurate, making life easier for your salespeople. What's more? Even after handing over the project to non-technical users, I remain invested in its success by offering contextual support. Let me help you optimize your sales process with automated data collection and processing.

$15 USD 40 päivässä

4,7

(26 arvostelua)

4,3

@MilosDelic0203

Dear Jonathan Y. We went through your project description and it seems like our team is a great fit for this job. We are an expert team which have many years of experience on Python, Web Scraping, Data Mining, Data Science, Data Engineer Please come over chat and discuss your requirement in a detailed way. Regards

$25 USD 7 päivässä

4,8

(4 arvostelua)

4,0

@dataspro

Nice to meet you TheGoodWork01, It is a pleasure to communicate with you. My name is Anthony Muñoz, I am the lead engineer for DSPro IT agency and I would like to offer you my professional services. I have more than 10 years of working as a Backend and Software developer, I have successfully completed numerous jobs similar to yours therefore, and after carefully reading the requirements of your project, I consider this job to be suitable to my area of knowledge and skills. I would love to work together to make this project a reality. I greatly appreciate the time provided and I remain pending for any questions or comments. Feel free to contact me. Greetings

$44 USD 40 päivässä