Find Jobs
Hire Freelancers

AWS API/S3/Lambda Developer Needed

$250-750 USD

Suljettu
Julkaistu yli 5 vuotta sitten

$250-750 USD

Maksettu toimituksen yhteydessä
Established b2b company with local, COLO, cloud and virtualized services needs help on a special AWS project. We need a multi-step utility/development project to utilize a few AWS services as part of a collective “project”. What We Currently Do: -We have hundreds of data partners. They provide us API endpoint access to download updated values/data. - We invoke their API in a .NET solution. - Response data is saved locally. - We have to run multiple API GET’s, liken to looping through page=1, page=2 sometimes. - Requires a good bit of computing power and resources, especially now that we are scaling. What We Want: - We want to invoke one (1) AWS API. By invoking this one API Endpoint all further processing will be done in AWS and return a “job status” code or results/total successful API’s, etc. - We want to pass to AWS API a list of all API URL’s and/or path-variables and/or query string parameters. - This batch list of urls that need to be invoked will be sent somehow to AWS, such as in a HTTPWebRequest Body (JSON). - AWS wil have to “loop through” or “process” list of URL’s. I’m not sure if this is Lambada or Batch, there is no preference except light-weight and serverless. - After AWS API returns a response (which is always in JSON format), we want each response saved as a .json file in an AWS S3 Bucket. - A method to easily query/index S3 Bucket contents. Some buckets will have millions of files. We need to be able to quickly determine if a filename exists (CloudSearch?) Considerations: - You will have to create AWS API Gateway and simply the process for passing querystring parameters for each URL (example: zip = 24018, pagenum = 1) - You will have to create AWS S3 Bucket and consider permissions, access, etc. - You will have to create some sort of “job” or “batch” fuction that will tell AWS API to update each URL in the .json body. - You will have to provide a simple and extremely responsive/quick way to return S3 Bucket file search properties. - You will have to create a new IAM user just for this project. - You must lock down the IAM user to only these resources. - You must limit the region for the IAM user to have access to only US-East1. - The IAM user will have rights to invoke AWS API, create/manage/run Lambda Functions and/or AWS Batch Jobs and S3 Bucket Read/Write permissions for the buckets that only pertain to this project. Pseudo-Code Diagram - Attached is a basic visual diagram of the flow/goal of thisi project. This is not meant to be static nor correct. We need help designing the most efficient and cost effective way to accomplish the goals outlined in the diagram. - It is not mean to be “accurate”, rather an overview of what needs to happen and the net result of how to save the URL response data into S3 Buckets.
Projektin tunnus (ID): 18116064

Tietoa projektista

7 ehdotukset
Etäprojekti
Aktiivinen 5 vuotta sitten

Haluatko ansaita rahaa?

Freelancerin tarjouskilpailun edut

Aseta budjettisi ja aikataulu
Saa maksu työstäsi
Kuvaile ehdotustasi
Rekisteröinti ja töihin tarjoaminen on ilmaista
7 freelancerit tarjoavat keskimäärin $684 USD tätä projektia
Käyttäjän avatar
amazon web services expert
$500 USD 1 päivässä
5,0 (32 arvostelua)
5,6
5,6
Käyttäjän avatar
Hi there, Though there're several points we need to discuss in detail for the moment the requirements look quite clear and straighforward to implement. Apparently, the main operation is not compute bound but rather a network bound task. AWS Batch is specifically designed for compute heavy tasks like rendering, analytics, evolution algos. so it's definitely not something you need. AWS Lambda is a super scalable service. Every time a function is invoked a new instance is spawned. For your use case at each invocation a GET request will be sent to the specified url and the response will be saved in a json file on s3. Since that's a very lightweight operation I don't think there's any use treating the whole thing as a job queue. It's most likely that by the time you get back a "job status code" lambda would already be done with the request. It seems to me API Gateway is really not what you need. SNS would be much more useful in a similar manner. You'd publish json messages, just like you've defined, to a certain topic1. This'd trigger subscriber lambda function. When the function is done and save the data to s3 it'd publish the result to another topic2. You may subsribe to topic2 beforehand to receive those messages. Cloudsearch with S3 would be just fine but I'm not sure if that's the best way to go for your use case. How exactly you want to search these files, by name, content, date, ect.? I expect this to take 10 days at most once the details are cleared.
$1 000 USD 10 päivässä
5,0 (14 arvostelua)
5,3
5,3
Käyttäjän avatar
Hi I am certified aws solution architect with 3+ years of industry experience. I have been working on deploying highly scalable, available, fault-tolerant and optimized solutions using aws clouds. I actually found your project interesting one and want to work on this with you. Let us further discuss the project. I have expertise in following areas. 1. Microservice application development using aws lambda, api gateway, SAM, dynamodb etc. 2. Microservice deployment using Jenkin and Docker, Kuberneetes in aws ECS. 3. EC2(Autoscaling, load balancing, bootstrape automation) 4. RDS, SNS, SQS, SWF, Data analytics stack etc. 5. Shell, python, java, cron jobs. I can do this job for you according to your expectations. Looking forward to hearing from you. Best Regards
$500 USD 10 päivässä
5,0 (8 arvostelua)
4,1
4,1
Käyttäjän avatar
Hi, I went through the requirement and what I feel is you need an async api which has to call multiple rest api's and store the result in some persistent store like dynamo db or S3. Couple of things which will help you to design your solution. 1) Use lambda (It supports execution upto 15mins now and multi threading is supported) 2) Use Dynamo DB instead of s3 for storing the result. It would be much convenient to do some basic CRUD operations in case you want on the reuslts 3) Use aws cloud formation to automate the entire deployment so that future changes can deployed easily. 4) Serverless application Model is awesome and simple. You must try for this task. Even if you are not interested in my proposal please give the above suggestions to the freelancer who is going to work on this. From my experience over 5 years in aws these things are definitely needed for your system in the long run,.
$497 USD 10 päivässä
4,8 (8 arvostelua)
3,8
3,8
Käyttäjän avatar
******************************************************************************************************************************************************************************************************************************************************************************
$750 USD 20 päivässä
0,0 (0 arvostelua)
0,0
0,0
Käyttäjän avatar
Hello there, We are a team of AWS experts with more than 12 years of rich industry experience .We provide DevOps as a service to wide variety of companies ranging from Small Scale to large organisations. Following are our DevOps services along with the plethora of DevOps tools we use : Container Services: Docker,Kubernetes,EcS,rkt Infrastructure provisioning:CloudFormation,Terraform Configuration Management:Ansible,Chef,PuppetLabs,Saltstack CI/CD:Concourse,AWS code pipeline,go,Jenkins Performance gathering:Collectd,Telegraf Metrics Aggregation and analytics:influxDb,graphana,Graphite Build system:Ant,Gradle,Grunt,Ivy,Maven,sbt Environment Building:VirtualBox Alert Management:Cloudwatch,Twilio,Nagios We can assure the best DevOps solutions at the best possible rates and minimal time frames. Thanks for reading the proposal. Regards Chhavi
$555 USD 10 päivässä
0,0 (0 arvostelua)
0,0
0,0
Käyttäjän avatar
I'm working with AWS for 5 years, and currently studying to get my AWS Solutions Architect certification. I'm ready to solution your project and deliver a working cloud application as outlined in the provided doc. I do have some questions, when is a good time to speak by phone? If you award me this project I'd provide an architecture diagram and overview how I'm planning to complete the project. Once you OK it, I'll deliver you the POC, and last but not least the production ready solution.
$988 USD 10 päivässä
0,0 (0 arvostelua)
0,0
0,0

Tietoja asiakkaasta

Maan UNITED STATES lippu
Waxhaw, United States
0,0
0
Liittynyt marrask. 4, 2018

Asiakkaan vahvistus

Muita töitä tältä asiakkaalta

aws api gateway developement
$250-750 USD
Kiitos! Olemme lähettäneet sinulle sähköpostitse linkin, jolla voit lunastaa ilmaisen krediittisi.
Jotain meni pieleen lähetettäessä sähköpostiasi. Yritä uudelleen.
Rekisteröitynyttä käyttäjää Ilmoitettua työtä yhteensä
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Ladataan esikatselua
Lupa myönnetty Geolocation.
Kirjautumisistuntosi on vanhentunut ja sinut on kirjattu ulos. Kirjaudu uudelleen sisään.