There is a simple project on Python testing.
Pyspark unit test cases need to be developed.
Roughly 10 functions and likely to take a few hours
for total work for experienced developer.
Any IDE like intellij or Pycharm will do.
Unit test library name is "Unittest" , using "mock".
Details of what the function does, variable details, and expected results.
Functions will involve pyspark dataframes and udfs (user defined functions).
Input1 (variable1) is json text data with multiple nested key value pairs.
Input2 is a list of keys from a json file (variable2) ,
output will be these attributes in json format (variable3).
Some functions will involve returning results.
Some are only to call other functions without a return value.
Set of python pyspark functions involving dataframes, uds,
general pyspark operations such as join, select, python lambdas,
reading and writing files from and to S3,
handling different spark data types like integers, arrays, strings
and different data formats such as csv, json.
7 freelanceria on tarjonnut keskimäärin ₹9301 tähän työhön
hi I'm professional python developer with 6 plus years of experience and I have experience in test automation also. have used unittest, pytest and robot framework also..so I can do this task..
Hi, I am an experienced Data Engineer with a solid background in Spark. I have also experience in writing units tests in Python and Scala. Let's have a call for more details about the project. Regards