Lead Data Engineer for Data Infrastructure

@mahads31

With over a decade of experience as a software engineer, I've specifically focused on data engineering for the past seven years. My specialty lies in SQL and Python, which facilitates efficient handling of complex data sets. Equally, my proficiency with Spark, PySpark and Scala ensures superior data processing and manipulation capabilities - essential qualities for any capable lead data engineer. Data modeling holds no mystery to me; in fact, I go beyond simple comprehension to bring Kimball and Data Vault methodologies into practice effectively. I believe in system optimization for enhanced performance, scalability, and also keeping costs down. To achieve this, I'm very comfortable working with cloud platforms like AWS or GCP. What sets me apart is my dedicated focus on using technology as an agent of scalable solutions that align with precise business requirements. Given that we live in a rapidly-evolving digital era where remote work has become crucial, my experience in remote collaboration and ability to adapt to global technologies make me the ideal fit for this role. Reach out now and let's discuss how together we can transform your functionality!

@Web1devloper

As a long-time data enthusiast, I've been fortunate enough to work with powerful tools such as Python, SQL and Snowflake, Databricks and BigQuery. My experience also includes ETL/ELT workflows using Airflow, dbt, and Prefect which aligns perfectly with your requirements. To add to that, I have advanced skills in designing data models for analytics and ML use cases, and have worked across multiple sectors including finance, healthcare, e-commerce, and SaaS; all of which has given me extensive knowledge on leveraging data for better business decisions. Another key area where I could greatly contribute is my proficiency in optimizing systems for performance, scalability and cost through expert utilization of Terraform and familiarity with CI/CD workflows. I have hands-on experience with deployment/kubernetes (AWS/GCP/Azure), which I believe will be crucial in ensuring a smooth remote functioning of our architecture while being able to do monthly visits to your Coimbatore office. Considering my 8+ years of experience in Data Analytics & Science, I hold an advanced degree in a relevant field (if that does matter to you) and am an avid open-source contributor. Coupling this with a deep-seated passion for understanding complex datasets and turning them into actionable insights would certainly make me an ideal lead data engineer for this project. Let's maximize the potential of your data together!

@hoanj

? Hi there, I am excited about the opportunity to apply for the Lead Data Engineer position with a focus on Data Platforms & Analytics Infrastructure. With over 8 years of experience in Data Engineering, including 4 years in a lead role, I am confident in my ability to meet and exceed the requirements of this role. I have a strong proficiency in SQL, Python, and PySpark, along with hands-on experience in designing and implementing Lakehouse architecture using Delta Lake and Iceberg. My expertise extends to building scalable batch and real-time data pipelines, optimizing systems for performance and scalability, and mentoring team members to enforce engineering best practices. In approaching this project, I propose to leverage my skills in data modeling (Kimball / Data Vault), streaming systems (Kafka / Flink), and cloud platforms such as AWS. My approach will focus on developing robust ETL/ELT workflows using Airflow and ensuring data quality, governance, and monitoring throughout the data lifecycle. I am eager to contribute to your team's success by delivering reliable and high-performance data solutions that support analytics and AI/ML use cases. Let's connect to discuss how my experience and skills align with your project needs. Huynh Pham

@richardt057

Hello, Scaling a data platform to handle complex AI/ML workloads requires a Systems Engineering approach that balances high-velocity ingestion with strict data governance. As a Software Engineer and Founder, my background in managing information systems ecosystems has honed my focus on structural data organization, warehouse optimization, and building persistent data pipelines. My Strategy for Your Infrastructure: Scalable Lakehouse & ETL Pipeline: I will design your batch and real-time pipelines using Python and SQL, optimizing data models for analytical performance and cost-efficiency. Robust Data Governance: Drawing on my experience with production database schemas (PostgreSQL/SQLite), I will enforce strict monitoring, data validation, and automated error handling across all workflows. Engineering Leadership: I bring a "Founder’s Mindset" to guide your technical team, implementing clean documentation and scalable architectural best practices. Let's discuss how we can build a resilient platform for your data lifecycle. Best regards, Richard TCHODIE Systems & Software Engineer | Data Platforms Architect

@sumitamitdas1

Hello, I am a highly experienced Data Engineer / Big Data Consultant with 15+ years of IT experience specializing in AWS Glue, PySpark, Snowflake SQL, Big Data Hive, ETL Development, Data Warehousing, and Cloud Data Engineering. I have extensive hands-on experience in designing, developing, and optimizing large-scale data pipelines, ETL/ELT workflows, and data transformation solutions for enterprise-grade applications. My Technical Expertise: ✔ AWS Glue – Building scalable ETL pipelines, Glue Jobs, workflow orchestration, and optimization. ✔ PySpark – Processing large-scale datasets, performance tuning, Spark transformations, joins, partitioning, and optimization. ✔ Snowflake SQL – Stored Procedures, complex SQL development, performance tuning, data modeling, Snowflake architecture, ELT pipelines. ✔ Big Data Hive – HiveQL development, partitioned tables, optimization, big data processing. ✔ Cloud & Data Engineering – AWS S3, ETL pipelines, data integration, data migration, reporting datasets, and analytics solutions. I have successfully delivered multiple enterprise projects involving large-volume data processing, cloud migration, and data platform modernization with a strong focus on performance, scalability, and quality delivery. I am confident I can deliver this project efficiently and according to your requirements. I would be happy to discuss the project details and provide the best solution approach. Looking forward to working with you. Best Regards

@Pushpak2509

Hello Sir/Madam, I am interested in the Lead Data Engineer opportunity. I currently work as a Data Engineer with hands-on experience in SQL, Python, PySpark, DBT, Snowflake, and AWS-based data workflows. I have been involved in migrating and modernizing data pipelines, building ETL/ELT solutions, optimizing SQL transformations, and supporting scalable analytics platforms. My experience includes: • Developing ETL/ELT pipelines using SQL, Python, DBT, and PySpark • Working with Snowflake and cloud-based data environments • Data transformation, validation, and performance optimization • Supporting reporting and analytics use cases • Understanding of data modeling concepts and scalable pipeline design • Collaborating with business and technical teams for data delivery I am also continuously expanding my knowledge in modern data engineering technologies such as Databricks, Airflow, Spark optimization, and AI data engineering workflows. Although my experience is slightly below the requested range, I am confident in my technical capabilities, quick learning ability, and commitment to delivering high-quality solutions. I am available to discuss the role further and can join immediately. Thank you for your time and consideration. Best Regards, Pushpak

@Monishap12

I worked as a Data Engineer and Data Analyst with around 3 years of experience in both backend and frontend development. On the backend side, I worked with Azure Databricks, Azure Data Factory, ADLS Gen2, PySpark, SQL, and Unity Catalog. We followed Medallion Architecture where raw data was stored in the Bronze layer, transformed and cleansed in the Silver layer, and business-ready curated tables were maintained in the Gold layer. I developed and maintained ETL pipelines using Azure Data Factory and PySpark notebooks in Databricks for data ingestion, transformation, validation, and aggregation. I also worked with ADLS Gen2 for data storage and Azure Key Vault for secure credential management.

@sumedhkmr1

I do have 7 years of overall experience as python Pyspark developer and Azure Databricks Data Engineer. I do have proficiency in creating scalable data pipelines using ADF and Databricks. My working always being timely and target based so I'll definitely finish the work timely.

@teja3653

With experience in Data Engineering using Python, SQL, PySpark, and ETL development, I have worked on building and maintaining data pipelines, transforming large datasets, and supporting analytics workflows. I am familiar with Databricks, Spark, Airflow, and cloud-based data platforms, along with data warehousing concepts and performance optimization. I have experience collaborating with teams on scalable data solutions and handling batch processing workflows. I am eager to contribute to this project, learn from the team, and deliver reliable results within timelines. I am comfortable working remotely and can support occasional visits to the Coimbatore office if required.