Informatica powercenter etl jobs

...pipeline—from ingestion through transformation to storage—so that the front-end team always has clean, query-ready information. You’ll also dive into exploratory and diagnostic analysis, helping us uncover trends that inform product decisions and power the visual stories we show users. The stack is flexible, but you should be comfortable with modern data-engineering tools (think Python or Scala for ETL, SQL-based warehouses, stream processing frameworks, containerised deployments, and cloud services such as AWS or GCP). On the web side we are leaning toward a REST-or GraphQL-driven API that feeds a JavaScript visualisation layer. If you have a preferred alternative that achieves the same performance and scalability goals, I’m open to it. By the end o...

...infrastructure to operationalize, monitor, and scale models built by data science teams. Key Responsibilities Pipeline & Feature Engineering: Design, automate, and scale robust data pipelines for AI/ML workloads. Build and manage feature stores to support model training and inference. Data Integration: Ingest and transform structured and unstructured data sources using AWS Glue, Snowflake, and Informatica Data Management Cloud (IDMC). MLOps & Automation: Implement DevOps/MLOps practices and CI/CD pipelines (via GitHub Actions). Monitor pipeline performance, troubleshoot production issues, and mitigate model drift. Governance-by-Design: Implement strict data quality checks, metadata tagging, and end-to-end lineage tracking in compliance with enterprise data govern...

I already have the raw HR analytics dataset from Kaggle sitting in a MySQL database, populated each time my existing Python ETL script runs. The broad pipeline works; now I want to drill down on one thing only: how entry-level attrition is evolving over time. Here’s what I need from you: • A clean, well-commented SQL query (or set of queries) that calculates monthly and quarterly attrition rates for entry-level employees. • A short Python (pandas) routine that executes those queries, pushes the results to an Excel file, and time-stamps each export so historical runs are preserved. Acceptance criteria 1. SQL returns the correct counts and percentages when spot-checked against sample rows in MySQL. 2. The Python script runs from the command line without ...

I have a clean, production-ready operational dataset and I now want to turn it into an engaging, fully interactive Power BI dashboard. The data connection is already set up—no ETL work is needed—so the focus is purely on translating raw numbers into clear visuals, drill-downs and KPIs that management can act on immediately. Because the team is based in Hyderabad, in-person collaboration is required. You’ll work in Power BI Desktop and publish to our Power BI Service workspace, adding any DAX measures. I have dashboard with me, but it is creating lot of performance issues. So want to get it optimized.

...in his own n8n cloud instance. • Incorporate quick comparisons to Zapier and Make whenever helpful so he understands when to choose each platform. • Provide small homework tasks, sample data, and code snippets, then review them in the next call. • Gradually guide him through a capstone project that pulls real-world APIs relevant to analytics (e.g., marketing or finance data), automates the ETL, and produces a dashboard-ready output. Acceptance criteria 1. By the end of the engagement he can independently create, debug, and document multi-step automations using webhooks, API credentials, and custom functions in n8n. 2. All lesson recordings, workflow files (.json), and code snippets are shared in an organised Google Drive or GitHub repo. 3. A brief prog...

...propose the right technical solution Experience with enterprise tools (CRMs, ERPs, internal databases) a strong plus Excellent problem-solving skills and clear technical communication Skills Required: Skills Python, Advanced Python, Business Process Automation, Complex Workflow Automation, Enterprise Automation, System Integration, API Integration, REST API, Webhooks, OAuth, Middleware Development, ETL, Data Pipeline, Database Integration, SQL, NoSQL, CRM Integration, ERP Integration, Microsoft Power Automate, Zapier, Make, Integromat, n8n, RPA, Process Re-engineering, Conditional Logic, Decision Automation, Multi-System Automation, Error Handling, Exception Handling, Logging, Monitoring, Scalable Automation, Cloud Automation, Google Workspace, Microsoft 365...

...Databricks cost/performance optimization and architectural trade-off judgment Hands-on: DBRX, Agent Bricks, Genie, Unity Catalog, Delta Lake, MLflow Token-efficient agentic AI/LLM system design Can implement full production solutions, not just advise Deliverables: [e.g., architecture doc, working pipeline, cost audit] Skills: Apache Spark, Machine Learning, Data Engineering, AI, Cloud Computing, Python, ETL Can you please explain your experience in databricks platform, end to end deployment, monitoring, ml lifecycle management, experimentation, model provisioning, model serving, alerting monitoring, retraining, data drift, model drift, vm design for databricks workspace, databricks optimization and performance techniques, canary deployment on databricks, databricks genie, agent...

...contract. The primary focus of this role is implementing enterprise data masking solutions as part of Data Migration and ETL execution across Enterprise Data Hub (EDH) environments. Candidates must have prior hands-on experience designing and implementing data masking during large-scale data migration and ETL/ELT projects. Experience in SQL Server Dynamic Data Masking (DDM), Azure Data Factory (ADF), Snowflake, and Databricks is essential. The role requires collaboration with US stakeholders and flexibility to work during overlapping US business hours. --- Primary Requirement (Mandatory) Proven experience implementing Data Masking as part of Data Migration and ETL/ELT execution in Enterprise Data Hub (EDH) environments. Hands-on experience masking sensitive d...

...Catalog * Workflows * Delta Live Tables * Auto Loader * Photon * Serverless * Lakehouse * Medallion Architecture --- ### Delta Lake * Time Travel * Merge * Vacuum * Optimize * Z-Ordering * Transaction Log * Schema Evolution * Schema Enforcement --- ### Snowflake * Warehouse * Virtual Warehouse * Clustering * Streams * Tasks * Dynamic Tables * Iceberg * Time Travel --- ### System Design * ETL Design * Data Lake Design * CDC * Streaming Pipelines * Incremental Loads --- Each topic should contain: * 100–300 interview questions * Detailed answers * Diagrams * Code examples * Best practices * Real interview scenarios * Frequently asked interview questions --- ## 5. Powerful Search Users should be able to search: * Questions * Topics * Technologies * Keywords Exa...

I have several Excel spreadsheets that must be imported accurately into my existing MySQL database. The sheets are clean and well-structu...brief field-mapping guide I’ve prepared Deliverables I expect back: • All spreadsheet records populated in MySQL, matching the given mapping • A short log noting any rows that required correction or could not be imported • SQL export file (.sql) of the final tables for my backup I routinely work with phpMyAdmin and simple SQL scripts, so feel free to use those tools, the MySQL CLI, or any ETL software you prefer—accuracy is what matters most. Once the staging data is confirmed, I will migrate it to production. If this sounds straightforward to you and you can turn it around quickly, let me know your timeframe an...

I have several Excel spreadsheets that must be imported accurately into my existing MySQL database. The sheets are clean and well-structu...brief field-mapping guide I’ve prepared Deliverables I expect back: • All spreadsheet records populated in MySQL, matching the given mapping • A short log noting any rows that required correction or could not be imported • SQL export file (.sql) of the final tables for my backup I routinely work with phpMyAdmin and simple SQL scripts, so feel free to use those tools, the MySQL CLI, or any ETL software you prefer—accuracy is what matters most. Once the staging data is confirmed, I will migrate it to production. If this sounds straightforward to you and you can turn it around quickly, let me know your timeframe an...

I have several Excel spreadsheets that must be imported accurately into my existing MySQL database. The sheets are clean and well-structu...brief field-mapping guide I’ve prepared Deliverables I expect back: • All spreadsheet records populated in MySQL, matching the given mapping • A short log noting any rows that required correction or could not be imported • SQL export file (.sql) of the final tables for my backup I routinely work with phpMyAdmin and simple SQL scripts, so feel free to use those tools, the MySQL CLI, or any ETL software you prefer—accuracy is what matters most. Once the staging data is confirmed, I will migrate it to production. If this sounds straightforward to you and you can turn it around quickly, let me know your timeframe an...

I have several Excel spreadsheets that must be imported accurately into my existing MySQL database. The sheets are clean and well-structu...brief field-mapping guide I’ve prepared Deliverables I expect back: • All spreadsheet records populated in MySQL, matching the given mapping • A short log noting any rows that required correction or could not be imported • SQL export file (.sql) of the final tables for my backup I routinely work with phpMyAdmin and simple SQL scripts, so feel free to use those tools, the MySQL CLI, or any ETL software you prefer—accuracy is what matters most. Once the staging data is confirmed, I will migrate it to production. If this sounds straightforward to you and you can turn it around quickly, let me know your timeframe an...

I have several Excel spreadsheets that must be imported accurately into my existing MySQL database. The sheets are clean and well-structu...brief field-mapping guide I’ve prepared Deliverables I expect back: • All spreadsheet records populated in MySQL, matching the given mapping • A short log noting any rows that required correction or could not be imported • SQL export file (.sql) of the final tables for my backup I routinely work with phpMyAdmin and simple SQL scripts, so feel free to use those tools, the MySQL CLI, or any ETL software you prefer—accuracy is what matters most. Once the staging data is confirmed, I will migrate it to production. If this sounds straightforward to you and you can turn it around quickly, let me know your timeframe an...

I have several Excel spreadsheets that must be imported accurately into my existing MySQL database. The sheets are clean and well-structu...brief field-mapping guide I’ve prepared Deliverables I expect back: • All spreadsheet records populated in MySQL, matching the given mapping • A short log noting any rows that required correction or could not be imported • SQL export file (.sql) of the final tables for my backup I routinely work with phpMyAdmin and simple SQL scripts, so feel free to use those tools, the MySQL CLI, or any ETL software you prefer—accuracy is what matters most. Once the staging data is confirmed, I will migrate it to production. If this sounds straightforward to you and you can turn it around quickly, let me know your timeframe an...

I’m preparing a new data workflow an...databases, and several spreadsheet feeds. The end goal is a robust, easily maintained pipeline that moves data accurately and on schedule, ready for downstream analytics and reporting. Here’s how I see the collaboration: • Discovery: review my existing systems, identify the cleanest integration path, and document the plan. • Build: develop the connectors or scripts, set up any middleware or ETL tooling you recommend, and configure automated scheduling and basic monitoring. • Handover: provide clear deployment notes plus a brief walkthrough so I can operate and extend the solution confidently. If you’ve implemented similar multi-source integrations and can point to tangible results, I’d love to hear ...

...that also supports real-time analytics. What I need • A dimensional warehouse schema that accommodates SAP BW, relational, and REST data. • Production-ready ETL pipelines (Python,SQL, or an established tool such as Azure Data Factory, Informatica, Talend—use what you know best) that extract, transform, and load data with minimal latency. • Incremental load logic and data-quality checks so the warehouse stays in sync without full reloads. • Documentation covering schema design, job orchestration, and how to add new sources. Acceptance criteria 1. All listed sources land in the warehouse on an automated schedule with ETL runtimes under 15 minutes. 2. KPI dashboard refresh proves sub-minute query response on at least three combin...

كراسة الشروط والمواصفات الفنية: أتمتة البيع وربط منصات الإعلانات بروبوت واتساب ذكي (AI Agent) (يفضل شخص يتحدث اللغه العربيه ) 1. ربط وتزامن البيانات (Webhooks & ETL) * المطلوب: ربط نماذج تجميع بيانات العملاء (Lead Forms) من منصات: Meta, Google, TikTok, Snapchat, LinkedIn * * آلية النقل: النقل عبر Webhooks لضمان السرعة، بحد أقصى للتحديث والتزامن كل 5 دقائق إلى شيت جوجل رئيسي (Master Google Sheet). * * أعمدة الشيت الموحد (Database Schema): يجب أن يحتوي الشيت على الأعمدة التالية بدقة: * [تاريخ الإضافة، منصة الإعلان، الاسم، رقم الهاتف، الإيميل، الكورس المستهدف، المسمى الوظيفي، اسم المؤسسة، حالة العميل (Lead Status)] * أداة الأتمتة: يفضل استخدام n8n أ لإدارة الـ Workflows والـ OAuth الخاص بالمنصات. 2. ربط الـ WhatsApp Business API والـ AI Agent ربط الشيت الموحد بمزود خدمة وا...

...archival logic that keeps the data tidy and fast. • Process flow diagrams or swim-lanes that clearly map the journey from data entry to RAG display. • A concise data dictionary covering every operational data element we will store, plus the rules that flip a field from Green to Amber or Red. • Recommended tech stack or tools (for example, PostgreSQL vs. MySQL, dashboard layer, possible use of ETL scripts) with pros and cons so I can make an informed decision. Acceptance Criteria – The scope covers the full onboarding journey end-to-end and matches my operational data focus. – All RAG logic is documented in plain language plus simple pseudocode or formulas. – I can hand the document to a dev team and have them start building with no ...

I'm a AI Data Engineer open to contract or full-time roles in the US ( remote, hybrid). I already get inbound contracting calls without applying ( linkedin premium, other platforms ) — so visibility isn't the problem. What I'm missing is quality of calls, getting calls that ...applying ( linkedin premium, other platforms ) — so visibility isn't the problem. What I'm missing is quality of calls, getting calls that value my experience. I'm looking to work with someone familiar for recruiting space in USA who can help with: <phase1>Application strategy <phase2> actual applications What I bring: solid data engineering experience of 15+ years across SQL, Python, pipelines/ETL, and cloud data platforms, databricks, AI ( agentic, G...

Senior Data Migration Engineer (Contract) We are seeking an experienced Python/Django Data Migration Engineer to design and build a scalable migration framework for our legal case management platform. You will develop ETL pipelines, data mapping tools, validation, reconciliation and import connectors for systems such as Clio, LEAP, Actionstep, ProClaim and CSV/Excel. Strong experience with Python, Django, PostgreSQL, APIs, Celery/background jobs and large-scale data migrations is essential. Experience building migration tooling for SaaS or LegalTech products is highly desirable. The solution must be robust, auditable, scalable and reusable for future migration connectors. You will be required to build tool and once built you will be instructed to do the migration if capable, there...

I need an end-to-end ETL pipeline that moves data from our on-premise databases into Google Cloud, transforms it, and lands it cleanly in BigQuery. The core stack must be Airflow for orchestration, Dataproc (running PySpark) for heavy transformations, and native BigQuery SQL for final modelling and reporting layers. You will design and implement: • Secure ingestion from the on-prem source into GCS staging • Airflow DAGs that trigger Dataproc jobs, handle retries, logging and alerting • PySpark transformation scripts on Dataproc, tuned for performance and cost • BigQuery SQL models that expose the refined tables • Parameterised configuration so environments can be promoted from dev to prod without code changes Acceptance criteria: – A ...

Freelance Analytics Engineer (Snowflake / Data Integration) We’re looking for a freelance Analytics Engineer / Data Consultant to help unify data from multiple systems into a single source of truth. Our stack JobWatch (core operations, Snowflake partner) Xero (finance) Odoo (ERP / CRM) Asset Panda (assets) Task Set up / optimise Snowflake as central warehouse Connect systems via ETL/ELT pipelines (Fivetran, Airbyte, APIs) Build a clean data model (jobs, customers, revenue, costs) Deliver initial insights (e.g. job profitability dashboard) Ideal Requirements Strong Snowflake + SQL Experience integrating SaaS tools Familiar with modern data stack (ELT, dbt a plus) Commercial / business mindset Details Freelance / project-based Remote Potential for ongoing work

...Develop, train, and deploy machine learning and deep learning models - Work with large datasets to build predictive and analytical solutions - Integrate AI/ML models into production systems - Experience with frameworks such as TensorFlow, PyTorch, or Scikit-learn - Strong background in NLP, computer vision, or generative AI is a plus 4. Data Engineer - Build and maintain robust data pipelines and ETL processes - Design and manage data warehouses and data lakes - Work with large-scale distributed systems such as Apache Spark, Kafka, or Airflow - Collaborate with data scientists and analysts to support data needs - Experience with SQL, NoSQL databases, and cloud data services (BigQuery, Redshift, Snowflake) All candidates should have strong communication skills, the ability to wo...

Project Title: End-to-End Horse Racing AI Data Platform I am building an AI-powered horse racing analytics platform focused on Indian race clubs. I need an experienced developer/data engineer to build the entire backend data infrastructure. Scope of work: 1. Data acquisition (legal and authorized sources only) * Identify official, licensed, or publicly available data sources * Create automated data collection pipelines * Build daily data update mechanisms 2. Historical data collection Collect and maintain historical data for: * Bangalore * Hyderabad * Mysore * Chennai * Mumbai * Pune * Kolkata * Other available Indian race clubs 3. Data fields required Horse details: * Horse name * Horse age * Sex * Owner * Stable * Equipment changes Race details: * Race date * Race club * R...

...to refresh properly. We need someone who can perform a paid diagnostic, identify root cause, recommend stabilization steps, and potentially continue with longer-term fractional support. This is not basic Jet Reports report writing and not a general Power BI dashboard project. We need backend experience with Jet Analytics or Jet Data Manager, SQL Server, SSAS / OLAP cubes, SQL Server Agent jobs, ETL troubleshooting, data warehouse design, and Dynamics NAV / Navision data structures. Current environment: - Dynamics NAV 2016 - Heavily customized NAV environment - LS Retail embedded - Jet Reports / Jet Analytics - SQL Server - SSAS / OLAP cube environment Initial scope: - Review Jet Analytics / Jet Data Manager configuration - Review SQL Server jobs, schedules, logs, and failure h...

Python, SQL, ETL, PySpark, Spark SQL, AWS EMR, AWS Lambda, AWS Step Functions, Amazon S3 (Data Lake – Raw & Processed Zones), AWS CloudWatch, AWS SNS, Pandas, Excel, Veeva CRM Project Overview Designed and implemented an end-to-end AWS-based data engineering pipeline to bifurcate, process, and deliver pharmaceutical sales, HCP, call activity, territory, and marketing data for Europe (EU) and Russia (RU) regions into Veeva CRM. The solution automated data ingestion from external APIs, validated and transformed high-volume datasets using Spark on EMR, and enforced multi-layer data quality checks based on business rules. Final curated datasets were delivered to Veeva CRM to support daily call planning, HCP targeting, territory alignment, and field sales insights, enabling acc...

...new metrics later is straightforward. Dashboard & Visuals Power BI, Tableau or a similarly robust BI tool should drive the front end, giving drill-down capability for hub, date range and shift while remaining presentation-ready for top-level management reviews. Clean, consistent visuals and export-to-slide/PDF functionality are essential. Deliverables 1. Documented data architecture with ETL/ELT workflow. 2. Fully populated master data table(s) with refresh schedule. 3. Interactive dashboard covering: – Productivity, efficiency, quality KPIs – Hub-to-hub comparison (west region focus) – Escalation and roster matrices 4. Quick-start user guide and hand-off session. Acceptance Criteria • Data refreshes without manual in...

...archival logic that keeps the data tidy and fast. • Process flow diagrams or swim-lanes that clearly map the journey from data entry to RAG display. • A concise data dictionary covering every operational data element we will store, plus the rules that flip a field from Green to Amber or Red. • Recommended tech stack or tools (for example, PostgreSQL vs. MySQL, dashboard layer, possible use of ETL scripts) with pros and cons so I can make an informed decision. Acceptance Criteria – The scope covers the full onboarding journey end-to-end and matches my operational data focus. – All RAG logic is documented in plain language plus simple pseudocode or formulas. – I can hand the document to a dev team and have them start building with no ...

Business Problem: A global pizza chain lacked deep operational visibility into peak consumer ordering hours, order sizing configurations, and product menu performance. Tools Used: Power BI, Power Query ETL, Time Intelligence DAX, Python (for initial dataset profile) KPIs Tracked: Total Revenue ($329K), Average Order Value ($38), Total Pizzas Sold (20K), Total Orders (9K), Average Pizzas Per Order (2.33). Insights Discovered: Peak Demand Windows: Orders spike drastically during weekend blocks (Friday and Saturday evenings), with Wednesday peaking as the highest volume weekday (1,227 total orders). Sizing Preference: Large-sized pizzas dominate consumer demand, contributing to 45.89% of the overall sizing mix. Category Leadership: The Classic Pizza category serves as the highest v...

...build, and maintain scalable data pipelines. • Develop ETL/ELT processes for collecting and transforming data. • Manage data warehouses and optimize data storage solutions. • Ensure data quality, consistency, and governance. • Collaborate with analysts, marketers, and engineering teams to deliver actionable insights. • Monitor and improve data infrastructure performance. Requirements • 3+ years of experience in data engineering. • Strong proficiency in Python and SQL. • Experience with data pipeline tools such as Airflow, Prefect, or similar. • Experience with cloud data platforms (AWS, GCP, Azure). • Knowledge of data warehousing technologies such as BigQuery, Snowflake, or Redshift. • Understanding of data model...

...data back the other way should we decide to revert. This is a true end-to-end migration: tables, indexes, constraints, sequences, views, procedures, functions, triggers, and the historical data itself all have to arrive intact on the new platform, with type mappings and performance characteristics that feel native to SQL Server. You will plan the cut-over, build the conversion scripts, run the ETL (Oracle Data Pump, SQL*Plus, SSMA, SSIS—whatever combination you prefer), and document every decision so the team here can maintain the system afterward. Zero data loss and verifiable referential integrity are non-negotiable, and the production window for the final switch needs to stay under two hours. Deliverables • Schema conversion scripts and repeatable migration proc...

...availability, throughput, and latency; proactively troubleshoot issues Perform capacity planning, tuning, upgrades, patching, and disaster recovery activities Develop and support event streaming pipelines for real-time and near-real-time data processing Integrate Kafka with API Gateways (APIGW), microservices, and backend systems Implement Kafka producers, consumers, and Kafka Connect connectors for ETL and data movement Collaborate with development teams to define event schemas, topics, and data contracts Apply Kafka security best practices: authentication, authorization, encryption, and auditing Document Kafka architectures, configurations, and operational procedures Required Skills & Experience 3+ years of hands-on experience administering and developing with Apache Kafk...

I have a dataset of financial figures drawn from our internal systems, and I need a focused variance analysis that pinpoints where actual results diverge from plan. The task is squarely about data analysis rather than ETL work; however, you may do light cleaning if it helps you reach accurate conclusions. Here is what I expect: • A concise explanation of your methodology (tools such as Excel-Power Query, Python pandas, or R are all fine). • A set of variance calculations broken down by the dimensions you recommend (e.g., period, cost center, product line). • Visual aids—charts or dashboards—that make those variances instantly clear to leadership. • A brief narrative highlighting the main drivers behind the largest positive and negative gaps. ...

...can pick it up without asking you questions What I'm looking for: Genuine self-direction. You research, decide, and deliver without hand-holding, and you know when to flag a blocker instead of spinning your wheels. Solid Azure fundamentals, and the resourcefulness to teach yourself what you don't already know General data literacy: comfortable with SQL, basic data modeling, how data pipelines (ETL/ELT) work, and how BI and reporting fit together. One of my gigs is data-heavy, so this matters even if your strength is elsewhere. Strong technical writing. Your runbook should let someone else execute it cold. Scripting in PowerShell and/or Python Real working knowledge in ONE of these, plus eagerness to learn the other: (A) Power BI + Databricks + Python, or (B) Entra ID ...

I have a presentation starting shortly and need 6-7 eye-catching PowerPoint slides that explain a data-transformation pipeline. The look should feel fresh and creative rather than corporate or minimalistic. Please weave in clear icons, concise illustrations, and a few punchy charts or graphs so each stage of the pipeline is instantly understood. The flow will follow the classic ETL rhythm—source data, transform steps, and final load—so structure the visuals around that sequence while keeping text to a minimum. Deliverables (within 45 minutes of hire): • A PPTX file containing 6–7 fully designed slides, ready to present • Consistent colour palette and typography across all slides • Editable vector icons/graphics so I can tweak labels later if ...

### **Project Title:** South Africa Census Data Extraction & Formatting (2011 Household Income by Ward Level) ### **Pro...228 800 * R 1 228 801 - R 2 457 600 * R 2 457 601 or more #### **Data Validation Requirement:** * The total row count must match the total number of valid South African wards (approx. 4,460 to 4,468 entries). * Missing data, null values, or unmapped ward boundaries must be clearly flagged as NaN rather than left blank or filled with zeroes. #### **Skills Required:** * Data Extraction / ETL * Web Scraping (Python / BeautifulSoup / Requests) * Excel / CSV Formatting * Experience with South African geographic data (Stats SA / MDB shapefiles) is a major advantage. Please state your estimated turnaround time and your approach to handling the extraction...

...availability, throughput, and latency; proactively troubleshoot issues Perform capacity planning, tuning, upgrades, patching, and disaster recovery activities Develop and support event streaming pipelines for real-time and near-real-time data processing Integrate Kafka with API Gateways (APIGW), microservices, and backend systems Implement Kafka producers, consumers, and Kafka Connect connectors for ETL and data movement Collaborate with development teams to define event schemas, topics, and data contracts Apply Kafka security best practices: authentication, authorization, encryption, and auditing Document Kafka architectures, configurations, and operational procedures Required Skills & Experience 3+ years of hands-on experience administering and developing with ...

...parameters for future products Everything should slot straight into the production server with minimal downtime; I’ll give admin access to the staging server first then to live server once verified, the moment we agree on the structure. Deliverable is the complete Loan & Savings report pack, fully tested and ready for staff to run. Please include a brief outline of how you plan to approach the ETL queries and any similar Pentaho-on-Mifos work you’ve completed. Looking forward to getting this live quickly. PS: I already have some of the Pentaho templates installed and I am on the latest Fineract version....

...preferred; frameworks like FastAPI/Django). • Familiar with ML infra: model serving (TorchServe, TensorFlow Serving, or containerised endpoints), CI/CD, Docker, Kubernetes or serverless deployments. • Cloud experience (AWS/Azure/GCP) for training, storage and secure hosting. • Experience integrating with REST APIs, web scraping, and handling supplier portals. Data engineering & pipelines • ETL/data‑pipeline experience: ingesting PDFs/DWGs/photos, normalising formats, annotation pipelines, data versioning. • Familiar with label tools and managing annotation quality. Product & UX • Ability to design simple review/edit UI for estimators (annotated drawings, override edits) or to integrate with existing tools. • Deliver audit...

I need a skilled React developer comfortable working inside a modern code-base to create a small web application that helps me map external data into an internal schema. The idea is simple: a user uploads or fetches a payload, sees both the source and destination fields side-by-side, then pairs t...expected: • A project in a Git repo containing the mapping interface, API routes for CRUD on mappings, and README instructions • Sample mappings created from the supplied datasets to show everything working end-to-end I’ll be available to answer domain questions quickly, and I prefer incremental pushes so I can test early. If you’ve built anything similar—drag-and-drop field mappers, visual ETL tools, or dashboard integrations—please mention it whe...

...including Azure and Databricks. --- Key Responsibilities - Design and develop scalable solutions in SAP BW 7.5 on HANA - Build and optimize data models, ETL processes, and reporting solutions - Perform performance tuning and ensure data integrity & security - Contribute to migration to SAP BW/4HANA and cloud platforms - Work with Azure, Databricks, and modern SAP technologies - Collaborate in Agile teams with cross-functional stakeholders - Drive continuous improvement and innovation in data solutions --- Required Skills & Experience - 3+ years of hands-on experience in SAP BW 7.5 on HANA - Strong expertise in: - Data Modeling & ETL processes - ABAP & AMDP development - Performance optimization in HANA - Experience or exposure to: - SAP BW/4HA...

Key Responsibilities: experience 14+Years ● Design, develop, test, and maintain scalable ETL data pipelines using Python. ● Architect the enterprise solutions with various technologies like Kafka, multi-cloud services, auto-scaling using GKE, Load balancers, APIGEE proxy API management, DBT, using LLMs as needed in the solution, redaction of sensitive information, DLP (Data Loss Prevention) etc. ● Work extensively on Google Cloud Platform (GCP) services such as: ○ Data-flow for real-time and batch data processing ○ Cloud Functions for lightweight serverless compute ○ BigQuery for data warehousing and analytics ○ Cloud Composer for orchestration of data workflows (on Apache Airflow) ○ Google Cloud Storage (GCS) for managing data at scale ○ IAM for access control and security ○ Cloud...

I'm looking for a skilled azure data engineer to develop robust data pipelines. The primary data source will be databases. Requirements: - Design and implement efficient data pipelines. - Extract data from various databases. - Ensure data quality and integrity. - Collaborate on detailed projec...engineer to develop robust data pipelines. The primary data source will be databases. Requirements: - Design and implement efficient data pipelines. - Extract data from various databases. - Ensure data quality and integrity. - Collaborate on detailed project proposals. Ideal Skills and Experience: -Azure data factory -azure databricks - Proficiency in SQL and database management. - Experience with ETL tools. - Strong problem-solving skills. - Prior experience in data pipeline develop...

ETL Pipeline Development and Machine Learning Model for Crop Growth Prediction Project Overview We are seeking an experienced Data Engineer and Machine Learning Engineer to develop a complete data pipeline and predictive machine learning solution for crop growth forecasting using tabular agricultural data. The deliverables should include a production-ready ETL pipeline, a trained and evaluated machine learning model, and deployment-ready artifacts suitable for integration into a larger agricultural analytics platform. Scope of Work 1. ETL Pipeline Development Design and implement a robust ETL (Extract, Transform, Load) pipeline that: * Ingests crop-related tabular datasets from CSV files, databases, or APIs. * Performs data cleaning and validation. * H...

My current workload centers on Oracle Data Integrator (ODI) projects that pull data from APIs and other cloud services into Oracle Cloud Infrastructure. I need practical, hands-on support to strengthen three areas: 1. Building a reusable Oracle Data Quality (DQ) Framework that plugs neatly into OCI Data Integration. 2. Automating the ETL workflow in ODI—from source ingest through load—so every run is fully tested without manual intervention. 3. Embedding robust data-validation checkpoints throughout the pipeline to ensure accuracy, completeness, and early error detection. You will be working exclusively in the ODI ecosystem, orchestrating API-driven loads, and codifying data-quality rules that can be triggered automatically. Reusable test cases, clear logging, and ...

...Installation and deployment documentation * Automated ingestion scripts * Deduplication pipeline * Text extraction pipeline * Chunking pipeline * Final JSONL corpus generation workflow * Server setup guide * Knowledge transfer session Required Skills * Advanced Python * Large-scale data processing * JSONL / GZIP / Zstandard processing * Async programming (aiohttp, asyncio) * Data engineering * ETL pipeline development * Linux server administration * Elasticsearch/OpenSearch * PostgreSQL or SQLite * Text extraction from PDF and EPUB * RAG and vector database fundamentals * Git and deployment workflows Preferred Experience * Anna's Archive datasets * LibGen metadata processing * Digital library projects * Knowledge graph or search systems * Qdrant, Weaviate, Milvus, or pgv...

...based on machine signals and historical downtime • Schedule optimisation that balances tool changes, order priority and energy costs 3. Inventory management • Demand-forecasting engine that feeds reorder alerts for resin, pigments and packaging • Dynamic stock-level dashboards linked to our existing ERP (SQL back-end, REST API available) Deliverables • Data-pipeline design and ETL scripts (Python or Node) • Trained computer-vision model (TensorFlow/PyTorch + OpenCV) for surface and dimensional inspection, deployable on edge GPUs • Optimisation modules written in Python (scikit-learn, Optuna, or similar) with documented inference APIs • Inventory-forecast microservice with integration hooks to our ERP • Web-bas...

...improvements in analytics and reporting processes. Required Skills & Qualifications * Strong expertise in Microsoft Power BI, including dashboards, reports, and data modeling. * Minimum **5 years of experience in Power BI Development. * Minimum **5 years of experience in Microsoft SQL Server Development * Strong SQL skills with experience writing complex and optimized queries. * Expertise in ETL processes, data transformation, and data analysis. * Strong knowledge of DAX (Data Analysis Expressions). * Understanding of data visualization best practices. * Experience with Microsoft Azure and Azure Data Factory. * Good communication and stakeholder management skills. * Bachelor's degree in Computer Science, Information Technology, or a related field. Preferred Skills ...

...management. The role involves writing optimized SQL queries, designing data models, creating ETL workflows, and transforming raw data into actionable insights. The candidate will collaborate with cross-functional teams to understand business requirements, maintain data accuracy, and optimize reporting performance. The position is based in Chennai. Qualifications Strong expertise in SQL Server including queries, stored procedures, views, joins, indexing, and performance optimization Experience with Microsoft Power BI Desktop and Power BI Service Strong knowledge of SQL, DAX, Power Query, and Data Modeling Experience in creating dashboards, reports, KPIs, and data visualizations Understanding of ETL processes and Data Warehousing concepts Ability to extract, clean, and tran...