Lead Data Engineer

Ormae Llp; Pune District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

ORMAE is seeking a seasoned Lead Data Engineer with 6+ years in core data engineering to design scalable real-time and batch data platforms, with strong supply chain domain expertise. You will drive data ingestion pipelines, complex transformations, and data quality across REST APIs, databases, files, and streaming sources, guiding the team on best practices and architecture.

Join a fast-growing consultancy with a global footprint, offering opportunities to work across industries and scale

Qualifications

• 6+ years of hands-on data engineering experience.
• Supply chain data management and transformation.
• Strong Python and PySpark development skills.
• Advanced SQL skills, including complex joins, and query optimization.
• ETL and ELT design patterns.
• Experience with REST APIs, JSON, CSV, MongoDB and OLAP engines.
• Experience with developing Agents and Agentic design.
• Strong understanding of data modelling, schema design, partitioning, and schema evolution.
• Experience implementing pipeline observability, data validation, monitoring, and alerting.
• Experience with Git, CI/CD, Docker, and Kubernetes.

Responsibilities

• Design and build real-time, AI-based supply chain batch data ingestion pipelines.
• Build complex data transformation, cleansing, enrichment, and reconciliation workflows.
• Integrate data from REST APIs, databases, files, enterprise systems, and streaming sources.
• Develop ETL tools and validation routines for complex supply chain workflows.
• Develop reusable Python and SQL components for ingestion and transformation.
• Implement schema evolution, incremental loads, retries, error handling, and data recovery.
• Understand agentic structure and create data solutions and models for optimisation engines.
• Implement data quality checks, monitoring, logging, alerting, and audit controls.
• Review technical designs and guide data engineers on implementation standards.

Skills

Python

PySpark

SQL

ETL/ELT

REST APIs

Data ingestion

Data modelling

Supply chain data

Data quality checks

Git CI/CD

Docker

Kubernetes

Tools

Git

Docker

Kubernetes

MongoDB

REST APIs

JSON

CSV

OLAP engines

Job description

We are looking for a seasoned Lead Data Engineer with 6+ years of core data engineering experience to design and build scalable, real-time and batch data platforms with Supply Chain expertise

The role requires strong expertise in data analytics, python, advanced SQL, complex data transformations, system integrations, and supply chain workflow.

Key Responsibilities

• Design and build real-time, AI based supply chain batch data ingestion pipelines.
• Build complex data transformation, cleansing, enrichment, and reconciliation workflows.
• Integrate data from REST APIs, databases, files, enterprise systems, and streaming sources.
• Develop ETL tools and validation routines for complex supply chain workflows.
• Develop reusable Python and SQL components for ingestion and transformation.
• Implement schema evolution, incremental loads, retries, error handling, and data recovery.
• Understand agentic structure and create data solutions and models for optimisation engines.
• Implement data quality checks, monitoring, logging, alerting, and audit controls.
• Review technical designs and guide data engineers on implementation standards.

Required Skills

• 6+ years of hands-on data engineering experience.
• Strong experience with Supply chain data management and transformation.
• Strong Python and PySpark development skills.
• Advanced SQL skills, including complex joins, and query optimization.
• Strong understanding of ETL and ELT design patterns.
• Experience with REST APIs, JSON, CSV, MongoDB and OLAP engines.
• Experience with developing Agents and Agentic design.
• Strong understanding of data modelling, schema design, partitioning, and schema evolution.
• Experience implementing pipeline observability, data validation, monitoring, and alerting.
• Experience with Git, CI/CD, Docker, and Kubernetes.

Who we are:

ORMAE is a fast-growing consulting firm specializing in Data Science, Optimization, and Generative AI solutions. Founded and led by Dr. Amit Garg, a Guinness World Record holder in Mathematics and a PhD in Operations Research, ORMAE helps organizations solve complex business challenges through advanced analytics and optimization across industries, including Logistics & Transportation, Automotive, Retail, E-commerce, ITeS, Healthcare, and Pharmaceuticals.

Our Client List and Global footprint:

With a global footprint spanning the USA, Germany, UAE, Saudi Arabia, Japan, and India, ORMAE has partnered with 110+ clients, including 12+ billion-dollar enterprises, and successfully delivered over 100 Data Science and Optimization projects. ORMAE also offers RouteCap, its proprietary route optimization platform for multi-depot routing and dynamic Gantt chart planning.

ORMAE is backed by 3 angel investors from UAE. The company is at a steady growth rate and has successfully completed 10 years, and we are growing strong, consistently acquiring new logos and delighting existing customers.

What Is in it For Me - Candidate Benefits:

• Meritocracy based Career Progression
• Global Exposure
• Livewire Environment
• Great Technical cross training & mentorship
• Happening start up - in the garden city of Bengaluru