Lead Data Engineer

Gradera; Hyderabad; On-site; INR 3,000,000 - 6,000,000; Full time

Job summary

Gradera in Hyderabad, India, seeks a hands-on Lead Data Engineer to shape our data foundation for digital twin and AI-powered platforms. You will lead the data engineering team, design scalable pipelines, and drive real-time processing and governance using Databricks, Unity Catalog, and Delta Lake.

You will mentor engineers, promote DataOps, and partner with ML and simulation teams to deliver production-grade data products with quality, observability, and sub-second latency where required.

Qualifications

• 10+ years of hands-on data engineering experience with 4+ years in a technical leadership role.
• Track record of architecting and delivering scalable, production-grade data platforms.
• Experience with modern DataOps practices (CI/CD for data, automated testing, pipeline monitoring).
• Demonstrated ability to mentor and grow data engineers.
• Experience working in cross-functional, agile teams.

Responsibilities

• Provide technical leadership, mentorship, and guidance to the data engineering team.
• Lead design and evolution of data pipelines, Lakehouse architecture, and real-time event processing systems.
• Architect data foundations for digital twin platforms, AI/ML workloads, and operational analytics.
• Ensure delivery of high-quality, well-governed data products with strong observability.

Skills

Technical leadership

DataOps

Mentoring

Real-time pipelines

Data governance

Observability

Cross-functional collaboration

Tools

Databricks

Unity Catalog

Teradata

Apache Kafka

Delta Live Tables

Structured Streaming

TimescaleDB

Python

Scala

SQL

Great Expectations

Job description

Job Description:

About Gradera

Gradera defines a new category of enterprise transformation called Software-Orchestrated Services™ - where software orchestrates human expertise, digital workers, and enterprise systems to deliver governed outcomes at scale. As an AI Native Services firm, we help enterprises redesign how work gets done across operations, product, engineering, customer experience, data, and enterprise workflows to move beyond fragmented AI pilots and disconnected automation toward measurable business outcomes.

Overview

We are seeking a hands-on, collaborative Lead Data Engineer to drive the technical direction, architecture, and delivery of our data foundation for digital twin and AI-powered enterprise platforms. As Lead Data Engineer, you will provide technical leadership and guidance to the data engineering team, ensuring the delivery of robust, scalable, high-performance data pipelines that power real-time operational intelligence and simulation-ready data products. This role partners closely with data architects, simulation engineers, ML engineers, and platform teams.

Our core data platform stack includes:

• Databricks as the single point of truth for all data
• Databricks SQL for analytical queries
• Unity Catalog for metadata management and governance
• Terradata for data warehouse and business intelligence.
• Apache Kafka for real-time event ingestion and streaming
• Structured Streaming for continuous data processing
• Delta Live Tables for declarative, quality-enforced pipelines
• TimescaleDB for time-series operational data
• Python, Scala, and SQL for data transformation and orchestration
• AI/ML workflows on Databricks models
• Great Expectations and Delta Live Tables expectations for data validation
• Unity Catalog for metadata management and compliance
• Work with Time-series databases is a plus

Key Responsibilities

• Provide technical leadership, mentorship, and guidance to the data engineering team
• Lead the design, implementation, and evolution of data pipelines, Lakehouse architecture, and real-time event processing systems
• Architect and build data foundations that serve digital twin platforms, AI/ML workloads, and operational analytics
• Ensure delivery of high-quality, well-governed, and performant data products using Unity Catalog and Delta Lake
• Drive adoption of DataOps best practices, including CI/CD for data pipelines, automated testing, and monitoring
• Champion data hygiene, quality frameworks, and observability across the data estate
• Oversee real-time data ingestion from diverse operational systems and IoT sources
• Guide the use of modern data engineering tools, architectural patterns, and emerging technologies
• Own the implementation and quality of data lineage, cataloging, and governance to enable trust and compliance
• Collaborate with simulation engineers and ML teams to deliver simulation-ready and feature-engineered datasets
• Ensure engineering rigor, code quality, and documentation standards are met across the data stack
• Facilitate clear communication, knowledge sharing, and effective documentation within the team
• Support team growth through coaching, feedback, and skills development
• Drive development efficiency and alignment with product team priorities through clear prioritization frameworks and delivery metrics.

Preferred Qualifications

• 10+ years of hands-on data engineering experience, with 4+ years in a technical leadership role
• Track record of architecting and delivering scalable, production-grade data platforms
• Experience with modern DataOps practices (CI/CD for data, automated testing, pipeline monitoring)
• Demonstrated ability to mentor and grow data engineers
• Experience working in cross-functional, agile teams

Highly Desirable

• Experience building data foundations for digital twin or simulation platforms
• Familiarity with OpenUSD data pipelines or FMI/FMU data integration
• Experience with physics-informed or operational ML feature engineering
• Track record of delivering data platforms with sub-second latency for real-time operational use cases
• Experience thriving in fast-paced, ambiguous environments and balancing rapid delivery with technical excellence
• Experience leading or working with distributed, multidisciplinary teams
• Exposure to industrial domains such as Manufacturing, Logistics, or Transportation is a plus.

Requirements