Lead Data Engineer – Analytics & AI

Abjayon; Hyderabad; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

Abjayon seeks a senior data engineering lead to drive detailed technical design and delivery of enterprise data, analytics, and AI/ML solutions. You will translate architectures into practical designs, define data models and pipelines, and guide teams to scalable, production-ready systems.

You'll mentor engineers, establish engineering standards, and coordinate across data engineering, analytics, and AI/ML groups to solve complex delivery challenges.

Qualifications

• Significant professional experience in data engineering, data warehousing, or data architecture with leadership in enterprise projects.
• Strong hands-on knowledge of SQL, data modelling, ETL/ELT, data pipelines, and warehouse or lakehouse architectures.
• Ability to translate high-level designs into detailed technical designs for implementation.
• Proven capability to lead engineering delivery, review work, and diagnose complex issues.

Responsibilities

• Turn architecture into working solutions by detailing data models, mappings, transformations, and interfaces.
• Own data engineering design quality for scalable warehouses, lakehouses, and curated datasets.
• Lead complex data engineering delivery, guiding developers, reviewing designs, and enforcing standards.
• Design for scale and reliability across batch, incremental, CDC, and streaming workloads.
• Troubleshoot data, integration, and performance issues including post-go-live problems.
• Collaborate with analytics and AI/ML teams to support predictive analytics and AI use cases.

Skills

data engineering

data warehousing

data architecture

SQL

ETL/ELT

data pipelines

Tools

Databricks

Snowflake

Oracle ADW

Redshift

Kafka

Airflow

Job description

You'll lead the detailed technical design and engineering delivery of enterprise data, analytics, and AI/ML solutions. Working closely with Solution Architects, you'll turn high-level architecture into practical designs that engineering teams can build, operate, and scale.

This is a hands-on technical leadership role. You'll have significant influence over data models, pipelines, engineering standards, and implementation quality while helping Data Engineering and AI/ML teams solve complex delivery and production challenges.

What You'll Do

• Turn architecture into working solutions. You'll translate high-level designs into detailed data models, mappings, transformations, interfaces, pipeline designs, and implementation specifications that developers can execute against.
• Own data engineering design quality. You'll design scalable data warehouse, lakehouse, and curated data structures that support reporting, dashboards, analytics, KPIs, and AI/ML use cases.
• Lead complex data engineering delivery. You'll guide developers through implementation, review designs and code, establish practical engineering standards, and help ensure solutions are maintainable and production-ready.
• Design for scale and reliability. You'll make appropriate design choices across batch, incremental, CDC, and streaming workloads while considering performance, scalability, data quality, and operational support.
• Solve difficult technical problems. You'll troubleshoot complex data, integration, and performance issues, including production and post-go-live problems where the root cause may span multiple systems.
• Connect data engineering with analytics and AI. You'll work with analytics and AI/ML teams to ensure data models and curated datasets support predictive analytics, forecasting, anomaly detection, visualization, and other advanced use cases.
• Raise the technical capability of the team. You'll mentor engineers, provide technical direction, encourage sound engineering practices, and help teams adopt appropriate new technologies.

What You'll Need

Must-haves

• Significant professional experience in data engineering, data warehousing, or data architecture, including experience leading the technical design or delivery of enterprise data solutions.
• Strong hands-on knowledge of SQL, data modelling, ETL/ELT, data pipelines, and enterprise data warehouse or lakehouse architectures.
• Experience translating solution architecture or high-level designs into detailed technical designs that engineering teams can implement.
• Ability to lead engineering delivery, review technical work, and diagnose complex integration, scalability, performance, and production issues.
• Strong communication and problem-solving skills, including the ability to work effectively with architects, developers, analytics teams, and other stakeholders.

Nice-to-haves

• Hands-on experience with technologies such as Python, Spark/PySpark, Databricks, Snowflake, Oracle ADW, Redshift, Azure Synapse, Kafka, Airflow, ADF, AWS Glue, or similar platforms.
• Experience with cloud platforms such as AWS, Azure, or OCI, along with Git and CI/CD practices.
• Exposure to AI/ML data engineering, MLOps, GenAI/LLM-enabled analytics, data governance, lineage, quality, or observability.
• Experience with the utilities industry, particularly metering, customer, outage, grid, or asset data; Oracle Utilities C2M/NMS exposure is an additional advantage.

What We Offer

You'll have the opportunity to take technical ownership of complex enterprise data initiatives, influence engineering standards, and grow further into senior technical architecture or engineering leadership roles.

You'll work across modern data, analytics, and AI technologies rather than being limited to a single platform or toolset.

The Team

You'll work closely with Solution Architects and collaborate with Data Engineering, Analytics, and AI/ML teams. The role sits between architecture and implementation: you'll be expected to understand the broader solution while remaining close enough to the engineering work to make practical technical decisions.

The role can involve technically complex environments, unfamiliar technologies, and production issues that require structured troubleshooting rather than straightforward implementation.