Lead Data Engineer (10+ years with MS Fabric, Azure, ADF, Data Platforms only)

C T Corporation System (DE); Chennai; On-site; INR 2,400,000 - 4,200,000; Full time

Job summary

C T Corporation System (DE) is seeking a Lead Data Engineer in Chennai to shape and grow a modern, cloud-based data platform spanning Databricks, Snowflake, and Microsoft Fabric. You will drive architecture, set standards, and mentor a team of data engineers to deliver scalable, secure data products.

The role emphasizes AI-first engineering, building reusable data frameworks, and partnering across teams to ensure data quality, governance, and operational excellence.

Qualifications

• Experience leading data engineering teams
• Strong background in data modeling, data governance and data quality
• Hands-on engineering leader capable of shaping architecture and standards
• Proven ability to mentor engineers and raise technical depth

Responsibilities

• Provide hands-on technical leadership for a growing team of data engineers
• Set engineering standards and reusable patterns across architecture, coding, testing, data modeling, documentation, observability, security, and deployment
• Lead architecture discussions, technical design reviews, and code reviews
• Mentor engineers and develop technical depth, ownership, problem-solving, and engineering judgment
• Translate complex initiatives into clear technical approaches and guide teams from design through production
• Foster a culture of ownership, collaboration, curiosity, experimentation, continuous learning, and accountability

Skills

Data architecture leadership

Mentoring

Data engineering

Observability

Security

Deployment

Tools

Databricks

Snowflake

Microsoft Fabric

Cloud-native services

Job description

Job Summary

We are looking for a Lead Data Engineer to help shape and grow our next-generation data engineering capability. This is a hands-on technical leadership role for an experienced engineer who can set technical direction, establish engineering standards, mentor and elevate other engineers, and build enterprise-scale data solutions . You will help design and evolve a modern, scalable, cloud-based data platform that may span technologies such as Microsoft Fabric, Databricks, Snowflake, and cloud-native services. Rather than being tied to a single technology stack, we are looking for someone who understands modern data architecture principles and can make thoughtful technology choices based on business, data, scalability, interoperability, and operational needs. Just as importantly, we are looking for an AI-first engineering leader who can help the team rethink how we design, build, test, operate, and continuously improve data products using AI.

Key Responsibilities

Lead Engineering Excellence

• Provide hands-on technical leadership for a growing team of data engineers.
• Set engineering standards and reusable patterns across architecture, coding, testing, data modeling, documentation, observability, security, and deployment.
• Lead architecture discussions, technical design reviews, and code reviews.
• Mentor engineers and develop technical depth, ownership, problem-solving, and engineering judgment across the team.
• Translate complex initiatives into clear technical approaches and guide teams from design through production.
• Foster a culture of ownership, collaboration, curiosity, experimentation, continuous learning, and accountability .

Build & Evolve a Modern Data Platform

• Lead the architecture and engineering of scalable, secure, and governed enterprise data platforms.
• Design solutions across lakehouse, data warehouse, data lake, streaming, and data-product architectures based on the needs of the use case.
• Build batch, incremental, near-real-time, and event-driven data processing solutions.
• Create reusable engineering frameworks for ingestion, transformation, orchestration, data quality, metadata, and data delivery.
• Design scalable data models supporting operational, analytical, reporting, and AI use cases.
• Establish patterns that promote interoperability and reduce unnecessary platform and vendor dependencies.
• Evaluate technology choices based on performance, scalability, reliability, security, cost, maintainability, and business value.
• Remain hands-on and contribute directly to engineering solutions when needed.

Champion AI-First Engineering

• Champion the use of AI across the engineering lifecycle, including requirements, architecture, development, testing, documentation, troubleshooting, and operations.
• Establish responsible practices for reviewing, validating, securing, and governing AI-generated engineering artifacts.
• Promote spec-driven development , using well-defined requirements, data contracts, acceptance criteria, and automated tests.
• Design data platforms and products that are ready to support AI agents, RAG, semantic models, machine learning, and conversational analytics .
• Evaluate emerging AI and data technologies through focused experimentation and translate successful approaches into reusable engineering patterns.
• Help engineers use AI to increase productivity and innovation without compromising security, quality, or engineering accountability.

Build Reliable & Trusted Data Products

• Establish automated data quality, reconciliation, validation, and testing practices.
• Build observability across data pipelines, datasets, APIs, and platform services.
• Define operational expectations and reliability measures for critical data products.
• Lead root-cause analysis for significant issues and ensure corrective actions address systemic problems.
• Continuously improve platform reliability, resilience, performance, and operational efficiency.
• Treat data as a product with clear ownership, quality expectations, documentation, and consumer needs.

Design Enterprise Integration & Data Access

• Design secure, scalable integration patterns across enterprise applications, data platforms, and external data sources.
• Develop and guide the implementation of REST and GraphQL APIs.
• Design event-driven and serverless integration solutions.
• Establish standards for API lifecycle management, authentication, versioning, monitoring, and governance.
• Enable data access through appropriate patterns including APIs, data sharing, event streams, and governed analytical interfaces.
• Apply modern identity and access patterns such as OAuth 2.0, OpenID Connect, JWT, managed identities, service principals, or equivalent approaches.

Embed Governance, Security & Compliance

• Build data governance, lineage, metadata, data quality, security, privacy, and auditability into platform designs.
• Ensure engineering solutions align with applicable governance, risk, security, and compliance requirements.
• Apply appropriate engineering rigor to KYC, UCC, p