Lead Data Engineer - Azure Databricks & API Platforms

Encora; United States; Hybrid; USD 140,000 - 190,000; Full time

Job summary

Coforge is seeking a Lead Data Engineer to own end-to-end delivery of strategic data products within a modern cloud data platform. You will design low-latency API-driven data services, build scalable Python backends, and manage data pipelines with Azure Databricks and DBT, while guiding a team of data and ML engineers.

You’ll drive architecture decisions, ensure data quality and observability, and lead CI/CD adoption. Open to LATAM-based candidates with strong leadership and communication skills.

Qualifications

• 8+ years of experience in Data Engineering, Software Engineering, or related technical roles.
• At least 3 years of experience leading engineering teams, including senior, mid-level, and junior professionals.
• Strong backend development experience with proven delivery of production-grade REST APIs and backend services.
• Excellent Python programming with strong object-oriented principles.
• Extensive experience with PyTest, unit testing and software quality practices.
• Hands-on Azure Databricks and DBT expertise.
• Advanced SQL skills and data modeling knowledge.
• Experience with Terraform and IaC deployments.
• CI/CD pipelines, Git-based version control, and modern software engineering practices.
• Proven leadership in technical delivery while remaining hands-on.

Responsibilities

• Lead the RTS delivery squad and own end-to-end delivery of strategic data products on a modern cloud data platform.
• Design and oversee architecture of low-latency, API-driven data solutions.
• Build and maintain scalable backend services and REST APIs with Python and OOP.
• Develop data pipelines using Azure Databricks, DBT, SQL and cloud-native tech.
• Own curated data layers ensuring availability, scalability and performance.
• Lead Terraform-based infrastructure deployment and IaC practices.
• Mentor and develop a team of data and ML engineers; conduct code reviews and set standards.
• Collaborate with Product/Engineering/Business to define technical solutions and roadmaps.
• Ensure data quality, observability, monitoring, and SLA compliance across data products.
• Drive CI/CD adoption and oversee release management for pipelines and platform components.

Skills

Python

REST APIs

Azure Databricks

DBT

SQL

Terraform

Data Engineering

Data Modeling

CI/CD

Team Leadership

Job description

Coforge is seeking a Lead Data Engineer to own end-to-end delivery of strategic data products within a modern cloud data platform. You will design low-latency API-driven data services, build scalable Python backends, and manage data pipelines with Azure Databricks and DBT, while guiding a team of data and ML engineers.

You’ll drive architecture decisions, ensure data quality and observability, and lead CI/CD adoption. Open to LATAM-based candidates with strong leadership and communication skills.