Technical Lead

Carelon Global Solutions

Gurugram District, Bengaluru

Hybrid

INR 3,000,000 - 5,500,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Carelon Global Solutions in Gurugram is seeking a senior Data Engineer/Technical Lead to drive scalable data engineering initiatives. You will architect and implement lakehouse solutions using Databricks, Python, and Spark, while guiding teams across batch and real-time processing projects.

The role requires strong Snowflake, Iceberg, and cloud platform expertise, plus leadership and stakeholder communication skills for large-scale data modernization programs. On-site in India.

Qualifications

  • B.Tech/MCA with CS background or equivalent experience.
  • 7 years total experience with minimum 3 years in Databricks.
  • Hands-on Databricks design and implementation of scalable lakehouse solutions.
  • strong Python and PySpark for building enterprise data pipelines.
  • Snowflake data modeling, performance tuning, query optimization, and cost management.
  • Experience with Apache Iceberg, Delta Lake, and open table formats.
  • Strong knowledge of AWS/GCP cloud services and platform operations.
  • Experience building batch and real-time data processing with Spark, Kafka, and cloud services.
  • Proven ETL/ELT design, data integration, and cloud migration experience.
  • Experience leading development teams and data modernization programs.
  • Excellent troubleshooting and stakeholder communication skills.
  • Expert Databricks knowledge: Lakehouse, Delta Lake, Unity Catalog, Workflows, optimization.

Responsibilities

  • Lead development and implementation of scalable data engineering solutions.
  • Drive end-to-end data pipeline development using Databricks and Python.
  • Design batch and real-time data processing frameworks.
  • Review code and establish engineering best practices.
  • Guide teams on performance optimization and reliability improvements.
  • Build and maintain ETL/ELT pipelines using Databricks, Spark, and Python.
  • Develop scalable data lakehouse architectures.
  • Implement data quality, governance, and observability frameworks.
  • Optimize Snowflake workloads and data models.
  • Design Iceberg-based data lakes for large-scale analytics.
  • Work with AWS/GCP cloud services.
  • Develop cloud-native data solutions and CI/CD awareness.
  • Support platform modernization and cloud migration initiatives.
  • Partner with architects, product owners, and business stakeholders.
  • Translate business requirements into technical solutions.
  • Provide effort estimation and delivery planning.

Skills

Databricks
Python
PySpark
Snowflake
Delta Lake
Iceberg
Open table formats
AWS
GCP
Spark
Kafka
ETL/ELT
Data governance
Data modeling
Unity Catalog
Workflows
SQL

Education

B.Tech degree/MCA with computer science background or equivalent experience.

Tools

CI/CD
Terraform
CloudFormation

Job description

QUALIFICATION
  • B.Tech degree/MCA with computer science background or equivalent experience.
EXPERIENCE
  • 7 years of total and minimum 3 years of relevant experience in Databricks.
  • Proven hands-on experience with Databricks , including designing and implementing scalable data lakehouse solutions.
SKILLS
  • Strong expertise in Python and PySpark for developing enterprise-grade data pipelines and distributed processing frameworks.
  • Extensive experience with Snowflake, including data modeling, performance tuning, query optimization, and cost management.
  • Hands-on experience with Apache Iceberg, Delta Lake, and modern open table formats for large-scale data lake implementations.
  • Strong understanding of AWS/GCP cloud services, architecture design, deployment, and platform operations.
  • Experience in building batch and real-time data processing pipelines using Spark, Kafka, and cloud-native services.
  • Proven expertise in ETL/ELT design, data integration, and cloud migration initiatives.
  • Experience leading development teams and delivering data modernization programs.
  • Strong troubleshooting and analytical skills with the ability to resolve complex production issues.
  • Excellent stakeholder management and communication skills, working across technical and business teams.
  • Expert knowledge of Databricks, including Lakehouse Architecture, Delta Lake, Unity Catalog, Workflows, and Performance Optimization.
  • Strong proficiency in Python, PySpark, Spark SQL, and distributed data processing.
  • Extensive hands-on experience with Snowflake, including Snowpipe, Streams, Tasks, Data Sharing, and Query Optimization.
  • Strong expertise in Apache Iceberg, open table formats, schema evolution, partitioning, and time-travel capabilities.
  • Experience designing and implementing large-scale Data Engineering and ETL/ELT Pipelines.
  • Hands-on experience with AWS/Azure/GCP cloud-native data services.
  • Strong SQL and data modeling skills across OLTP and analytical environments.
  • Experience with Real-Time Data Processing using Kafka, Event Hubs, Kinesis, or Pub/Sub.
  • Knowledge of Data Governance, Metadata Management, Data Quality, and Security frameworks.
Good to Have
  • Experience with Terraform, CloudFormation, or Infrastructure as Code (IaC).
  • Knowledge of Docker, Kubernetes, and containerized data platforms.
  • Experience with CI/CD tools such as GitHub Actions, Jenkins, Azure DevOps, or GitLab.
  • Exposure to AI/ML, MLOps, and GenAI data platform integration.
  • Knowledge of Data Mesh, Data Fabric, and modern enterprise data architectures.
  • Databricks, Snowflake, and Cloud platform certifications.
  • Expert knowledge of Databricks, including Lakehouse Architecture, Delta Lake, Unity Catalog, Workflows, and Performance Optimization.
  • Strong proficiency in Python, PySpark, Spark SQL, and distributed data processing.
  • Extensive hands-on experience with Snowflake, including Snowpipe, Streams, Tasks, Data Sharing, and Query Optimization.
  • Strong expertise in Apache Iceberg, open table formats, schema evolution, partitioning, and time-travel capabilities.
  • Experience designing and implementing large-scale Data Engineering and ETL/ELT Pipelines.
  • Hands-on experience with AWS/Azure/GCP cloud-native data services.
  • Strong SQL and data modeling skills across OLTP and analytical environments.
  • Experience with Real-Time Data Processing using Kafka, Event Hubs, Kinesis, or Pub/Sub.
  • Knowledge of Data Governance, Metadata Management, Data Quality, and Security frameworks.
Key Responsibilities
Technical Leadership
  • Lead development and implementation of scalable data engineering solutions.
  • Drive end-to-end data pipeline development using Databricks and Python.
  • Design batch and real-time data processing frameworks.
  • Review code and establish engineering best practices.
  • Guide teams on performance optimization and reliability improvements.
Data Engineering & Analytics
  • Build and maintain ETL/ELT pipelines using Databricks, Spark, and Python.
  • Develop scalable data lakehouse architectures.
  • Implement data quality, governance, and observability frameworks.
  • Optimize Snowflake workloads and data models.
  • Design Iceberg-based data lakes for large-scale analytics.
Cloud & Platform Engineering
  • Work with AWS/GCP cloud services.
  • Develop cloud-native data solutions using managed services.
  • Awareness of CI/CD pipelines and Infrastructure as Code.
  • Support platform modernization and cloud migration initiatives.
Stakeholder Management
  • Partner with architects, product owners, and business stakeholders.
  • Translate business requirements into technical solutions.
  • Provide effort estimation and delivery planning.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Senior Architect
Databricks Senior Architect

Vrinda Global • Bengaluru

On-site
INR 3,000,000 - 6,000,000
Databricks Data Engineer
Databricks Data Engineer

Emergys • Pune District

On-site
INR 1,400,000 - 2,100,000
Senior Data Engineer
Senior Data Engineer

CoffeeBeans • Bengaluru

On-site
INR 1,800,000 - 2,400,000
Data Engineer, Data Platforms
Data Engineer, Data Platforms

S&P Global Market Intelligence • Hyderabad, Chennai

On-site
INR 2,500,000 - 4,500,000
Senior Databricks Engineer / Tech Lead
Senior Databricks Engineer / Tech Lead

Innover Digital Inc. • Bengaluru

On-site
INR 1,800,000 - 3,000,000
Data Engineer (Databricks ) Senior Manager
Data Engineer (Databricks ) Senior Manager

PwC • Pune District, Gurugram District, Bengaluru

Hybrid
INR 3,500,000 - 5,500,000
Technical Lead (Databricks)
Technical Lead (Databricks)

UNAVAILABLE • Pune District

On-site
INR 3,500,000 - 5,200,000
Senior Data Engineer
Senior Data Engineer

Amgen Technology • Hyderabad

Hybrid
INR 2,500,000 - 4,000,000
Senior Data Engineer
Senior Data Engineer

S&P Global Inc. • Hyderabad

On-site
INR 1,200,000 - 2,400,000
Databricks Architect
Databricks Architect

Infosys • Bengaluru

On-site
INR 4,000,000 - 7,000,000