Lead Data Engineer — Azure Databricks & Spark

Cognizant; United States; On-site; USD 150,000 - 210,000; Full time

Job summary

Cognizant seeks an experienced Lead Data Engineer to architect and implement scalable data platforms in the hotel and hospitality domain. You will lead end-to-end data engineering efforts, drive best practices, and collaborate with stakeholders to translate analytics requirements into robust technical solutions.

The role emphasizes data quality, governance, and cost-efficient Databricks and Spark workloads across enterprise data products.

Qualifications

• Strong hands-on experience with Azure Databricks and Spark ecosystems.
• Expertise in PySpark development and distributed data processing.
• Proven ability to design and maintain large-scale data pipelines.
• Experience with data modeling, data integration, ETL/ELT processes.
• Knowledge of data quality, monitoring, and validation frameworks.

Responsibilities

• Lead the end-to-end design and implementation of scalable data engineering solutions using Azure Databricks, Spark, and PySpark.
• Design, develop, and optimize robust data pipelines and processing workloads for performance, reliability, and cost efficiency.
• Provide technical direction on PySpark/Spark development standards, data modeling, and architecture patterns.

Skills

Azure Databricks

Apache Spark

PySpark

Large-scale data pipelines

Data modeling

Data integration

ETL/ELT

Data governance

Data quality

Performance tuning

Cluster optimization

Mentoring

Stakeholder management

Translate requirements

Cloud architecture

Job description

Cognizant seeks an experienced Lead Data Engineer to architect and implement scalable data platforms in the hotel and hospitality domain. You will lead end-to-end data engineering efforts, drive best practices, and collaborate with stakeholders to translate analytics requirements into robust technical solutions.

The role emphasizes data quality, governance, and cost-efficient Databricks and Spark workloads across enterprise data products.