Senior Databricks Data Engineer — NYC

Capgemini; New York (NY); On-site; USD 89,000 - 166,000; Full time

Job summary

Capgemini in New York City is seeking a Senior Data Engineer with expertise in Databricks, Spark, and cloud data platforms to lead data engineering initiatives and client delivery. You will design scalable pipelines, manage project delivery, and optimize cost and performance while collaborating with cross-functional teams.

The role requires 5+ years in ETL/ELT design, strong Python/SQL, and hands-on experience with cloud data platforms (AWS/Azure/GCP).

Qualifications

• 5+ years building ETL/ELT pipelines with Databricks, Spark, and PySpark.
• Strong Python and SQL skills with PySpark proficiency.
• 5+ years in project/delivery management.
• Experience with Delta Lake, lakehouse arch, streaming data.
• Hands-on with AWS, Azure, or GCP.
• CI/CD, Git, DevOps practices, and performance tuning.

Responsibilities

• Design, build, and optimize scalable data pipelines and lakehouse solutions.
• Lead project delivery, resource planning, risk management, and stakeholder communication.
• Manage support operations, ensuring SLA compliance and high customer satisfaction.
• Collaborate with business, analytics, and engineering teams to deliver data-driven solutions.
• Develop dashboards, reports, and automation initiatives to improve operational efficiency.
• Monitor cloud costs (GCP/Snowflake) and drive cost optimization efforts.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

Delivery management

Delta Lake

Data Lakehouse

Streaming data

Data warehousing

AWS

Azure

GCP

CI/CD

Git

DevOps

Performance tuning

Job description

Capgemini in New York City is seeking a Senior Data Engineer with expertise in Databricks, Spark, and cloud data platforms to lead data engineering initiatives and client delivery. You will design scalable pipelines, manage project delivery, and optimize cost and performance while collaborating with cross-functional teams.

The role requires 5+ years in ETL/ELT design, strong Python/SQL, and hands-on experience with cloud data platforms (AWS/Azure/GCP).