Senior Databricks Engineer: Spark on AWS & Delta Lake

Citi; New York (NY); On-site; USD 37,000 - 68,000; Full time

Job summary

Citi is seeking a Senior Databricks Engineer to contribute to the data platform modernization on Databricks with AWS. You will lead complex Spark implementations, optimize pipelines, and help transition from legacy Hadoop to cloud-native patterns.

The role requires hands-on Spark expertise, strong architectural sense, and collaboration with platform teams and DevOps to deliver scalable, production-ready solutions.

Qualifications

• 10+ years in data engineering or distributed systems.
• Strong Apache Spark (JavaSpark / PySpark) and Delta Lake on AWS.
• Experience with SQL and AWS services for large-scale processing.
• Ability to modernize legacy data platforms to cloud-based architectures.

Responsibilities

• Refactor Spark pipelines to Databricks native architectures.
• Eliminate Hadoop dependencies and adopt AWS patterns.
• Develop Databricks features and Workflows for orchestration and auto-scaling.
• Contribute to architecture and detailed designs; define data models and pipelines.
• Ensure solutions are scalable, maintainable, and production-ready.
• Follow Databricks and Spark engineering standards; write clean, modular code.
• Collaborate with senior architects, platform teams, and DevOps engineers.
• Develop unit, integration, and data-validation tests; support releases.

Skills

Spark

Databricks on AWS

Delta Lake

SQL

AWS services

Data engineering

Education

Bachelor’s degree

Job description

Citi is seeking a Senior Databricks Engineer to contribute to the data platform modernization on Databricks with AWS. You will lead complex Spark implementations, optimize pipelines, and help transition from legacy Hadoop to cloud-native patterns.

The role requires hands-on Spark expertise, strong architectural sense, and collaboration with platform teams and DevOps to deliver scalable, production-ready solutions.