Remote Databricks Data Engineer: Spark, Delta Lake

Leidos; Atlanta (GA); On-site; USD 87,000 - 157,000; Full time

Job summary

Leidos is seeking a Databricks Data Engineer to design, develop, and maintain scalable data engineering solutions on the Databricks Lakehouse Platform for the NHSN support contract. This role will build ETL/ELT pipelines, optimize Delta Lake tables, and implement Medallion Architecture to support analytics, BI, and AI initiatives across the organization.

You will collaborate with data architects, data scientists, and application teams to deliver secure, reliable data platforms in a cloud-native

Qualifications

• Hands-on experience with Databricks and Apache Spark.
• Strong Python and PySpark development skills.
• Advanced SQL and data modeling capabilities.
• Experience designing scalable ETL/ELT pipelines.
• Knowledge of Delta Lake and Lakehouse architecture.
• Experience with cloud platforms (Azure, AWS, or GCP).
• Git-based source control and CI/CD deployment experience.
• Strong analytical, troubleshooting, and problem-solving abilities.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Apache Spark.
• Build data ingestion frameworks to integrate structured, semi-structured, and unstructured data from various enterprise systems.
• Develop data transformation logic using PySpark and Spark SQL.
• Design, implement, and optimize Delta Lake tables for high performance, reliability, and scalability.
• Implement Medallion Architecture (Bronze, Silver, Gold) for enterprise data processing.
• Collaborate with cross-functional teams to understand business requirements and deliver high-quality data solutions.

Skills

Python

PySpark

SQL

Databricks

Apache Spark

Git & CI/CD

Analytical skills

Problem solving

Education

Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics

Tools

Airflow

Delta Lake

Job description

Leidos is seeking a Databricks Data Engineer to design, develop, and maintain scalable data engineering solutions on the Databricks Lakehouse Platform for the NHSN support contract. This role will build ETL/ELT pipelines, optimize Delta Lake tables, and implement Medallion Architecture to support analytics, BI, and AI initiatives across the organization.

You will collaborate with data architects, data scientists, and application teams to deliver secure, reliable data platforms in a cloud-native