Senior Data Engineer — Azure, Spark & Databricks

Prometheus Federal Services (PFS); Fairfax (VA); On-site; USD 130,000 - 155,000; Full time

Benefits offered by this job

Health, dental & vision benefits

Retirement plan

Paid time off

Job summary

Prometheus Federal Services (PFS) seeks an experienced Data Engineer to support VA programs. You will design, optimize, and maintain scalable ETL/ELT pipelines and databases across on‑prem and Azure cloud environments, focusing on performance, data quality, and governance.

Responsibilities include building pipelines with SQL, Spark, Databricks and Azure data services, implementing monitoring, security, backup strategies, and collaborating with developers, data scientists, and stakeholders to

Qualifications

• Bachelor’s degree required.
• 5+ years of data engineering, ETL/ELT, or DBA experience.
• Strong SQL, Python, PySpark, Spark, Databricks proficiency for ingestion, cleaning, transformation and performance tuning.

Responsibilities

• Design, build, deploy, and maintain scalable ETL/ELT pipelines using SQL, Spark, Databricks and Azure-native tools.
• Implement workflow orchestration, scheduling, and monitoring for automated data delivery.
• Develop data quality monitoring, lineage tracking, and data integrity across pipelines.

Skills

SQL

Python

PySpark

Spark

Databricks

Education

Bachelor’s degree

Tools

Azure Data Factory

Azure Synapse

Azure Data Lake Storage

Azure SQL

SQL Server

PostgreSQL

GitHub

GitHub Actions

Job description

Prometheus Federal Services (PFS) seeks an experienced Data Engineer to support VA programs. You will design, optimize, and maintain scalable ETL/ELT pipelines and databases across on‑prem and Azure cloud environments, focusing on performance, data quality, and governance.

Responsibilities include building pipelines with SQL, Spark, Databricks and Azure data services, implementing monitoring, security, backup strategies, and collaborating with developers, data scientists, and stakeholders to