Senior Data Engineer: Azure, Spark & Data Pipelines

prometheusfederalservices; United States; Remote; USD 130,000 - 155,000; Full time

Benefits offered by this job

Health, dental, and vision insurance

Retirement plan

Paid time off

Job summary

Prometheus Federal Services (PFS) is seeking an experienced Data Engineer to design, optimize, and maintain scalable data pipelines and databases across on‑prem and cloud environments for VA programs.

You will collaborate with developers, data scientists, and stakeholders to deliver reliable, high‑performance data and analytics products while improving data accessibility, quality, and operational efficiency.

Qualifications

• Bachelor’s degree required.
• 5+ years in data engineering, ETL/ELT, or DBA.
• Strong SQL, Python, PySpark, Spark, Databricks experience for ingestion, cleaning, and transformation including performance tuning.

Responsibilities

• Design, build, deploy, and maintain scalable ETL/ELT pipelines using SQL, Spark, Databricks, and Azure-native tools.
• Implement workflow orchestration, scheduling, and monitoring for automated data delivery.
• Develop data quality monitoring and observability tools to track lineage and integrity.

Skills

SQL

Python

PySpark

Spark

Databricks

Azure

Performance tuning

Education

Bachelor’s degree

Tools

GitHub

GitHub Actions

Azure Data Factory

Azure Synapse

Azure Data Lake Storage

Azure SQL

SQL Server

PostgreSQL

Job description

Prometheus Federal Services (PFS) is seeking an experienced Data Engineer to design, optimize, and maintain scalable data pipelines and databases across on‑prem and cloud environments for VA programs.

You will collaborate with developers, data scientists, and stakeholders to deliver reliable, high‑performance data and analytics products while improving data accessibility, quality, and operational efficiency.