Senior Data Engineer - Azure & Databricks

Prometheus Federal Services; United States; Remote; USD 130,000 - 155,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

Retirement plan

Paid time off

Job summary

Prometheus Federal Services seeks an experienced Data Engineer to design, optimize, and maintain scalable data pipelines and databases for VA programs. You will work across on‑prem and cloud environments, focusing on data quality, performance, and accessibility.

The role requires strong SQL, Python, PySpark, Spark, and Databricks expertise, plus Azure Data Factory/Synapse/ADLS. Collaboration with developers and stakeholders ensures reliable analytics products and governance.

Qualifications

• Bachelor’s degree required.
• 5+ years of data engineering, ETL/ELT, or DBA experience.
• Strong SQL, Python, PySpark, Spark, and Databricks skills with Azure data services.
• Experience administering production relational databases and data pipelines.
• Ability to work with VA platforms and enterprise data assets.

Responsibilities

• Design, build, deploy, and maintain scalable ETL/ELT pipelines using SQL, Spark, Databricks, and Azure-native tools.
• Implement workflow orchestration, scheduling, and monitoring for automated data delivery.
• Develop data quality monitoring and observability tools to track lineage and integrity.
• Optimize ingestion, transformation, and storage for performance and cost.
• Administer and tune relational databases; ensure security and auditing aligned with VA requirements.
• Support backup/restore, disaster recovery, and high availability configurations.
• Create dashboards and alerts to proactively identify data issues; collaborate with stakeholders.

Skills

SQL

Python

PySpark

Spark

Databricks

Azure analytics

DBA best practices

Data quality

Data orchestration

Education

Bachelor’s degree

Tools

Azure Data Factory

Azure Synapse

Azure Data Lake Storage

Azure SQL

GitHub

GitLab

GitHub Actions

Job description

Prometheus Federal Services seeks an experienced Data Engineer to design, optimize, and maintain scalable data pipelines and databases for VA programs. You will work across on‑prem and cloud environments, focusing on data quality, performance, and accessibility.

The role requires strong SQL, Python, PySpark, Spark, and Databricks expertise, plus Azure Data Factory/Synapse/ADLS. Collaboration with developers and stakeholders ensures reliable analytics products and governance.