Data Engineer - Health Data Pipelines & APIs (Secret)

Systems Planning & Analysis; United States; On-site; USD 120,000 - 146,000; Full time

Benefits offered by this job

Health insurance

Retirement plans

Life and disability insurance

Paid time off

Job summary

Systems Planning & Analysis seeks a Data Engineer to support DoD health and defense data initiatives. Design and maintain scalable data pipelines using Spark, Python, and SQL in a Databricks environment, integrating health, financial, and operations data into reusable data products.

You will build data models for enterprise reporting, develop API-based integrations, and ensure data quality, governance, and security.

Qualifications

• Active Secret clearance
• Bachelor's degree in CS/IT or related field
• 3–10 years of data/software engineering experience
• Strong proficiency with Python and SQL
• Experience with Spark or similar distributed data-processing frameworks
• Experience with Databricks
• Experience building production data pipelines
• Experience integrating data via REST APIs
• Familiarity with data governance, security and privacy considerations
• Git-based development workflows and modern software practices
• Comfort in terminal/CLI environments
• Experience using AI-assisted development tools
• Strong communication with stakeholders
• Ability to navigate ambiguity and drive tasks to resolution

Responsibilities

• Design, develop, and maintain scalable, production-ready data pipelines using Spark, Python, and SQL in a Databricks environment.
• Integrate and transform diverse health, financial, and operational datasets into reliable, reusable data products.
• Develop data models and analytics datasets that support enterprise reporting and operational decision-making.
• Build and maintain API-based integrations for ingesting and exchanging data across systems.
• Monitor, troubleshoot, and optimize scheduled workflows and production pipelines for performance and reliability.
• Implement data validation, quality checks, lineage, and governance practices to improve trust in data products.
• Contribute to automated testing, code reviews, documentation, and reusable engineering patterns.
• Collaborate with engineers, architects, analysts, and customer stakeholders to translate data needs into practical solutions.

Skills

Python

SQL

Apache Spark

Databricks

ETL/ELT

REST APIs

Git

CLI/Unix

AI-assisted tools

Data modeling

Communication

Education

Bachelor's degree in Computer Science, Information Systems, Software Engineering, or a related field

Tools

Databricks

Palantir Foundry

Advana

GitLab

CI/CD pipelines

AWS

Job description

Systems Planning & Analysis seeks a Data Engineer to support DoD health and defense data initiatives. Design and maintain scalable data pipelines using Spark, Python, and SQL in a Databricks environment, integrating health, financial, and operations data into reusable data products.

You will build data models for enterprise reporting, develop API-based integrations, and ensure data quality, governance, and security.