Data Engineer - Secure Data Pipelines (Secret Clearance)

Agile Defense; Arlington (VA); On-site; USD 110,000 - 160,000; Full time

Job summary

Agile Defense is seeking a Data Engineer to develop and maintain secure data pipelines and analytics-ready datasets. The role collaborates with data scientists, engineers, and mission stakeholders to process data using Databricks, Spark, Palantir Foundry, Maven Smart Systems, or related platforms.

Responsibilities include ETL/ELT pipeline support, data validation, and working within Agile frameworks. The position requires US citizenship and an active Secret clearance, with onsite work in SCIF

Qualifications

• Bachelor’s degree in computer science, data science, engineering, information systems, mathematics, or a related technical field.
• 3+ years of relevant data engineering, software development, data analytics, or enterprise‑platform experience.
• Experience with Python and SQL familiarity with data validation, data modeling, and data-quality practices; familiarity with Spark, PySpark, Databricks, or comparable technologies.
• Experience supporting ETL/ELT pipelines and data transformations.
• Understanding of version control, testing, documentation, and Agile development.
• Ability to collaborate with technical and nontechnical stakeholders.
• Active US government security clearance.

Responsibilities

• Supports the development and maintenance of secure data pipelines, ETL/ELT workflows, and analytics-ready datasets.
• Works with data scientists, engineers, and mission stakeholders to process structured and unstructured data using Databricks, Spark, Palantir Foundry, Maven Smart Systems, or comparable platforms.

Skills

3+ years data engineering

Python

SQL

Data validation

ETL/ELT

Agile

Stakeholder collaboration

Security clearance

Education

Bachelor's degree in CS/DS/Engineering

Tools

Databricks

Spark

Palantir Foundry

Maven Smart Systems

War Data Platform

Docker

CI/CD

MLflow

APIs

Terraform

AWS

Azure

Job description

Agile Defense is seeking a Data Engineer to develop and maintain secure data pipelines and analytics-ready datasets. The role collaborates with data scientists, engineers, and mission stakeholders to process data using Databricks, Spark, Palantir Foundry, Maven Smart Systems, or related platforms.

Responsibilities include ETL/ELT pipeline support, data validation, and working within Agile frameworks. The position requires US citizenship and an active Secret clearance, with onsite work in SCIF