Senior Data Engineer — AI Pipelines (Remote USA)

Defenseunicorns; Washington (District of Columbia); Remote; USD 149,000 - 201,000; Full time

Benefits offered by this job

Health insurance

Company-paid premiums

HSA

Life insurance

Disability insurance

401k retirement plan

Stock options

Home office budget

Paid holidays

Parental leave

Job summary

Defense Unicorns seeks a senior Data Engineer to embed with a government team and partners, building an AI-powered engineering ecosystem for mission-critical data workflows. You will own data ingestion, transformation, storage, and metadata across AI workloads in secure environments.

The role emphasizes creating repeatable data patterns and scalable foundations, collaborating with platform engineers on Kubernetes/OpenShift, CI/CD, and GitOps to move from prototype to production.

Qualifications

• Active TS/SCI clearance.
• 7+ years in data engineering, data platform engineering, or a closely related field.
• Strong proficiency with SQL and Python for data processing, automation, and integration.
• Hands-on experience building and operating production data pipelines and ETL/ELT workflows.
• Experience with relational databases and modern data storage patterns for structured and unstructured data.
• Experience integrating data through REST APIs, message/event systems, or other distributed integration patterns.
• Working knowledge of data modeling, schema design, data quality, lineage, metadata, and data governance concepts.
• Experience operating in cloud-native or containerized environments and collaborating with Kubernetes/platform engineering teams.
• Ability to troubleshoot data issues across infrastructure, applications, APIs, storage, networking, and access controls.
• Ability to work independently, rapidly learn new technologies, and translate ambiguous mission needs into working data solutions.
• Local to the National Capital Region and able to support onsite work at a government facility in Springfield, VA as mission needs increase.

Responsibilities

• Design, build, and maintain reliable data pipelines that ingest, transform, validate, and deliver data for AI models, agents, and mission applications.
• Integrate structured and unstructured data sources into enterprise data services, including relational, document, vector, and graph-oriented technologies.
• Work closely with AI/agent engineers to make data discoverable and usable for RAG, semantic search, model context, and tool-enabled agent workflows.
• Develop repeatable data ingestion and transformation patterns using Python, SQL, APIs, event-driven architectures, and workflow/orchestration technologies.
• Establish data quality, validation, provenance, lineage, and metadata practices so downstream users and AI systems can trust the information they consume.
• Design data access patterns that enforce appropriate identity, authorization, classification, and security boundaries while minimizing unnecessary friction for developers and mission users.
• Collaborate with platform engineers to operationalize data services within Kubernetes/OpenShift environments and integrate them into CI/CD and GitOps workflows.
• Help integrate enterprise data capabilities such as relational stores, JSON/document data, vector search, graph data, advanced analytics, and RAG services into the broader platform architecture.
• Troubleshoot end-to-end data issues spanning source systems, pipelines, storage, APIs, identity, networking, platform services, and AI applications.
• Work directly with government engineers and commercial technology partners to resolve data dependencies and remove blockers to mission delivery.
• Develop documentation, data contracts, schemas, architecture decision records, runbooks, and operational standards that make successful patterns repeatable across environments.
• Identify recurring data engineering challenges that can be standardized, automated, or productized rather than repeatedly solved through manual engineering.
• Operate effectively in a fast-moving, ambiguous environment where data sources, architectures, security requirements, and mission priorities will continue to evolve.

Skills

SQL

Python

ETL/ELT

Data pipelines

Kubernetes

Cloud environments

Data modeling

Data governance

Tools

Kubernetes

OpenShift

Airflow

Dagster

Argo Workflows

Job description

Defense Unicorns seeks a senior Data Engineer to embed with a government team and partners, building an AI-powered engineering ecosystem for mission-critical data workflows. You will own data ingestion, transformation, storage, and metadata across AI workloads in secure environments.

The role emphasizes creating repeatable data patterns and scalable foundations, collaborating with platform engineers on Kubernetes/OpenShift, CI/CD, and GitOps to move from prototype to production.