Senior Data Engineer

Defense Unicorns; Washington (District of Columbia); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Health Insurance

401k Retirement Plan

Company Stock Options

Home Office Budget

Paid Holidays

Parental Leave

Learning reimbursements

Job summary

Defense Unicorns is seeking a Senior Data Engineer to embed with a government team and partners, building an AI-powered engineering ecosystem. You will own end-to-end data lifecycle, making data reliable, secured, and ready for AI workloads in a mission-critical environment.

The role involves ingesting, transforming, storing, and provisioning data across relational, document, vector, and graph stores, plus ensuring governance, provenance, and security.

Qualifications

• Active TS/SCI clearance.
• 7+ years in data engineering or related field.
• Strong proficiency with SQL and Python for data processing, automation, and integration.
• Hands-on experience building and operating production data pipelines and ETL/ELT workflows.
• Experience with relational databases and modern data storage patterns for structured and unstructured data.
• Experience integrating data through REST APIs, message/event systems, or other distributed integration patterns.
• Knowledge of data modeling, schema design, data quality, lineage, metadata, and governance concepts.
• Experience operating in cloud-native or containerized environments and collaborating with Kubernetes/platform engineering teams.
• Ability to troubleshoot data issues across infrastructure, applications, APIs, storage, networking, and access controls.
• Ability to work independently, rapidly learn new technologies, and translate ambiguous mission needs into working data solutions.
• Local to the National Capital Region and onsite Springfield, VA as needed.

Responsibilities

• Design, build, and maintain reliable data pipelines for AI models, agents, and mission apps.
• Integrate structured and unstructured data sources into enterprise data services.
• Collaborate with AI/agent engineers to enable RAG, semantic search, and tool-enabled workflows.
• Develop repeatable ingestion and transformation patterns using Python, SQL, APIs, and event-driven architectures.
• Establish data quality, provenance, lineage, and metadata practices for trust.
• Design data access patterns with proper identity, authorization, and security boundaries.
• Work with platform engineers to operationalize data services in Kubernetes/OpenShift and CI/CD workflows.
• Integrate data capabilities like relational stores, JSON/document data, vector search, and graphs into the platform.
• Troubleshoot end-to-end data issues across source systems, pipelines, storage, APIs, and AI apps.
• Collaborate with government engineers and partners to resolve data dependencies and blockers.
• Develop documentation, data contracts, schemas, runbooks, and standards for repeatable patterns.
• Identify recurring data engineering challenges that can be standardized or automated.
• Operate in a fast-moving, evolving environment.

Skills

SQL

Python

Data pipelines

ETL/ELT

Data governance

Cloud native

Security/compliance

Tools

PostgreSQL/MySQL

REST APIs

Kubernetes

OpenShift

Airflow

Dagster

Kafka

Job description

Senior Data Engineer

AI Engineering | Forward Deployed Engineering

Role Description

Defense Unicorns is seeking a senior Data Engineer to embed with a government team and its technology partners building an emerging AI-powered engineering ecosystem.

This role will focus on making enterprise data usable, discoverable, governed, and accessible to AI models, agents, and mission applications. The engineer will work across data ingestion, transformation, storage, metadata, APIs, search, vector and graph technologies, and the interfaces between the data layer and AI workloads.

The ideal candidate is a hands‑on data engineer who can operate across the full data lifecycle and understands that data availability and integration are often the primary blockers to delivering AI capabilities. They should be comfortable entering an evolving architecture, learning unfamiliar systems quickly, working directly with commercial technology partners, and solving difficult problems involving data quality, lineage, access, performance, and security.

This is not a traditional analytics or business intelligence role. The focus is on building the data foundations and repeatable patterns that allow AI and agentic capabilities to access trusted data and move from prototype to production in secure and classified environments.

Responsibilities

• Design, build, and maintain reliable data pipelines that ingest, transform, validate, and deliver data for AI models, agents, and mission applications.
• Integrate structured and unstructured data sources into enterprise data services, including relational, document, vector, and graph-oriented technologies.
• Work closely with AI/agent engineers to make data discoverable and usable for RAG, semantic search, model context, and tool-enabled agent workflows.
• Develop repeatable data ingestion and transformation patterns using Python, SQL, APIs, event‑driven architectures, and workflow/orchestration technologies.
• Establish data quality, validation, provenance, lineage, and metadata practices so downstream users and AI systems can trust the information they consume.
• Design data access patterns that enforce appropriate identity, authorization, classification, and security boundaries while minimizing unnecessary friction for developers and mission users.
• Collaborate with platform engineers to operationalize data services within Kubernetes/OpenShift environments and integrate them into CI/CD and GitOps workflows.
• Help integrate enterprise data capabilities such as relational stores, JSON/document data, vector search, graph data, advanced analytics, and RAG services into the broader platform architecture.
• Troubleshoot end‑to‑end data issues spanning source systems, pipelines, storage, APIs, identity, networking, platform services, and AI applications.
• Work directly with government engineers and commercial technology partners to resolve data dependencies and remove blockers to mission delivery.
• Develop documentation, data contracts, schemas, architecture decision records, runbooks, and operational standards that make successful patterns repeatable across environments.
• Identify recurring data engineering challenges that can be standardized, automated, or productized rather than repeatedly solved through manual engineering.
• Operate effectively in a fast-moving, ambiguous environment where data sources, architectures, security requirements, and mission priorities will continue to evolve.

Minimum Experience and Qualifications

• Active TS/SCI clearance.
• 7+ years of experience in data engineering, data platform engineering, or a closely related field.
• Strong proficiency with SQL and Python for data processing, automation, and integration.
• Hands‑on experience building and operating production data pipelines and ETL/ELT workflows.
• Experience with relational databases and modern data storage patterns for structured and unstructured data.
• Experience integrating data through REST APIs, message/event systems, or other distributed integration patterns.
• Working knowledge of data modeling, schema design, data quality, lineage, metadata, and data governance concepts.
• Experience operating in cloud‑native or containerized environments and collaborating with Kubernetes/platform engineering teams.
• Ability to troubleshoot data issues across infrastructure, applications, APIs, storage, networking, and access controls.
• Ability to work independently, rapidly learn new technologies, and translate ambiguous mission needs into working data solutions.
• Local to the National Capital Region and able to support onsite work at a government facility in Springfield, VA as mission needs increase.

Preferred Experience and Qualifications

• Experience supporting AI/ML workloads, including RAG, embeddings, vector search, model context, or agentic applications.
• Experience with vector databases, graph databases, document databases, knowledge graphs, or semantic search technologies.
• Experience with Oracle or comparable enterprise data platforms supporting relational, JSON/document, vector, graph, and analytics workloads.
• Experience designing data products or platforms for high-scale, low-latency, or mission-critical workloads.
• Experience with data orchestration and workflow tools such as Airflow, Dagster, Argo Workflows, or comparable technologies.
• Experience with streaming/event platforms such as Kafka, Redpanda, NATS, or similar technologies.
• Experience deploying data services using Kubernetes, Helm, GitOps, Infrastructure as Code, and automated CI/CD pipelines.
• Experience supporting data and AI systems in Secret or TS/SCI environments and across multiple security domains.
• Experience with disconnected or air‑gapped environments and the challenges of moving data, metadata, and software across security boundaries.
• Familiarity with NIST 800-53, RMF/ATO processes, zero-trust architectures, and data security/compliance practices.
• Experience with enterprise search, RAG pipelines, data catalogs, data contracts, or knowledge‑graph architectures.
• Familiarity with UDS, Zarf, Pepr, Iron Bank, or similar secure software delivery technologies.
• Experience working on multidisciplinary teams consisting of government engineers, FDEs, OEM professional services teams, and multiple technology vendors.
• Demonstrated ability to take a fragmented or difficult data environment and turn it into a reliable, reusable foundation for mission applications and AI.

Mission Focus

Make trusted data available where AI needs it, while establishing the repeatable, secure data patterns required to move from experimentation to operational capability.

Who We Are

Defense Unicorns delivers mission value by streamlining software delivery so our customers can focus on the most important challenges. We share a vision of freedom and security for the advancement of progress and innovation. Our commitment to this vision, and to our mission‑driven customers, means a commitment to speed, user experience and optionality, without compromising security. Our team is composed of innovators, software engineers, and veterans with decades of experience delivering technology programs across the federal market.

What We Do

We create and deliver secure solutions for continuous software integration and delivery. Defense Unicorns consolidates the best practices for security pipelines, testing, and deployment automation in order to meet the high security requirements valued by mission owners. Our solutions are agnostic by design and we believe that growing a robust ecosystem of secure, cloud‑native software solutions can help enterprise customers inside and outside the federal market buy and integrate software more easily.

Who We Serve

Defense Unicorns’ customers are mission‑focused leaders across public and private enterprises. We proudly support defense and civil agencies across the U.S. government and we work closely with the creators of leading‑edge software solutions to deliver value to the mission‑owner by improving the security and consumability of commercial software products.

What We Work On

• Kubernetes
• Cloud Environments (AWS/GCP and Azure)
• Infrastructure‑as‑code (like Terraform/Pulumi)
• Continuous Delivery and automation tooling
• GitOps
• Containers
• CNCF projects and open source products and packages
• Helm/Kustomize-Value Stream Mapping
• Building and improving security delivery
• Building Kubernetes and cloud native applications

Benefits Our Unicorns Enjoy

Health:

• Medical/Dental/Vision
• Premiums are 100% Company Paid
• Health Savings Account
• Life Insurance
• Disability Insurance

Financial:

• 401k Retirement Plan
• Company Stock Options
• Home Office Budget

Leave:

• We offer all full-time Unicorns Flexible Time Off (FTO) plus all Federal Holidays, one week for Thanksgiving, and two weeks for Christmas and New Year’s
• Paid Parental Leave

Learning:

• Reimbursement for approved trainings/subscriptions
• Conferences (travel, lodging, and fees)

Don’t have all the preferred experience or qualifications? Studies show that underrepresented groups like women and people of color are less likely to apply to jobs if they don’t meet every requirement listed.

At Defense Unicorns, we’re committed to diversity. If you’re enthusiastic about the role but don’t match every criteria, we encourage you to apply. You could be the perfect fit for this or another role! Defense Unicorns is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, disability, sex, sexual orientation, gender identity or expression, age, national origin, veteran status, genetic information, union status and/or beliefs, or any other characteristic protected by federal, state, or local law.

CCPA DISCLOSURE