Senior Data Engineer - Remote, DoD Data Lakehouse

Omni Federal; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Omni Federal is seeking a Senior Data Engineer to design, build, and operate modern data pipelines for DoD-focused programs. This remote role requires active DoD Secret clearance and delivering production-grade streaming and batch solutions for data lakehouse workloads.

You will work in an agile environment, own pipelines end-to-end, and collaborate with data providers and consumers while adhering to DevSecOps practices, Terraform IaC, and AWS data services.

Qualifications

• 8+ years of professional data engineering experience with production pipelines.
• Hands-on design and operation of streaming and batch data pipelines using Debezium, MSK, Spark, and Airflow.
• Proficiency with Python and SQL and AWS data services.
• Ability to work independently and communicate with technical and non-technical stakeholders.

Responsibilities

• Design, build, and operate streaming (Debezium to Amazon MSK) and batch (Spark via AWS EMR) data pipelines for the program’s Iceberg-based data lakehouse.
• Corroborate onboarded data against existing lakehouse data to validate quality and lineage.
• Maintain metadata/catalog using AWS Glue and API Gateway integrations for discoverability by downstream partners.
• Translate requirements into working data solutions with minimal oversight.
• Engage with data providers and data consumers to understand context and communicate it clearly.
• Participate in agile delivery (Kanban/Scrum) and contribute to sprint planning and code reviews.
• Maintain infrastructure as code (Terraform) and follow DevSecOps CI/CD pipelines.
• Support data governance, quality, and auditability per government standards.

Skills

Python

SQL

Agile delivery

Communication

Tools

Debezium

Amazon MSK

Apache Spark

Apache Airflow

AWS EMR

Glue

Terraform

API Gateway

Iceberg

Kubernetes

S3

Aurora

Job description

Omni Federal is seeking a Senior Data Engineer to design, build, and operate modern data pipelines for DoD-focused programs. This remote role requires active DoD Secret clearance and delivering production-grade streaming and batch solutions for data lakehouse workloads.

You will work in an agile environment, own pipelines end-to-end, and collaborate with data providers and consumers while adhering to DevSecOps practices, Terraform IaC, and AWS data services.