Senior Data Engineer (TS/SCI) for Federal Missions

Metaphase-Consulting; Washington (District of Columbia); Hybrid; USD 140,000 - 200,000; Full time

Job summary

MetaPhase Consulting seeks a Senior Data Engineer to design and sustain scalable data pipelines, data stores, and analytic products for a secure federal mission environment. You will convert diverse data into governed information assets to support visual analytics and investigative workflows.

You will lead batch and streaming pipelines, build robust data models, and implement ETL/ELT processes with strong data quality, metadata, and security controls across AWS/Azure-like platforms in regulated

Qualifications

• Minimum of 10 years of relevant experience in data engineering, data architecture, data integration, or a closely related technical discipline.
• Bachelor’s degree in computer science, data science, information systems, engineering, mathematics, or a related field.
• Active Top Secret clearance with eligibility for SCI access. U.S. Citizenship is required.
• Advanced proficiency in Python, SQL, and database development, with hands‑on experience designing relational data stores and optimizing data-processing performance.
• Experience building and operating ETL or ELT pipelines, including batch and streaming data processing, data validation, error handling, and monitoring.
• Experience integrating diverse data sources and managing data quality, metadata, lineage, access controls, and security requirements in a Federal or similarly regulated environment.
• Ability to communicate technical data concepts clearly and collaborate effectively with mission users, analysts, engineers, and delivery leadership.

Responsibilities

• Design, develop, and optimize batch and real-time data pipelines that ingest, validate, transform, and publish structured, semi-structured, and unstructured data.
• Build and maintain large-scale relational databases, cloud storage, and data models that support high-performance web and desktop visual analytic applications.
• Develop ETL and ELT processes, data-quality checks, metadata, lineage, and monitoring that make complex data reliable and usable across the mission lifecycle.
• Engineer scalable data services, APIs, and curated datasets that enable bulk analysis of relationships, advanced graphics, visualization, and high-performance computing workloads.
• Optimize query performance, database structures, indexing, and data-processing workflows while integrating new sources with existing warehouse and analytic environments.
• Collaborate directly with analysts, software engineers, architects, and mission users to refine data requirements, test delivered capabilities, and improve data products through iterative delivery.
• Apply secure engineering practices for data access, auditability, configuration management, and operational resilience across open-source, COTS, and GOTS technologies.

Skills

Python

SQL

Data modeling

Data integration

Cloud data platforms

Education

Bachelor's degree in CS/DS/IS/Engineering/Math

Tools

PostgreSQL

OpenSearch

Elasticsearch

Kafka

NiFi

Cribl

Docker

Kubernetes

IaC

Job description

MetaPhase Consulting seeks a Senior Data Engineer to design and sustain scalable data pipelines, data stores, and analytic products for a secure federal mission environment. You will convert diverse data into governed information assets to support visual analytics and investigative workflows.

You will lead batch and streaming pipelines, build robust data models, and implement ETL/ELT processes with strong data quality, metadata, and security controls across AWS/Azure-like platforms in regulated