Mission-Critical Data Engineer — DoD AI Pipelines (Secret)

Hatch IT; Washington (District of Columbia); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

401k matching

Medical/dental/vision insurance (PPO/H

Education reimbursement

Life insurance

Generous PTO

Onsite gym

Commuter Benefits Plan

Job summary

Hatch IT is seeking an experienced Data Engineer to support design, development, and deployment of scalable AI-enabled data solutions for the DoD program. You will work across data ingestion, preprocessing, feature engineering, and ML workflows in secure, containerized environments, collaborating with product managers, platform and DevSecOps engineers, data scientists, and mission stakeholders.

The role requires Secret clearance with ability to obtain TS/SCI, and onsite work in Washington, DC.

Qualifications

• Bachelor's degree plus 3 years of recent specialized experience; or
• Associate's degree plus 7 years of recent specialized experience; or
• Major certification plus 7 years of recent specialized experience; or
• 11 years of recent specialized experience.
• Experience with data visualization and data storytelling using tools such as Palantir MSS Workshop and Slate applications.
• Proficiency with Python, SQL, and distributed data frameworks, including technologies such as Spark, Databricks, and PySpark.
• Experience developing machine learning models from training through deployment using industry-standard tools and libraries such as scikit-learn, TensorFlow, and XGBoost.
• Strong technical communication skills with the ability to explain complex concepts to non-technical audiences.

Responsibilities

• Design, develop, and maintain reusable services for data ingestion, transformation, preprocessing, and feature engineering supporting AI/ML workflows.
• Build scalable data pipelines and workflows supporting structured and unstructured mission data.
• Implement data science capabilities such as entity resolution, classification, clustering, prediction, anomaly detection, pattern recognition, and decision-support functions.
• Develop services within secure, containerized environments using established CI/CD, version-control, testing, and documentation standards.
• Collaborate with DevSecOps engineers to integrate data and ML services into secure production environments using technologies such as Databricks, Docker, and Terraform.
• Ensure production services meet applicable performance, reliability, security, and architectural requirements for DoD enterprise and cloud-native environments.
• Develop and deploy standalone and embedded machine learning models supporting mission decision-making, automation, anomaly detection, and pattern recognition.
• Select and implement appropriate modeling approaches using Python, Spark, and cloud-native ML frameworks such as SageMaker and MLflow.
• Maintain reproducibility and interpretability of model outputs to support mission transparency and audit requirements.
• Package model-inference services using documented APIs for integration with end-user applications, operational dashboards, and other mission capabilities.
• Conduct exploratory data analysis to identify patterns, trends, data gaps, and opportunities across structured and unstructured datasets.
• Develop data visualizations, analytical outputs, and interpretive summaries supporting stakeholder understanding and product-team decisions.
• Translate analytical findings into actionable recommendations using visual, narrative, and quantitative communication methods.
• Develop and contribute reusable analysis templates, queries, and analytical workflows to improve delivery efficiency.
• Engage product managers and mission users to define data, analytical, and model requirements aligned with operational objectives.
• Collaborate with software, platform, and DevSecOps engineers to ensure data science components align with technical constraints, architecture, and deployment patterns.
• Participate in Agile sprint planning, retrospectives, demonstrations, and related delivery activities.
• Maintain documentation supporting technical accountability, reproducibility, operational handoff, and sustainment.

Skills

Python

SQL

Spark

Databricks

PySpark

Data visualization

Data storytelling

Palantir Foundry

CI/CD

Education

Bachelor's degree + 3 years experience

Associate's degree + 7 years experience

Major certification + 7 years experience

11 years of recent specialized experience

Tools

Databricks

Docker

Terraform

Job description

Hatch IT is seeking an experienced Data Engineer to support design, development, and deployment of scalable AI-enabled data solutions for the DoD program. You will work across data ingestion, preprocessing, feature engineering, and ML workflows in secure, containerized environments, collaborating with product managers, platform and DevSecOps engineers, data scientists, and mission stakeholders.

The role requires Secret clearance with ability to obtain TS/SCI, and onsite work in Washington, DC.