Data Engineer - Mission-Critical AI Pipelines

Remote; San Francisco (CA); Hybrid; USD 150,000 - 230,000; Full time

Benefits offered by this job

Unlimited PTO

Sick time

Comprehensive health benefits

11 paid holidays

Professional development

Flat structure

Mission-driven work

Growth opportunities

401(k) with company match

Job summary

Spear AI is a growing defense contracting company delivering AI/ML data infrastructure to support national security. The Data Engineer role focuses on building scalable data pipelines, modeling data for AI/ML workloads, and ensuring data quality and governance across cloud platforms.

Join a flat, mission-driven team with opportunities to collaborate directly with leadership and impact warfighter capabilities.

Qualifications

• 5+ years of experience in data engineering or related field.
• Active TS/SCI clearance with ability to obtain CI Polygraph.
• Proficiency in Python, SQL, and data processing frameworks (Apache Spark, Kafka, etc.).
• Experience with cloud platforms (AWS preferred, Azure, GCP) and containerization technologies.
• Knowledge of data warehousing concepts and modern data stack tools.
• Understanding of version control systems and CI/CD practices.

Responsibilities

• Develop and maintain ETL/ELT pipelines for processing large-scale datasets.
• Design data models and schemas optimized for AI/ML workloads.
• Implement data quality monitoring and validation frameworks.
• Collaborate with AI engineers to understand data requirements and optimize data delivery.
• Troubleshoot data pipeline issues and implement performance improvements.
• Maintain documentation for data flows, transformations, and pipeline architectures.
• Support data governance initiatives and ensure compliance with data handling policies.

Skills

Python

SQL

Apache Spark

Kafka

Cloud platforms

CI/CD

Version control

AWS

Military experience

Tools

AWS

Azure

GCP

Docker/Kubernetes

Job description

Spear AI is a growing defense contracting company delivering AI/ML data infrastructure to support national security. The Data Engineer role focuses on building scalable data pipelines, modeling data for AI/ML workloads, and ensuring data quality and governance across cloud platforms.

Join a flat, mission-driven team with opportunities to collaborate directly with leadership and impact warfighter capabilities.