DoD Data Engineer: ETL Pipelines & Spark Analytics

Redhorse Corporation; Arlington (VA); On-site; USD 110,000 - 150,000; Full time

Job summary

Redhorse Corporation seeks a proactive Mid-Level Data Engineer to join our team supporting the Office of the Under Secretary of Defense for Acquisition & Sustainment. You will design and optimize data pipelines and analytics solutions across enterprise DoD data environments, enabling data-driven decision making.

You will work with Databricks, Qlik, and cloud platforms to deliver scalable ETL/ELT workflows, maintain pipeline health, and ensure STIG compliance.

Qualifications

• Bachelor's degree in a STEM field with 3+ years of data engineering experience (or Master's with 2+ years).
• Experience developing ETL/ELT pipelines and processing large-scale data using Spark.
• Experience in cloud environments (AWS, Azure, MilCloud 2.0).
• Proficiency in Python, SQL, or Scala.
• Familiarity with DoD STIGs and data security practices.
• Knowledge of data modeling and automated configuration management.

Responsibilities

• Ingest, configure, and transform structured, semi-structured, and unstructured datasets to meet mission partner and stakeholder requirements.
• Build, automate, and maintain scalable ETL/ELT pipelines supporting DevOps and rapid analytics delivery.
• Maintain data pipeline operations, monitor environment health, and implement automated performance optimization and cost-reduction practices.
• Prototype new data capabilities, evaluate emerging technologies, and deploy end-to-end data solutions.
• Collaborate with cross-functional technical teams to troubleshoot pipeline bottlenecks, optimize database structures, and implement secure data retrieval methods.
• Implement and automate DoD STIGs across data systems to ensure strict compliance and system security.

Skills

ETL/ELT pipelines

Apache Spark

Python/SQL/Scala

Cloud environments (AWS/Azure/MilCloud

STIGs compliance

Data modeling

Education

Bachelor's degree in Computer Science / Data Engineering / Information Systems

Master's degree

Tools

Databricks

Qlik Sense/QlikView

Jira / GitLab / Jenkins CI/CD

Advana DoD Data Environment

Job description

Redhorse Corporation seeks a proactive Mid-Level Data Engineer to join our team supporting the Office of the Under Secretary of Defense for Acquisition & Sustainment. You will design and optimize data pipelines and analytics solutions across enterprise DoD data environments, enabling data-driven decision making.

You will work with Databricks, Qlik, and cloud platforms to deliver scalable ETL/ELT workflows, maintain pipeline health, and ensure STIG compliance.