Data & Software Engineer: Cloud, AI Pipelines, ETL

NS2 Mission; North Dakota; On-site; USD 140,000 - 190,000; Full time

Job summary

NS2 Mission is seeking a Data & Software Engineer to design, develop, and maintain scalable data pipelines and software solutions supporting mission operations in a cloud-first environment. You will collaborate with a small, highly skilled team to leverage Python, Spark, and AWS services to transform complex data into actionable intelligence and drive modernization across platforms.

Security clearance consideration is noted (TS/SCI with an FSP).

Qualifications

• 5+ years of experience building production data pipelines and scalable ETL solutions.
• Advanced Python development experience, including Pandas and NumPy.
• Experience developing with Apache Spark and PySpark.
• Experience deploying containerized applications using Docker or Podman.
• Experience working with AWS services including S3, Lambda, and Step Functions.
• Strong SQL experience, including Trino, MySQL, and PostgreSQL with performance tuning and schema design.
• Experience working with NoSQL technologies such as DynamoDB.
• Experience with workflow orchestration tools including Apache Airflow and Apache Iceberg.
• Familiarity with Infrastructure as Code using Terraform or CloudFormation.
• Experience using Git and modern source control practices.
• Experience with Bash scripting for automation and data processing.
• Understanding of data governance, security, privacy, lineage, and compliance principles.

Responsibilities

• Design, develop, and maintain scalable end-to-end data pipelines using Python and modern data engineering frameworks.
• Build and optimize ETL workflows that process structured, unstructured, and geospatial data at scale.
• Deploy and manage containerized applications within AWS cloud environments.
• Develop and optimize SQL and NoSQL data solutions, including database design, performance tuning, and query optimization.
• Integrate AI/ML models and services into production data workflows.
• Utilize orchestration tools to automate data pipelines and manage Spark jobs.
• Maintain data governance, lineage, cataloging, and compliance best practices across enterprise datasets.
• Collaborate with stakeholders to gather requirements, design technical solutions, and support platform modernization initiatives.
• Troubleshoot pipeline failures, performance bottlenecks, and data quality issues while contributing to engineering best practices and technical documentation.

Skills

Python

Pandas

NumPy

Apache Spark

PySpark

Docker

AWS

SQL

NoSQL

Airflow

Terraform

Git

Bash

Data governance

Security & privacy

Tools

Docker

Podman

AWS

S3

Lambda

Step Functions

Apache Airflow

Apache Iceberg

Terraform

CloudFormation

Git

Bash

Spark

PostgreSQL

MySQL

DynamoDB

PostGIS

Job description

NS2 Mission is seeking a Data & Software Engineer to design, develop, and maintain scalable data pipelines and software solutions supporting mission operations in a cloud-first environment. You will collaborate with a small, highly skilled team to leverage Python, Spark, and AWS services to transform complex data into actionable intelligence and drive modernization across platforms.

Security clearance consideration is noted (TS/SCI with an FSP).