Data Engineer - Cloud Pipelines & Streaming (Remote)

ProfilHQ; United States; Remote; USD 149,000 - 201,000; Full time

Benefits offered by this job

Medical/Dental/Vision

Company Stock Options

Home Office Budget

Flexible Time Off

Paid Holidays

Job summary

Defense Unicorns is hiring a data engineering architect to design and operate data workflows and streaming across Kubernetes, AWS, and on-prem environments. The role focuses on building scalable pipelines, secure data exchanges, and compliant architectures for government workloads.

You will work with platform teams to automate infrastructure, enforce security, and optimize performance, while ensuring DoD standards. Remote-USA with partial travel is expected.

Qualifications

• Demonstrated experience (minimum 5 years) in building and maintaining production data pipelines, databases, and data exchange services on Kubernetes.
• Experience in designing, deploying, and operating Kafka / Redpanda data streaming solutions on Kubernetes.
• Experience in creating data pipelines from scratch with the ability to process data in large volumes and manage data stores.
• Familiarity with containerization and orchestration tools like Docker and Kubernetes; knowledge of API security and AWS security.

Responsibilities

• Designing data workflows, pipelines, and pathways to support ETL/ELT across AWS cloud and on-prem environments.
• Implementing systems engineering principles to ensure reliability, scalability, and maintainability of data layer systems including backup/recovery strategies.
• Architecting, deploying, and operating data streaming technologies (Kafka, Redpanda, Kinesis) on Kubernetes with declarative definitions.
• Collaborating with platform teams to implement best practices for configuration, security, and performance in Kubernetes and AWS.
• Automating infrastructure provisioning, monitoring, and scaling using Terraform, and aligning with cybersecurity SMEs to defend data layers.
• Ensuring data layer systems comply with DoD regulations, standards, and best practices.

Skills

Data pipelines

Kubernetes

Kafka

Redpanda

Kinesis

Terraform

Security

DoD compliance

Automation

CI/CD

Tools

Terraform

Pulumi

Docker

Kubernetes

Job description

Defense Unicorns is hiring a data engineering architect to design and operate data workflows and streaming across Kubernetes, AWS, and on-prem environments. The role focuses on building scalable pipelines, secure data exchanges, and compliant architectures for government workloads.

You will work with platform teams to automate infrastructure, enforce security, and optimize performance, while ensuring DoD standards. Remote-USA with partial travel is expected.