Senior Data Engineer - Remote with Secret Clearance

VivSoft; Northern (KY); Remote; USD 160,000 - 180,000; Full time

Benefits offered by this job

Comprehensive Medical benefits

Life Insurance

Paid Time Off

401K Retirement Plan with employer 1:1

Professional Development Training

Job summary

VivSoft is seeking a Senior Data Engineer to lead design and operation of a scalable data platform for USAF programs. You will architect modern lakehouse on AWS using Iceberg and Spark, enabling AI/ML analytics and operational reporting.

Ideal candidates have 8+ years in data engineering, strong Python/Scala, and deep experience with AWS services, Airflow, Terraform, and governance. You will mentor engineers and collaborate with data science and cloud teams.

Qualifications

• Active Secret Clearance required.
• 8+ years in data engineering or large-scale data platforms.
• Strong Spark performance tuning and memory optimization.
• Proficient in Python or Scala and advanced SQL.
• Experience with Iceberg (or Delta Lake).
• Familiar with Athena, Trino, Spark SQL engines.
• AWS services experience (S3, Glue, EMR, SQS, etc.).
• Airflow for workflow orchestration and Terraform IaC.
• Kubernetes experience and data visualization (Grafana).
• Data governance, lineage, and access controls understanding.
• Strong leadership, mentoring, and stakeholder communication.

Responsibilities

• Architect cloud-based lakehouse on Iceberg, S3, and Glue Catalog.
• Develop and optimize Spark data pipelines on EMR and Kubernetes.
• Design event-driven ingestion using S3 events and SQS.
• Optimize queries across Athena, Trino, and Spark SQL.
• Orchestrate ETL with Airflow and manage IaC with Terraform.
• Develop, publish, monitor, and maintain data products and dashboards.
• Implement data quality, governance, lineage, and access controls.
• Create technical docs, architectural designs, and runbooks.

Skills

Secret Clearance

8+ years

Apache Spark

Python/Scala

SQL

Iceberg/Delta Lake

Athena/Trino/Spark SQL

AWS

Airflow

Terraform IaC

Kubernetes

Grafana dashboards

Data governance

Leadership mentoring

Tools

Terraform

Kubernetes

Trino

dbt

Grafana

Airflow

Job description

VivSoft is seeking a Senior Data Engineer to lead design and operation of a scalable data platform for USAF programs. You will architect modern lakehouse on AWS using Iceberg and Spark, enabling AI/ML analytics and operational reporting.

Ideal candidates have 8+ years in data engineering, strong Python/Scala, and deep experience with AWS services, Airflow, Terraform, and governance. You will mentor engineers and collaborate with data science and cloud teams.