Remote Senior Data Engineer: Lakehouse & AI Data Platforms

VivSoft Technologies LLC; Oregon; Hybrid; USD 160,000 - 180,000; Full time

Benefits offered by this job

Medical, Dental, Vision (employer-paid

Life Insurance

Paid Time Off and Holidays

401K with employer match

Professional Development Reimbursement

Job summary

VivSoft Technologies LLC is hiring a Senior Data Engineer to design, build, and operate a secure data platform on AWS. You will lead the lakehouse architecture using Apache Iceberg and Spark, enabling advanced analytics, AI/ML initiatives, and operational reporting across DoD-focused programs.

The role requires deep expertise in distributed data platforms, cloud infrastructure, and technical leadership, with mentoring responsibilities for engineers and collaboration with data scientists and

Qualifications

• Active Secret Clearance required.
• 8+ years of data engineering or large-scale data platform experience.
• Expertise in Apache Spark including tuning and optimization.
• Strong Python or Scala and advanced SQL skills.
• Experience with Apache Iceberg or Delta Lake.
• Experience with AWS services: S3, Glue, EMR, Athena, SQS.
• Experience with Airflow for orchestration.
• Terraform and IaC experience; Kubernetes familiarity.
• Strong leadership and mentoring abilities.

Responsibilities

• Architect and manage cloud-based lakehouse using Iceberg, S3, and Glue Catalog.
• Develop Spark data pipelines on EMR and Kubernetes.
• Design event-driven ingestion with S3 events and SQS.
• Optimize queries across Athena, Trino, and Spark SQL.
• OrchestrateETL workflows with Apache Airflow.
• Manage infrastructure deployment with Terraform.
• Develop and publish data products and dashboards.
• Implement data quality, governance, lineage, and access controls.
• Create technical runbooks and architectural designs.
• Mentor junior engineers and collaborate with data scientists.

Skills

Active Secret Clearance

Apache Spark

Python

Scala

SQL

AWS

Airflow

Kubernetes

Data Governance

Tools

Apache Iceberg

Grafana

Job description

VivSoft Technologies LLC is hiring a Senior Data Engineer to design, build, and operate a secure data platform on AWS. You will lead the lakehouse architecture using Apache Iceberg and Spark, enabling advanced analytics, AI/ML initiatives, and operational reporting across DoD-focused programs.

The role requires deep expertise in distributed data platforms, cloud infrastructure, and technical leadership, with mentoring responsibilities for engineers and collaboration with data scientists and