Senior Data Engineer, Lakehouse & DataOps Leader

United States Digital Space LLC; United States; Remote; USD 120,000 - 178,000; Full time

Benefits offered by this job

Medical, dental & vision coverage

Incentive programs

Life insurance

401k contributions

Job summary

United States Digital Space LLC is seeking a Senior Data Engineer to design, build, and operate enterprise-scale data solutions on an AWS Lakehouse platform. The role focuses on batch and streaming ingestion, ELT/ETL, data quality, and trusted data products, collaborating with product owners and platform teams.

Responsibilities include developing reusable pipelines with Python and PySpark, orchestrating with Airflow, implementing DataOps CI/CD, and guiding engineering best practices.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or related field, or equivalent work experience.
• 8+ years of overall IT experience.
• 5+ years designing and developing enterprise-scale data engineering solutions.
• Strong experience building scalable data pipelines with Python and PySpark.
• Experience with AWS Glue, dbt, Apache Spark, or comparable technologies.
• Strong data warehousing concepts, dimensional modeling, and Lakehouse architectures.
• Experience with AWS, Azure, or GCP (AWS preferred).
• Strong SQL and familiarity with NoSQL databases is a plus.
• Experience with Spark/EMR/Hadoop-based platforms and Git-based workflows.
• Strong analytical, problem-solving, and cross-team communication skills.

Responsibilities

• Design and implement ELT/ETL solutions for batch and streaming ingestion.
• Develop reusable data processing frameworks and configuration-driven pipelines using Python and PySpark.
• Build and maintain scalable orchestration workflows (Airflow) for production data delivery.
• Implement data quality checks, validation frameworks, and monitoring.
• Apply DataOps practices: Git-based development, CI/CD/CT for data pipelines.
• Contribute to data lifecycle practices with platform and governance teams.
• Support cloud migration of legacy data flows into Lakehouse patterns.
• Collaborate with stakeholders to map technical designs to business requirements and consumption needs.
• Establish and document engineering standards; participate in Agile ceremonies.
• Provide technical leadership: mentor engineers, design/code reviews, reliability improvements.

Skills

Python

PySpark

ELT/ETL

Airflow

DataOps

Git

CI/CD

Agile

SQL

Education

Bachelor's degree in CS/INFO/SYS/Engineering

Tools

AWS Glue

dbt

Spark (Apache/Spark runtimes)

EMR

Airflow

Job description

United States Digital Space LLC is seeking a Senior Data Engineer to design, build, and operate enterprise-scale data solutions on an AWS Lakehouse platform. The role focuses on batch and streaming ingestion, ELT/ETL, data quality, and trusted data products, collaborating with product owners and platform teams.

Responsibilities include developing reusable pipelines with Python and PySpark, orchestrating with Airflow, implementing DataOps CI/CD, and guiding engineering best practices.