Senior Data Engineer — Azure, Databricks & ML Pipelines

United States Digital Space LLC; United States; Remote; USD 140,000 - 170,000; Full time

Job summary

United States Digital Space LLC is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and ML-ready infrastructure on Azure and Databricks. This is a hands-on engineering role: you'll own the full data pipeline lifecycle — ingestion, transformation, orchestration, and deployment — while supporting machine learning workflows with clean, reliable data.

If you're comfortable owning infrastructure decisions and writing production-quality Python at scale, this role

Qualifications

• Experience designing and building scalable data pipelines on Azure and Databricks.
• Production-quality Python for data processing and automation.
• Knowledge of data governance, security, and access control best practices.
• Experience with CI/CD for data/ML workflows.
• Strong SQL skills and working with relational/distributed data systems.

Responsibilities

• Design, build, and maintain data pipelines using Databricks and Azure-native services.
• Develop and optimize ETL/ELT processes to support analytics and ML workloads.
• Build and maintain CI/CD pipelines for data engineering and ML deployment workflows.
• Write clean, production-quality Python for data processing and pipeline automation.
• Support ML teams with well-structured datasets and feature pipelines.
• Design and manage data architecture across Azure data services (Data Factory, Data Lake, Synapse).
• Monitor pipeline performance, troubleshoot data quality issues, and implement reliability improvements.
• Implement data governance, security, and access control best practices.
• Collaborate with data scientists, analysts, and software engineers to align data infra with business needs.
• Participate in code reviews and architectural discussions.

Skills

Python

SQL

CI/CD

Data orchestration

Data governance

Analytics

Problem solving

Independent work

Tools

Databricks

Azure Data Factory

Azure Data Lake

Azure Synapse

Terraform

Bicep

ARM templates

Kafka

Event Hubs

MLflow

Azure ML

Job description

United States Digital Space LLC is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and ML-ready infrastructure on Azure and Databricks. This is a hands-on engineering role: you'll own the full data pipeline lifecycle — ingestion, transformation, orchestration, and deployment — while supporting machine learning workflows with clean, reliable data.

If you're comfortable owning infrastructure decisions and writing production-quality Python at scale, this role