Senior Data Engineer: Databricks, Delta Lake & Multi-Cloud

SugarAI; Denver (CO); Hybrid; USD 155,000 - 185,000; Full time

Benefits offered by this job

Excellent healthcare

401(k) match

Online legal services

Financial planning services

Pet insurance

Travel and entertainment discounts

Health and wellness reimbursement

Career development resources

Employee referral bonus program

Merit-based growth opportunities

Job summary

SugarAI, a Denver-based AI CRM pioneer, seeks a Senior Data Engineer to own Databricks pipelines for Sugar Predict, driving reliability, cost efficiency, and platform growth across Azure and AWS. You will partner with ML engineers, product, and enterprise architecture to ensure fast, clean data at global scale.

The role requires 4+ years in data engineering, 2+ years on Databricks, and a strong background in Spark, Delta Lake, and multi-tenant data architectures.

Qualifications

• 4+ years of data engineering experience.
• At least 2 years on Databricks or Apache Spark across Azure and/or AWS.
• Proficiency in PySpark, SQL, and Python with production pipelines under SLA.
• Hands-on with Delta Lake including schema evolution and ACID transactions.
• Experience with PostgreSQL in production data pipelines.
• Experience with multi-tenant architectures and data privacy.
• Ability to collaborate across data science, product, and infrastructure teams.

Responsibilities

• Own Databricks production support for the Sugar Predict data platform.
• Monitor, alert, and respond to incidents across production data flows.
• Report SLA performance metrics for data pipelines.
• Identify and implement pipeline cost optimizations and throughput improvements.
• Migrate legacy ETL/ELT pipelines to Databricks and build automation.
• Support onboarding of new customers by provisioning tenant data pipelines.
• Design and build high-performance Databricks pipelines ingesting ERP/CRM data at scale.
• Own Delta Lake architecture including schema design and data quality.
• Enforce security practices across Databricks, including Unity Catalog and RBAC.
• Implement data quality monitoring and observability for ML inputs.
• Apply CI/CD for data pipelines with versioning and automated tests.

Skills

Databricks

PySpark

SQL

Python

Delta Lake

PostgreSQL

ETL tooling

Multi-tenant architectures

Data governance

Cloud (Azure/AWS)

Tools

Azure

AWS

Job description

SugarAI, a Denver-based AI CRM pioneer, seeks a Senior Data Engineer to own Databricks pipelines for Sugar Predict, driving reliability, cost efficiency, and platform growth across Azure and AWS. You will partner with ML engineers, product, and enterprise architecture to ensure fast, clean data at global scale.

The role requires 4+ years in data engineering, 2+ years on Databricks, and a strong background in Spark, Delta Lake, and multi-tenant data architectures.