Remote Senior Databricks Data Engineer: AI Pipelines

Far Coder; Northern (KY); Hybrid; USD 150,000 - 180,000; Full time

Job summary

Livefront is seeking a Senior Databricks Data Engineer to design and deploy production-grade data pipelines and Lakehouse architectures in a remote, US-based role.

You’ll own end-to-end data workflows, work with Lakeflow, Autoloader, Delta Lake, and Unity Catalog, and collaborate with data scientists, engineers, and stakeholders to deliver scalable data platforms that power AI-enabled digital products.

Qualifications

• 7-10 years of data engineering experience with at least 5 years in production Databricks environments, preferably in a consulting or client delivery context.
• Solid working knowledge of AWS and Azure cloud services relevant to Databricks deployments - storage, networking, IAM, and compute - with GCP familiarity a plus.
• Deep, production-grade Databricks expertise across Lakeflow, Autoloader, Structured Streaming, and Unity Catalog.

Responsibilities

• Design and build production data pipelines using Lakeflow Declarative Pipelines and Structured Streaming with end-to-end ownership.
• Architect and implement Lakehouse solutions on Databricks tailored to analytics and AI needs.
• Build and maintain Databricks transformation layers with data quality constraints and SLAs.
• Design data and AI foundations (Unity Catalog, Feature Store, MLflow) powering production ML and AI-enabled products.
• Collaborate with product and backend engineers to design data models and APIs for the platform.
• Consult with clients to understand data challenges and develop sustainable data strategies.

Skills

Databricks

SQL

Python

CI/CD

Kafka

Java

Rust

AWS

Tools

Lakeflow Declarative Pipelines

Autoloader

Structured Streaming

Unity Catalog

dbt

Delta Lake

Job description

Livefront is seeking a Senior Databricks Data Engineer to design and deploy production-grade data pipelines and Lakehouse architectures in a remote, US-based role.

You’ll own end-to-end data workflows, work with Lakeflow, Autoloader, Delta Lake, and Unity Catalog, and collaborate with data scientists, engineers, and stakeholders to deliver scalable data platforms that power AI-enabled digital products.