Data Platform Engineer Databricks

Luxoft; Irvine (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Luxoft in Irvine, CA seeks a senior data platform engineer to design, build, and scale data capabilities within Databricks and related technologies. You will create reusable data products and self-service data tooling, and design robust pipelines with governance and observability.

Collaborate with researchers to prototype solutions and establish production-ready patterns, accelerating research-to-production delivery while reducing friction between research and engineering workflows.

Qualifications

• 8+ years of platform and data solutions experience.
• Hands-on Databricks experience (workspace, notebooks, jobs) and building platform capabilities.
• Proficient in Python.
• Proficient in SQL.
• Experience with Modern CI/CD and IaC (Terraform strongly preferred).
• Data Governance and observability (Unity Catalog, lineage, monitoring).
• Orchestration with Airflow.
• AI fluency: use of AI agents as a primary tool to design, build, test, and support solutions.

Responsibilities

• Design, build, and enhance platform capabilities within Databricks and related technologies.
• Build reusable data products and self-service capabilities.
• Design and maintain data pipelines and integration frameworks.
• Improve data quality, reliability, lineage, and monitoring.
• Accelerate research-to-production delivery: prototype solutions; establish patterns that ease productionization; reduce friction between research and engineering workflows.

Skills

Databricks

Python

SQL

Terraform IaC

CI/CD

Data governance

Airflow

AI fluency

Tools

Databricks Platform

Job description

Responsibilities

• Design, build, and enhance platform capabilities within Databricks and related technologies.
• Build reusable data products and self-service capabilities.
• Design and maintain data pipelines and integration frameworks.
• Improve data quality, reliability, lineage, and monitoring.
• Accelerate research-to-production delivery: prototype solutions alongside researchers; establish patterns that make productionization easier and faster; reduce friction between research and engineering workflows.

Mandatory Skills Description

• 8+ years of platform and data solutions experience
• Hands-on Databricks experience (workspace, notebooks, jobs) and building platform capabilities
• Python
• SQL
• Modern CI/CD, IaC (Terraform strongly preferred)
• Data Governance and observability (Unity Catalog, lineage, monitoring)
• Orchestration (Airflow)
• AI fluency: use of AI agents (Claude Code preferred) as a primary tool to design, build, test, and support solutions

Nice-to-Have Skills Description

• Financial domain experience, especially quant research