Databricks Platform Engineer: Data Pipelines & AI

Luxoft

New York, Northern (NY, KY)

Hybrid

USD 140,000 - 200,000

Full time

42 hours ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Luxoft is seeking a senior platform/data engineer contractor in New York to design and implement scalable data capabilities across Databricks. You will craft reusable data products and self-service tooling to accelerate researchers from prototype to production.

The role emphasizes Python/SQL, CI/CD, Terraform, data governance, and orchestration with Airflow, plus an eye for AI-assisted design and monitoring.

Qualifications

8+ years of platform and data solutions experience.
Hands-on Databricks experience (workspace, notebooks, jobs) and building platform capabilities.
Python.
SQL.
Modern CI/CD, IaC (Terraform strongly preferred).
Data Governance and observability (Unity Catalog, lineage, monitoring).
Orchestration (Airflow).
AI fluency: use of AI agents (Claude Code preferred) as a primary tool to design, build, test, and support solutions.
Nice-to-have: financial domain experience, API development, PowerBI.

Responsibilities

Design, build, and enhance platform capabilities within Databricks and related technologies.
Build reusable data products and self-service capabilities.
Design and maintain data pipelines and integration frameworks.
Improve data quality, reliability, lineage, and monitoring.
Accelerate research-to-production delivery: prototype solutions alongside researchers; establish patterns that make productionization easier and faster; reduce friction between research and engineering workflows.