Data Platform Engineer (Databricks)

Luxoft; New York, Northern (NY, KY); Hybrid; USD 140,000 - 200,000; Full time

Job summary

Luxoft is seeking a senior platform/data engineer contractor in New York to design and implement scalable data capabilities across Databricks. You will craft reusable data products and self-service tooling to accelerate researchers from prototype to production.

The role emphasizes Python/SQL, CI/CD, Terraform, data governance, and orchestration with Airflow, plus an eye for AI-assisted design and monitoring.

Qualifications

• 8+ years of platform and data solutions experience.
• Hands-on Databricks experience (workspace, notebooks, jobs) and building platform capabilities.
• Python.
• SQL.
• Modern CI/CD, IaC (Terraform strongly preferred).
• Data Governance and observability (Unity Catalog, lineage, monitoring).
• Orchestration (Airflow).
• AI fluency: use of AI agents (Claude Code preferred) as a primary tool to design, build, test, and support solutions.
• Nice-to-have: financial domain experience, API development, PowerBI.

Responsibilities

• Design, build, and enhance platform capabilities within Databricks and related technologies.
• Build reusable data products and self-service capabilities.
• Design and maintain data pipelines and integration frameworks.
• Improve data quality, reliability, lineage, and monitoring.
• Accelerate research-to-production delivery: prototype solutions alongside researchers; establish patterns that make productionization easier and faster; reduce friction between research and engineering workflows.

Skills

Platform engineering

Databricks

Python

SQL

CI/CD

IaC (Terraform)

Data governance

Observability

Airflow

AI tooling

Quant/Finance domain

API development

PowerBI

Tools

Terraform

Airflow

Job description

Project description

We are seeking a seasoned professional whose role sits at the intersection of platform engineering, data engineering, and data product development. You will build reusable platform capabilities and data products that let quantitative researchers discover, access, and consume data across a complex, disparate landscape, and then accelerate the good prototypes to production.This is a contractor position.

Responsibilities

• Design, build, and enhance platform capabilities within Databricks and related technologies.
• Build reusable data products and self-service capabilities.
• Design and maintain data pipelines and integration frameworks.
• Improve data quality, reliability, lineage, and monitoring.
• Accelerate research-to-production delivery: prototype solutions alongside researchers; establish patterns that make productionization easier and faster; reduce friction between research and engineering workflows.

SKILLS

Must have

• 8+ years of platform and data solutions experience
• Hands-on Databricks experience (workspace, notebooks, jobs) and building platform capabilities
• Python
• SQL
• Modern CI/CD, IaC (Terraform strongly preferred)
• Data Governance and observability (Unity Catalog, lineage, monitoring)
• Orchestration (Airflow)
• AI fluency: use of AI agents (Claude Code preferred) as a primary tool to design, build, test, and support solutions

Nice to have

• Financial domain experience, especially quant research
• API development
• PowerBI