Data Platform Engineer (Databricks)

Luxoft; New York (NY); On-site; USD 150,000 - 230,000; Full time

Job summary

Luxoft is seeking a seasoned professional at the intersection of platform engineering, data engineering, and data product development. You will build reusable platform capabilities and data products that let quantitative researchers discover, access, and consume data across a complex landscape, and accelerate prototypes to production.

You will design and maintain data pipelines, improve data quality, lineage, and monitoring, and champion production-ready patterns to reduce friction between

Qualifications

• 8+ years of platform and data solutions experience.
• Hands-on Databricks experience (workspace, notebooks, jobs) and building platform capabilities.
• Proficiency in Python and SQL.
• Experience with modern CI/CD and IaC (Terraform strongly preferred).
• Data governance and observability (Unity Catalog, lineage, monitoring).
• Orchestration with Airflow.
• AI fluency: use of AI agents as a primary tool to design, build, test, and support solutions.

Responsibilities

• Design, build, and enhance platform capabilities within Databricks and related technologies.
• Build reusable data products and self-service capabilities.
• Design and maintain data pipelines and integration frameworks.
• Improve data quality, reliability, lineage, and monitoring.
• Accelerate research-to-production delivery: prototype solutions alongside researchers; establish patterns that make productionization easier and faster; reduce friction between research and engineering workflows.

Skills

Databricks

Python

SQL

CI/CD

Terraform

Data governance

Airflow

AI agents

Tools

Terraform

Job description

We are seeking a seasoned professional whose role sits at the intersection of platform engineering, data engineering, and data product development. You will build reusable platform capabilities and data products that let quantitative researchers discover, access, and consume data across a complex, disparate landscape, and then accelerate the good prototypes to production.

Responsibilities:

• Design, build, and enhance platform capabilities within Databricks and related technologies.
• Build reusable data products and self-service capabilities.
• Design and maintain data pipelines and integration frameworks.
• Improve data quality, reliability, lineage, and monitoring.
• Accelerate research-to-production delivery: prototype solutions alongside researchers; establish patterns that make productionization easier and faster; reduce friction between research and engineering workflows.

Mandatory Skills Description:

• 8+ years of platform and data solutions experience
• Hands-on Databricks experience (workspace, notebooks, jobs) and building platform capabilities
• Python
• SQL
• Modern CI/CD, IaC (Terraform strongly preferred)
• Data Governance and observability (Unity Catalog, lineage, monitoring)
• Orchestration (Airflow)
• AI fluency: use of AI agents (Claude Code preferred) as a primary tool to design, build, test, and support solutions

Nice-to-Have Skills Description:

• Financial domain experience, especially quant research