DevOps Engineer

GQR; New York (NY); On-site; USD 120,000 - 150,000; Full time

Job summary

A leading tech firm is seeking a hands-on Azure Databricks Platform Engineer in New York City. The role involves designing and automating Azure Databricks infrastructure, managing clusters and enabling scalable data processing. Ideal candidates will have over 3 years of experience with Azure and Databricks, as well as skills in CI/CD and troubleshooting. The position offers opportunities for collaboration across engineering and data teams, focusing on building a reliable cloud data platform.

Qualifications

• 3+ years of experience in Azure and Databricks.
• Strong understanding of Azure cloud infrastructure.
• Experience with data processing using Apache Spark.

Responsibilities

• Design and automate Azure Databricks infrastructure.
• Manage clusters and workspaces for optimal performance.
• Enable scalable data processing in a collaborative environment.

Skills

Azure + Databricks

PySpark

Spark SQL

Structured Streaming

Infrastructure-as-Code (Bicep/ARM)

CI/CD with Azure DevOps

Troubleshooting across cloud, data, and networking

Job description

We’re hiring a hands-on Azure Databricks Platform Engineer to build and scale a cloud data platform within a DevOps environment.

You’ll design and automate Azure Databricks infrastructure, manage clusters and workspaces, and enable scalable data processing using Apache Spark. This role is highly collaborative, working across engineering, data, and security teams to deliver a reliable, production-grade platform.

Azure + Databricks (3+ years)

PySpark, Spark SQL, Structured Streaming

Infrastructure-as-Code (Bicep/ARM)

CI/CD with Azure DevOps

Strong troubleshooting across cloud, data, and networking

This is ideal for someone who enjoys automation, platform ownership, and solving complex data infrastructure challenges.