Senior Databricks Platform Engineer

TechDigital Group; Irving (TX); On-site; USD 120,000 - 150,000; Full time

Job summary

A leading tech company is seeking a Databricks Developer to lead the installation, configuration, and support of Databricks on GCP. The ideal candidate will have over 8 years in data engineering with a strong proficiency in PySpark, SQL, and Delta Lake. Responsibilities include developing scalable ETL pipelines, monitoring platform health, troubleshooting issues, and maintaining technical documentation. This role is essential for optimizing data workflows and ensuring optimal performance.

Qualifications

• 8+ years of experience in data engineering, with at least 3 years on Databricks.
• Strong proficiency in PySpark, SQL, and Delta Lake.
• Hands-on experience with GCP Dataproc.

Responsibilities

• Lead the installation and configuration of Databricks on GCP cloud platforms.
• Monitor platform health, performance, and cost optimisation.
• Implement governance, logging, and auditing mechanisms.
• Design and develop scalable ETL/ELT pipelines using PySpark, SQL, and Delta Lake.
• Collaborate with data engineers and analysts to enhance data workflows and models.
• Optimize existing notebooks and jobs for performance and reliability.
• Provide L2/L3 support for Databricks-related issues and incidents.
• Troubleshoot cluster failures, job errors, and performance bottlenecks.
• Maintain technical documentation for platform setup, operations, and development standards.

Skills

Databricks Developer

DataBric Admin

PySpark

SQL

Delta Lake

GCP Dataproc

Job description

A leading tech company is seeking a Databricks Developer to lead the installation, configuration, and support of Databricks on GCP. The ideal candidate will have over 8 years in data engineering with a strong proficiency in PySpark, SQL, and Delta Lake. Responsibilities include developing scalable ETL pipelines, monitoring platform health, troubleshooting issues, and maintaining technical documentation. This role is essential for optimizing data workflows and ensuring optimal performance.