Azure Databricks Platform Engineer - Security & Automation

TechDigital Group; Atlanta (GA); On-site; USD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking an Azure Databricks Administrator to enhance their data management capabilities. In this pivotal role, you will deploy and manage Azure Databricks workspaces, ensuring high availability and performance while optimizing resource utilization. Your expertise in Apache Spark and CI/CD automation will be crucial in driving performance improvements and cost efficiency. This innovative firm values collaboration, working closely with data engineers and scientists to support robust data pipelines and analytics workloads. If you're passionate about data governance and performance optimization, this opportunity is perfect for you.

Qualifications

• 5+ years of Azure Databricks administration and performance optimization experience.
• Expertise in Apache Spark, PySpark, SQL, and Scala is essential.

Responsibilities

• Deploy and manage Azure Databricks workspaces for optimal performance.
• Collaborate with teams to support data pipelines and analytics workloads.

Skills

Azure Databricks administration

Apache Spark

PySpark

SQL

Scala

Infrastructure as Code

CI/CD pipeline automation

Problem-solving skills

Communication skills

Tools

Terraform

Azure Monitor

Prometheus

Grafana

Azure DevOps

GitHub Actions

Jenkins

Databricks REST API

Job description

An established industry player is seeking an Azure Databricks Administrator to enhance their data management capabilities. In this pivotal role, you will deploy and manage Azure Databricks workspaces, ensuring high availability and performance while optimizing resource utilization. Your expertise in Apache Spark and CI/CD automation will be crucial in driving performance improvements and cost efficiency. This innovative firm values collaboration, working closely with data engineers and scientists to support robust data pipelines and analytics workloads. If you're passionate about data governance and performance optimization, this opportunity is perfect for you.