Azure Databricks Engineer - DWH & Datalake Automation

Programmers.io; Louisville (KY); On-site; USD 110,000 - 150,000; Full time

Job summary

Programmers.io is seeking an experienced Azure Databricks specialist in Louisville, KY. The role focuses on building data warehouses and data lakes in the Azure cloud using Spark, ADF, Terraform, and CI/CD, with strong Python and SQL scripting skills.

You will collaborate with DevOps and development teams to automate deployment of infrastructure, maintain CI/CD pipelines, and implement IaC practices with Terraform and GitHub.

Qualifications

• 5+ years of experience in the IT/Technology and having atleast 3+ years of experience in Azure Databricks
• Hands on experience in development of Azure cloud DWH & datalakes using Azure technologies like Spark (Scala or Python), ADF, Azure Databricks, Terraform & CI/CD.
• Strong programming skills in Pyspark, Python, writing complex queries in Postgresql, T-SQL
• 6+ years of experience in coding complex U-SQL, Spark (Scala or Python). T-SQL, postgresql
• Should have experience in Azure Storages like ADLS Gen2, Azure Postgresql
• Design and develop automation to support continuous delivery and continuous integration process when it comes to cloud infrastructure.

Skills

Azure Databricks

Spark

Python

PySpark

SQL

CI/CD

Terraform

PostgreSQL

T-SQL

Azure Storage

Tools

ADF (Azure Data Factory)

Terraform

CI/CD pipelines

GitHub Enterprise

Spark

Job description

Programmers.io is seeking an experienced Azure Databricks specialist in Louisville, KY. The role focuses on building data warehouses and data lakes in the Azure cloud using Spark, ADF, Terraform, and CI/CD, with strong Python and SQL scripting skills.

You will collaborate with DevOps and development teams to automate deployment of infrastructure, maintain CI/CD pipelines, and implement IaC practices with Terraform and GitHub.