Azure Databricks Engineer: Data Pipelines & Governance

Quantum Technologies LLC

Cincinnati (OH)

On-site

USD 81,000 - 115,000

Full time

26 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Quantum Technologies. LLC is seeking a data engineer to design and implement scalable data pipelines on Azure Databricks. You will manage Unity Catalog governance, write Python/PySpark processing, and automate CI/CD with GitHub Actions in a modern cloud data stack.

The role emphasizes data architecture, lineage, and security, with hands-on DevOps practices for deployment and collaboration on GitHub. 10-month contract with potential extension in the Cincinnati area.

Qualifications

  • Strong hands-on experience with Azure Databricks.
  • Experience managing data governance and Unity Catalog access control.
  • Strong proficiency in Python and PySpark for distributed data processing.
  • Experience designing, developing, and optimizing scalable data pipelines.
  • Good understanding of data architecture, data lineage, and metadata management.
  • Experience with GitHub, including branching strategies, pull requests, and code reviews.
  • Hands-on experience with GitHub Actions and CI/CD pipeline implementation.
  • Familiarity with DevOps practices for data engineering deployments.
  • Understanding of data security and governance best practices.
  • Experience working with modern cloud-based data platforms.
  • Familiarity with AI-assisted development tools such as GitHub Copilot and Databricks Assistant.

Responsibilities

  • Work with Azure Databricks to build data pipelines.
  • Manage data governance and Unity Catalog access.
  • Develop Python and PySpark data processing solutions.
  • Design, develop, and optimize scalable data pipelines.
  • Ensure data architecture, lineage, and metadata are well managed.
  • Collaborate via GitHub with PRs and reviews.
  • Implement CI/CD pipelines with GitHub Actions.
  • Apply DevOps practices to data deployments.
  • Follow data security and governance best practices.
  • Work with cloud-based data platforms and AI-assisted tools.

Skills

Azure Databricks
Python
PySpark
Data pipelines
Data governance
Unity Catalog
GitHub
CI/CD
GitHub Actions
DevOps
Data security
Cloud platforms

Tools

Azure
Databricks
GitHub
GitHub Actions
CI/CD tooling

Job description

Quantum Technologies. LLC is seeking a data engineer to design and implement scalable data pipelines on Azure Databricks. You will manage Unity Catalog governance, write Python/PySpark processing, and automate CI/CD with GitHub Actions in a modern cloud data stack.

The role emphasizes data architecture, lineage, and security, with hands-on DevOps practices for deployment and collaboration on GitHub. 10-month contract with potential extension in the Cincinnati area.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Databricks Engineer: Governance, PySpark & Pipelines
Azure Databricks Engineer: Governance, PySpark & Pipelines

Quantum Technologies LLC • Cincinnati (OH)

On-site
USD 110,000 - 165,000
Azure Databricks Engineer: Data Pipelines & Governance
Azure Databricks Engineer: Data Pipelines & Governance

Valiant Technologies LLC • Cincinnati (OH)

On-site
USD 90,000 - 110,000
Azure Databricks Engineer: Data Pipelines & Governance
Azure Databricks Engineer: Data Pipelines & Governance

MACHINE LEARNING TECHNOLOGIES LLC • Cincinnati (OH)

On-site
USD 78,000 - 112,000
Azure Databricks Engineer - Data Pipelines & Governance
Azure Databricks Engineer - Data Pipelines & Governance

Ethereum Technologies LLC • Cincinnati (OH)

On-site
USD 81,000 - 115,000
Lead Azure Databricks Engineer – Data Pipelines & Governance
Lead Azure Databricks Engineer – Data Pipelines & Governance

Robotics Technologies • Cincinnati (OH)

On-site
USD 91,000 - 124,000
Azure Databricks Engineer: Pipelines, Governance & CI/CD
Azure Databricks Engineer: Pipelines, Governance & CI/CD

Robotic Process Automation LLC • Cincinnati (OH)

On-site
USD 94,000 - 103,000
Azure Data Engineer — Pipelines, Databricks & Power BI
Azure Data Engineer — Pipelines, Databricks & Power BI

Quantum Technologies. LLC • Columbia Township (MO)

On-site
USD 90,000 - 130,000
Azure Databricks Engineer: Data Pipelines and Unity Catalog
Azure Databricks Engineer: Data Pipelines and Unity Catalog

Ethereum Technologies LLC • Cincinnati (OH)

On-site
USD 120,000 - 160,000
Azure Databricks Engineer - Data Pipelines & Unity Catalog
Azure Databricks Engineer - Data Pipelines & Unity Catalog

Robotic Process Automation LLC • Cincinnati (OH)

On-site
USD 120,000 - 180,000
Azure Data Engineer — Lead Data Pipelines & Databricks
Azure Data Engineer — Lead Data Pipelines & Databricks

Quantum Technologies LLC • Pennsylvania

On-site
USD 91,000 - 124,000