Azure Databricks Engineer: Data Pipelines & Governance

Valiant Technologies LLC

Cincinnati (OH)

On-site

USD 90,000 - 110,000

Full time

26 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Valiant Technologies LLC seeks a data engineer to design, build, and optimize scalable data pipelines on Azure Databricks. You will manage data governance with Unity Catalog, write Python and PySpark code for distributed processing, and implement GitHub-driven CI/CD pipelines. Expect collaboration with cloud data platforms and DevOps practices.

The role emphasizes data security, metadata management, and modern tooling to enable AI-assisted development.

Qualifications

  • Hands-on experience with Azure Databricks and Unity Catalog.
  • Strong Python and PySpark for distributed data processing.
  • Design, develop, and optimize scalable data pipelines.
  • Knowledge of data architecture, lineage, and metadata management.
  • GitHub workflows, PRs, and code reviews; CI/CD and DevOps for data deployments.
  • Understanding of data security and governance best practices.

Responsibilities

  • Design, develop, and optimize scalable data pipelines on cloud platforms.
  • Manage data governance and access control using Unity Catalog.
  • Develop with Python and PySpark for distributed data processing.
  • Collaborate on GitHub-based workflows, PRs, and code reviews.
  • Implement CI/CD pipelines and DevOps practices for data deployments.
  • Maintain data architecture, lineage, and metadata documentation.
  • Apply data security and governance best practices across projects.
  • Work with modern cloud data platforms and AI-assisted tooling.

Skills

Azure Databricks
PySpark
Python
Data pipelines
Data governance
GitHub
CI/CD
DevOps for data eng
Data security
Cloud data platforms

Tools

Databricks
Unity Catalog
GitHub Actions
Azure

Job description

Valiant Technologies LLC seeks a data engineer to design, build, and optimize scalable data pipelines on Azure Databricks. You will manage data governance with Unity Catalog, write Python and PySpark code for distributed processing, and implement GitHub-driven CI/CD pipelines. Expect collaboration with cloud data platforms and DevOps practices.

The role emphasizes data security, metadata management, and modern tooling to enable AI-assisted development.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Databricks Engineer - Governance, PySpark & CI/CD Expert
Azure Databricks Engineer - Governance, PySpark & CI/CD Expert

Valiant Technologies LLC • Cincinnati (OH)

On-site
USD 120,000 - 155,000
Azure Databricks Engineer: Data Pipelines & Governance
Azure Databricks Engineer: Data Pipelines & Governance

Quantum Technologies LLC • Cincinnati (OH)

On-site
USD 81,000 - 115,000
Azure Databricks Engineer - Data Pipelines & Governance
Azure Databricks Engineer - Data Pipelines & Governance

Ethereum Technologies LLC • Cincinnati (OH)

On-site
USD 81,000 - 115,000
Senior Azure Data Engineer: Pipelines, Databricks & SQL
Senior Azure Data Engineer: Pipelines, Databricks & SQL

Valiant Technologies LLC • Houston (TX)

On-site
USD 120,000 - 170,000
Azure Databricks Engineer: Data Pipelines & Governance
Azure Databricks Engineer: Data Pipelines & Governance

MACHINE LEARNING TECHNOLOGIES LLC • Cincinnati (OH)

On-site
USD 78,000 - 112,000
Azure Data Engineer: Cloud Data Pipelines & Spark Pro
Azure Data Engineer: Cloud Data Pipelines & Spark Pro

Valiant Technologies LLC • Austin (TX)

On-site
USD 110,000 - 160,000
Azure Databricks Engineer - Scalable Data Pipelines & Lakehouse
Azure Databricks Engineer - Scalable Data Pipelines & Lakehouse

Valiant Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 170,000
Azure Databricks Engineer: Pipelines, Governance & CI/CD
Azure Databricks Engineer: Pipelines, Governance & CI/CD

Robotic Process Automation LLC • Cincinnati (OH)

On-site
USD 94,000 - 103,000
Azure Databricks Engineer - Data Pipelines & Unity Catalog
Azure Databricks Engineer - Data Pipelines & Unity Catalog

Robotic Process Automation LLC • Cincinnati (OH)

On-site
USD 120,000 - 180,000
Azure Databricks Engineer: Governance, PySpark & Pipelines
Azure Databricks Engineer: Governance, PySpark & Pipelines

Quantum Technologies LLC • Cincinnati (OH)

On-site
USD 110,000 - 165,000