Remote Databricks Platform Engineer (Cloud Data Infra)

Zachary Piper, LLC; Zachary (LA); Remote; USD 115,000 - 130,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

401(k)

PTO

Paid Holidays

Job summary

Piper Companies is seeking a Databricks Infrastructure Engineer to design, build, and manage enterprise cloud data infrastructure supporting forecasting, analytics, and ML initiatives.

The role focuses on administering Databricks environments, implementing automation, securing platforms, and optimizing data pipelines for reliability and cost efficiency across Azure/AWS/GCP environments.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field or equivalent experience.
• 3+ years in cloud data platform engineering, infrastructure administration, or Databricks platform management.
• Hands-on Databricks administration: clusters, jobs, workflows, SQL warehouses, Unity Catalog, workspace configs.
• Experience with Azure, AWS, or Google Cloud; IaC using Terraform; Git-based deployment workflows.
• Proficiency in Python, SQL, scripting; strong data engineering knowledge and security controls.

Responsibilities

• Design, configure, and maintain Databricks platform infrastructure for analytics and forecasting workloads.
• Manage Databricks workspaces, clusters, SQL warehouses, jobs, and Unity Catalog access controls.
• Develop IaC solutions with Terraform and cloud automation; implement CI/CD for notebooks and pipelines.
• Support data ingestion, transformation, feature engineering, and ML workloads in the cloud.
• Monitor performance, optimize costs, and enforce security, IAM, and data governance.

Skills

Python

SQL

Scripting

Terraform

CI/CD

Cloud computing

Data governance

IAM

Education

Bachelor's degree in CS/IT/Engineering

Tools

Terraform

GitHub Actions

Azure DevOps

Databricks

CI/CD pipelines

Job description

Piper Companies is seeking a Databricks Infrastructure Engineer to design, build, and manage enterprise cloud data infrastructure supporting forecasting, analytics, and ML initiatives.

The role focuses on administering Databricks environments, implementing automation, securing platforms, and optimizing data pipelines for reliability and cost efficiency across Azure/AWS/GCP environments.