AWS DevOps Lead: Data Platform & Databricks

UsefulBI

Alameda (CA)

Hybrid

USD 140,000 - 180,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

UsefulBI is seeking an experienced AWS DevOps Lead/Architect to lead cloud infrastructure, DevOps automation, and platform engineering for enterprise data platforms. The role combines 70% DevOps/Platform Engineering and 30% Data Engineering support, with emphasis on AWS, CI/CD, Kubernetes, and Databricks.

The candidate will design scalable pipelines, manage IaC, automate Python workflows, and ensure security and performance across cloud environments.

Qualifications

  • 3 days per week onsite in Alameda, CA.
  • Hands-on AWS cloud services experience (S3, IAM, Glue, Lambda, EC2, Redshift).
  • Proficiency with Terraform for provisioning and automation.
  • Strong Python programming for data engineering and automation.
  • Extensive Databricks experience (Spark, Delta Lake, orchestration).
  • Solid data engineering fundamentals: ETL/ELT, data modeling.
  • CI/CD tooling experience (GitHub Actions, Jenkins, or equivalents).
  • SQL and relational database proficiency.

Responsibilities

  • Design, develop, and optimize scalable data pipelines with Databricks.
  • Build and manage cloud infrastructure using IaC with Terraform on AWS.
  • Develop automation scripts and data workflows in Python.
  • Maintain cloud-based data solutions using AWS services (S3, IAM, Glue, Lambda, EC2, Redshift).
  • Ensure high availability, scalability, and performance of data platforms.
  • Collaborate with analysts and engineers for end-to-end data lifecycle management.
  • Implement CI/CD pipelines for automated deployment of infra and data workflows.
  • Monitor, troubleshoot, and optimize production systems; enforce security best practices.

Skills

AWS
Terraform
Python
Databricks
Kubernetes
CI/CD
SQL
Spark
EKS

Tools

Terraform
GitHub Actions
Jenkins
GitLab CI/CD
Azure DevOps

Job description

UsefulBI is seeking an experienced AWS DevOps Lead/Architect to lead cloud infrastructure, DevOps automation, and platform engineering for enterprise data platforms. The role combines 70% DevOps/Platform Engineering and 30% Data Engineering support, with emphasis on AWS, CI/CD, Kubernetes, and Databricks.

The candidate will design scalable pipelines, manage IaC, automate Python workflows, and ensure security and performance across cloud environments.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AWS DevOps Lead/Architect
AWS DevOps Lead/Architect

UsefulBI • Alameda (CA)

Hybrid
USD 140,000 - 180,000
Lead DevSecOps Architect for Cloud & Databricks Platforms
Lead DevSecOps Architect for Cloud & Databricks Platforms

Analytica Inc. • Bethesda (MD)

On-site
USD 160,000 - 230,000
Bonuses
Health insurance
Training funds
+1
Lead Databricks Architect – Multi-Cloud Data Platform, CI/CD
Lead Databricks Architect – Multi-Cloud Data Platform, CI/CD

Artingdigital • United States

Remote
USD 140,000 - 200,000
Senior DataOps Engineer: Databricks & AWS Platform Lead
Senior DataOps Engineer: Databricks & AWS Platform Lead

Oportun • United States

Remote
USD 140,000 - 170,000
Databricks Data Platform Architect
Databricks Data Platform Architect

UsefulBI Corporation • Raleigh (NC)

On-site
USD 130,000 - 180,000
DevOps Engineer
DevOps Engineer

GQR • New York (NY)

On-site
USD 120,000 - 150,000
AWS Tech Lead with DataBricks
AWS Tech Lead with DataBricks

The Dignify Solutions, LLC • Raritan (NJ)

On-site
USD 110,000 - 140,000
AWS Databricks Platform Architect — Production & Governance
AWS Databricks Platform Architect — Production & Governance

Valiant Technologies LLC • Raritan (NJ)

On-site
USD 140,000 - 180,000
AWS DevOps Engineer — Databricks & Infra Automation
AWS DevOps Engineer — Databricks & Infra Automation

IQuest Solutions Corporation • Dallas (TX)

On-site
USD 120,000 - 180,000
Databricks Lead Architect - Multi-Cloud Data Platforms
Databricks Lead Architect - Multi-Cloud Data Platforms

Arting Digital Private Limited • United States

Remote
USD 160,000 - 220,000