Databricks Platform Architect - AWS Lakehouse & DevOps

UsefulBI Corporation; Raleigh (NC); On-site; USD 130,000 - 180,000; Full time

Job summary

A leading technology firm is seeking a Databricks Data Platform Architect to design and manage their enterprise lakehouse platform on AWS. This role emphasizes DevOps and Infrastructure Engineering to ensure the reliability and scalability of the Databricks environment. Responsibilities include implementing CI/CD processes, optimizing performance, and establishing data engineering standards. Ideal candidates will have extensive experience with Databricks and strong skills in Apache Spark, Python, SQL, and AWS services.

Qualifications

• 8+ years in Data Engineering/Platform Engineering/Cloud Infrastructure.
• 4+ years hands-on with Databricks in production.
• Strong expertise in Apache Spark, Python, and SQL.
• Deep knowledge of AWS services including S3 and IAM.

Responsibilities

• Architect and manage Databricks on AWS end-to-end.
• Implement CI/CD for notebooks, jobs, and pipelines.
• Design lakehouse architecture (bronze/silver/gold).
• Drive cost optimization and compute efficiency strategies.

Skills

Apache Spark

Python

SQL

AWS

Terraform

CI/CD

Performance tuning

Tools

Databricks

CloudWatch

Datadog

Job description

A leading technology firm is seeking a Databricks Data Platform Architect to design and manage their enterprise lakehouse platform on AWS. This role emphasizes DevOps and Infrastructure Engineering to ensure the reliability and scalability of the Databricks environment. Responsibilities include implementing CI/CD processes, optimizing performance, and establishing data engineering standards. Ideal candidates will have extensive experience with Databricks and strong skills in Apache Spark, Python, SQL, and AWS services.