Databricks Data Platform Architect

UsefulBI Corporation

Raleigh (NC)

In loco

USD 130.000 - 180.000

Tempo pieno

14 giorni+

Aumenta le possibilità di ricevere una risposta da questo datore di lavoro

Il tuo curriculum e una lettera di presentazione personalizzati, che corrispondono esattamente a ciò che sta cercando.

Supera i filtri ATS

Descrizione del lavoro

A leading technology firm is seeking a Databricks Data Platform Architect to design and manage their enterprise lakehouse platform on AWS. This role emphasizes DevOps and Infrastructure Engineering to ensure the reliability and scalability of the Databricks environment. Responsibilities include implementing CI/CD processes, optimizing performance, and establishing data engineering standards. Ideal candidates will have extensive experience with Databricks and strong skills in Apache Spark, Python, SQL, and AWS services.

Competenze

  • 8+ years in Data Engineering/Platform Engineering/Cloud Infrastructure.
  • 4+ years hands-on with Databricks in production.
  • Strong expertise in Apache Spark, Python, and SQL.
  • Deep knowledge of AWS services including S3 and IAM.

Mansioni

  • Architect and manage Databricks on AWS end-to-end.
  • Implement CI/CD for notebooks, jobs, and pipelines.
  • Design lakehouse architecture (bronze/silver/gold).
  • Drive cost optimization and compute efficiency strategies.

Conoscenze

Apache Spark
Python
SQL
AWS
Terraform
CI/CD
Performance tuning

Strumenti

Databricks
CloudWatch
Datadog

Descrizione del lavoro

Databricks Data Platform Architect (DevOps & Infrastructure Focus)

Location: Alameda, CA – Onsite (5 days/week)

About the Role

We are looking for a hands‑on Databricks Data Platform Architect to design, build, and operate our enterprise lakehouse platform on AWS.

This role combines DevOps, Infrastructure Engineering, and Data Engineering architecture.

You will own the reliability, scalability, security, and performance of our Databricks environment and enable data teams to build high-quality, production‑grade pipelines.

If you enjoy building platforms from the ground up, automating everything, and solving performance problems at scale, this role is for you.

What You’ll Own
Platform & Infrastructure Architecture (Core Focus)
  • Architect and manage Databricks on AWS end-to-end
  • Implement CI/CD for notebooks, jobs, and pipelines
  • Define cluster policies, autoscaling, and environment isolation (dev/stage/prod)
  • Set up monitoring, logging, and alerting (CloudWatch/Datadog)
  • Drive cost optimization and compute efficiency strategies
  • Ensure high availability, reliability, and production uptime
  • Design lakehouse architecture (bronze/silver/gold)
  • Optimize Spark/Delta workloads for performance and scalability
  • Establish data engineering standards and best practices
  • Support batch and streaming workloads
  • Troubleshoot performance bottlenecks and production incidents
Security, Governance & Compliance
  • Implement role-based access control using Unity Catalog
  • Secure S3, IAM, and external locations
  • Enforce data access policies and PII protection
  • Enable audit logging and compliance guardrails
  • Partner with security and governance teams to ensure best practices
Required Qualifications
  • 8+ years in Data Engineering / Platform Engineering / Cloud Infrastructure
  • 4+ years hands‑on with Databricks in production
  • Strong expertise in Apache Spark, Python, and SQL
  • Deep knowledge of AWS (S3, IAM, VPC, CloudWatch, networking)
  • Experience with Terraform or other IaC tools
  • CI/CD implementation experience for data platforms
  • Strong troubleshooting and performance tuning skills
Preferred
  • Experience with Delta Lake optimization (ZORDER, OPTIMIZE, partitioning)
  • Workflow orchestration tools (Airflow/Databricks Workflows)
  • Experience with enterprise data governance tools (Unity Catalog/Atlan/Collibra)
  • Knowledge of GDPR/HIPAA or regulated environments
  • Prior experience building platforms for multiple teams
What Success Looks Like
  • Clear environment separation and automation
  • Self‑service data platform for engineers and analysts
  • Strong security and governance with minimal friction
Ottieni la revisione del curriculum gratis e riservata.

o trascina qui il file.

Similar jobs

Offerte di lavoro simili che vale la pena confrontare

Databricks Architect (Lead Data Platform Architect)
Databricks Architect (Lead Data Platform Architect)

iLink Digital • New York (NY)

In loco
USD 180.000 - 240.000
AWS DevOps Lead/Architect
AWS DevOps Lead/Architect

UsefulBI • Alameda (CA)

Ibrido
USD 140.000 - 180.000
Databricks Data Architect
Databricks Data Architect

NewVision • San Francisco (CA)

Ibrido
USD 140.000 - 190.000
Hybrid work model
Databricks Data Architect
Databricks Data Architect

NewVision Software • Stati Uniti

Remoto
USD 120.000 - 180.000
Senior Databricks Engineer
Senior Databricks Engineer

Remote • San Francisco (CA)

In loco
USD 150.000 - 190.000
Sr. Staff Platform/Data Reliability Engineer, Databricks (R5537)
Sr. Staff Platform/Data Reliability Engineer, Databricks (R5537)

Remote • San Francisco (CA)

In loco
USD 200.000 - 270.000
Databricks Architect
Databricks Architect

Rapid Eagle Inc • Atlanta (GA)

In loco
USD 150.000 - 210.000
401(k) matching
Dental insurance
Health insurance
Data Team Lead
Data Team Lead

Incedo Inc. • San Rafael (CA)

Ibrido
USD 130.000 - 150.000
Medical insurance
Vision insurance
401(k)
Databricks Platform Engineer
Databricks Platform Engineer

EXL Talent Acquisition Team • New York (NY)

In loco
USD 180.000 - 240.000
Databricks Architect
Databricks Architect

Applexus Technologies (P) Ltd • Federal Way (WA)

In loco
USD 150.000 - 200.000