Databricks on AWS Platform Architect

Stefanini North America and APAC; Dallas (TX); On-site; USD 180,000 - 240,000; Full time

Job summary

Stefanini North America and APAC seeks a Databricks on AWS Platform Architect in Dallas to lead a secure, scalable Lakehouse on AWS. You will design data platforms with S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.

Responsibilities include defining networking (VPC, PrivateLink, IAM), establishing Dev/Test/UAT/Prod environments via Terraform, and implementing data governance with Unity Catalog.

Qualifications

• Databricks Platform Administrator or Databricks Certified Data Engineer certification is required.

Responsibilities

• Architect and manage a secure, scalable, and highly available Databricks Lakehouse Platform on AWS.
• Design data platforms leveraging AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.
• Define networking architecture including VPC, PrivateLink, IAM roles, security groups, and encryption standards.
• Establish multi-environment strategies (Dev, Test, UAT, Prod) with automated provisioning through Terraform and Infrastructure as Code (IaC).
• Implement enterprise-wide data governance, lineage, metadata management, and compliance controls using Unity Catalog.
• Design high-performance data ingestion and processing architectures supporting batch, streaming, and real-time analytics workloads.
• Enable AI and GenAI capabilities through Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.
• Define CI/CD, monitoring, logging, and observability frameworks using GitHub, Jenkins, CloudWatch, and Databricks Workflows.
• Optimize platform performance, reliability, scalability, and cloud costs through FinOps and workload optimization practices.
• Provide architectural leadership and best practices for Lakehouse modernization, advanced analytics, AI governance, and enterprise data platform adoption.

Education

Databricks Platform Administrator / Databricks Certified Data Engineer

AWS Certification (DevOps/Solution Architect)

Terraform

Tools

Databricks

AWS S3

Delta Lake

Unity Catalog

Terraform

GitHub

CI/CD

CloudWatch

Mosaic AI

Genie

Vector Search

MLflow

Kafka

Airflow

Kubernetes

Job description

Details

Job Description

Role- Databricks on AWS Platform Architect

Location- Dallas TX

Databricks on AWS Platform Architect – Platform Architecture (10 Bullet Summary)

• Architect and manage a secure, scalable, and highly available Databricks Lakehouse Platform on AWS.


• Design data platforms leveraging AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.


• Define networking architecture including VPC, PrivateLink, IAM roles, security groups, and encryption standards.


• Establish multi-environment strategies (Dev, Test, UAT, Prod) with automated provisioning through Terraform and Infrastructure as Code (IaC).


• Implement enterprise-wide data governance, lineage, metadata management, and compliance controls using Unity Catalog.


• Design high-performance data ingestion and processing architectures supporting batch, streaming, and real-time analytics workloads.


• Enable AI and GenAI capabilities through Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.


• Define CI/CD, monitoring, logging, and observability frameworks using GitHub, Jenkins, CloudWatch, and Databricks Workflows.


• Optimize platform performance, reliability, scalability, and cloud costs through FinOps and workload optimization practices.


• Provide architectural leadership and best practices for Lakehouse modernization, advanced analytics, AI governance, and enterprise data platform adoption.

Core Technologies: Databricks, AWS S3, Delta Lake, Unity Catalog, Spark/PySpark, Terraform, GitHub, CI/CD, CloudWatch, Mosaic AI, Genie, Vector Search, MLflow, Kafka, Airflow, Kubernetes.

Job Requirements

Details

Certification :

• DataBricks Platform Administrator /Databricks Certified Data Engineer/


• AWS Certification (DevOps/Solution Architect)


• Terraform