Platform Architect with Databricks

Themesoft Inc; EE. UU.; A distancia; USD 170.000 - 230.000; Jornada completa

Descripción de la vacante

Themsoft Inc. is seeking a Platform Architect with Databricks to lead the Azure Databricks Platform's architecture, governance, and security.

You will design and implement scalable Databricks environments, enforce data governance, and integrate with ADF, ADLS Gen2, Key Vault, and Entra ID. Collaborate with Data Engineering, Security, Networking, and Operations teams to optimize performance, reliability, and cost.

Formación

• Hands-on with Unity Catalog, Delta Lake, Spark and Databricks components.
• Extensive experience with Azure data services (ADF, ADLS Gen2, Key Vault).
• Experience implementing governance, IAM, and data protection controls.
• Strong networking know-how in Azure (VNets, NSGs, private endpoints).
• Databricks performance tuning, cost optimization, capacity planning, troubleshooting.
• Proficient in CI/CD (Azure DevOps, GitHub Actions), REST APIs, scripting (PowerShell, Python, SQL).
• Security, compliance, and enterprise-scale platform management expertise.

Responsabilidades

• Lead architecture and governance of the Azure Databricks Platform.
• Design secure, scalable Databricks environments using Unity Catalog.
• Implement data governance and fine-grained access controls.
• Manage integrations with ADF, ADLS Gen2, Key Vault, and Entra ID.
• Drive platform performance, reliability, and cost management.
• Collaborate with Data Engineering, Security, Networking, and Operations.
• Define architecture standards and operational runbooks.

Conocimientos

Unity Catalog

Delta Lake

Spark

Databricks Clusters

Databricks Jobs

Databricks Notebooks

Databricks Repos

Azure Data Factory

ADLS Gen2

Key Vault

Entra ID/Azure AD

Azure Monitor

Log Analytics

SCIM

SSO

RBAC

ACLs

Data Governance

Azure Networking

VNets

NSGs

Private Endpoints

Firewall Rules

DNS

Routing

Performance tuning

Cost optimization

Capacity planning

Troubleshooting

CI/CD

Azure DevOps

GitHub Actions

REST APIs

PowerShell

Python

Scala

SQL

Azure CLI

Cloud security

Compliance

Enterprise-scale platform management

Herramientas

Azure DevOps

GitHub Actions

Descripción del empleo

Position: Platform Architect with Databricks

Location: United States – Remote

Job Description:

• Hands-on experience with Unity Catalog, Delta Lake, Spark, Databricks Clusters, Jobs, Notebooks, and Repos
• Extensive experience with Azure Data Factory (ADF), ADLS Gen2, Azure Key Vault, Entra ID (Azure AD), Azure Monitor, and Log Analytics
• Experience implementing SCIM, SSO, RBAC, ACLs, and Data Governance
• Strong knowledge of Azure Networking including VNets, NSGs, Private Endpoints, Firewall Rules, DNS, and Routing
• Expertise in Databricks performance tuning, cost optimization, capacity planning, and troubleshooting
• Experience with CI/CD, Azure DevOps, GitHub Actions, REST APIs, PowerShell, Python, Scala, SQL, and Azure CLI
• Strong understanding of cloud security, compliance, and enterprise-scale platform management

Key Responsibilities:

• Lead architecture, deployment, governance, and administration of Azure Databricks Platform.
• Design secure and scalable Databricks environments using Private Networking and Unity Catalog.
• Implement Data Governance, Fine-Grained Access Controls, and Security best practices.
• Manage integrations with ADF, ADLS Gen2, Key Vault, and Entra ID.
• Drive platform performance optimization, monitoring, reliability, and cost management.
• Collaborate with Data Engineering, Security, Networking, and Operations teams.
• Define architecture standards, operational runbooks, and platform best practices.

Work Authorization

Applicants must be legally authorized to work in the United States.

Themesoft Inc is an Equal Employment Opportunity employer. We consider qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other status protected by applicable federal, state, or local law.