Senior Databricks Platform Architect - Remote

Caylent; México; A distancia; USD 140.000 - 210.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Medical Insurance for you and eligible

Generous holidays and flexible PTO

Paid for exams and certifications

Peer bonus awards

State of the art laptop and tools

Equipment & Office Stipend

Annual Learning & Development stipend

Travel up to 25%

Descripción de la vacante

Caylent is seeking a Senior Data Architect to design and harden enterprise-scale Databricks data platforms within client engagements. You will lead architecture design, build guardrails, automate processes, and ensure governance to enable safe, scalable data solutions across business teams.

The role focuses on Databricks-based platform development with AWS best-practice patterns, emphasizing real-time and batch pipelines, metadata integration, and production readiness. 25% travel may be required.

Formación

• End-to-end modern data platform architecture design (lakehouse, ingestion, processing, access layers).
• Hands-on Databricks experience: workspace, Unity Catalog, lakehouse patterns, governance tooling, optimize Spark configurations and Delta Lake storage.
• Experience architecting scalable cloud-native systems (AWS).
• Advanced distributed processing and streaming architectures.
• Platform security, networking, and automation; cost estimation and performance optimization.
• Leading multi-database migration and modernization initiatives, including onto Databricks.
• Deep expertise in OLTP/OLAP system design and schema evolution.
• Advanced performance tuning, HA/DR, and security patterns.
• Cloud-native database architecture across managed services.
• Migration strategy, cutover planning, and risk mitigation.

Responsabilidades

• Lead architectural design for Databricks-based data platforms or modernization programs, including workspace consolidation and governance patterns.
• Ensure designs align with AWS leading-practice architecture patterns.
• Design high-volume batch and real-time streaming pipelines using PySpark and Structured Streaming.
• Integrate Databricks Unity Catalog with external metadata repositories.
• Implement automation (ConfigOps), CI/CD, and guardrail patterns for safe platform extension.
• Build and maintain certified data products, semantic layer, and lineage exposure for self-service and agent-use cases.
• Ensure production readiness and scalability of platform components.
• Communicate architectural decisions clearly to technical and business stakeholders; create docs and roadmaps.
• Collaborate with client teams to translate use-cases into platform capabilities and guardrails.

Conocimientos

Databricks

AWS

PySpark

Structured Streaming

Data Platform Architecture

IaC (Terraform)

Governance

Team Leadership

Cloud Data Migration

Herramientas

Terraform

Databricks

Spark

Unity Catalog

Descripción del empleo

Caylent is seeking a Senior Data Architect to design and harden enterprise-scale Databricks data platforms within client engagements. You will lead architecture design, build guardrails, automate processes, and ensure governance to enable safe, scalable data solutions across business teams.

The role focuses on Databricks-based platform development with AWS best-practice patterns, emphasizing real-time and batch pipelines, metadata integration, and production readiness. 25% travel may be required.