Senior Azure Databricks Architect | Data Platform Leader

Pyramid Consulting, Inc; Miami (FL); On-site; USD 150,000 - 160,000; Full time

Job summary

Pyramid Consulting, Inc. is seeking an Azure Databricks Architect for a fulltime onsite role in Miami, FL. The candidate will own end-to-end data platform architecture, emphasize performance, governance, and cost efficiency, and collaborate with analysts and engineering teams.

Requires 12+ years in data engineering/architecture, hands-on Azure Databricks, Spark, Python/Scala, SQL, and Delta Lake, plus Azure DevOps/CI/CD experience. Certifications and financial services knowledge are a plus.

Qualifications

• 12+ years of experience in data engineering or architecture.
• Hands-on with Azure Databricks, Spark, Python/Scala, SQL, and Delta Lake.
• Design cloud data hubs, data lakes, and analytics stores.
• Experience with Azure DevOps, CI/CD, and automated deployments.
• Ability to implement solutions rather than only oversee.
• Strong communication and stakeholder collaboration.
• Experience in financial services or insurance.
• Knowledge of Event Hub, Cosmos DB, Azure Synapse, or similar services.
• Databricks or Microsoft Azure certifications.
• Experience with reusable frameworks, shell scripting, and production support.

Responsibilities

• Own the end-to-end architecture for Azure Databricks-based data platforms, including ingestion, transformation, storage, serving, and governance layers.
• Design reusable frameworks, ELT patterns, and data models that improve delivery speed, consistency, and maintainability.
• Build and optimize pipelines and Spark jobs directly, with a strong focus on performance, reliability, and cost efficiency.
• Architect solutions using Azure Databricks, Delta Lake, ADLS, Azure Data Factory, and related Azure platform services.
• Establish governance, security and code-quality standards for the platform.
• Partner closely with business analysts, functional teams, and engineering teams to translate requirements into working solutions.
• Guide troubleshooting, code reviews, and production support for complex data workloads.
• Drive modernization and cloud-native adoption across data and analytics capabilities

Skills

Azure Databricks

Spark

Python/Scala

SQL

Delta Lake

Azure DevOps

CI/CD

Communication

Job description

Pyramid Consulting, Inc. is seeking an Azure Databricks Architect for a fulltime onsite role in Miami, FL. The candidate will own end-to-end data platform architecture, emphasize performance, governance, and cost efficiency, and collaborate with analysts and engineering teams.

Requires 12+ years in data engineering/architecture, hands-on Azure Databricks, Spark, Python/Scala, SQL, and Delta Lake, plus Azure DevOps/CI/CD experience. Certifications and financial services knowledge are a plus.