Azure Data Analytics Architect — Lead Databricks & Pipelines

Quantum Technologies. LLC; Houston (TX); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Quantum Technologies LLC seeks an experienced data architect to lead analytics solutions on Azure Databricks and Fabric. You will design, implement, and optimize data pipelines, collaborating with stakeholders and IT teams to ensure scalable, secure data architecture.

The role emphasizes leadership, governance, and cutting-edge cloud data practices. Responsibilities include overseeing end-to-end data workflows, integrating with Azure services, and mentoring engineers while maintaining

Qualifications

• 8+ years in developing data ingestion and transformation pipelines using Databricks / Synapse notebooks and Azure Data Factory.
• Delta tables/Delta Lake and ADLS Gen2 experience.
• Experience with Auto Loader and Delta Live Tables for data ingestion/transformation.
• Databricks SQL for building/optimizing queries.
• Integration of Databricks with Synapse, ADLS Gen2, and Power BI for end-to-end analytics.

Responsibilities

• Lead the architecture, design, and implementation of advanced analytics solutions on Azure Databricks / Fabric.
• Work with business stakeholders and IT teams to gather requirements and deliver effective solutions.
• Oversee end-to-end data solution implementation aligned with business needs and best practices.
• Lead development of data pipelines and ETL processes using Azure Databricks, PySpark, and related tools.
• Integrate Databricks with Azure services and on-prem systems.
• Provide technical leadership and mentorship to the data engineering team.
• Ensure architecture/documentation and governance/compliance are maintained.
• Uphold code quality, data security, and scalability best practices.
• Stay updated with Databricks and related tech to drive innovation.

Skills

Azure Databricks

Delta Lake

Spark

PySpark

Azure Data Lake

Azure Synapse

Azure Data Factory

SQL

NoSQL

CI/CD

Git

Cloud security

Power BI

Spark Streaming

ARM templates

Tools

Azure DevOps

ARM templates

Job description

Quantum Technologies LLC seeks an experienced data architect to lead analytics solutions on Azure Databricks and Fabric. You will design, implement, and optimize data pipelines, collaborating with stakeholders and IT teams to ensure scalable, secure data architecture.

The role emphasizes leadership, governance, and cutting-edge cloud data practices. Responsibilities include overseeing end-to-end data workflows, integrating with Azure services, and mentoring engineers while maintaining