ETL Architect

Quantum Technologies. LLC; Tallahassee (FL); On-site; USD 110,000 - 170,000; Full time

Job summary

Quantum Technologies LLC is seeking a senior Data Engineer to design and optimize large-scale ETL/ELT pipelines in a Microsoft Azure environment. You will implement automated workflows with Azure Data Factory, Databricks, Synapse, and Purview, ensure data quality, security, and governance, and mentor junior engineers while collaborating with analytics teams.

This role emphasizes performance, reliability, and scalable architecture to support Power BI reporting and data insights.

Qualifications

• 7+ years of experience in ETL development / data engineering.
• 3+ years hands-on experience with Azure Data Factory, Databricks, Synapse, and Purview.
• Strong experience building large-scale, high-availability ETL pipelines.
• Advanced skills in Spark, Python, and/or Scala.
• Strong SQL skills with complex datasets.
• Solid understanding of data governance, security, and RBAC in Azure.

Responsibilities

• Design, build, and optimize high-performance ETL/ELT pipelines across multiple data sources.
• Develop automated workflows using Azure Data Factory (ADF) for reliable data ingestion and transformation.
• Ensure secure, fault-tolerant data movement from source to target systems.
• Work with ADF, Databricks, Azure Synapse Analytics, and Azure Purview.
• Load and manage large structured and unstructured datasets in Azure Synapse.
• Support analytics and reporting use cases (Power BI consumption layer).
• Act as a technical lead for ETL/data engineering initiatives.
• Collaborate with data engineers, analysts, security teams, and business stakeholders.
• Mentor junior engineers and enforce ETL best practices.
• Create and maintain detailed technical documentation.

Skills

ETL development

Azure Data Factory

Databricks

Spark

Python

Scala

SQL

RBAC in Azure

Tools

Azure Data Factory

Databricks

Synapse

Purview

Job description

Key Responsibilities

• Design, build, and optimize high-performance ETL/ELT pipelines across multiple data sources.
• Develop automated workflows using Azure Data Factory (ADF) for reliable data ingestion and transformation.
• Ensure secure, fault-tolerant data movement from source to target systems.

Data Integration & Transformation

• Build complex transformation logic using Azure Databricks (Spark with Python/Scala).
• Implement robust data validation, quality checks, and error handling.
• Ensure accurate, timely, and trusted data for analytics and reporting.
• Work extensively with ADF, Databricks, Azure Synapse Analytics, and Azure Purview.
• Load and manage large structured and unstructured datasets in Azure Synapse.
• Support analytics and reporting use cases (Power BI consumption layer).

Performance & Optimization

• Tune ETL pipelines to reduce latency and improve throughput.
• Design scalable architectures that meet high-availability and performance SLAs.

Security, Governance & Compliance

• Implement RBAC, data security, and compliance controls across the Azure ecosystem.
• Use Azure Purview for data cataloging, lineage, and governance enforcement.
• Ensure compliance with departmental and regulatory data policies.
• Act as a technical lead for ETL/data engineering initiatives.
• Collaborate with data engineers, analysts, security teams, and business stakeholders.
• Mentor junior engineers and enforce ETL best practices.

Documentation & Standards

• Create and maintain detailed technical documentation.
• Define and enforce standards for ETL development, data handling, and security.

Required Qualifications

• 7+ years of experience in ETL development / data engineering.
• 3+ years hands-on experience with Azure Data Factory, Databricks, Synapse, and Purview.
• Strong experience building large-scale, high-availability ETL pipelines.
• Advanced skills in Spark, Python, and/or Scala.
• Strong SQL skills with complex datasets.
• Solid understanding of data governance, security, and RBAC in Azure.

Preferred Qualifications

• Microsoft certifications:
  • Azure Data Engineer Associate
  • Azure Solutions Architect Expert
  • Azure Fundamentals

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.