Azure Data Engineer - ETL/ELT, Spark & Synapse

Cloud Analytics Technologies, LLC; Austin (TX); On-site; USD 120,000 - 180,000; Full time

Job summary

Cloud Analytics Technologies, LLC in Austin, TX is seeking a Senior Data Engineer to design and implement scalable ETL/ELT pipelines on Azure. You will build data lakehouse solutions using ADLS Gen2, Synapse, Databricks, and Python-based transformations.

You will collaborate with data architects and stakeholders to ensure data quality, security, and governance, optimize Spark jobs and SQL queries, and implement CI/CD deployments with Azure DevOps.

Qualifications

• Strong experience with Microsoft Azure cloud data services and data engineering.
• Hands-on experience with Azure Data Factory (ADF) and Databricks.
• Proficient in Python and PySpark for data processing and scripting.
• Strong SQL skills and query optimization, with data warehousing knowledge.
• Experience with Azure Synapse Analytics and ADLS Gen2.
• Familiarity with data governance, security, and quality practices.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory.
• Build and optimize data processing solutions with Azure Databricks and PySpark.
• Develop data transformation workflows using Python and SQL.
• Implement data ingestion from multiple sources including databases, APIs, files, and cloud platforms.
• Build and maintain data lake and data warehouse solutions using ADLS Gen2 and Azure Synapse.
• Collaborate with data architects, analysts, and stakeholders to ensure data quality and compliance.
• Troubleshoot production data pipelines and ensure operational stability.

Skills

Azure data services

Azure Data Factory

Azure Databricks

Python

PySpark

SQL / query optimization

Azure Synapse Analytics

ADLS Gen2

Data warehousing

ETL/ELT pipelines

Azure DevOps

Data governance & security

Delta Lake

Tools

Git

Azure DevOps

Delta Lake

Job description

Cloud Analytics Technologies, LLC in Austin, TX is seeking a Senior Data Engineer to design and implement scalable ETL/ELT pipelines on Azure. You will build data lakehouse solutions using ADLS Gen2, Synapse, Databricks, and Python-based transformations.

You will collaborate with data architects and stakeholders to ensure data quality, security, and governance, optimize Spark jobs and SQL queries, and implement CI/CD deployments with Azure DevOps.