Azure Data Engineer: Scalable Pipelines & IaC

Ideate Technologies; Charlotte (NC); On-site; USD 110,000 - 165,000; Full time

Job summary

Ideate Technologies is seeking a skilled Data Platform Engineer to design, build, and maintain scalable data pipelines on Microsoft Azure. You will work with Azure Data Factory, Databricks, Synapse Analytics, and Azure SQL Database to deliver reliable analytics solutions.

You will implement Infrastructure as Code with ARM templates or Terraform, automate deployments with PowerShell or Python, and collaborate with DevOps and security teams to ensure secure, cost-efficient cloud architectures.

Qualifications

• Experience designing, developing, and maintaining scalable data pipelines.
• Strong knowledge of Microsoft Azure services and cloud architecture.
• IaC experience with ARM templates or Terraform.
• Proficiency in scripting languages (PowerShell, Python).
• Understanding of DevOps, CI/CD, and cloud security best practices.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Azure Data Factory, Databricks, and Synapse.
• Design and manage cloud-based apps and infrastructure.
• Develop IaC to automate provisioning and management.
• Create and maintain automation scripts.
• Monitor and optimize cloud resources for performance and cost.
• Collaborate with DevOps and security teams on secure cloud architectures.
• Troubleshoot Azure services, data pipelines, and infrastructure issues.

Skills

Azure

Cloud architecture

Data pipelines

Python

PySpark

SQL

Scala

Java

Hive

Terraform

ARM templates

PowerShell

CI/CD

Cloud security

Databricks

Azure Data Factory

Synapse Analytics

Azure SQL Database

Tools

Azure Data Factory

Synapse Analytics

Databricks

Azure SQL Database

Job description

Ideate Technologies is seeking a skilled Data Platform Engineer to design, build, and maintain scalable data pipelines on Microsoft Azure. You will work with Azure Data Factory, Databricks, Synapse Analytics, and Azure SQL Database to deliver reliable analytics solutions.

You will implement Infrastructure as Code with ARM templates or Terraform, automate deployments with PowerShell or Python, and collaborate with DevOps and security teams to ensure secure, cost-efficient cloud architectures.