Azure Cloud Data Engineer

Ideate Technologies; Charlotte (NC); On-site; USD 110,000 - 165,000; Full time

Job summary

Ideate Technologies is seeking a skilled Data Platform Engineer to design, build, and maintain scalable data pipelines on Microsoft Azure. You will work with Azure Data Factory, Databricks, Synapse Analytics, and Azure SQL Database to deliver reliable analytics solutions.

You will implement Infrastructure as Code with ARM templates or Terraform, automate deployments with PowerShell or Python, and collaborate with DevOps and security teams to ensure secure, cost-efficient cloud architectures.

Qualifications

• Experience designing, developing, and maintaining scalable data pipelines.
• Strong knowledge of Microsoft Azure services and cloud architecture.
• IaC experience with ARM templates or Terraform.
• Proficiency in scripting languages (PowerShell, Python).
• Understanding of DevOps, CI/CD, and cloud security best practices.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Azure Data Factory, Databricks, and Synapse.
• Design and manage cloud-based apps and infrastructure.
• Develop IaC to automate provisioning and management.
• Create and maintain automation scripts.
• Monitor and optimize cloud resources for performance and cost.
• Collaborate with DevOps and security teams on secure cloud architectures.
• Troubleshoot Azure services, data pipelines, and infrastructure issues.

Skills

Azure

Cloud architecture

Data pipelines

Python

PySpark

SQL

Scala

Java

Hive

Terraform

ARM templates

PowerShell

CI/CD

Cloud security

Databricks

Azure Data Factory

Synapse Analytics

Azure SQL Database

Tools

Azure Data Factory

Synapse Analytics

Databricks

Azure SQL Database

Job description

• Strong knowledge of Microsoft Azure services, cloud architecture, and cloud computing.
• Experience with Azure data services, including:Azure Data Factory (ADF)
• Strong programming skills in Python, PySpark, Spark, SQL, Scala, Java, and Hive.
• Experience designing, developing, and maintaining scalable data pipelines.
• Knowledge of Azure Virtual Machines, Azure Storage, and Azure Networking.
• Experience with Infrastructure as Code (IaC) using ARM templates or Terraform.
• Proficiency in PowerShell, Python, or similar scripting languages.
• Understanding of DevOps practices, CI/CD pipelines, and cloud security best practices.
• Strong problem-solving, analytical thinking, and attention to detail.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory, Databricks, Azure SQL Database, and Synapse Analytics.
• Design, implement, and manage cloud-based applications and infrastructure.
• Develop Infrastructure as Code to automate cloud resource provisioning and management.
• Create and maintain scripts for automation and deployment.
• Monitor and optimize cloud resources for performance and cost efficiency.
• Collaborate with DevOps and security teams to implement secure and scalable cloud architectures.
• Troubleshoot and resolve issues related to Azure services, data pipelines, and infrastructure.

Preferred Qualifications

• Microsoft Azure certifications, such as AZ-104 or AZ-303.
• Experience with cloud infrastructure automation and management.
• Strong communication, adaptability, and teamwork skills.