Azure Data Engineer - Full Time Only (12 yrs. + Exp)

Programmers.io; Dallas (TX); Hybrid; USD 90,000 - 120,000; Full time

Job summary

Programmers.io is seeking a Data Engineer in Dallas, TX or Charlotte, NC. The role emphasizes designing and building scalable data solutions using Azure and Apache Spark.

The ideal candidate will implement robust data pipelines and collaborate with stakeholders for business intelligence needs. Applicants are expected to have a strong technical background, particularly in Azure services, and experience with data governance tools.

Qualifications

• Strong hands-on experience with Microsoft Azure and Spark.
• Proficiency in Python and SQL is essential.
• Experience in designing data pipelines and understanding of data warehousing.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure services.
• Build and optimize Spark-based data processing applications.
• Implement ETL/ELT workflows for both batch and real-time data processing.

Skills

Azure

Apache Spark

Python

SQL

Data Warehousing

CI/CD

Tools

Azure Data Factory

Databricks

Power BI

Job description

Location: Dallas, TX or Charlotte, NC (3 Days Onsite/week)

Job Description

Role Overview

We are looking for a Data Engineer with strong experience in Azure and Apache Spark to design and build scalable, high-performance data solutions. The ideal candidate will work on modern cloud data platforms, ensuring reliable data pipelines and supporting analytics and business intelligence needs. Familiarity with Databricks and Collibra will be an added advantage.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure services
• Build and optimize Spark-based data processing applications
• Implement ETL/ELT workflows for batch and real-time data processing
• Work with Azure Data Factory (ADF) or similar tools for orchestration
• Store and manage data in Azure Data Lake (ADLS) or cloud storage systems
• Perform data transformation, cleansing, and validation
• Ensure data quality, governance, and security compliance
• Collaborate with analysts, data scientists, and business stakeholders
• Monitor, debug, and optimize data workflows

Required Skills & Qualifications

Technical Skills

• Strong hands-on experience with Microsoft Azure (ADF, ADLS, Synapse or similar)
• Solid experience in Apache Spark (PySpark preferred)
• Proficiency in Python and SQL
• Experience in designing data pipelines and data models
• Good understanding of data warehousing concepts
• Experience with version control and CI/CD pipelines (e.g., Azure DevOps)

Nice-to-Have Skills

• Experience with Azure Databricks for distributed data processing
• Exposure to Collibra or similar data governance/catalog tools
• Knowledge of Delta Lake architecture
• Familiarity with streaming technologies (Kafka, Event Hub)
• Experience with BI tools such as Power BI