Azure Data Engineer Lead - Scalable ETL Pipelines

SoftServe; Mexico; On-site; PHP 5,643,000 - 8,150,000; Full time

Job summary

SoftServe is seeking an experienced Data Engineer to design and deliver scalable data solutions in Azure environments. You will combine hands-on data engineering with technical leadership, translating business needs into reliable data solutions across the delivery lifecycle.

You will design ETL pipelines, own implementation quality, and collaborate with clients and architects to ensure high performance, quality, and maintainability of data workflows.

Qualifications

• 5+ years of experience in Data Engineering with ETL pipelines and data integration solutions.
• Advanced proficiency in SQL and Python for data processing and optimization.
• Strong experience with Spark and PySpark for distributed data processing.
• Hands-on experience with Azure data technologies and cloud-based data platforms.
• Advanced proficiency with Azure Data Factory and/or Apache Airflow for workflow orchestration and scheduling.
• Strong understanding of Data Architecture, Data Modeling, Data Quality, and data integration.
• Proven ability to provide technical guidance to engineering teams as a reference.
• Strong English communication skills for daily interaction with international clients.

Responsibilities

• Design and deliver scalable ETL and data integration pipelines across Azure data environments.
• Own technical implementation and solution quality for reliable, scalable data processing.
• Collaborate with clients, Solution Architects, and engineering teams to translate requirements into data solutions.
• Build and optimize data processing workflows using SQL, Python, Spark/PySpark, Azure Data Factory, and Apache Airflow.
• Develop data models and schemas aligned with business requirements and downstream consumption needs.
• Implement data quality and validation controls to maintain data integrity across platforms.
• Optimize SQL queries, ETL processes, and data workflows for performance and reliability.
• Guide the Data Engineering team on technical decisions, engineering practices, troubleshooting, and delivery.

Skills

Data engineering

SQL

Python

Spark / PySpark

Data modeling

Leadership

Tools

Azure Data Factory

Apache Airflow

Power BI

Hadoop

Kafka

Job description

SoftServe is seeking an experienced Data Engineer to design and deliver scalable data solutions in Azure environments. You will combine hands-on data engineering with technical leadership, translating business needs into reliable data solutions across the delivery lifecycle.

You will design ETL pipelines, own implementation quality, and collaborate with clients and architects to ensure high performance, quality, and maintainability of data workflows.