Lead Data Engineer (Azure)

SoftServe; Lake Oswego, Northern (OR, KY); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Flexible work model

Health coverage

Learning opportunities

Job summary

SoftServe is seeking a senior Data Engineer to design and deliver scalable data solutions in Azure, acting as a technical reference for the engineering team. You will translate business needs into reliable data solutions across the delivery lifecycle with a focus on data integration, performance, and quality.

You will lead ETL pipeline design, collaborate with clients and architects, and guide the team on technical decisions, architecture, and delivery excellence across data environments.

Qualifications

• 5+ years of experience in Data Engineering with ETL pipelines.
• Proficiency in SQL and Python for data processing and optimization.
• Strong experience with Spark/PySpark for distributed data processing.
• Hands-on experience with Azure data technologies and cloud-based data platforms.
• Advanced proficiency with Azure Synapse, Azure Data Factory and/or Apache Airflow for workflow orchestration and scheduling.
• Strong understanding of Data Architecture, Data Modeling, Data Quality, Data Integration, and SQL and ETL performance optimization.
• Proven ability to provide technical guidance to engineering teams.
• Strong working proficiency in English for daily communication with international clients and teams.
• Exposure to Microsoft Fabric, Hadoop, Hive, Kafka, Power BI, or Tableau is a plus.
• Experience with healthcare or pharmaceutical data environments is an advantage.

Responsibilities

• Design and develop scalable ETL and data integration pipelines across Azure data environments.
• Own technical implementation and solution quality, ensuring reliable, scalable, and maintainable data processing.
• Collaborate with clients, Solution Architects, and engineering teams to translate business requirements into effective data solutions.
• Build and optimize data processing workflows using SQL, Python, Spark, PySpark, Azure Data Factory, and Apache Airflow.
• Develop data models and schemas aligned with business requirements, architecture standards, and downstream consumption needs.
• Implement data quality and validation controls to maintain data integrity and consistency across platforms.
• Optimize SQL queries, ETL processes, and data workflows to improve performance and reliability.
• Guide the Data Engineering team on technical decisions, engineering practices, troubleshooting, and successful solution delivery.

Skills

SQL

Python

Spark

PySpark

Azure Synapse

Azure Data Factory

Apache Airflow

Data modeling

Data quality

Data integration

ETL optimization

English communication

Client-facing

Healthcare data

Tools

Azure Data Factory

Apache Airflow

Spark

PySpark

Power BI

Tableau

Hadoop

Kafka

Hive

Microsoft Fabric

Job description

In this role, you will design and deliver scalable data solutions within Azure environments while serving as a technical reference for the engineering team. Combining hands‑on Data Engineering with technical leadership, you will work closely with clients and architects to translate business needs into reliable solutions across the delivery lifecycle, with a focus on data integration, performance, and quality.

RESPONSIBILITIES

• Design and develop scalable ETL and data integration pipelines across Azure data environments
• Own technical implementation and solution quality, ensuring reliable, scalable, and maintainable data processing
• Collaborate with clients, Solution Architects, and engineering teams to translate business requirements into effective data solutions
• Build and optimize data processing workflows using SQL, Python, Spark, PySpark, Azure Data Factory, and Apache Airflow
• Develop data models and schemas aligned with business requirements, architecture standards, and downstream consumption needs
• Implement data quality and validation controls to maintain data integrity and consistency across platforms
• Optimize SQL queries, ETL processes, and data workflows to improve performance and reliability
• Guide the Data Engineering team on technical decisions, engineering practices, troubleshooting, and successful solution delivery

REQUIREMENTS

• 5+ years of experience in Data Engineering, including hands‑on development of ETL pipelines and data integration solutions
• Advanced proficiency in SQL and Python for data processing, transformation, and optimization
• Strong experience with Spark and PySpark for distributed data processing
• Hands‑on experience with Azure data technologies and cloud‑based data platforms
• Advanced proficiency with Azure Synapse, Azure Data Factory and/or Apache Airflow for workflow orchestration and scheduling
• Strong understanding of Data Architecture, Data Modeling, Data Quality, Data Integration, and SQL and ETL performance optimization
• Proven ability to provide technical guidance to engineering teams and serve as a technical reference throughout project delivery
• Strong working proficiency in English for daily communication with international clients and teams
• Client‑facing, communication, analytical, and problem‑solving skills
• Exposure to Microsoft Fabric, Hadoop, Hive, Kafka, Power BI, or Tableau as a plus
• Experience with healthcare or pharmaceutical data environments as an advantage

SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.

About us

We are a digital engineering and technology consulting company where expertise grows alongside people. For more than 30 years, we have been elevating technology: helping organizations navigate complex business challenges by combining deep engineering knowledge with thoughtful, research‑backed innovation. Our teams work across key areas: digital engineering, data and analytics, Cloud, and AI/ML. In each, we deliver practical, scalable solutions rooted in real business needs and measurable human impact.

You bring your perspective and ambition. We create an environment where your work meets clarity, confidence, and purpose.

We offer

Flexible Work Model

Work from home, from the office, or in a hybrid format that supports focus and collaboration.

Competitive, market‑based pay, benchmarked by role and location — plus health coverage, paid time off, wellness support, and learning opportunities.

Approachable leaders who communicate openly, keep teams close to the strategy, and support long‑term planning.

Advanced tech communities

Stay close to AI/ML, Cloud, Quantum Computing, IoT, and Robotics communities, with projects built on modern frameworks.

More opportunities available

Browse all open positions to find the best fit for your experience.