Lead Big Data Engineer: Databricks Lakehouse on Azure

SoftServe

Polska

Hybrid

PLN 180,000 - 240,000

Full time

9 days ago

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Job summary

SoftServe is seeking a data engineer to design and develop scalable data solutions using the Databricks Lakehouse platform within Azure cloud environments. You will collaborate with technical and business stakeholders to deliver high-quality batch and streaming data pipelines, contribute to architectural decisions, and support projects from PoC to production.

You will work with Azure services such as Data Factory, Synapse, Data Lake, and DevOps, applying advanced SQL and PySpark to optimize data

Qualifications

Hands-on experience with Python and PySpark for data engineering solutions.
Strong knowledge of Databricks Lakehouse architecture and related concepts.
2+ years of experience designing data models, building ETL pipelines, and solving business problems through data solutions.
Practical experience with Azure cloud technologies, including Azure Data Factory, Azure DevOps, Azure Synapse, and Azure Data Lake Services.
Advanced SQL skills and experience working with relational databases.
Understanding of database and data warehouse design best practices.
Experience designing, building, and scaling both batch and streaming data pipelines.
Ability to evaluate technical options, conduct trade-off analysis, and solve complex problems.
Strong communication and stakeholder management skills.
Upper-intermediate or higher level of English.

Responsibilities

Design and develop scalable batch and streaming ETL pipelines using Python and PySpark.
Build and optimize data solutions based on Databricks Lakehouse architecture and best practices.
Develop and maintain data models to support business and analytical needs.
Work with Azure cloud services, including Azure Data Factory, Azure Synapse, Azure Data Lake Services, and Azure DevOps.
Implement and optimize advanced SQL solutions for data processing and analytics.
Participate in architecture discussions, conduct trade-off analysis, and recommend optimal technical solutions.
Collaborate closely with technical and business stakeholders to understand requirements and deliver effective data solutions.
Support continuous improvement, knowledge sharing, and engineering best practices within the team.
Contribute across the full project lifecycle, from PoC and MVP stages to production implementation.