Lead BigData Engineer (Databricks + Azure)
SoftServe
Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.
Job summary
SoftServe is seeking a data engineer to design and develop scalable data solutions using the Databricks Lakehouse platform within Azure cloud environments. You will collaborate with technical and business stakeholders to deliver high-quality batch and streaming data pipelines, contribute to architectural decisions, and support projects from PoC to production.
You will work with Azure services such as Data Factory, Synapse, Data Lake, and DevOps, applying advanced SQL and PySpark to optimize data
Qualifications
- Hands-on experience with Python and PySpark for data engineering solutions.
- Strong knowledge of Databricks Lakehouse architecture and related concepts.
- 2+ years of experience designing data models, building ETL pipelines, and solving business problems through data solutions.
- Practical experience with Azure cloud technologies, including Azure Data Factory, Azure DevOps, Azure Synapse, and Azure Data Lake Services.
- Advanced SQL skills and experience working with relational databases.
- Understanding of database and data warehouse design best practices.
- Experience designing, building, and scaling both batch and streaming data pipelines.
- Ability to evaluate technical options, conduct trade-off analysis, and solve complex problems.
- Strong communication and stakeholder management skills.
- Upper-intermediate or higher level of English.
Responsibilities
- Design and develop scalable batch and streaming ETL pipelines using Python and PySpark.
- Build and optimize data solutions based on Databricks Lakehouse architecture and best practices.
- Develop and maintain data models to support business and analytical needs.
- Work with Azure cloud services, including Azure Data Factory, Azure Synapse, Azure Data Lake Services, and Azure DevOps.
- Implement and optimize advanced SQL solutions for data processing and analytics.
- Participate in architecture discussions, conduct trade-off analysis, and recommend optimal technical solutions.
- Collaborate closely with technical and business stakeholders to understand requirements and deliver effective data solutions.
- Support continuous improvement, knowledge sharing, and engineering best practices within the team.
- Contribute across the full project lifecycle, from PoC and MVP stages to production implementation.
Skills
Tools
Job description
In this role, you will lead the design and development of scalable data solutions using the Databricks Lakehouse platform within Azure cloud environments. You will collaborate with technical and business stakeholders to deliver high-quality batch and streaming data pipelines, contribute to architectural decisions, and support the full project lifecycle, from proof of concept to enterprise-scale implementation within a collaborative and innovation-driven team.
- Design and develop scalable batch and streaming ETL pipelines using Python and PySpark
- Build and optimize data solutions based on Databricks Lakehouse architecture and best practices
- Develop and maintain data models to support business and analytical needs
- Work with Azure cloud services, including Azure Data Factory, Azure Synapse, Azure Data Lake Services, and Azure DevOps
- Implement and optimize advanced SQL solutions for data processing and analytics
- Participate in architecture discussions, conduct trade-off analysis, and recommend optimal technical solutions
- Collaborate closely with technical and business stakeholders to understand requirements and deliver effective data solutions
- Support continuous improvement, knowledge sharing, and engineering best practices within the team
- Contribute across the full project lifecycle, from PoC and MVP stages to production implementation
- Hands-on experience with Python and PySpark for data engineering solutions
- Strong knowledge of Databricks Lakehouse architecture and related concepts
- 2+ years of experience designing data models, building ETL pipelines, and solving business problems through data solutions
- Practical experience with Azure cloud technologies, including Azure Data Factory, Azure DevOps, Azure Synapse, and Azure Data Lake Services
- Advanced SQL skills and experience working with relational databases
- Understanding of database and data warehouse design best practices
- Experience designing, building, and scaling both batch and streaming data pipelines
- Ability to evaluate technical options, conduct trade-off analysis, and solve complex problems
- Strong communication and stakeholder management skills
- Upper-intermediate or higher level of English
SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.
We are a digital engineering and technology consulting company where expertise grows alongsidepeople. For more than 30years, we have been elevating technology: helping organizations navigate complex businesschallenges by combining deepengineering knowledge with thoughtful, research-backed innovation. Our teams work across keyareas: digitalengineering, data and analytics, Сloud, and AI/ML. In each, we deliver practical, scalablesolutions rooted in realbusiness needs and measurable human impact.
You bring your perspective and ambition. We create an environment where your work meetsclarity, confidence, andpurpose.
Work from home, from the office, or in a hybrid format that supports focus andcollaboration.
Competitive, market-based pay, benchmarked by role andlocation — plus health coverage, paid time off, wellness support, and learningopportunities.
Approachable leaders who communicate openly, keep teams close to the strategy, andsupport long-term planning.
Stay close to AI/ML, Cloud, Quantum Computing, IoT, andRobotics communities, with projects built on modern frameworks.
Browse all open positions to find the best fit for your experience.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
