Lead Data Software Engineer (Databricks)

EPAM Systems Inc; España; On-site; EUR 70,000 - 100,000; Full time

Job summary

EPAM Systems Inc in Malaga is seeking a Senior Data Software Engineer (Databricks) to design scalable data pipelines and empower actionable insights across business units.

You will collaborate with data engineers and software developers to build Python-based workflows in Databricks, monitor performance, and translate business needs into robust architectures that support decision making.

Qualifications

• Design scalable data pipelines across ingestion and transformation needs.
• Able to translate business needs into technical data solutions.
• Collaborate with software engineers and project managers to align work with timelines.

Responsibilities

• Architect and implement scalable data pipelines for diverse data needs.
• Collaborate with software engineering teams to integrate data workflows into systems.
• Engage with project managers to align pipeline development with timelines.
• Develop, test, and maintain Python-based data workflows in the Databricks platform.
• Monitor pipeline performance and troubleshoot data issues to ensure quality.
• Share data insights and status updates with stakeholders to inform decisions.
• Participate in planning to design future-proof data infrastructure and integration strategies.

Skills

Data pipeline design

Cross-functional collaboration

Data analysis

Tools

Databricks

Python

Job description

What makes a number trustworthy enough to drive decisions across an enterprise? The Senior Data Software Engineer (Databricks) role in Malaga sits at the intersection of data engineering and analytics, where robust pipelines support seamless integration and actionable insights for multiple business units. Working closely with a team of data engineers and software developers, you shape the flow of information that powers business strategy. The challenge is to build infrastructure that adapts as business needs evolve. The person in this role decides how data becomes a reliable asset for the company.

Responsibilities

• Architect and implement scalable data pipelines to support diverse data ingestion and transformation needs
• Collaborate with software engineering teams to ensure smooth integration of data workflows into broader systems
• Engage regularly with project managers to align pipeline development with project timelines and objectives
• Develop, test, and maintain Python-based data workflows within the Databricks platform
• Monitor pipeline performance and troubleshoot data processing issues to maintain data quality and reliability
• Share data insights and pipeline status updates with business stakeholders to inform decision-making
• Participate in planning sessions to design future-proof data infrastructure and integration strategies

Requirements

• Proficient in designing and optimizing scalable data workflows using Databricks and Python
• Skilled in resolving complex data integration challenges across heterogeneous data sources
• Experienced in collaborating with cross-functional teams including software engineers and project managers
• Capable of translating business requirements into technical data solutions and pipeline architectures
• Adept at analysing data to extract meaningful insights and communicating findings to stakeholders