Senior Databricks Data Pipeline Architect

EPAM Systems; Málaga; On-site; EUR 90,000 - 120,000; Full time

Benefits offered by this job

Private health insurance

EPAM Employees Stock Purchase Plan

100% paid sick leave

Referral Program

Professional certification

Language courses

Job summary

EPAM Systems Inc. in Málaga, Spain, is seeking a Senior Data Software Engineer (Databricks) to architect and maintain scalable data pipelines and Python-based workflows within the Databricks platform.

You will collaborate with data engineers, software developers, and project managers to deliver reliable data infrastructure and actionable insights for multiple business units. You will design future-proof data integration strategies, monitor pipeline performance, and communicate findings to

Qualifications

• Experience with Databricks and Python to build scalable data workflows.
• Ability to resolve complex data integration across heterogeneous sources.
• Experience collaborating with software engineers and project managers.
• Translate business requirements into technical data solutions.

Responsibilities

• Architect and implement scalable data pipelines to support diverse data ingestion and transformation needs.
• Collaborate with software engineering teams to ensure smooth integration of data workflows into broader systems.
• Engage regularly with project managers to align pipeline development with project timelines and objectives.
• Develop, test, and maintain Python-based data workflows within the Databricks platform.
• Monitor pipeline performance and troubleshoot data processing issues to maintain data quality and reliability.
• Share data insights and pipeline status updates with business stakeholders to inform decision-making.
• Participate in planning sessions to design future-proof data infrastructure and integration strategies.

Skills

Databricks

Python

Data pipelines

Cross-functional collaboration

Data analysis

Tools

SQL

Airflow

Job description

EPAM Systems Inc. in Málaga, Spain, is seeking a Senior Data Software Engineer (Databricks) to architect and maintain scalable data pipelines and Python-based workflows within the Databricks platform.

You will collaborate with data engineers, software developers, and project managers to deliver reliable data infrastructure and actionable insights for multiple business units. You will design future-proof data integration strategies, monitor pipeline performance, and communicate findings to