Data Engineer with Pyspark

Stafide; Madrid; On-site; EUR 60,000 - 90,000; Full time

Job summary

Zohorecruit in Madrid, Spain, is seeking a Data Engineer specialized in PySpark to design, build and optimize scalable data pipelines and models. The role emphasizes batch and streaming processing, data quality, and collaboration with cross-functional teams in an Agile environment.

You will work with Python, PySpark and SQL on large datasets, contribute to data platform improvements, and drive performance and reliability across data workloads.

Qualifications

• 5+ years of professional experience in Data Engineering or a related field.
• Strong hands-on experience with PySpark as a core data processing technology.
• Strong proficiency in Python, PySpark, and SQL.
• Hands-on experience developing and maintaining batch and streaming data pipelines.
• Experience with large-scale data processing and transformation.
• Good understanding of data modeling, data integration, and data optimization.
• Experience with cloud-based data platforms and modern data engineering architectures.
• Experience with data quality, validation, troubleshooting, and performance optimization.
• Familiarity with CI/CD and modern software engineering practices.
• Experience working in Agile development and delivery environments.
• Strong communication, collaboration, analytical, and problem-solving skills.

Responsibilities

• Design, develop, and maintain scalable and reliable data processing solutions using PySpark.
• Build and manage robust batch and streaming data pipelines.
• Develop efficient data transformation and processing solutions using Python, PySpark, and SQL.
• Design, develop, and optimize data models to support scalable and high-performance data processing.
• Work with large datasets and complex data processing workloads.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
• Implement data quality, validation, and error-handling mechanisms across data pipelines.
• Collaborate with engineering, architecture, and business teams to deliver reliable data solutions.
• Contribute to Agile development practices and continuously improve data engineering standards and processes.

Skills

PySpark

Python

SQL

Data pipelines

Data modeling

Cloud platforms

CI/CD

Agile

Communication

Job description

As a Data Engineer with PySpark, you will:

• Design, develop, and maintain scalable and reliable data processing solutions using PySpark.
• Build and manage robust batch and streaming data pipelines.
• Develop efficient data transformation and processing solutions using Python, PySpark, and SQL.
• Design, develop, and optimize data models to support scalable and high-performance data processing.
• Work with large datasets and complex data processing workloads.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
• Implement data quality, validation, and error-handling mechanisms across data pipelines.
• Collaborate with engineering, architecture, and business teams to deliver reliable data solutions.
• Contribute to Agile development practices and continuously improve data engineering standards and processes.

What You Bring to the Table:

• 5+ years of professional experience in Data Engineering or a related field.
• Strong hands-on experience with PySpark as a core data processing technology.
• Strong proficiency in Python, PySpark, and SQL.
• Hands-on experience developing and maintaining batch and streaming data pipelines.
• Experience working with large-scale data processing and transformation.
• Good understanding of data modeling, data integration, and data optimization.
• Experience with cloud-based data platforms and modern data engineering architectures.
• Experience with data quality, validation, troubleshooting, and performance optimization.
• Familiarity with CI/CD and modern software engineering practices.
• Experience working in Agile development and delivery environments.
• Strong communication, collaboration, analytical, and problem-solving skills.

You Should Possess the Ability to:

• Develop scalable and high-performance data pipelines using PySpark.
• Build efficient data transformations using Python, PySpark, and SQL.
• Process and manage large volumes of structured and unstructured data.
• Optimize Spark jobs, data processing performance, and resource utilization.
• Troubleshoot complex data pipeline and processing issues.
• Design reliable and maintainable data engineering solutions.
• Implement data quality and validation processes.
• Work effectively with technical and business stakeholders.
• Apply best practices for code quality, scalability, maintainability, and reliability.
• Continuously improve data engineering processes and solutions.

What We Bring to the Table:

• Opportunity to work on enterprise-scale data engineering initiatives in Spain.
• Exposure to modern data platforms, PySpark, cloud technologies, and large-scale data processing.
• A collaborative Agile environment focused on technical excellence and innovation.
• Opportunities to work on complex data pipelines and data transformation solutions.
• Continuous learning and opportunities for technical and professional growth.
• A culture focused on quality, ownership, scalability, and sustainable data engineering solutions.