Data Engineer: PySpark & Spark Optimization (Hybrid, Kraków)

Vertex Agility; Kraków; Hybrid; PLN 120,000 - 160,000; Full time

Benefits offered by this job

Private medical care

Life insurance

Multisport card

Job summary

Vertex Agility in Kraków is seeking an experienced Data Engineer to join a team focused on building and optimizing high-performance data processing solutions. You’ll transform experimental code into scalable, reliable production workflows using PySpark and SQL.

The role emphasizes Spark performance tuning, Hadoop ecosystem work (YARN), and data pipeline design. Hybrid work arrangement requires 6 days/month in the office; private medical care and long-term projects are offered.

Qualifications

• Experience with PySpark and Spark for data processing.
• Experience designing and optimizing data pipelines.
• Strong SQL development and data analysis skills.
• Hands-on experience with Hadoop ecosystem and Spark on YARN.
• GCP experience is a plus.
• Troubleshooting and analytical abilities.

Responsibilities

• Develop and maintain data processing solutions using PySpark and Spark.
• Convert experimental code into production-ready workflows.
• Design, implement, and optimize data processing pipelines.
• Perform advanced SQL development and data analysis.
• Optimize Spark applications for performance and scalability.
• Work within Hadoop ecosystem, including Spark on YARN.
• Troubleshoot and improve data processing workflows.
• Support scalable and reliable data solutions.
• Strong experience with PySpark and data engineering.
• Advanced SQL skills.
• Strong Spark development and optimization.
• Hands-on Hadoop ecosystem experience.
• Experience working with Spark on YARN.
• Strong understanding of data processing and distributed computing.
• Experience developing performant and scalable data workflows.

Skills

PySpark

Advanced SQL

Spark

Hadoop ecosystem

YARN

Data engineering

Troubleshooting

Distributed computing

Analytical skills

Tools

Hadoop

Spark on YARN

Job description

Vertex Agility in Kraków is seeking an experienced Data Engineer to join a team focused on building and optimizing high-performance data processing solutions. You’ll transform experimental code into scalable, reliable production workflows using PySpark and SQL.

The role emphasizes Spark performance tuning, Hadoop ecosystem work (YARN), and data pipeline design. Hybrid work arrangement requires 6 days/month in the office; private medical care and long-term projects are offered.