Senior Data Platform Lead: GCP, PySpark & AI Pipelines

JobCubby; Kraków; Hybrid; PLN 194,000 - 298,000; Full time

Benefits offered by this job

Hybrid work in Kraków

Medical, sport, lunch subsidy

Online training

Social events

Job summary

JobCubby in Kraków is seeking a Senior Data Engineering Lead to drive data migration projects and guide an engineering pod. You will design scalable data solutions in GCP, promote coding standards and ensure high performance and reliability.

You will mentor engineers, collaborate with data analysts and business analysts, and contribute to planning, sprints and reviews. Strong PySpark/Scala, GCP, Airflow, and Unix/Linux skills are essential, along with excellent English communication.

Qualifications

• 10+ years of experience in software design, data engineering and development in Agile/DevOps environments.
• Strong development and solution design experience with PySpark and Scala.
• Hands-on experience with Google Cloud Platform and building data pipelines in GCP.
• Experience with Apache Spark, Spark SQL, Hadoop, YARN, Hive, MapReduce and ETL frameworks.
• Experience with Python, SQL, PL/SQL and data integration.
• Experience with Apache Airflow or similar scheduling tools.
• Experience with REST APIs and RESTful services.
• Strong knowledge of Unix and Linux environments.
• Experience with Git, GitHub, Jenkins, Ansible and Jira.
• Experience with automated testing, deployment and monitoring.
• Understanding of cloud architecture and cloud design patterns.
• Ability to lead an engineering pod and guide the work of other engineers.
• Ability to work autonomously and act as a technical subject‑matter expert.
• Strong communication and presentation skills for technical, senior and non‑technical audiences.
• Strong written and spoken English.

Responsibilities

• Provide technical leadership for data migration projects, including solution design, approach and estimation.
• Design and develop scalable data pipelines using GCP, PySpark and Scala.
• Provide guidance on environment setup, system architecture and cloud design patterns.
• Implement solutions focused on performance, scalability, availability, accuracy and monitoring.
• Promote coding standards, conduct code reviews and support knowledge sharing.
• Mentor engineers and oversee the technical delivery of the engineering pod.
• Work with Business Analysts to ensure requirements are correctly interpreted and implemented.
• Support production environments, troubleshoot issues and communicate findings to development teams.
• Participate in planning sessions, status meetings, sprint reviews and retrospectives.
• Identify opportunities to use AI enablers to improve traditional engineering workflows.

Skills

PySpark

Scala

Google Cloud Platform

Apache Spark

Hadoop/YARN/Hive/MapReduce

SQL/PLSQL

Airflow

REST APIs

Unix/Linux

Git/GitHub/Jenkins/Ansible/Jira

CI/CD & Monitoring

Communication & Presentation

Tools

Jira

GitHub

Jenkins

Ansible

Elasticsearch

Job description

JobCubby in Kraków is seeking a Senior Data Engineering Lead to drive data migration projects and guide an engineering pod. You will design scalable data solutions in GCP, promote coding standards and ensure high performance and reliability.

You will mentor engineers, collaborate with data analysts and business analysts, and contribute to planning, sprints and reviews. Strong PySpark/Scala, GCP, Airflow, and Unix/Linux skills are essential, along with excellent English communication.