Lead Data Consultant

GFT Technologies Poland; Kraków; Hybrid; PLN 320,000 - 520,000; Full time

Job summary

GFT Technologies Poland is seeking an experienced data engineering leader to drive data migration projects and build scalable pipelines on Google Cloud Platform. You will guide the engineering pod, promote best practices, and collaborate with analysts to translate requirements into robust solutions.

The role emphasizes hands-on design with PySpark/Scala, cloud patterns, and AI-enabled workflow improvements. Hybrid work from Kraków is expected, with frequent collaboration across teams.

Qualifications

• 10+ years of experience in software design, data engineering and Agile/DevOps environments.
• Strong development and solution design experience with PySpark or Scala.
• Hands-on experience with Google Cloud Platform and building data pipelines in GCP.
• Experience with Apache Spark, Spark SQL, Hadoop, YARN, Hive, MapReduce and ETL frameworks.
• Experience with Python, SQL, PL/SQL and data integration.
• Experience with Apache Airflow or similar scheduling tools.
• Experience with REST APIs and RESTful services.
• Strong knowledge of Unix and Linux environments.
• Experience with Git, GitHub, Jenkins, Ansible and Jira.
• Experience with automated testing, deployment and monitoring.
• Understanding of cloud architecture and cloud design patterns.
• Ability to lead an engineering pod and guide the work of other engineers.
• Ability to work autonomously and act as a technical subject-matter expert.
• Strong communication and presentation skills for technical, senior and non-technical audiences.
• Strong written and spoken English.
• Openness to work hybrid from our client's office (Kraków) 8 days a month.

Responsibilities

• Lead the technical delivery of data migration projects, including solution design, approach and estimation.
• Design and develop scalable data pipelines using GCP, PySpark and Scala.
• Provide guidance on environment setup, system architecture and cloud design patterns.
• Implement solutions focused on performance, scalability, availability, accuracy and monitoring.
• Promote coding standards, conduct code reviews and support knowledge sharing.
• Mentor engineers and oversee the technical delivery of the engineering pod.
• Work with Business Analysts to ensure requirements are correctly interpreted and implemented.
• Support production environments, troubleshoot issues and communicate findings to development teams.
• Participate in planning sessions, status meetings, sprint reviews and retrospectives.
• Identify opportunities to use AI enablers to improve traditional engineering workflows.

Skills

Data migration leadership

Data engineering

Agile / DevOps

PySpark

Scala

Python

SQL / PLSQL

GCP

Apache Spark

Hadoop

YARN

Hive

MapReduce

REST APIs

Unix/Linux

Git

GitHub

Jenkins

Ansible

Airflow

Automation

Communication

Tools

GCP

Apache Spark

Hadoop

YARN

Hive

MapReduce

GitHub

Jenkins

Ansible

Airflow

Jira

Job description

You will lead the technical delivery of data migration and engineering initiatives, working closely with Engineers, Data Analysts and Business Analysts. You will design scalable data solutions in GCP, guide the engineering pod, promote development standards and support the team throughout the Agile development process.

• Provide technical leadership for data migration projects, including solution design, approach and estimation
• Design and develop scalable data pipelines using GCP, PySpark and Scala
• Provide guidance on environment setup, system architecture and cloud design patterns
• Implement solutions focused on performance, scalability, availability, accuracy and monitoring
• Promote coding standards, conduct code reviews and support knowledge sharing
• Mentor engineers and oversee the technical delivery of the engineering pod
• Work with Business Analysts to ensure requirements are correctly interpreted and implemented
• Support production environments, troubleshoot issues and communicate findings to development teams
• Participate in planning sessions, status meetings, sprint reviews and retrospectives
• Identify opportunities to use AI enablers to improve traditional engineering workflows

Requirements

• At least 10 years of experience in software design, data engineering and development in Agile and DevOps environments
• Strong development and solution design experience with PySpark or Scala
• Hands-on experience with Google Cloud Platform and building data pipelines in GCP
• Experience with Apache Spark, Spark SQL, Hadoop, YARN, Hive, MapReduce and ETL frameworks
• Experience with Python, SQL, PL/SQL and data integration
• Experience with Apache Airflow or similar scheduling tools
• Experience with REST APIs and RESTful services
• Strong knowledge of Unix and Linux environments
• Experience with Git, GitHub, Jenkins, Ansible and Jira
• Experience with automated testing, deployment and monitoring
• Understanding of cloud architecture and cloud design patterns
• Ability to lead an engineering pod and guide the work of other engineers
• Ability to work autonomously and act as a technical subject-matter expert
• Strong communication and presentation skills for technical, senior and non-technical audiences
• Strong written and spoken English
• Openness to work hybrid from our client's office (Kraków) 8 days a month

Nice to have

• Experience with Elasticsearch
• Experience with data ingestion solutions
• Experience working with Scrum and Kanban
• Interest in AI-based tools that can accelerate engineering workflows