data engineer (GCP)

HireHi; Kraków; On-site; PLN 180,000 - 280,000; Full time

Benefits offered by this job

B2B контракт

AI и автоматизация проектов

Job summary

Capco ищет инженера данных для разработки масштабируемых конвейеров и обработки данных с использованием Python и Apache Spark. Нужно работать с Hadoop, Linux, SQL и GCP, обеспечивая качество данных и документацию.

Требуется 2–4+ года опыта, знание английского уровня не ниже B2, умение работать в Agile и обсуждать требования с командой. Предусмотрен B2B контракт и участие в проектах в финансовом секторе.

Qualifications

• 2–4+ года коммерческого опыта в Data Engineering или аналогичной роли

Responsibilities

• Проектирование, разработка и поддержка масштабируемых конвейеров данных и процессов обработки
• Разработка трансформаций и процессов обработки с использованием Python и Apache Spark
• Работа с крупными наборами данных в распределённых средах Hadoop
• Разработка облачных решений на Google Cloud Platform (GCP)
• Поддержка внедрения и тестирования дата-решений и их документации

Skills

Python

Apache Spark

Linux

SQL

Git

Agile

English B2

Tools

Docker

Kubernetes

Airflow

BigQuery

Job description

Описание

Capco is a global technology and management consulting company that helps financial institutions address challenges across banking, payments, capital markets, wealth, and asset management.

Задачи

• Design, develop, and maintain scalable data pipelines and data processing solutions
• Develop data transformation and processing workflows using Python and Apache Spark
• Work with large-scale datasets in distributed environments using Hadoop and related technologies
• Build and support cloud-based data solutions on Google Cloud Platform (GCP)
• Develop reliable ingestion processes integrating data from multiple source systems
• Implement data transformations, validation rules, and data quality checks
• Troubleshoot data pipeline issues and support performance optimization
• Work with Linux-based environments, including scripting, deployment, and operational activities
• Collaborate with Data Engineers, Architects, Analysts, and other project stakeholders to translate business requirements into technical solutions
• Participate in code reviews and follow software engineering and data engineering best practices
• Create and maintain technical documentation covering data flows, dependencies, configurations, and operational procedures
• Support deployment, testing, stabilization, and ongoing maintenance of data solutions

Требования

• 2–4+ Years of commercial experience in Data Engineering or a similar role
• Good hands-on programming skills in Python
• Practical experience with Apache Spark, including building and maintaining data processing jobs
• Experience working with Hadoop or distributed data processing ecosystems
• Good knowledge of Linux and command-line environments
• Commercial experience with Google Cloud Platform (GCP) and relevant data services
• Good understanding of ETL/ELT processes, data pipelines, and data transformation concepts
• Working knowledge of SQL and relational data concepts
• Understanding of data quality, monitoring, and troubleshooting practices
• Familiarity with Git and modern software development practices
• Ability to work effectively in an Agile environment and collaborate with distributed teams
• Good communication skills
• English at a minimum B2 level
• Будет плюсом: Experience with GCP services such as BigQuery, Cloud Storage, Dataproc, Dataflow, or Pub/Sub, experience in the financial/banking domain, experience with orchestration tools such as Apache Airflow, familiarity with CI/CD processes for data solutions, knowledge of data modelling and data warehouse concepts, experience working with financial services or banking clients, familiarity with containerization technologies such as Docker or Kubernetes

Условия

• B2B contract
• Innovative AI and automation initiatives for leading financial institutions