Cloud Data Pipelines Engineer

Link Group; Warszawa; Hybrid; PLN 120,000 - 160,000; Full time

Job summary

Link Group seeks a data-focused software engineer with advanced Python and/or R skills to build and optimize cloud-native data pipelines. You will containerize workloads, orchestrate with Airflow/Prefect, and integrate with GCP or Azure data services to deliver scalable, reliable analytics solutions.

The role emphasizes strong SQL/data warehouse experience, CI/CD with GitHub Actions, and solid fundamentals in version control, testing, and API design. Immediate start in Warsaw, Poland.

Qualifications

• Advanced proficiency in Python and/or R, with strong experience writing clean, modular, and well-tested code.
• Cloud Platforms: Hands-on production experience with Google Cloud Platform (GCP) or Microsoft Azure.
• Containerization & CI/CD: Practical experience with Docker, Kubernetes, and CI/CD pipelines (GitHub Actions, GitLab CI).
• Data Stores: Proficiency with SQL and relational/non-relational data warehouses (PostgreSQL, BigQuery, Snowflake, or Azure SQL).
• Software Engineering Foundations: Solid grasp of version control (Git), automated testing practices, and API design principles.
• R/Python Workload Execution: Containerize, deploy, and orchestrate custom R and Python scripts to run reliably in cloud-native serverless or containerized environments.
• Cloud Integration: Leverage managed cloud services on GCP (Cloud Functions, Cloud Run, BigQuery, Dataflow) or Azure (Azure Data Factory, Azure Functions, Databricks, Synapse Analytics).
• Workflow Orchestration: Implement and maintain workflow automation tools (such as Apache Airflow or Prefect) to schedule and monitor script execution.
• Data Quality & Reliability: Implement automated logging, error handling, and testing frameworks to validate data integrity and script outputs.
• Performance Optimization: Tune data processing jobs and cloud resource allocation for cost-efficiency, execution speed, and scalability.
• Start ASAP

Skills

Python

R

SQL

English

Tools

GCP

Docker

GitHub Actions

Git

Kubernetes

PostgreSQL

BigQuery

Snowflake

Job description

• Python
• R
• GCP
• Docker
• GitHub Actions
• SQL
• Data warehouses
• BigQuery
• Snowflake
• Git
• Automated testing
• API
• Angielski
• Programming: Advanced proficiency in Python and/or R , with strong experience writing clean, modular, and well-tested code.
• Cloud Platforms: Hands-on production experience with either Google Cloud Platform (GCP) or Microsoft Azure .
• Containerization & CI/CD: Practical experience with Docker , Kubernetes , and CI/CD pipelines (GitHub Actions, GitLab CI).
• Data Stores: Proficiency with SQL and relational/non-relational data warehouses (PostgreSQL, BigQuery, Snowflake, or Azure SQL).
• Software Engineering Foundations: Solid grasp of version control (Git), automated testing practices, and API design principles.
• R/Python Workload Execution: Containerize, deploy, and orchestrate custom R and Python scripts to run reliably in cloud-native serverless or containerized environments.
• Cloud Integration: Leverage managed cloud services on GCP (Cloud Functions, Cloud Run, BigQuery, Dataflow) or Azure (Azure Data Factory, Azure Functions, Databricks, Synapse Analytics).
• Workflow Orchestration: Implement and maintain workflow automation tools (such as Apache Airflow or Prefect) to schedule and monitor script execution.
• Data Quality & Reliability: Implement automated logging, error handling, and testing frameworks to validate data integrity and script outputs.
• Performance Optimization: Tune data processing jobs and cloud resource allocation for cost-efficiency, execution speed, and scalability.
• Start ASAP