Data Engineer

CodiLime; Polska; Híbrido; PLN 150.000 - 240.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Flexible hours

Remote/hybrid work

Training budget

Hands-on onboarding

Collaborative environment

Project variety

Descripción de la vacante

CodiLime is seeking a data engineer to join the team behind a large-scale data platform. You will design, code, test, and operate production Python systems—pipelines, libraries, and services—that move and transform data at scale.

You will work with Snowflake, dbt, Airflow, and Spark on Azure Databricks, building reliable, maintainable software powering enterprise datasets. This role offers remote/hybrid options and opportunities to grow within a collaborative Europe-India distributed team.

Formación

• Proven experience building scalable data pipelines.
• Strong Python and SQL proficiency.
• Experience with cloud data platforms and data modeling.

Responsabilidades

• Design, code, test, and operate production Python data pipelines and services.
• Collaborate with Data Architecture Lead and engineers to maintain data quality.
• Build reliable, maintainable software powering a global data platform.

Conocimientos

Python

SQL

Data engineering

ETL pipelines

Herramientas

Snowflake

dbt

Airflow

Kubernetes

Azure Databricks

Pandas

Polars

PostgreSQL

Redis

FastAPI

Descripción del empleo

Get to know us better

CodiLime is a software and network engineering industry expert and the first-choice service partner for top global networking hardware providers, software providers and telecoms. We create proofs-of-concept, help our clients build new products, nurture existing ones and provide services in production environments. Our clients include both tech startups and big players in various industries and geographic locations (US, Japan, Israel, Europe).

While no longer a startup - we have 250+ people on board and have been operating since 2011 we’ll keep our people-oriented culture. Our values are simple:

• Act to deliver.
• Disrupt to grow.
• Team up to win.

The project and the team

You will join the team behind a large-scale, centralized data platform built for a global consulting organization. The platform is the shared source of company data behind several of the firm’s internal products, and is used daily by consultants for company research.

This is fundamentally a data engineering role combined with software engineering: you’ll be designing, coding, testing, and operating production Python systems - pipelines, libraries, and services - that move, transform, and serve data at scale. This is not a role focused on configuring tools or writing one-off queries. You’ll be building reliable, maintainable software that powers our data platform. The goal is a unified, enterprise-grade dataset of 300M+ company records, integrated from 10+ external and internal sources.

The platform delivers firm-level and site-level data - firmographics, technographics, and hierarchical relationships (parent company, subsidiary, site) - alongside key business metrics such as revenue, CAGR, EBITDA, headcount, M&A activity, competitors, industry classification, and web traffic. Data needs to stay accurate, well-structured, and fast to query as both the dataset and the number of consumers keep growing.

Technology stack:

• Languages: Python, SQL
• Data platform: Snowflake, dbt
• Workflow orchestration: Apache Airflow (complex DAGs), running on Kubernetes
• Data processing: Apache Spark on Azure Databricks
• Data tooling and DBs: pandas, Polars, PyArrow, DuckDB, PySpark, PostgreSQL, Redis
• Cloud: Azure (AKS, Blob Storage, ACR, Databricks, OpenSearch, Azure AI Search)
• API & services: FastAPI (REST, async), API Gateway
• Testing & code quality: pytest, mypy/pyright, ruff/black, sqlfluff, SonarQube
• Schema validation: Pydantic
• Dependency & environment management: uv, Poetry
• CI/CD & infrastructure: GitHub Actions, Docker, Kubernetes
• AI-Assisted Development: Cursor, Claude Code, ChatGPT Enterprise
• Future direction: agentic AI systems, LangChain, Azure OpenAI integration

What else you should know:

• Team: Data Architecture Lead, Data Engineers, DataOps Engineers, Backend Engineer, Product Owner, collaboration with Frontend Engineers and Data Science and AI Engineers
• Distributed team across Europe and India
• Agile, collaborative environment; given the platform's organization-wide impact, we’re looking for a mature, proactive, results-driven approach
• Code quality is enforced through testing, typing, and tooling - not just code review

We work on multiple interesting projects at a time, so it may happen that we’ll invite you to an interview for another project if we see that your competencies and profile are well suited for it.

More reasons to join us

• Flexible working hours and approach to work: fully remotely, in the office or hybrid
• Professional growth supported by internal training sessions and a training budget
• Solid onboarding with a hands-on approach to give you an easy start
• A great atmosphere among professionals who are passionate about their work
• The ability to change the project you work on