Data Engineer

Andersen; Warszawa; Hybrid; PLN 181,000 - 336,000; Full time

Benefits offered by this job

Private health insurance

Sports compensation

Referral program

AWS/PMP certification funding

Mentoring and onboarding program

Hybrid/remote work options

Job summary

Andersen is looking for a Data Engineer to modernize a cloud-based quality intelligence platform and build scalable data solutions for analytics, security, and AI capabilities. The role involves handling batch pipelines, data ingestion from multiple sources, and shaping data for ML/analytics in Redshift.

You will own end-to-end data workflows, mentor junior engineers, and contribute to a hybrid work model with strong emphasis on reliability and scalability.

Qualifications

• 4+ years of end-to-end ELT/ETL data pipeline experience in cloud data warehouses.
• Expert-level SQL skills and production Redshift experience.
• Experience with incremental loads, CDC, SCD2, and near-real-time processing.
• Experience migrating legacy ETL to cloud platforms with minimal disruption.
• Experience with MySQL sources (incl. MariaDB).
• Proficient in AWS S3 and IAM for data landing and access control.
• Familiar with Airflow and dbt.

Responsibilities

• Monitor batch pipelines and debug failures in production.
• Trace failures across bronze/silver/gold layers and validate in a test env.
• Set up failure alerting for batch runs.
• Progress toward near-real-time data refresh with incremental loads.
• Ingest new data sources: connect, land in S3, define Redshift schema, grant IAM access.
• Expose data to ML engineers via views/materialized views.
• Review PRs and mentor junior engineers, ensuring performance budgets.
• Onboard quickly to align with platform architecture and philosophy.

Skills

SQL

Airflow

dbt

Redshift

English proficiency

Tools

AWS S3

IAM

Snowflake

SQL Server

Job description

Andersen is hiring a Data Engineer for a project modernizing a cloud-based quality platform and building scalable data solutions to support analytics, security, and AI capabilities. The customer is a technology company developing scalable software solutions for organizations across multiple industries. Its products help businesses improve operational efficiency, optimize workflows, and support digital transformation through modern technologies and data-driven approaches. The company focuses on delivering reliable, user-oriented solutions that enable customers to respond effectively to changing business and technology requirements.

The project is focused on modernizing a cloud-based quality intelligence platform by enhancing its API ecosystem, security, data architecture, and AI capabilities. It includes implementing secure public API access, reengineering the data warehouse, improving automated testing and CI processes, and building an agent orchestration platform while supporting SOC 2 compliance.

Responsibilities:

• Monitoring and troubleshooting the batch pipelines day to day — the panel named this as the first responsibility: check every run, and when something fails, debug it ("not really glamorous work at first").
• Tracing failures through the medallion layers (bronze/silver/gold), distinguishing source-side changes from pipeline bugs, fixing what's understandable, and validating in a test environment before production.
• Setting up failure alerting — e.g. sensor notifications into a teams channel showing whether a batch succeeded or failed, with the error surfaced.
• Working toward making the refresh cadence of the data more frequent (near-real-time) through incremental-load strategies — named by the panel as a goal the role will grow into.
• Handling requests to ingest new data and add new sources independently: setting up connections and credentials, landing data in S3, deciding on the Redshift schema and target tables, and granting IAM access to downstream consumers.
• Extracting from the platform's sources (Postgres, MySQL, Google Sheets, Salesforce, HR software) via Glue, load into Redshift, and do transformations in dbt inside the warehouse, feeding the published layer.
• Provisioning data to ML engineers and analysts on request (creating views / materialized views, managing access), shaping it around what they need to see.
• Reviewing PRs and mentoring junior engineers — checking naming/readability, reuse, and that added functionality doesn't blow the ETL's runtime budget.
• Onboarding well enough to review PRs and troubleshooting in a way that matches the platform's overall architecture and modern philosophy — not just patch the error.

Must-haves:

• Commercial experience in Data Engineering, owning end-to-end ELT/ETL pipelines and cloud data-warehouse architecture for 4+ years.
• Expert-level SQL skills.
• Production experience with Amazon Redshift, including schema design, distribution/sort keys, WLM, query tuning, and reducing refresh latency on a live warehouse.
• Strong MPP data warehouse experience; Snowflake and/or Azure Synapse SQL Pool experience is valued but does not fully substitute for Redshift experience.
• Experience with incremental loads and CDC, including watermarks, SCD2, change-data-capture, and moving from full rebuilds toward near-real-time processing.
• Experience re-architecting legacy ETL pipelines and data warehouses, including migration from script-based or on-prem legacy systems to scalable cloud platforms with minimal disruption.
• Experience using MySQL as a source system, including incremental extraction from relational OLTP sources; MariaDB familiarity.
• Experience with AWS S3 and IAM for data landing and access management.
• Production experience with Airflow or equivalent orchestration tools and dbt.
• Ability to work fully independently as the sole owner of a platform, including identifying and prioritizing technical debt without formal ticket tracking.
• Strong monitoring and failure recovery practices, including checking pipeline runs, debugging failures, and maintaining a healthy batch platform.
• Level of English – from Upper-Intermediate and above.

Nice-to-haves:

• CDC tooling (e.g. Debezium / binlog-based) in production.
• Mentoring junior engineers; PR-review rigor (naming, readability, reuse, runtime budgets).

Reasons why this job would be interesting to you:

• Experience in teamwork with leaders in FinTech, Healthcare, Retail, Telecom, and others. Andersen cooperates with such businesses as Samsung, Siemens, Johnson & Johnson, BNP Paribas, Ryanair, Mercedes, TUI, Verivox, Allianz, T-Systems, etc..
• The opportunity to change the project and/or develop expertise in an interesting business domain.
• Job conditions – you can work both fully remotely and from the office or can choose a hybrid variant.
• Guarantee of professional, financial, and career growth! The company has introduced systems of mentoring and adaptation for each new employee.
• The opportunity to earn up to an additional 1,000 EUR per month, depending on the level of expertise, which will be included in the annual bonus, by participating in the company's activities.
• Access to the corporate training portal, where the entire knowledge base of the company is collected and which is constantly updated.
• Bright corporate life (parties / pizza days / PlayStation / fruits / coffee / snacks / movies).
• Certification compensation (AWS, PMP, etc).
• Referral program.
• Private health insurance and sports compensation, depending on the type of employment.

Your personal data is protected in accordance with GDPR regulations. Learn more: https://andersenlab.com/privacy-policy/pl