data engineer for enterprise data platforms

Enfint; Deutschland; Hybrid; EUR 90.000 - 130.000; Vollzeit

Benefits dieser Stelle

Medical healthcare

In-office, hybrid, or remote options

Learning stipend

Well-being program

Team events

Zusammenfassung

Exadel is seeking an experienced Data Engineer to design and maintain scalable data pipelines on Google Cloud Platform, from Bronze to Gold layers and ERP integrations.

You will implement SQL-based ELT in BigQuery, build and optimize transformations, create fact and dimension tables with SCD, and develop Airflow DAGs (Cloud Composer) with CI/CD automation.

The role offers in-office, hybrid, or remote options, plus ongoing learning, top-tier equipment, and an international project environment.

Qualifikationen

• Extensive experience with Google BigQuery and GCP.
• Hands-on Apache Airflow, especially Cloud Composer.
• Experience with BigQuery, AlloyDB/PostgreSQL, and Teradata.
• Proven experience with SQL-based ELT, SFTP, and API integrations, including NetSuite and PSPs.
• Knowledge of data governance frameworks and OpenMetadata, including lineage and catalog.
• Understanding of IAM/RBAC.
• Strong CI/CD and DevOps practices using GitHub, Jenkins, Flyway, and Terraform.
• Experience building metadata-driven data quality frameworks with BigQuery and Airflow.
• Upper-Intermediate English.

Aufgaben

• Develop and maintain Bronze, Silver, and Gold data pipelines using SQL-based ELT patterns.
• Implement data ingestion from SFTP, APIs, and databases into BigQuery.
• Build and optimize transformation logic for the curated Silver Mart layer using dimensional modeling.
• Create fact and dimension tables, including SCD handling and surrogate key logic.
• Implement data quality checks and integrate them with pipeline execution.
• Support Airflow DAG development for pipeline orchestration, scheduling, and monitoring.
• Enable reprocessing and idempotency so pipelines can be reliably rerun from Bronze.
• Collaborate on schema evolution and deployments using CI/CD tools.
• Optimize BigQuery performance and cost through partitioning, clustering, and query tuning.
• Implement and maintain data integrations with downstream systems such as ERP and reporting tools.
• Contribute to metadata, lineage, and documentation in the data catalog.
• Support testing, reconciliation, and parallel-run validation during migration from legacy systems.
• Troubleshoot pipeline failures and support monitoring and alerting processes.

Kenntnisse

BigQuery

GCP

Apache Airflow

SQL-based ELT

CI/CD / DevOps

Data governance

OpenMetadata

IAM/RBAC

GitHub

Jenkins

Flyway

Terraform

NetSuite

ERP integrations

Open metadata

SCD modeling

Tools

Cloud Composer

GitHub

Jenkins

Flyway

Terraform

Jobbeschreibung

Описание

Exadel is an AI-first global technology company with over 25 years of engineering leadership, more than 2,000 team members, and over 500 active projects for Fortune 500 clients. It builds AI platforms and supports enterprise digital transformation.

Задачи

• Develop and maintain Bronze, Silver, and Gold data pipelines using SQL-based ELT patterns;
• Implement data ingestion from SFTP, APIs, and databases into BigQuery;
• Build and optimize transformation logic for the curated Silver Mart layer using dimensional modeling principles;
• Create fact and dimension tables, including SCD handling and surrogate key logic;
• Implement data quality checks and integrate them with pipeline execution;
• Support Airflow DAG development for pipeline orchestration, scheduling, and monitoring;
• Enable reprocessing and idempotency so pipelines can be reliably rerun from Bronze;
• Collaborate on schema evolution and deployments using CI/CD tools;
• Optimize BigQuery performance and cost through partitioning, clustering, and query tuning;
• Implement and maintain data integrations with downstream systems such as ERP and reporting tools;
• Contribute to metadata, lineage, and documentation in the data catalog;
• Support testing, reconciliation, and parallel-run validation during migration from legacy systems;
• Troubleshoot pipeline failures and support monitoring and alerting processes.

Требования

• Strong experience with Google BigQuery and GCP;
• Hands-on expertise with Apache Airflow, especially Cloud Composer;
• Experience with BigQuery, AlloyDB / PostgreSQL, and Teradata;
• Proven experience with SQL-based ELT, SFTP, and API integrations, including NetSuite and PSPs;
• Knowledge of data governance frameworks and OpenMetadata, including lineage and catalog;
• Understanding of IAM/RBAC;
• Strong understanding of CI/CD and DevOps practices using GitHub, Jenkins, Flyway, and Terraform;
• Experience building and maintaining metadata-driven data quality frameworks with BigQuery and Airflow;
• Upper-Intermediate English.

Условия

• In-office, hybrid, or remote flexibility;
• Medical healthcare;
• Recognition program;
• Ongoing learning and reimbursement;
• Well-being program;
• Team events and local benefits;
• Sports compensation;
• Referral bonuses;
• Top-tier equipment provision;
• International projects.