Senior Data Engineer

Bayer; Województwo mazowieckie; Presencial; PLN 180.000 - 320.000; Jornada completa

Descripción de la vacante

Bayer poszukuje Senior Data Engineer w zespole Data & AI, koncentrującego się na tworzeniu i eksploatacji potoków danych GxP-validated dla badań klinicznych w data lakehouse. Wymaga to silnych umiejętności programistycznych w Pythonie i PySpark, znajomości narzędzi Databricks oraz zakresu bezpieczeństwa danych.

Do kluczowych zadań należy projektowanie, implementacja i operowanie niezawodnych potoków danych, współpraca z interesariuszami w metodyce Agile oraz promowanie dobrych praktyk inżynierii

Formación

• Wymagane tytuł licencjata z kierunków IT/np. informatyka lub zbliżony.
• Minimum 5 lat doświadczenia w data engineering i operacjach danych.
• Wysoki standard kodu, dokumentacja i metadane.
• Doświadczenie z Databricks i zarządzaniem dostępem.

Responsabilidades

• Prowadzenie budowy i utrzymania potoków danych w Clinical Data Lakehouse.
• Współpraca z biznesem i IT w zwinnej organizacji.
• Projektowanie i implementacja wzorców integracyjnych i semantyki danych.
• Wdrażanie IaC i współpraca z zespołem infra.

Conocimientos

Databricks

DBX Asset Bundles

Lakeflow

Unity Catalog

Governance model

Databricks CLI

API design

OpenSharing/Delta Sharing

Iceberg-Sharing

ODBC/GraphQL/OData/REST

Event streaming

Python / PySpark

Kapitan

Git / GitHub Actions / CI/CD

Data security practices

Analytical & communication skills

Educación

Bachelor’s degree in computer science, data science, information technology, or related field

Herramientas

Databricks

Terraform

AWS

Descripción del empleo

O projekcie:

Senior Data Engineer (CDL)

As a Senior Data Engineer, you are part of Bayer's Data & AI Team with a focus on creating data solutions that provide timely, high-quality data to scientists, analysts, and AI agents. You will closely collaborate with experts from business and IT to design, implement and operate reliable GxP-validated data pipelines that enable timely and secure access to high-quality data for clinical studies in our Clinical Data Lakehouse, combining a solid understanding for the clinical data flow with excellent technical and conceptual skills.

Wymagania:

• Bachelor’s degree in computer science, data science, information technology, or a related field.
• 5+ years of experience in data engineering, data operations, or a similar role, delivering production-grade data pipelines and services
• Strong ownership of deliverables, high standards for code review and peer mentorship, and a commitment to clear documentation, metadata/lineage publishing, and sustainable data engineering standards and practices
• Proficiency with Databricks, including DBX Asset Bundles, Lakeflow Jobs, Unity Catalog, Access control and governance model and Databricks CLI
• Experience with API design patterns and Ability to integrate data from source systems using various interface patterns, such as OpenSharing/Delta Sharing, Iceberg-Sharing, ODBC, GraphQL, OData, REST, or event streaming
• Strong engineering skills (data engineering, software engineering, or cloud engineering)
• Proficiency in Python programming, especially PySpark and Kapitan
• Experience in Git, including Github Actions, CI/CD Pipelines (declarative YAML), Branching strategies, and trunk-based development
• Understanding of data security principles and practices
• Strong analytical and communication skills
• Good communication and collaboration abilities to work independently in a self-organized team
• High level of accuracy and attention to detail

Preferred

• Experience with GxP validation and Computerized Systems Validation
• Experience in Requirements Engineering
• Hands-on experience with Infrastructure-as-code, ideally Terraform on AWS
• In-depth knowledge of FAIR Data Principles, Linked Data principles, and Data Mesh architecture concepts, with demonstrated ability to apply them in practice
• Experience with semantic layer concepts in modern data platforms such as Databricks, Snowflake, or Microsoft Fabric
• Experience designing and implementing Model Context Protocol (MCP) servers
• Hands-on experience with GraphRAG solution design and implementation
• Knowledge of data privacy regulations (e.g., GDPR, CCPA)
• Cloud engineering experience (e.g. AWS, Azure)
• Knowledge in the pharmaceutical domain (e.g. LOINC, SNOMED CT, omics, targets, assays, clinical studies, RWE, etc.)

Codzienne zadania:

• Lead the development and operation of reliable data pipelines to store, transform, share and analyze clinical data, enabling an efficient clinical dataflow in compliance to legal and regulatory requirements.
• Collaborate closely with relevant stakeholders in an agile setup to capture, specify and address data-related requirements and dependencies to ensure a fit-for-purpose technical implementation in a healthy data ecosystem.
• Actively exchange with peers, embracing and applying best-practices to maintain a coherent, consolidated and scalable data landscape.
• Contribute to the design and implementation of integration patterns, FAIR data principles and semantics to enable efficient, compliant, and secure data consumption by humans, systems, and AI agents within and beyond the primary data context.
• Collaborate with infrastructure experts to ensure a robust, efficient, secure and fit-for-purpose setup, leveraging infrastructure-as-code and composable architecture.
• Mentor less-experienced collagues and uphold high standards in requirements specification, engineering practices, code review, testing, and documentation