Senior Data Engineer - RealTime Pipelines & Data Governance

Rigby AG; Zürich; On-site; CHF 140,000 - 190,000; Full time

Job summary

Rigby AG in Zürich, Switzerland seeks an experienced senior data engineer to design, build, and operate real-time data pipelines and backend services for decisioning and reporting. You will model data, ensure governance, and deliver secure, scalable solutions using Python/Scala, SQL, and cloud-native technologies, including Databricks, Azure, Kafka, and Kubernetes.

Join a team focused on data quality, security, and robust pipelines across structured and unstructured data, leveraging modern data

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Analytics, Information Systems, or a related technical field; or equivalent training and professional experience.
• 5+ years of experience in ETL/ELT, data warehousing, BI, and integrating data processing/workflow tools into pipeline design.
• 5+ years building and maintaining end-to-end data systems using Python, Scala, or similar languages.
• Strong SQL skills for data analysis and root-cause investigation.
• Experience with Azure cloud data technologies in CI/CD environments.
• Experience with Databricks, PySpark, Spark Streaming, and Delta Lake.
• Hands-on with structured, semi-structured, and unstructured data across RDBMS, data warehouses, in-memory caches, and document stores.
• Familiarity with Azure Data Lake and Blob Storage.
• Solid data modelling knowledge (Data Vault a plus) and strong design/testing skills.
• Experience with event-driven architectures (Kafka, Event Hubs, Flink) and containerised microservices (Kubernetes, Docker, Helm Charts).
• Practical use of dbt and BI tools; strong communication and technical leadership.

Responsibilities

• Design, develop, and maintain data pipelines and backend services for real-time decisioning, reporting, data collecting, and related functions.
• Data Requirements and Modeling; Data Management and Transformation.
• Produce high-quality, well-tested, and secure code.
• Develop and maintain software to improve data governance and security.
• Develop processes to ensure Data Security and Data Quality.

Skills

ETL/ELT

Data Warehousing

Business Intelligence

Python

Scala

SQL

Azure Cloud

CI/CD

Data Governance

Data Modelling

Data Vault

Event-driven Architectures

Kafka

Azure Data Lake

Databricks

PySpark

Spark Streaming

Delta Lake

Kubernetes

Docker

dbt

BI Tools

Education

Bachelor’s or Master’s degree in Computer Science, Data Analytics, Information Systems, or a related technical field; or equivalent training and professional experience

Tools

Databricks

PySpark

Spark Streaming

Delta Lake

Azure

CI/CD Tools

Kafka

Event Hubs

Apache Flink

Kubernetes

Docker

Helm Charts

dbt

BI Tools

Job description

Rigby AG in Zürich, Switzerland seeks an experienced senior data engineer to design, build, and operate real-time data pipelines and backend services for decisioning and reporting. You will model data, ensure governance, and deliver secure, scalable solutions using Python/Scala, SQL, and cloud-native technologies, including Databricks, Azure, Kafka, and Kubernetes.

Join a team focused on data quality, security, and robust pipelines across structured and unstructured data, leveraging modern data