Data Engineer - Zurich Asuransi Indonesia

Zurich 56 Company Ltd; Daerah Khusus Ibukota Jakarta; On-site; IDR 408.763.000 - 654.023.000; Full time

Job summary

A leading data engineering firm in Indonesia is looking for a Data Engineer to design and implement ETL/ELT pipelines using Spark SQL and Python. The role requires strong proficiency in dimensional modeling and experience with automated testing and CI/CD pipelines. Join us as we innovate in data solutions and support data-driven decision-making.

Qualifications

• 3+ years’ experience with SparkSQL, Python, and PySpark for data engineering workflow.
• Strong proficiency in dimensional modeling and star schema design.
• Experience implementing automated testing and CI/CD pipelines.

Responsibilities

• Design and implement ETL/ELT pipelines using Spark SQL and Python.
• Develop dimensional data models following star schema methodology.
• Optimize Spark SQL and DataFrame operations.

Skills

SparkSQL

Python

PySpark

Dimensional modeling

Automated testing

CI/CD pipelines

GitHub operations

Cloud data services

IDE tools

Databricks

Job description

Job Summary

Responsible for identification, assessment, and design of specific data engineering solutions, infrastructure, and systems to support data-driven decision-making and analysis. They enable the organization to effectively manage, process, and analyze data, thereby supporting data-driven decision-making and unlocking valuable insights from data.

Key Requirements

• 3+ years’ experience with SparkSQL, Python, and PySpark for data engineering workflow
• Strong proficiency in dimensional modeling and star schema design for analytical workloads
• Experience implementing automated testing and CI/CD pipelines for data workflows
• Familiarity with GitHub operations and collaborative development practices
• Ability to optimize engineering workflow jobs for performance and cost efficiency
• Experience with cloud data services and infrastructure (AWS, Azure, or GCP)
• Proficiency with IDE tools such as Visual Studio Code
• Experience with Databricks platform is a plus

Key Accountabilities

Designs, develops, and validates data processes. Develops data pipelines and supports their implementation, ensuring data solutions align with business objectives. Looks ahead to understand future technology options for the business. Serves as a technical expert in specific processes or product areas, conducting process reviews and initiating changes to contribute to continuous improvement of services, efficiency, and quality. Researches external primary data sources, selects relevant information, evaluates key technology themes, and makes recommendations to inform policy and/or product development in the IT area.

Key Responsibilities

• Design and implement ETL/ELT pipelines using Spark SQL and Python within Databricks Medallion architecture
• Develop dimensional data models following star schema methodology with proper fact and dimension table design, SCD implementation, and optimization for analytical workloads
• Optimize Spark SQL and DataFrame operations through partitioning strategies, clustering, and join optimizations to maximize performance and minimize costs
• Build comprehensive data quality frameworks with automated validation checks, statistical profiling, exception handling, and data reconciliation processes
• Establish CI/CD pipelines incorporating version control and automated testing (unit, integration, smoke tests, etc.)
• Implement data governance standards, including row-level and column-level security policies for access control and compliance
• Create and maintain technical documentation such as ERDs, schema specifications, data lineage diagrams, and metadata repositories

At Zurich, we like to think outside the box and challenge the status quo. We focus on the positives and ask, "What can go right?"

We are an equal opportunity employer who values the uniqueness of each employee—it's what makes our team great!
Join us as we explore new ways to protect our customers and the planet.

• Location(s): ID - Head Office - MT Haryono