Data Engineer

RBC Shared Services Malaysia Sdn Bhd; Putrajaya; On-site; MYR 120,000 - 200,000; Full time

Job summary

RBC Shared Services Malaysia Sdn Bhd is hiring a Data Engineer to join the Operational Data, Information, and AI team. You will design, develop, and optimize cloud data Lakehouse, building scalable ELT pipelines with Python, SQL, Snowflake, and AWS.

You will also implement dbt transformations, data governance, and CI/CD for data workflows, collaborating with product owners and data scientists to deliver analytics and AI-focused data products across the enterprise.

Qualifications

• 3+ years of hands-on data engineering experience.
• Strong technical proficiency in Hadoop ecosystem (HDFS, YARN, Hive, Spark).
• SQL: Advanced query writing, optimization, indexing, stored procedures.
• Python: Production-quality code for data pipelines, automation, and scripting.
• AWS: S3, Glue, Lambda, Redshift.
• Snowflake: Data warehousing, virtual warehouses, clustering, security.
• Proven track record of delivering production data solutions in enterprise environments.
• Strong understanding of data modeling (dimensional, normalized, Data Vault).
• Experience with data pipeline orchestration tools (Airflow, Dagster, Luigi).
• Proficiency with Git version control and collaborative development workflows.
• Experience with containerization (Docker) and orchestration (Kubernetes).
• Demonstrated ability to work independently, take ownership, and drive projects to completion.
• Excellent problem-solving skills and attention to detail.

Responsibilities

• Design, develop, optimize, and maintain robust ELT pipelines using Python, SQL, Snowflake, Airflow, and Spark to process large-scale datasets.
• Develop data transformations using dbt for analytics workflows, ensuring modularity, testability, and documentation.
• Build batch and real-time data ingestion frameworks integrating with Snowflake and AWS services.
• Work with data product owners to develop analytical, AI/ML, and GenAI focused data products for enterprise usage.
• Write complex SQL for data modeling, aggregation, and performance optimization across Snowflake and relational databases.
• Develop streaming data solutions using Kafka, Kinesis, or similar technologies.
• Data quality and governance: Implement data validation, data quality checks, and governance practices to ensure data integrity and compliance with enterprise policy and data contracts.
• Build and maintain CI/CD pipelines for data engineering workflows using enterprise approved tools
• Containerize data applications using Docker and orchestrate with Kubernetes or AWS ECS
• Establish monitoring, logging, and alerting for production data pipelines
• Drive SRE practices including incident management, root cause analysis, and continuous improvement
• Document technical designs, data flows, and operational runbooks

Skills

Hadoop ecosystem

SQL

Python

AWS

Snowflake

Data modeling

Airflow

Kubernetes

Git

Spark

Tools

Snowflake

dbt

Airflow

Dagster

Luigi

Kafka

Kinesis

Docker

Kubernetes

Git

Spark

Hadoop

Hive

HDFS

Redshift

S3

Glue

Job description

We are seeking a talented Data Engineer to join the Operational Data, Information, and AI team in Personal and Commercial Operations. The candidate will play a critical role in the design, implementation, and optimization of our cloud data Lakehouse. You'll work with cutting-edge technologies to design, develop, and optimize scalable data pipelines that handle large volumes of structured and unstructured data and serve as the data foundation for transformational Business Intelligence, Machine Learning, and AI initiatives.

This role offers:

Technical depth: Hands-on work with cutting-edge data technologies (Snowflake, dbt, Iceberg, AWS)

Visibility: Contribute to strategic, enterprise-wide data initiatives

Innovation: Implement modern data engineering patterns including Lakehouse architecture and analytics engineering

WHAT WILL YOU DO?

Hands-On Technical Delivery

Design, develop, optimize, and maintain robust ELT pipelines using Python, SQL, Snowflake, Airflow, and Spark to process large-scale datasets Develop and optimize data transformations using dbt for analytics workflows, ensuring modularity, testability, and documentation

Build batch and real-time data ingestion frameworks integrating with Snowflake and AWS services

Work with data product owners to develop analytical, AI/ML, and GenAI focused data products for enterprise usage.

Write complex SQL for data modeling, aggregation, and performance optimization across Snowflake and relational databases.

Develop streaming data solutions using Kafka, Kinesis, or similar technologies.

Data quality and governance: Implement data validation, data quality checks, and governance practices to ensure data integrity and compliance with enterprise policy and data contracts.

Build and maintain CI/CD pipelines for data engineering workflows using enterprise approved tools

Containerize data applications using Docker and orchestrate with Kubernetes or AWS ECS

Establish monitoring, logging, and alerting for production data pipelines

Drive SRE practices including incident management, root cause analysis, and continuous improvement

Collaboration & Stakeholder Engagement

Work closely with product owners, data scientists, analysts, and business stakeholders to understand requirements and deliver solutions

Partner with enterprise architects and platform teams to ensure alignment with technology standards

Provide technical estimates, capacity planning inputs, and risk assessments for project initiatives

Document technical designs, data flows, and operational runbooks

WHAT DO YOU NEED TO SUCCEED?

Must-Have:

3+ years of hands-on data engineering experience

Strong technical proficiency in:

Hadoop ecosystem: HDFS, YARN, Hive, Spark

SQL: Advanced query writing, optimization, indexing, stored procedures

Python: Production-quality code for data pipelines, automation, and scripting

AWS: S3, Glue, Lambda, Redshift,

Snowflake: Data warehousing, virtual warehouses, clustering, security

Proven track record of delivering production data solutions in enterprise environments

Strong understanding of data modeling (dimensional, normalized, Data Vault)

Experience with data pipeline orchestration tools (Airflow, Dagster, Luigi)

Proficiency with Git version control and collaborative development workflows

Experience with containerization (Docker) and orchestration (Kubernetes)

Demonstrated ability to work independently, take ownership, and drive projects to completion

Excellent problem-solving skills and attention to detail

Nice-to-Have:

Experience with NoSQL databases (MongoDB, DynamoDB, Cassandra)

Knowledge of data streaming platforms (Apache Kafka, AWS Kinesis)

Exposure to data governance and metadata management tools

Experience with data quality frameworks and observability platforms (Great Expectations, Monte Carlo, Datadog)

Familiarity with machine learning pipelines and MLOps

Background in financial services or highly regulated industries

AWS, Snowflake, or dbt certifications

Experience with Polars or Pandas for data manipulation

WHAT'S IN IT FOR YOU?

We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, and stock where applicable

Leaders who support your development through coaching and learning opportunities

Ability to make a difference and lasting impact

Work in a dynamic, collaborative, progressive, and high-performing team

World-class training program in financial services and technology

Flexible work/life balance options

Opportunities to do challenging, meaningful work

Opportunities to take on progressively greater accountabilities

Access to cutting-edge technologies and modern data stack

Working hours: 1.00pm - 10.00pm MYT

You live a full life shaped by your ideas. By your family. Career. Community. Your unique place in the world. You want an employer that helps you realize your vision for life with a meaningful career and supports your desire for more.

At RBC, we get it. We’re committed to creating an environment that will help people realize their full potential. Whether it’s developing new skills, opportunities to innovate and grow, modern and comprehensive benefits, or the flexibility to enjoy the moments that matter, RBC is a great place to build your career and life.

It’s your career. Your life. Where do you want to go?

RBC. Make It Yours

You live a full life shaped by your ideas. By your family. Career. Community. Your unique place in the world. You want an employer that helps you realize your vision for life with a meaningful career and supports your desire for more.

At RBC, we get it. We’re committed to creating an environment that will help people realize their full potential. Whether it’s developing new skills, opportunities to innovate and grow, modern and comprehensive benefits, or the flexibility to enjoy the moments that matter, RBC is a great place to build your career and life.

It’s your career. Your life. Where do you want to go?