Data Engineer

Focal Consulting; Kuala Lumpur; On-site; MYR 120,000 - 180,000; Full time

Benefits offered by this job

13-Month Salary

Yearly Performance Bonus

Monthly Transport Allowance

Monthly Rental Subsidies

Job summary

Focal Consulting in Malaysia is seeking a Data Engineer to design, build, and scale a modern AWS data platform and robust ingestion pipelines.

You will implement real-time and batch pipelines using CDC with Debezium/Kafka, PySpark, and SQLMesh or DBT, while managing S3 and RDS and orchestrators like Airflow or Dagster.

The role emphasizes data modeling, quality, and continuous improvement, with 3–5+ years of experience and a strong engineering mindset.

Qualifications

• Bachelor’s or Master’s Degree in Computer Science, Engineering, or a related field.
• 3–5+ years of proven experience in Data Engineering or related roles.
• Strong proficiency in SQL, Python, PySpark, and AWS data pipelines.
• Experience with CDC concepts (Debezium/Kafka) and data transformation tools (SQLMesh, DBT).
• Familiarity with AWS RDS and data warehousing concepts.
• Knowledge of Kafka ecosystem, Airflow or Dagster, Docker & Kubernetes is a plus.
• Ownership-minded, independent, and able to balance fast delivery with scalable design.

Responsibilities

• Pipeline architecture and development for real-time and batch ingestion using CDC, PySpark, and SQLMesh/DBT.
• AWS platform management: build data platform components (S3, RDS) and orchestrate with Airflow or Dagster.
• Data modeling and warehousing: create raw/staged/curated layers and optimize storage and queries.
• Quality and reliability: implement validation, monitor pipelines, and troubleshoot production issues.
• Continuous improvement: evaluate new tech via POCs to improve scalability and cost efficiency.

Skills

SQL

Python

PySpark

CDC concepts

Data modeling

Data warehousing

AWS data pipelines

RDS

Education

Bachelor’s or Master’s Degree in Computer Science, Engineering, or a related field

Tools

SQLMesh

DBT

Airflow

Dagster

Docker

Kubernetes

Iceberg

Delta Lake

S3

MinIO

ClickHouse

Kafka ecosystem

Job description

Role Summary

Design, build, and scale a modern AWS data platform and robust data ingestion pipelines.

Roles & Responsibilities

• Pipeline Architecture & Development:Design and implement scalable real-time and batch data ingestion pipelines using CDC approaches, PySpark, and SQLMesh or DBT.
• AWS Platform Management:Build and manage AWS data platform components (S3, RDS) and implement orchestration and scheduling tools like Airflow or Dagster.
• Data Modeling & Warehousing:Develop structured data lake layers (raw/staged/curated) and optimize storage formats and query performance.
• Quality & Reliability:Implement robust data validation processes, monitor pipeline health, and systematically troubleshoot production issues.
• Continuous Improvement:Evaluate new technologies through POCs to enhance system architecture, scalability, and cost efficiency.

Requirements

• Education:Bachelor’s or Master’s Degree in Computer Science, Engineering, or a related field.
• Experience:3–5+ years of proven experience in Data Engineering or related roles.
• Technical Skills:Strong proficiency in SQL, Python, PySpark, transformation tools (SQLMesh, DBT), AWS data pipelines, CDC concepts (Debezium/Kafka), and AWS RDS.
• Core Knowledge:Deep understanding of data architecture, data modeling, warehousing concepts, and distributed systems fundamentals.
• Preferred Skills:Familiarity with the Kafka ecosystem, orchestration tools (Airflow, Dagster), open table formats (Iceberg, Delta Lake), Docker & Kubernetes, S3/MinIO, and ClickHouse.
• Candidate Profile:Ownership-driven with a strong engineering mindset, highly independent, and capable of balancing fast delivery with scalable system design.

Benefits

• 13-Month Salary:Guaranteed additional month of base salary.
• Yearly Performance Bonus:Rewarded annually based on individual and company achievements.
• Monthly Transport Allowance:Financial support to cover daily commuting expenses.
• Monthly Rental Subsidies:Monthly allowance provided to support your housing and accommodation needs.

Please note: Only shortlisted candidates will be notified.