Data Engineer – Hybrid Cloud Pipelines & Spark

Experian Asia Pacific

Kuala Lumpur

Hybrid

MYR 180,000 - 280,000

Full time

11 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Hybrid work arrangement: home and офис

Inclusive paid leave

People-first and inclusive workplace

Paid Internship

Job summary

Experian Asia Pacific is seeking a talented data engineer to join the Ascend Sandbox team, a cloud-based analytics platform that enables clients and data scientists to explore, model, and derive insights from data in a secure environment.

You will design, build, and scale batch (and some streaming) data pipelines, leveraging Python, PySpark, SQL, and AWS data services to deliver trusted datasets for analytics and downstream applications across the APAC region.

Qualifications

Degree in Computer Science, Data Engineering or equivalent.
At least 6 years of data engineering or data-pipelines experience.
Strong Python and PySpark for large-scale processing.
Strong SQL skills with complex transformations and performance tuning.
Experience designing and operating scalable ETL/ELT pipelines.
Experience with AWS data services (S3, Glue, Athena, Redshift, EMR).
Knowledge of data lake/lakehouse concepts and governance.

Responsibilities

Design, build, test, deploy and operate scalable data pipelines for Ascend Sandbox.
Develop Python/PySpark/SQL solutions for large datasets with accuracy and efficiency.
Design data models and reusable data products for analytics and downstream apps.
Optimize AWS-based data solutions (S3, Glue, Athena, Redshift, EMR) for cost and performance.
Implement data governance, lineage, and security across pipelines.
Collaborate with clients and data scientists to derive scalable data-processing solutions.
Investigate and fix production data issues and improve reliability.