[Expression of Interest] Data Engineer

Experian; Kuala Lumpur; Hybrid; MYR 180,000 - 240,000; Full time

Benefits offered by this job

Hybrid work

Inclusive paid leave

People-first workplace

Paid Internship

Job summary

Experian Malaysia is seeking a talented Data Engineer to join the Ascend Sandbox team, helping scale a cloud-based analytical platform across the APAC region. You will build and optimize data pipelines, model data products for analytics, and collaborate with internal and client teams to deliver reliable, governed data processing.

The role emphasizes Python and PySpark development, SQL proficiency, and AWS services.

Qualifications

• Degree in Computer Science, Data Engineering or equivalent.
• 6+ years of data engineering experience with data pipelines.
• Proficient in Python and PySpark for large-scale processing.
• Strong SQL skills and performance tuning.
• Experience with AWS data services and data governance is a plus.

Responsibilities

• Design, build, test, deploy and operate scalable data pipelines (batch and streaming).
• Develop Python, PySpark, and SQL solutions for large datasets.
• Design data models and reusable datasets for analytics and data science.
• Optimize AWS-based data solutions (S3, Glue, Athena, Redshift, EMR).
• Implement data governance, metadata, lineage and security requirements.
• Collaborate with clients and teams to translate requirements into solutions.

Skills

Python

PySpark

SQL

AWS

Airflow

Data Pipelines

Education

Bachelor's degree in CS/Data Eng

Tools

AWS

S3

Glue

Redshift

EMR

MWAA

Job description

• Schedule: Full Time

Company Description

Experian is a global data and technology company, powering opportunities for people and businesses around the world. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more. Experian invests in people and new advanced technologies to unlock the power of data. We have an amazing team of 25,200 people in 32 countries.

Our uniqueness is that we celebrate yours. Experian's people first, inclusive and purpose-driven culture is multi award-winning; World's Best Workplaces 2025 (Fortune Global Top 25), Great Place To Work 26 countries to name a few. Check out Experian Life on social or explore our Careers Site to understand why. Experian is also proud to be an Equal Opportunity and Aff…

• Hybrid work arrangement: home and office
• Inclusive paid leave
• People-first and inclusive workplace
• Paid Internship

#LI-Hybrid

Job Description

Role Summary

We are looking for a talented and driven engineer to join the Ascend Sandbox team - Experian's cloud-based analytical platform that empowers clients and data scientists to explore, model, and derive insights from data in a secure, governed environment. You will play a key role in building, maintaining, and scaling the platform that serves both internal teams and external clients across the APAC region.

What you’ll be doing

• Design, build, test, deploy and operate scalable batch and, where applicable, streaming data pipelines for the Ascend Sandbox analytical platform.
• Develop high-quality Python, PySpark and SQL solutions to ingest, transform and process large-scale datasets with accuracy, reliability and efficiency.
• Design data models and reusable, trusted datasets/data products for analytics, data science and downstream applications.
• Engineer and optimise AWS-based data solutions using relevant services such as Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift and Amazon EMR, according to the platform architecture.
• Apply data lake and lakehouse engineering practices, including efficient storage formats, partitioning, schema evolution and storage/query optimisation.
• Design robust ETL/ELT patterns, including incremental processing, dependency management, idempotency, retries, backfills and failure recovery.
• Implement workflow orchestration and scheduling for data pipelines using the platform's approved tooling, such as Airflow/MWAA, AWS Step Functions or AWS Glue Workflows.
• Build automated data quality controls and monitoring for completeness, validity, reconciliation, schema compliance, freshness, pipeline health and processing SLAs/SLOs.
• Optimise Spark workloads and distributed data processing for performance, scalability and cost, including partitioning, joins, shuffles, skew and resource utilisation.
• Implement practical data governance, metadata, lineage, cataloguing, security and access-control requirements across data pipelines and datasets.
• Collaborate with clients, business users, data scientists and technical teams to translate business and data requirements into scalable data‑processing solutions and integrations.
• Investigate and resolve production data and pipeline issues, perform root-cause analysis, and implement preventive improvements to reliability and operational efficiency.
• Ensure data engineering solutions comply with established architecture, coding, testing, security and quality standards, and contribute to code reviews, CI/CD and automated deployment practices.

Qualifications

• Must possess a Degree / Advanced Diploma in Computer Science, Data Engineering, Software Engineering or equivalent. Seniority will be commensurate with relevant experience and qualifications.
• At least 6 years of experience in data engineering or developing data-intensive systems, including the design, build and production support of data pipelines.
• Strong hands-on experience with Python and PySpark for large-scale and distributed data processing.
• Strong SQL skills, including complex transformations, joins, window functions, query optimisation and performance tuning.
• Hands-on experience designing and operating scalable ETL/ELT data pipelines, including incremental loads, retries, backfills, dependency management and failure recovery.
• Experience with data modelling for analytical use cases, including dimensional modelling, facts/dimensions, normalization/denormalization and handling historical data.
• Hands-on experience with AWS data services relevant to data engineering, such as Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift and Amazon EMR; exposure to Lambda, Kinesis/MSK, DMS or Lake Formation is an advantage where relevant.
• Experience with data lake/lakehouse concepts, storage layout, partitioning, schema evolution and efficient data formats such as Parquet; familiarity with JSON, CSV and Avro is useful.
• Strong understanding of Spark performance engineering, including partitioning, shuffles, joins, data skew, parallelism, memory/resource optimisation and troubleshooting.
• Experience with pipeline orchestration and workflow scheduling using tools such as Airflow/MWAA, AWS Step Functions, AWS Glue Workflows or equivalent.
• Experience implementing automated data quality checks and data observability, including monitoring pipeline health, data freshness, volumes, failures and quality anomalies.
• Practical knowledge of data governance, metadata management, data cataloguing, lineage and schema management.
• Practical understanding of AWS data security, including IAM/least privilege, encryption/KMS, secrets management, S3 security and data-access controls.
• Familiarity with Infrastructure as Code such as Terraform, AWS CloudFormation or AWS CDK is an advantage.
• Understanding of cloud cost and performance optimisation for storage, compute, Spark workloads and query patterns.
• Experience with Git-based workflows, branching strategies, code reviews, CI/CD pipelines and automated deployment. Familiarity with Jira, Bitbucket, Jenkins and Checkmarx is an advantage.
• Experience writing automated unit, integration and data-pipeline tests for Python/PySpark transformations and data quality.
• Strong testing, debugging, troubleshooting and root-cause analysis skills in production data environments.
• Good understanding of Agile, SCRUM, SDLC and software engineering practices.
• Strong communication and stakeholder management skills with the ability to engage clients, business users, data scientists and technical teams.
• Strong analytical thinking, problem‑solving, teamwork and willingness to learn, with the ability to work independently when required and deliver under tight deadlines.
• Familiarity with AI tools for coding assistance, documentation or automation is a plus.

Additional Information

Recruitment Fraud Awareness - Experian's recruitment process is conducted only through authorised channels. Recruitment communications will only be sent from an @ experian.com email address. Experian will never ask candidates to make any payment as part of an application, interview, assessment, onboarding, or recruitment process.

Experian Careers - Creating a better tomorrow together

Experian Careers - Creating a better tomorrow together

By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply