[Expression of Interest] Data Engineer

Experian; Petaling Jaya; Hybrid; MYR 100,000 - 201,000; Full time

Benefits offered by this job

Hybrid work arrangement: home and offi

Inclusive paid leave

People-first and inclusive workplace

Paid Internship

Job summary

Experian is seeking a talented data engineer to join the Ascend Sandbox team, Experian's cloud-based analytical platform. You will build, maintain and scale data pipelines serving internal teams and external clients across the APAC region.

Join a hybrid work environment to work on scalable ETL/ELT pipelines, data lakehouse practices and production-grade data solutions using Python, PySpark and AWS services.

Qualifications

• Degree in Computer Science, Data Engineering or related field.
• At least 6 years of experience in data engineering or data-intensive systems.
• Strong hands-on experience with Python and PySpark for large-scale processing.
• Strong SQL skills and data modelling for analytics.
• Experience with AWS data services: S3, Glue, Athena, Redshift, EMR.

Responsibilities

• Design, build, test, deploy and operate scalable data pipelines.
• Develop Python, PySpark and SQL solutions for large datasets.
• Design data models and reusable datasets for analytics and data science.
• Engineer and optimise AWS-based data solutions (S3, Glue, Athena, Redshift, EMR).
• Apply data lake/lakehouse practices including partitioning and formats (Parquet).
• Implement ETL/ELT patterns with incremental loads, retries and backfills.
• Implement workflow orchestration using Airflow/MWAA or Step Functions.
• Build automated data quality checks and monitoring for pipeline health.
• Collaborate with clients, data scientists and engineers to translate requirements.
• Ensure code quality, security and CI/CD in deployments.

Skills

Python

PySpark

SQL

ETL/ELT design

Data modelling

Spark performance engineering

Problem solving

Communication

Education

Degree in Computer Science or equivalent

Tools

Amazon S3

AWS Glue

Amazon Athena

Amazon Redshift

Amazon EMR

Airflow/MWAA

Terraform

AWS Step Functions

Jira

Bitbucket

Job description

• Schedule: Full Time

Company Description

Experian is a global data and technology company, powering opportunities for people and businesses around the world. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more. Experian invests in people and new advanced technologies to unlock the power of data. We have an amazing team of 25,200 people in 32 countries.

Our uniqueness is that we celebrate yours. Experian's people first, inclusive and purpose-driven culture is multi award-winning; World's Best Workplaces 2025 (Fortune Global Top 25), Great Place To Work 26 countries to name a few. Check out Experian Life on social or explore our Careers Site to understand why. Experian is also proud to be an Equal Opportunity and Aff…

• Hybrid work arrangement: home and office
• Inclusive paid leave
• People-first and inclusive workplace
• Paid Internship

#LI-Hybrid

Job Description

Role Summary

We are looking for a talented and driven engineer to join the Ascend Sandbox team - Experian's cloud-based analytical platform that empowers clients and data scientists to explore, model, and derive insights from data in a secure, governed environment. You will play a key role in building, maintaining, and scaling the platform that serves both internal teams and external clients across the APAC region.

What you’ll be doing

• Design, build, test, deploy and operate scalable batch and, where applicable, streaming data pipelines for the Ascend Sandbox analytical platform.
• Develop high-quality Python, PySpark and SQL solutions to ingest, transform and process large-scale datasets with accuracy, reliability and efficiency.
• Design data models and reusable, trusted datasets/data products for analytics, data science and downstream applications.
• Engineer and optimise AWS-based data solutions using relevant services such as Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift and Amazon EMR, according to the platform architecture.
• Apply data lake and lakehouse engineering practices, including efficient storage formats, partitioning, schema evolution and storage/query optimisation.
• Design robust ETL/ELT patterns, including incremental processing, dependency management, idempotency, retries, backfills and failure recovery.
• Implement workflow orchestration and scheduling for data pipelines using the platform's approved tooling, such as Airflow/MWAA, AWS Step Functions or AWS Glue Workflows.
• Build automated data quality controls and monitoring for completeness, validity, reconciliation, schema compliance, freshness, pipeline health and processing SLAs/SLOs.
• Optimise Spark workloads and distributed data processing for performance, scalability and cost, including partitioning, joins, shuffles, skew and resource utilisation.
• Implement practical data governance, metadata, lineage, cataloguing, security and access-control requirements across data pipelines and datasets.
• Collaborate with clients, business users, data scientists and technical teams to translate business and data requirements into scalable data‑processing solutions and integrations.
• Investigate and resolve production data and pipeline issues, perform root-cause analysis, and implement preventive improvements to reliability and operational efficiency.
• Ensure data engineering solutions comply with established architecture, coding, testing, security and quality standards, and contribute to code reviews, CI/CD and automated deployment practices.

Qualifications

• Must possess a Degree / Advanced Diploma in Computer Science, Data Engineering, Software Engineering or equivalent. Seniority will be commensurate with relevant experience and qualifications.
• At least 6 years of experience in data engineering or developing data-intensive systems, including the design, build and production support of data pipelines.
• Strong hands-on experience with Python and PySpark for large-scale and distributed data processing.
• Strong SQL skills, including complex transformations, joins, window functions, query optimisation and performance tuning.
• Hands-on experience designing and operating scalable ETL/ELT data pipelines, including incremental loads, retries, backfills, dependency management and failure recovery.
• Experience with data modelling for analytical use cases, including dimensional modelling, facts/dimensions, normalization/denormalization and handling historical data.
• Hands-on experience with AWS data services relevant to data engineering, such as Amazon S3, AWS Glue, Amazon Athena, Amazon Redshift and Amazon EMR; exposure to Lambda, Kinesis/MSK, DMS or Lake Formation is an advantage where relevant.
• Experience with data lake/lakehouse concepts, storage layout, partitioning, schema evolution and efficient data formats such as Parquet; familiarity with JSON, CSV and Avro is useful.
• Strong understanding of Spark performance engineering, including partitioning, shuffles, joins, data skew, parallelism, memory/resource optimisation and troubleshooting.
• Experience with pipeline orchestration and workflow scheduling using tools such as Airflow/MWAA, AWS Step Functions, AWS Glue Workflows or equivalent.
• Experience implementing automated data quality checks and data observability, including monitoring pipeline health, data freshness, volumes, failures and quality anomalies.
• Practical knowledge of data governance, metadata management, data cataloguing, lineage and schema management.
• Practical understanding of AWS data security, including IAM/least privilege, encryption/KMS, secrets management, S3 security and data-access controls.
• Familiarity with Infrastructure as Code such as Terraform, AWS CloudFormation or AWS CDK is an advantage.
• Understanding of cloud cost and performance optimisation for storage, compute, Spark workloads and query patterns.
• Experience with Git-based workflows, branching strategies, code reviews, CI/CD pipelines and automated deployment. Familiarity with Jira, Bitbucket, Jenkins and Checkmarx is an advantage.
• Experience writing automated unit, integration and data-pipeline tests for Python/PySpark transformations and data quality.
• Strong testing, debugging, troubleshooting and root-cause analysis skills in production data environments.
• Good understanding of Agile, SCRUM, SDLC and software engineering practices.
• Strong communication and stakeholder management skills with the ability to engage clients, business users, data scientists and technical teams.
• Strong analytical thinking, problem‑solving, teamwork and willingness to learn, with the ability to work independently when required and deliver under tight deadlines.
• Familiarity with AI tools for coding assistance, documentation or automation is a plus.

Additional Information

Recruitment Fraud Awareness - Experian's recruitment process is conducted only through authorised channels. Recruitment communications will only be sent from an @ experian.com email address. Experian will never ask candidates to make any payment as part of an application, interview, assessment, onboarding, or recruitment process.

Experian Careers - Creating a better tomorrow together

Experian Careers - Creating a better tomorrow together

By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply