Data Resiliency Engineer - Data Lake

PayNet (Payments Network Malaysia); Kuala Lumpur; On-site; MYR 90,000 - 120,000; Full time

Job summary

PayNet (Payments Network Malaysia) is looking for a Data Resiliency Engineer to enhance the robustness of our data infrastructure. This role involves analyzing issues, implementing solutions, and ensuring a seamless user experience through proactive monitoring and incident management.

Qualifications include a Bachelor's degree in a related field, proficiency in AWS data services, strong Python skills, and experience with monitoring tools like Datadog and Opsgenie. Join us in maintaining operational excellence across our data-driven operations.

Qualifications

• Strong experience in root cause analysis for complex data issues.
• Proficiency in data lakes and managing large-scale data environments.
• Relevant certifications in AWS and data engineering.

Responsibilities

• Analyze issues to identify root causes and implement solutions.
• Oversee data ecosystem using tools like Datadog.
• Serve as primary contact for data-related inquiries.

Skills

AWS data services (e.g., S3, Glue, Athena, Quicksight, Lambda)

Python programming language

PySpark

Kubernetes and EKS

Apache Airflow

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Terraform

Datadog

Opsgenie

CloudWatch

Job description

Summary of Responsibilities

As a Data Resiliency Engineer, you will be at the forefront of maintaining and enhancing the robustness of our data infrastructure. You will analyze issues to identify root causes and implement solutions that ensure a seamless user experience. Your expertise in data ecosystem monitoring, incident management, and data quality enhancement will play a pivotal role in maintaining operational excellence. You will utilize advanced tools like Datadog and Opsgenie to proactively monitor and resolve issues, act as a subject matter expert for data‑related inquiries, and champion data quality initiatives. Additionally, you will work closely with the team to ensure our suite of payments reporting systems for participants as well as PayNet internal reports are accurate and aligned with evolving organizational needs, driving continuous improvement in our data‑driven operations.

Key Areas of Responsibilities

• Root Cause Analysis & Bug Fixing: Analyze issues to uncover root causes and implement effective solutions, ensuring a smooth user experience.
• Data Ecosystem Monitoring: Oversee the data environment using advanced monitoring tools like Datadog, proactively identifying and addressing issues before they escalate.
• Alert System Management: Collaborate with the team managing Opsgenie and other alert systems, ensuring timely responses to critical incidents and maintaining operational excellence.
• Data Lake Support: Serve as the primary contact for data‑related inquiries, including ETL incident management, reporting challenges, and providing actionable insights.
• Data Quality Monitoring: Own and enhance data quality monitoring tools, designing robust pipelines and frameworks to maintain the highest data integrity standards.
• Reporting Management: Lead report amendments and ensure reporting processes are accurate and meet the evolving needs of the organization.

Qualifications & Experience

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.
• Strong experience with AWS data services (e.g., S3, Glue, Athena, Quicksight, Lambda).
• Strong experience in Python programming language.
• Proficiency in PySpark for large‑scale data processing.
• Familiarity with workflow management tools (e.g., Apache Airflow).
• Familiarity with operating data tools on Kubernetes and EKS.
• Expertise in monitoring tools (e.g., Datadog, CloudWatch) to proactively track and resolve issues.
• Experience with alert management systems like Opsgenie or similar platforms.
• Strong understanding of data lakes and managing large‑scale data environments.
• Experience in performing root cause analysis for complex data issues and implementing effective bug fixes.
• Terraform or other IaC tools for infrastructure provisioning.
• Relevant certifications in AWS and data engineering.

Personal Qualities

• Self‑motivated problem solver who can work with minimal guidance.
• Excellent communication skills to articulate technical details clearly to non‑technical stakeholders.
• Detail‑oriented with a focus on data quality and reliability.
• Proven ability to work cross‑functionally with multiple teams (e.g., Data Engineering, Operations, Analytics).
• Passionate engineer looking to learn new technologies.