Senior Data Engineer- Malaysia

Infosys Consulting; Kuala Lumpur; On-site; MYR 180,000 - 300,000; Full time

Job summary

Infosys Consulting seeks an experienced Senior Data Engineer in Kuala Lumpur to design, build, and manage scalable data pipelines on AWS. You will optimize ETL architectures, collaborate with stakeholders, and ensure data integrity across large datasets in production.

The role requires strong PySpark and SQL skills, familiarity with Databricks, and experience delivering production-grade data solutions with CI/CD practices.

Qualifications

• 7 to 10 years of experience and above.
• Experience with data lake architectures, big data technologies, and data pipeline orchestration.
• Familiarity with CI/CD practices for data engineering.
• AWS Certification (e.g., AWS Certified Data Analytics – Specialty or Solutions Architect) is a plus.
• Strong problem-solving skills and attention to detail.

Responsibilities

• Design, develop, and maintain data pipelines and ETL workflows using AWS services.
• Implement orchestration and automation for data workflows.
• Work with large datasets to ensure data integrity, scalability, and performance.
• Collaborate with stakeholders to understand data requirements and deliver solutions.
• Deploy changes directly to production environments with confidence and accountability.
• Support migration efforts to Databricks and optimize workflows for performance.

Skills

AWS data services

PySpark

SQL

CI/CD

Databricks

Production rollout

Tools

AWS Glue

S3

Lambda

Step Functions

Job description

• Design, develop, and maintain data pipelines and ETL workflows using AWS services.
• Implement orchestration and automation for data workflows.
• Work with large datasets to ensure data integrity, scalability, and performance.
• Collaborate with stakeholders to understand data requirements and deliver solutions.
• Deploy changes directly to production environments with confidence and accountability.
• Support migration efforts to Databricks and optimize workflows for performance

Qualifications

• 7 to 10 years of experience and above
• Experience with data lake architectures, big data technologies, and data pipeline orchestration.
• Familiarity with CI/CD practices for data engineering.
• AWS Certification (e.g., AWS Certified Data Analytics – Specialty or Solutions Architect) is a plus.
• Strong problem-solving skills and attention to detail.

Key Skills and Requirements

• Expert-level fluency in AWS services relevant to data engineering, including Glue, S3, Lambda, Step Functions, and other related services.
• Strong proficiency in PySpark for distributed data processing.
• Advanced SQL skills for querying and optimizing data operations.
• Comfortable with pushing changes live without formal review, ensuring quality through self-validation.
• Databricks experience is a plus.
• Ability to work independently and manage tasks in a siloed environment.
• Currently based in Kuala Lumpur, Malaysia