Senior Data Engineer

Pulse Consulting Sdn Bhd; Kuala Lumpur; On-site; MYR 80,000 - 110,000; Full time

Job summary

A leading software house in Kuala Lumpur is seeking a Data Engineer to implement data pipelines and develop machine learning algorithms. The role requires a Bachelor's degree in Computer Science, experience with Python, SQL, and cloud technologies like GCP. Ideal candidates enjoy working in Agile teams and demonstrate strong coding skills focused on scalability and maintainability. This position offers opportunities to collaborate closely with a diverse team to enhance data applications.

Qualifications

• Relevant job experience as a Data Engineer.
• Ability to write efficient, well-tested code with a keen eye on scalability and maintainability.
• Enjoy working in an Agile team.

Responsibilities

• Implement stream and batch processing data pipelines.
• Develop machine learning algorithms.
• Design and implement data validation strategies.
• Enable measurable asset performance through A/B testing.
• Improve reliability, scalability, and performance of the recommendation infrastructure.
• Collaborate with Data Scientists and other stakeholders.

Skills

Programming with Python

SQL

Docker

RESTAPI

Git

Kubernetes

Cloud experience (GCP)

Elasticsearch

Terraform

Redis

Education

Bachelor Degree in Computer Science or related field

Job description

Our client is one of the global leading software houses (IT MNC) with core businesses in software development, SAP consultation and ITO services based in Kuala Lumpur, Malaysia.

Job Description

• Implement stream and batch processing data pipelines that feed article indexes and build machine learning models.
• Developing machine learning algorithms, from single recommendation algorithms to whole page optimization algorithms.
• Designing and implementing data validation strategies to ensure model performance over time.
• Enable measurable asset performance by implementing A/B testing and tracking key metrics.
• Implement measures to improve the reliability, scalability, and performance of the recommendation infrastructure to deliver a personalized experience to millions of users.
• Work closely with Data Scientists, Infrastructure Engineers, Product Owners and other stakeholders to develop a highly scalable and intelligent data application.

Qualifications

• At least a Bachelor Degree in computer science or related field.
• Relevant job experience as a Data Engineer.
• Solid skills in programming with Python.
• Good knowledge in SQL, Python (asyncio), Docker, RESTAPI, Git, Kubernetes, strong cloud experience (GCP, AgroCD, Gitlab CI/CD), Elasticsearch (BigTable)
• Ability to write efficient, well-tested code with a keen eye on scalability and maintainability.
• Experience with at least one cloud, GCP is a plus.
• Enjoy working in an Agile team, like to get involved, think "out of the box" and are keen to tackle new topics.
• Good to have knowledge in Terraform and Redis.

Additional Information

All communication will be handled with absolute privacy. Only shortlisted candidates will be notified.