Data Engineer

Mandrill Tech; Kuala Lumpur; On-site; MYR 90,000 - 120,000; Full time

Job summary

A fast-growing technology company in Kuala Lumpur seeks a Data Engineer to develop scalable data architectures, ensure data quality, and collaborate with cross-functional teams. Candidates should have a Bachelor's degree, at least 5 years of experience with data integration tools, and proficiency in SQL, Python, and data processing technologies. A Master's degree and familiarity with cloud platforms like AWS are preferred. The role offers excellent career development opportunities and a great work environment.

Benefits

Excellent career development opportunities

Great work environment

Qualifications

• 5+ years of experience with data integration tools and technologies.
• Experience with batch and real-time data processing technologies.
• Experience developing with Python for data processing and automation.

Responsibilities

• Develop scalable data architectures and systems for large volumes of data.
• Ensure data quality and integrity through validation and cleansing processes.
• Collaborate with team members to understand data needs and requirements.

Skills

Data management

SQL

ETL

Data pipelines

Python

Apache Spark

Data cleaning

PostgreSQL

AWS

Education

Bachelor's degree in Computer Science, Electrical Engineering, or related field

Master’s degree in Computer Science or related field

Tools

Apache Flink

Pandas

dbt

Job description

Established in Kuala Lumpur since 2015, Mandrill Tech is a fast growing technology company. It serves many clients from various lines of industry.

Being an MSC certified company, Mandrill Tech is recognized as leaders in ICT industry development in Malaysia. It reflects our commitment to be the frontliner of technology innovation and advancements.

The Role

You will be responsible for:

• Develop scalable data architectures and systems that can handle large volumes of data and support real-time processing and analytics.
• Collect, extract, transform, and load data from various sources, including databases, data warehouses, and data lakes.
• Ensure data quality and integrity through the use of validation and cleansing processes and implement data security and privacy measures.
• Collaborate with data scientists, analysts, and business users to understand their data needs and requirements and provide them with access to the data they need.
• Develop and maintain ETL processes, data pipelines, and other data integration tools and technologies.
• Perform data modelling and database design and optimize data structures and queries for performance and scalability.
• Monitor and maintain data infrastructure and systems to ensure they are operating at peak performance and availability.
• Stay up to date with new and emerging technologies and techniques in the data engineering field, and recommend new tools and technologies as needed.
• Participate in team meetings, code reviews, and other collaborative activities to ensure the successful development and implementation of data infrastructure and systems.

Ideal Profile

• Bachelor's degree in Computer Science, Electrical Engineering, or related field (required), Master’s degree in Computer Science or related field (preferred).
• Strong knowledge of data management and database technologies, such as SQL, NoSQL, and Spark.
• Minimum 5 years of experience with data integration tools and technologies, such as ETL, data pipelines, and data warehousing.
• Programming Languages: Experience developing with Python for data processing, scripting, and automation.
• Data Processing: Experience with batch and real-time data processing technologies, including Apache Spark and Apache Flink.
• Data Transformation: Proficient in data cleaning, modeling, and transformation using tools and libraries like Pandas, dbt, and Spark.
• Database: Hands-on experience with PostgreSQL.
• Cloud Familiarity: Familiarity with cloud platforms, preferably AWS.

What's on Offer?

• Work alongside & learn from best in class talent
• Excellent career development opportunities
• Great work environment