Senior Data Engineer

CloudMile; Kuala Lumpur; On-site; MYR 180,000 - 300,000; Full time

Benefits offered by this job

Competitive salary

Career growth opportunities

Cloud exposure on multi-cloud stacks

Job summary

CloudMile seeks a Senior Data Engineer who will also serve as Delivery Lead, guiding a small team of engineers on data projects. You will design scalable pipelines, models, and dashboards while ensuring governance and quality across cloud platforms. Strong leadership and stakeholder management are essential.

The role requires 5+ years in data/AI, 3+ years with cloud platforms (GCP preferred), and proficiency in Python/SQL, plus experience with modern data tools like Spark, Airflow, and dbt.

Qualifications

• Minimum 5 years of experience in data and AI projects.
• At least 3 years working with cloud platforms (GCP preferred).
• Strong programming in Python and SQL; experience with NoSQL databases.
• Experience leading a data engineering team and delivering projects.

Responsibilities

• Design scalable data pipelines for high performance and reliability.
• Develop data models and transformation logic to support analysis.
• Set up data warehouses and implement data governance controls.
• Build dashboards to provide actionable insights for stakeholders.
• Create data APIs and microservices for systems integration.
• Provide technical leadership and ensure best practices.

Skills

Python

SQL

NoSQL

Spark

Airflow

dbt

Kafka

Cloud data pipelines

Data visualization

Education

Bachelor's or Master's in Computer Science or Engineering

Tools

GCP BigQuery

Snowflake

Azure Synapse

AWS Redshift

Databricks

Fivetran

Confluent

MongoDB Atlas

Job description

We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. This individual will also take on the role as a Delivery Lead in data engineering projects, leading a small team of engineers, and ensuring seamless project execution. The ideal candidate will have a strong background in data engineering, cloud technologies, project delivery, and stakeholder management.

Key Responsibilities

• Data Pipeline Development: Design and implement scalable data integration pipelines, ensuring high performance and reliability.
• Data Modeling & Transformation: Develop robust data models and transformation logic to support data analysis and business needs.
• Data Warehouse & Governance: Set up and manage data warehouses, implement data governance controls, and ensure compliance with organisational standards.
• Dashboard Development: Create dashboards that provide actionable insights, helping stakeholders make informed decisions.
• API Development: Build scalable data APIs to facilitate data access and integration across systems.
• Technical Leadership: Lead a team of data engineers, providing technical guidance and ensuring best practices in data engineering and project execution.
• Stakeholder Management: Collaborate and communicate effectively with internal and external stakeholders to gather requirements, manage expectations, and ensure alignment with project goals.

Requirements

• Experience:
• At least 5 years of relevant experience in data and AI projects.
• Minimum 3 years of experience working with cloud platforms (preferably GCP; AWS or Azure is acceptable).
• Technical Skills:
• Cloud Data Platforms: Proficient in working with cloud-based data tools and platforms, including GCP BigQuery, Snowflake, Azure Synapse, AWS Redshift, Databricks, Fivetran, Confluent, and MongoDB Atlas.
• Big Data & Open Source Expertise: Leverage open-source big data technologies such as Spark, Hadoop, Apache Beam, Kafka, dbt, and Airflow to orchestrate, process, and analyse large datasets efficiently.
• Programming Skills: Strong programming abilities in Python, SQL, and NoSQL databases, with a focus on data transformation, processing, and querying.
• API & Microservices Development: Experience in designing and building scalable microservices and data APIs (considered a bonus).
• Generative AI Technologies: Familiarity with Generative AI tools and frameworks, with the ability to integrate them into data solutions (considered a bonus).
• Data Management & Governance: Good knowledge of data management practices and data governance frameworks, including implementing policies, ensuring data quality, and maintaining compliance (considered a bonus).
• Communication & Management Skills:
• Proven experience leading a team of data engineers and demonstrated ability to manage projects and ensure successful delivery.
• Excellent verbal and written communication skills, with the ability to convey complex technical concepts to both technical and non-technical audiences.
• Strong stakeholder management skills to navigate expectations and drive alignment across teams.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Certification in cloud platforms such as Google Cloud Professional Data Engineer, AWS Certified Big Data, or Azure Data Engineer Associate.

What We Offer

• Competitive salary and benefits package.
• Opportunities to work on cutting-edge data and AI projects on various cloud platforms.
• A collaborative and innovative work environment.
• Career growth and learning opportunities.