Senior Data Engineer

Cloud Mile Pte Ltd; Kuala Lumpur; On-site; MYR 180,000 - 280,000; Full time

Job summary

Cloud Mile Pte Ltd is seeking a Senior Data Engineer who will also act as Delivery Lead on data engineering projects in Kuala Lumpur. The role involves building scalable pipelines, data models, and governance while guiding a small team of engineers.

Ideal candidates have 5+ years in data/AI projects and at least 3 years on cloud platforms (GCP preferred). Strong Python/SQL/NoSQL skills and stakeholder management are essential.

Qualifications

• 5+ years in data or AI projects; strong delivery track record.
• ≥3 years on cloud platforms (GCP preferred; AWS/Azure also acceptable).
• Proficient in Python, SQL and NoSQL databases for transformation and processing.

Responsibilities

• Design and build scalable data pipelines and APIs for data access.
• Lead a team of data engineers and ensure best practices.
• Develop data models, governance, and dashboards for stakeholders.
• Collaborate with stakeholders to gather requirements and manage delivery.
• Mentor team on cloud data architectures and project execution.

Skills

Data Engineer

Cloud platforms

Python

SQL

NoSQL

Spark

Kafka

Airflow

dbt

Fivetran

Confluent

MongoDB Atlas

Generative AI

APIs

Education

Bachelor's or Master's degree in Computer Science, Engineering, or related field

Tools

GCP BigQuery

Snowflake

Azure Synapse

AWS Redshift

Databricks

Fivetran

Confluent

MongoDB Atlas

Spark

Hadoop

Apache Beam

Kafka

dbt

Airflow

Job description

We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. This individual will also take on the role as a Delivery Lead in data engineering projects, leading a small team of engineers, and ensuring seamless project execution. The ideal candidate will have a strong background in data engineering, cloud technologies, project delivery, and stakeholder management.

Key Responsibilities

• Data Pipeline Development: Design and implement scalable data integration pipelines, ensuring high performance and reliability.
• Data Modeling & Transformation: Develop robust data models and transformation logic to support data analysis and business needs.
• Data Warehouse & Governance: Set up and manage data warehouses, implement data governance controls, and ensure compliance with organisational standards.
• Dashboard Development: Create dashboards that provide actionable insights, helping stakeholders make informed decisions.
• API Development: Build scalable data APIs to facilitate data access and integration across systems.
• Technical Leadership: Lead a team of data engineers, providing technical guidance and ensuring best practices in data engineering and project execution.
• Stakeholder Management: Collaborate and communicate effectively with internal and external stakeholders to gather requirements, manage expectations, and ensure alignment with project goals.

Requirements

• Experience:
• At least 5 years of relevant experience in data and AI projects.
• Minimum 3 years of experience working with cloud platforms (preferably GCP; AWS or Azure is acceptable).
• Technical Skills:
• Cloud Data Platforms: Proficient in working with cloud-based data tools and platforms, including GCP BigQuery, Snowflake, Azure Synapse, AWS Redshift, Databricks, Fivetran, Confluent, and MongoDB Atlas.
• Big Data & Open Source Expertise: Leverage open-source big data technologies such as Spark, Hadoop, Apache Beam, Kafka, dbt, and Airflow to orchestrate, process, and analyse large datasets efficiently.
• Programming Skills: Strong programming abilities in Python, SQL, and NoSQL databases, with a focus on data transformation, processing, and querying.
• API & Microservices Development: Experience in designing and building scalable microservices and data APIs (considered a bonus).
• Generative AI Technologies: Familiarity with Generative AI tools and frameworks, with the ability to integrate them into data solutions (considered a bonus).
• Data Management & Governance: Good knowledge of data management practices and data governance frameworks, including implementing policies, ensuring data quality, and maintaining compliance (considered a bonus).
• Communication & Management Skills:
• Proven experience leading a team of data engineers and demonstrated ability to manage projects and ensure successful delivery.
• Excellent verbal and written communication skills, with the ability to convey complex technical concepts to both technical and non-technical audiences.
• Strong stakeholder management skills to navigate expectations and drive alignment across teams.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Certification in cloud platforms such as Google Cloud Professional Data Engineer, AWS Certified Big Data, or Azure Data Engineer Associate.

What We Offer

• Competitive salary and benefits package.
• Opportunities to work on cutting-edge data and AI projects on various cloud platforms.
• A collaborative and innovative work environment.
• Career growth and learning opportunities.