Data Engineer - QuantumBlack

QuantumBlack, AI by McKinsey; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

A leading company in AI consulting is seeking a Data Engineer to join their global Data Engineering community. You will work on high-impact projects, collaborating with cross-functional teams to build innovative machine learning systems. The role requires strong coding skills, experience in data pipeline development, and a collaborative mindset to drive business value through technology.

Qualifications

• At least 2 years of relevant professional experience.
• Proficiency in writing clean, scalable code in Python, Scala, or Java.
• Experience building production data pipelines for analytics.

Responsibilities

• Participate in problem-solving sessions to build impactful analytics solutions.
• Design and build GenAI applications collaboratively with data scientists.
• Create and manage secure data environments.

Skills

Problem Solving

Collaboration

Data Quality Assessment

Data Pipeline Development

Software Engineering Best Practices

Education

Degree in Computer Science, Engineering, Mathematics

Tools

Python

Scala

Java

Spark

Kafka

Snowflake

GCP DataProc

Azure DataFactory

AWS Glue

Apache Beam/Flink

PySpark

Job description

Who You'll Work With

You will be part of our global Data Engineering community and work in cross-functional Agile project teams alongside Data Scientists, Machine Learning Engineers, other Data Engineers, Project Managers, and industry experts.

You will collaborate closely with our clients, including data owners, users, fellow engineers, and C-level executives.

Who you are

You are a highly collaborative individual who enjoys solving problems that drive business value. You possess a strong sense of ownership and enjoy hands-on technical work. Our values resonate with yours.

Your Impact

You will work on real-world, high-impact projects across various industries, collaborating with QB/Labs teams to build complex and innovative ML systems that accelerate AI work and solve business problems efficiently.

This environment will foster your growth as a technologist and leader, helping you develop a perspective that connects technology with business value through diverse, multidisciplinary teams.

Our technology stack often includes Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro, Dask/RAPIDS, Docker, Kubernetes, and cloud platforms like AWS, GCP, and Azure.

As a Data Engineer, you will

1. Participate in problem-solving sessions with your team and clients to build impactful analytics solutions.
2. Contribute to R&D projects and internal asset development.
3. Design and build GenAI applications collaboratively with data scientists.
4. Map data fields to hypotheses and prepare data for advanced analytics models.
5. Assess data quality and understand the client's data landscape.
6. Create and manage secure data environments.
7. Design scalable, modular, and reproducible data pipelines for machine learning.
8. Support the development and maintenance of the technical platform for analytics engagements.

Your Qualifications and Skills

1. Degree in computer science, engineering, mathematics, or equivalent experience.
2. At least 2 years of relevant professional experience.
3. Proficiency in writing clean, scalable, and robust code in languages like Python, Scala, or Java.
4. Experience building production data pipelines for analytics.
5. Experience working with structured, semi-structured, and unstructured data.
6. Knowledge of software engineering best practices, including DevOps, DataOps, and MLOps, is a plus.
7. Familiarity with distributed computing frameworks, cloud platforms, containerization, and analytics libraries.
8. Experience with Big Data tools such as Spark, Kafka, Snowflake, GCP DataProc, Azure DataFactory, AWS Glue, Apache Beam/Flink, and PySpark.
9. Experience managing client relationships or with senior stakeholders is beneficial.