Lead Data Engineer Manager: Pipelines, Cloud & ML

Artefact; Pulau Untung Jawa; On-site; IDR 420,000,000 - 780,000,000; Full time

Job summary

Artefact is a data service provider specialising in data consulting and data-driven digital marketing. We transform data into business impact across the value chain and are experiencing rapid growth.

As a Data Engineer Manager, you will craft and maintain robust data pipelines using Python and SQL to ensure efficient ETL, scalable data platforms, and Cloud integrations. You will mentor engineers, collaborate with stakeholders, and integrate ML models with Spark and Kafka to enable real-time

Qualifications

• 8+ years of data engineering experience.
• Proficiency in Python, SQL, and database management.
• Experience with Data Pipelines ETL, Cloud Services (Azure, GCP, AWS), ML Modeling, Spark & Kafka.
• Data modelling and modern data platform design concepts.
• Building and maintaining data platforms in Azure/AWS or GCP.

Responsibilities

• Data Pipeline Development: Build and optimise pipelines for data flow.
• Database Management: Ensure data integrity and efficient storage.
• Cloud Services Integration: Architect solutions on Azure/GCP/AWS.
• ML Integration: Work with teams to embed ML models into pipelines.
• Spark & Kafka: Implement real-time processing and analytics.
• Mentor engineers and share best practices.

Skills

Python

SQL

Data pipelines ETL

Cloud: Azure/GCP/AWS

Spark

Kafka

Data modelling

Data platform design

Stakeholder communication

Mentoring

Education

Bachelor's degree in CS/EE/CE

Tools

Spark

Kafka

Job description

Artefact is a data service provider specialising in data consulting and data-driven digital marketing. We transform data into business impact across the value chain and are experiencing rapid growth.

As a Data Engineer Manager, you will craft and maintain robust data pipelines using Python and SQL to ensure efficient ETL, scalable data platforms, and Cloud integrations. You will mentor engineers, collaborate with stakeholders, and integrate ML models with Spark and Kafka to enable real-time