Data Engineer

KGCF; Indonesia; On-site; IDR 200.000.000 - 300.000.000; Full time

Job summary

A leading company in Indonesia is looking for a Data Engineer to optimize solutions for collecting and processing large datasets. You'll design scalable data infrastructure, build ETL processes, and utilize technologies like Kafka and Spark. The ideal candidate is passionate about big data and has a strong background in software engineering, capable of integrating systems across the company. This is a fantastic opportunity for those looking to work in an innovative environment focused on data-driven solutions.

Qualifications

• Practical experience in planning, sizing & migrating large enterprise workloads to cloud.
• Working knowledge of big data components like Hadoop, MapReduce, HDFS.
• Experience with Big Data ML toolkits like Mahout or SparkML is preferred.

Responsibilities

• Design and scale our data infrastructure as a data lake.
• Construct ETL processes for data pipelines.
• Collaborate with other Data Team members on project goals.

Skills

Passion in big data

Ability to design data systems

Knowledge of data warehousing

In-depth knowledge of SQL

Experience with messaging systems

Knowledge of big data components

Education

Bachelor's degree in technical majors

Tools

Kafka

Spark

BigQuery

Hadoop

Docker

Kubernetes

Job description

Kompas Gramedia has been part of Indonesia’s journey of enlightening people’s for the past 50+ years. Now, we are expanding the business through Media, Retail and Publishing, Hospitality, Manufactures, Event and Venue, also Educations. We are not only present as a business entity, but also contribute directly to build the character of human beings by our products and services to support our goal is to enlightening people's.

Job Description

The primary focus of Data Engineer will be on choosing optimal solutions to use for collecting, storing, processing, and analyzing of huge sets of data, then maintaining, implementing, and monitoring them. Data Engineer will also be responsible for integrating them with the architecture used across the company.

What you will be doing :

• Designing, constructing, maintaining, and scaling our data infrastructure. Our data infrastructure is data lake for all industry under Kompas Gramedia.
• Designing and constructingETL process as data pipeline for each data source inside Kompas Gramedia
• As a person who solve business case using data technology, such as Kafka, Spark, DataFlow, BigQuery, Airflow, etc.
• Ensure systems meet business requirements and industry practices
• Employ a variety of languages and tools (e.g. scripting languages) to marry systems together
• Collaborate with other Data Team members on project goals

Qualifications

• Passion in big data, software engineering, and systems.
• Ability to design and architect distributed data systems
• Knowledge of data warehousing and data pipeline concepts, such as ETL, data integration, data streaming
• In-depth knowledge of SQL and other database solutions (No SQL), such as MongoDB, Elastic Search, and so on
• Knowledgeable in Linux-based system administrating
• Practical experience in planning, sizing & migrating large enterprise workloads from on-premises to cloud platform.
• Working knowledge of big data components like Hadoop v2, MapReduce, HDFS, Hive, MapReduce, Spark, Apache Beam, Storm is preferred
• Experience with various messaging systems, such as Kafka, RabbitMQ, NSQ is preferred
• Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O is preferred
• Good knowledge on big data platform such as Cloudera, MapR, Hortonworks, or ELK (Elastic Search – Log stash – Kibana) is an added advantage
• Experience in DevOps tools such as Ansible, Docker, Kubernetes, Chef, Vagrant is a plus
• Knowledge of other open source database technologies is an added advantage.
• A bachelor degree in technical majors is preferred (e.g. Computer Science, Information Technology)