Data Engineer

PT Datacaraka Solusindo; Jakarta Barat; On-site; IDR 240,000,000 - 460,000,000; Full time

Benefits offered by this job

Restaurant d'entreprise

Job summary

PT Datacaraka Solusindo in Jakarta, Indonesia is seeking an experienced Data/Integration Engineer to design, build, and maintain scalable data pipelines using NiFi and Kafka.

You will optimize SQL queries with Impala and Hive, work with Cloudera CDP/CDH, and collaborate with DevOps on Docker and Kubernetes deployments, bringing strong problem solving and documentation skills. The role also requires teamwork, excellent communication, and a passion for reliable data platforms.

Qualifications

• Bachelor's Degree in Information Technology, Information Systems, Computer Science, or a related field.
• Minimum 3 years of experience as a Data Engineer, Integration Engineer, or in a similar role.
• Hands-on experience with Apache NiFi, including processors, controller services, and provenance.
• Strong knowledge of Apache Kafka, including cluster setup, topics, producers/consumers, and stream processing.
• Proficient in writing and optimizing SQL queries using Impala and/or Hive for big data analytics.
• Familiar with Cloudera Platform (CDP/CDH) and the Hadoop ecosystem, including HDFS, Hive, Spark, and Kudu.
• Experience with Docker and Kubernetes for application deployment.
• Knowledge of Java, Python, or Scala is an advantage.
• Excellent analytical thinking, problem-solving, and debugging skills.
• Strong communication and interpersonal skills with the ability to collaborate effectively across cross-functional teams.
• Able to produce clear, well-structured technical documentation.
• Capable of working independently as well as collaboratively in a team-oriented environment.

Responsibilities

• Design, develop, implement, and maintain scalable data pipelines using Apache NiFi for data ingestion, transformation, and routing.
• Manage and maintain Apache Kafka as an event streaming platform, including topic configuration, producers, consumers, and cluster administration.
• Develop, optimize, and maintain SQL queries in Impala to support large-scale data analytics and reporting.
• Integrate data pipelines from multiple sources, including databases, APIs, IoT devices, and cloud storage into enterprise analytics platforms.
• Monitor, troubleshoot, and optimize the performance, reliability, and security of Cloudera, NiFi, Kafka, and Impala environments.
• Collaborate with the DevOps team to deploy, manage, and optimize applications using Docker and Kubernetes.
• Perform root cause analysis and resolve issues related to data integration, processing, and system performance.
• Ensure data quality, reliability, and availability across the end-to-end data engineering lifecycle.

Skills

Apache NiFi

Apache Kafka

Impala

Hive

Cloudera CDP/CDH

Docker

Kubernetes

Java

Python

Scala

SQL queries

Big data analytics

Education

Bachelor's Degree in IT/CS

Tools

Docker

Kubernetes

Job description

PT Datacaraka Solusindo is a leading Information and Communication Technology (ICT) company based in Jakarta, Indonesia. We specialize in delivering innovative IT solutions and support services across various industries. Our mission is to provide high-quality technology solutions that help our clients achieve their business objectives efficiently and securely.

With a strong customer-centric approach, technical expertise, and commitment to excellence, we continuously embrace technological advancements to deliver exceptional value to our clients and partners.

Qualifications

Bachelor's Degree in Information Technology, Information Systems, Computer Science, or a related field.

Minimum 3 years of experience as a Data Engineer, Integration Engineer, or in a similar role.

Hands-on experience with Apache NiFi, including processors, controller services, and provenance.

Strong knowledge of Apache Kafka, including cluster setup, topics, producers/consumers, and stream processing.

Proficient in writing and optimizing SQL queries using Impala and/or Hive for big data analytics.

Familiar with Cloudera Platform (CDP/CDH) and the Hadoop ecosystem, including HDFS, Hive, Spark, and Kudu.

Experience with Docker and Kubernetes for application deployment.

Knowledge of Java, Python, or Scala is an advantage.

Excellent analytical thinking, problem-solving, and debugging skills.

Strong communication and interpersonal skills with the ability to collaborate effectively across cross-functional teams.

Able to produce clear, well-structured technical documentation.

Capable of working independently as well as collaboratively in a team-oriented environment.

Key Responsibilities

Design, develop, implement, and maintain scalable data pipelines using Apache NiFi for data ingestion, transformation, and routing.

Manage and maintain Apache Kafka as an event streaming platform, including topic configuration, producers, consumers, and cluster administration.

Develop, optimize, and maintain SQL queries in Impala to support large-scale data analytics and reporting.

Integrate data pipelines from multiple sources, including databases, APIs, IoT devices, and cloud storage into enterprise analytics platforms.

Monitor, troubleshoot, and optimize the performance, reliability, and security of Cloudera, NiFi, Kafka, and Impala environments.

Collaborate with the DevOps team to deploy, manage, and optimize applications using Docker and Kubernetes.

Perform root cause analysis and resolve issues related to data integration, processing, and system performance.

Ensure data quality, reliability, and availability across the end-to-end data engineering lifecycle.

PT. Datacaraka Solusindo is a fast growing IT Solution Company. Our vision is to help our clients win the competition by providing them complete IT Solutions for their needs. We invite you to be a part of our plan to develop a better IT ambience in Indonesia. We served several industries such as bankings, distribution, manufacturing, and retail. With us, you are trained to be a professional in IT industry. Through basic and advanced certifications, it's highly hoped that you can achieve better understanding about IT industry for your personal career track record and also to leverage our clients' business competitiveness. Take part in our massive growing team, professionals!

PT. Datacaraka Solusindo is a fast growing IT Solution Company. Our vision is to help our clients win the competition by providing them complete IT Solutions for their needs. We invite you to be a part of our plan to develop a better IT ambience in Indonesia. We served several industries such as bankings, distribution, manufacturing, and retail. With us, you are trained to be a professional in IT industry. Through basic and advanced certifications, it's highly hoped that you can achieve better understanding about IT industry for your personal career track record and also to leverage our clients' business competitiveness. Take part in our massive growing team, professionals!