Senior Apache Kafka Data Engineer

Accenture Southeast Asia; Kota Semarang; On-site; IDR 497.265.000 - 828.776.000; Full time

Job summary

A consulting firm based in Semarang is seeking a Data Engineer to work on data pipelines, automate processes using Azure, and analyze data for insights. The ideal candidate holds a Bachelor's degree in IT and has 2 years of experience. You will develop dashboards and reports, ensuring data quality and transformation. This role offers a dynamic environment with opportunities for new graduates as well.

Qualifications

• Minimum 2 years’ experience in Data Engineering fields (new graduates welcome for some openings).
• Experience working in a client-facing/consulting environment is a plus.
• Good communication skills in English.

Responsibilities

• Support data requirements including reports and dashboards.
• Design and develop the E2E Data Pipeline.
• Develop data pipeline automation using Azure technologies.
• Analyze source data and perform data ingestion.

Skills

Python

SQL

Spark

Cloud Architect

Data Visualization

DAX programming

Data Analysis

Education

Bachelor’s degree in Computer Science, Computer Engineering, IT, or related fields

Tools

Azure

Power BI

Tableau

Apache Spark

Kafka

Job description

As a Data Engineer, you will:

• Work across workstreams to support data requirements including reports and dashboards
• Analyze and perform data profiling to understand data patterns and discrepancies following Data Quality and Data Management processes
• Understand and follow best practices to design and develop the E2E Data Pipeline: data transformation, ingestion, processing, and surfacing of data for large-scale applications
• Develop data pipeline automation using Azure technologies stack, Databricks, Data Factory
• Understand business requirements to translate them into technical requirements that the system analysts and other technical team members can drive into the project design and delivery
• Analyze source data and perform data ingestion in both batch and real-time patterns via various methods; for example, file transfer, API, Data Streaming using Kafka and Spark Streaming
• Analyze and understand data processing and standardization requirements, develop ETL using Spark processing to transform data
• Understand data/reports and dashboards requirements, develop data export, data API, or data visualization using Power BI, Tableau, or other visualization tools

Qualifications

• Bachelor’s degree in Computer Science, Computer Engineer, IT, or related fields
• Minimum 2 years’ experience in Data Engineering fields (new graduates are also welcome for some of our job openings)
• Data Engineering skills: Python, SQL, Spark, Cloud Architect, Data & Solution Architect, API, Databricks, Azure
• Data Visualization skills: Power BI (or other visualization tools), DAX programming, API, Data Model, SQL, Story Telling and wireframe design
• Business Analyst skills: business knowledge, data profiling, basic data model design, data analysis, requirement analysis, SQL programming
• Basic knowledge in Data Lake/Data Warehousing/ Big data tools, Apache Spark, RDBMS and NoSQL, Knowledge Graph
• Experience working in a client-facing/consulting environment is a plus
• Team player, analytical and problem-solving skills
• Good communication skills in English