Senior Apache Kafka Data Engineer

7500 PT Accenture Company; Kota Semarang; On-site; IDR 180,000,000 - 300,000,000; Full time

Job summary

Accenture is seeking a Data Engineer to support data requirements across multiple workstreams, focusing on building end-to-end data pipelines and dashboards.

The role involves data profiling, transformation, and ingestion using Azure stack, Databricks, and Data Factory, with emphasis on translating business needs into technical designs.

Qualifications

• Data engineering skills include Python, SQL, Spark, Databricks and Azure.
• Power BI, storytelling with dashboards, API, data modeling and SQL programming are essential visualization skills.
• Business analyst capabilities include requirement analysis, data profiling and basic data modeling.
• Knowledge of Data Lake/Data Warehousing, Apache Spark, RDBMS/NoSQL, Knowledge Graph.

Responsibilities

• Work across workstreams to support data requirements including reports and dashboards.
• Analyze and perform data profiling to understand data patterns and discrepancies following Data Quality and Data Management processes.
• Understand and follow best practices to design and develop the E2E Data Pipeline: data transformation, ingestion, processing, and surfacing of data for large-scale applications.
• Develop data pipeline automation using Azure technologies stack, Databricks, Data Factory.
• Understand business requirements to translate them into technical requirements that the system analysts and other technical team members can drive into the project design and delivery.
• Analyze source data and perform data ingestion in both batch and real-time patterns via various methods; for example, file transfer, API, Data Streaming using Kafka and Spark Streaming.
• Analyze and understand data processing and standardization requirements, develop ETL using Spark processing to transform data.
• Understand data/reports and dashboards requirements, develop data export, data API, or data visualization using Power BI, Tableau, or other visualization tools

Skills

Data modeling

Data profiling

Requirements analysis

SQL programming

Education

Bachelor’s degree in Computer Science, Computer Engineer, IT, or related fields

Tools

Python

SQL

Spark

Databricks

Azure

Data Factory

Power BI

Tableau

APIs

Job description

Job Responsibilities

• Work across workstreams to support data requirements including reports and dashboards
• Analyze and perform data profiling to understand data patterns and discrepancies following Data Quality and Data Management processes
• Understand and follow best practices to design and develop the E2E Data Pipeline: data transformation, ingestion, processing, and surfacing of data for large-scale applications
• Develop data pipeline automation using Azure technologies stack, Databricks, Data Factory
• Understand business requirements to translate them into technical requirements that the system analysts and other technical team members can drive into the project design and delivery
• Analyze source data and perform data ingestion in both batch and real-time patterns via various methods; for example, file transfer, API, Data Streaming using Kafka and Spark Streaming
• Analyze and understand data processing and standardization requirements, develop ETL using Spark processing to transform data
• Understand data/reports and dashboards requirements, develop data export, data API, or data visualization using Power BI, Tableau, or other visualization tools

Requirements

Bachelor’s degree in Computer Science, Computer Engineer, IT, or related fields

Minimum 2 years’ experience in Data Engineering fields (new graduates are also welcome for some of our job openings)

• Data Engineering skills: Python, SQL, Spark, Cloud Architect, Data & Solution Architect, API, Databricks, Azure
• Data Visualization skills: Power BI (or other visualization tools), DAX programming, API, Data Model, SQL, Story Telling and wireframe design
• Business Analyst skills: business knowledge, data profiling, basic data model design, data analysis, requirement analysis, SQL programing
• Basic knowledge in Data Lake/Data Warehousing/ Big data tools, Apache Spark, RDBMS and NoSQL, Knowledge Graph

About Accenture

We work with one shared purpose: to deliver on the promise of technology and human ingenuity. Every day, more than 775,000 of us help our stakeholders continuously reinvent. Together, we drive positive change and deliver value to our clients, partners, shareholders, communities, and each other. We believe that delivering value requires innovation, and innovation thrives in an inclusive and diverse environment. We actively foster a workplace free from bias, where everyone feels a sense of belonging and is respected and empowered to do their best work. At Accenture, we see well-being holistically, supporting our people’s physical, mental, and financial health. We also provide opportunities to keep skills relevant through certifications, learning, and diverse work experiences. We’re proud to be consistently recognized as one of the World’s Best Workplaces™.

Visit us at www.accenture.com

Equal Employment Opportunity Statement

We believe that no one should be discriminated against because of their differences.All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, military veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicable law.Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities. Bring your incredible skills and join our global team of innovators. We come together from different backgrounds across the world and work with the latest technologies to create value and growth for our clients. With us, you’ll continue to learn and grow so you can advance in your career. Your personal dreams and ambitions are just as important to us; that’s why we offer support any way we can—when you thrive, we all thrive.

Explore your next step at Accenture Belong. Grow. Thrive. Join agreat place to work for reinventors who drive meaningful change for our clients, communities, and the world. Wo rld. Explore your next step at Accenture