Senior Apache Spark Data Engineer

Accenture Southeast Asia; Kota Semarang; On-site; IDR 129.094.000 - 193.643.000; Full time

Job summary

A leading technology consulting firm is seeking an Application Developer to design and build applications using Google Dataproc. The role involves collaborating with cross-functional teams and utilizing Apache Spark to deliver solutions. Ideal candidates will have strong experience in cloud technologies and data architectures, and the ability to work with various database models. Join us to drive impactful data-driven initiatives in a dynamic environment.

Qualifications

• Strong experience in Apache Spark and Java for Spark.
• Strong experience with multiple database models (SQL, NoSQL, OLTP, OLAP).
• Knowledge of cloud data platforms such as GCS, BigQuery, and Dataproc.

Responsibilities

• Design, build, and configure applications using Google Dataproc.
• Collaborate with cross-functional teams to deliver data-driven solutions.
• Utilize Apache Spark for data processing and analysis.

Skills

Apache Spark

Java

Cloud Dataproc

Data Streaming Architecture

SQL

NoSQL

Infrastructure as Code

Data analysis with large datasets

GCP Data Engineer Certification

Job description

As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements using Google Dataproc.

Your typical day will involve working with Apache Spark and collaborating with cross-functional teams to deliver impactful data-driven solutions.

Roles & Responsibilities:

1. Design, build, and configure applications to meet business process and application requirements using Google Dataproc.
2. Collaborate with cross-functional teams to deliver impactful data-driven solutions.
3. Utilize Apache Spark for data processing and analysis.
4. Develop and maintain technical documentation for applications.

Professional & Technical Skills:

1. Strong experience in Apache Spark and Java for Spark.
2. Strong experience with multiple database models (SQL, NoSQL, OLTP, and OLAP).
3. Strong experience with Data Streaming Architecture (Kafka, Spark, Airflow).
4. Strong knowledge of cloud data platforms and technologies such as GCS, BigQuery, Cloud Composer, Dataproc, and other cloud-native offerings.
5. Knowledge of Infrastructure as Code (IaC) and associated tools (Terraform, Ansible, etc.).
6. Experience pulling data from various data source types including Mainframe (EBCDIC), Fixed Length and delimited files, databases (SQL, NoSQL, Time-series).
7. Experience performing analysis with large datasets in a cloud-based environment, preferably with an understanding of Google’s Cloud Platform (GCP).
8. Comfortable communicating with various stakeholders (technical and non-technical).
9. GCP Data Engineer Certification is a nice to have.