Data Engineer

TUNTUN SEKURITAS INDONESIA; Pluit; On-site; IDR 200.000.000 - 300.000.000; Full time

Job summary

A leading Indonesian data solutions company is seeking a Data Engineer to develop and maintain data extraction frameworks. The ideal candidate will have over 2 years of experience, proficient in programming languages like Python and SQL, with a strong understanding of web scraping technologies and ETL processes. You will work closely with cross-functional teams to ensure quality data delivery and drive data-driven decision-making within the organization.

Qualifications

• More than 2 years of relevant experience or significant internship.
• Familiarity with ETL (Extract, Transform, Load) processes.
• Understanding of data warehousing concepts.
• Experience of extracting data from HTML and web crawling.

Responsibilities

• Extract data using web scraping and API integrations.
• Develop and maintain scalable data crawlers for diverse sources.
• Ensure data quality through validation processes.
• Document technical specifications and workflows.
• Collaborate with teams to understand data requirements.

Skills

Web scraping

API integrations

Data pipelines

Programming (Python, Java, SQL)

Data quality validation

Problem-solving

Analytical abilities

Job description

Overview

Extract data using web scraping and API integrations. Develop and maintain scalable data crawlers for diverse sources. Develop data pipelines for efficient data extraction, transformation, and loading (ETL) processes. Ensure data quality through validation processes. Document technical specifications and workflows. Collaborating with data analysts, data architects, and software engineers to understand data requirements and deliver relevant data sets. Participating in data governance and compliance efforts to meet regulatory requirements. Collaborating with cross-functional teams to drive data-driven decision-making within the organization.

Responsibilities

• Extract data using web scraping and API integrations.
• Develop and maintain scalable data crawlers for diverse sources.
• Developing and maintaining data pipelines for ETL processes.
• Ensure data quality through validation processes.
• Document technical specifications and workflows.
• Collaborate with data analysts, data architects, and software engineers to understand data requirements and deliver relevant data sets.
• Participate in data governance and compliance efforts to meet regulatory requirements.
• Collaborate with cross-functional teams to drive data-driven decision-making within the organization.

Qualifications

• More than 2 years of relevant experience or significant internship.
• Proficiency in programming languages like Python, Java, or SQL.
• Basic knowledge of data modeling and database management.
• Familiarity with ETL (Extract, Transform, Load) processes.
• Understanding of data warehousing concepts.
• Must have experience of extracting data from HTML, and web crawling.
• Strong problem-solving and analytical abilities.
• Eagerness to learn and adapt to new data technologies and tools.