Data Engineer

Integrity; Daerah Khusus Ibukota Jakarta; On-site; IDR 200.000.000 - 300.000.000; Full time

Job summary

Integrity Indonesia is seeking a passionate Data Engineer to design and build robust data pipelines, supporting data-driven decisions. In this role, you will manage data infrastructure, collaborate with data scientists, and ensure timely data availability. The ideal candidate will have a Bachelor's Degree in a relevant field and significant experience in data engineering, with skills in Python, data warehousing, and web scraping techniques.

Qualifications

• Minimum 2-5 years experience in data engineering.
• Proficiency in Bahasa Indonesia and English.
• Experience in data pipeline development using Apache Airflow, Kafka, or similar.

Responsibilities

• Build and maintain scalable data pipelines and ETL processes.
• Develop and manage data crawlers and data scraping engines.
• Ensure compliance with data security and processing standards.

Skills

Data Pipeline Development

Python

REST APIs

Data Warehousing

Web Scraping

Containerization

Data Security

Docker

Kubernetes

Education

Bachelor's Degree in Computer Science or related field

Post-graduate certification in Big Data or Cloud Technologies

Tools

Apache Airflow

Kafka

Spark

MongoDB

PostgreSQL

Elasticsearch

Scrapy

Selenium

Job description

Are you a passionate data engineer ready to build the infrastructure that powers our insights and decisions? As a Data Engineer, you will support the Data Scientist team by designing and building robust data pipelines, ensuring clean, structured, and accessible data for analytics, modeling, and decision-making
processes.

What You’ll Do:

• Build and Maintain Data Pipelines
  a. Develop, test, and maintain scalable ETL/ELT pipelines that collect and process raw data from various sources.
  b. Automate data workflows to ensure timely and accurate data availability.
  c. Integrate various data formats and APIs into centralized systems.
• Develop and Manage Data Crawlers
  a. Build and maintain scalable web crawlers and data scraping engines to collect information from public or internal websites and APIs.
  b. Ensure compliance with ethical and legal web scraping practices (robots.txt, rate limits, etc.).
  c. Monitor crawler performance and implement logging, error handling, and retry mechanisms.
• Data Preparation and Transformation
  a. Clean and preprocess raw data to ensure consistency and usability.
  b. Create structured datasets from semi-structured and unstructured data.
  c. Support feature engineering and data modeling needs of Data Scientists.
• Data Infrastructure and Tools Management
  a. Manage and maintain databases, data lakes, or data warehouses (e.g., SQL, NoSQL, cloud platforms).
  b. Set up monitoring and logging for data systems to ensure performance and reliability.
  c. Ensure systems scale with data volume and processing demand.
  d. Ensure metadata management and proper data cataloging.
• Collaboration and Support
  a. Work closely with Data Scientists to understand project data needs and deliver appropriate data sets.
  b. Partner with software engineers, developer, and business stakeholders for end-to-end data solutions.
  c. Provide documentation, version control, and data lineage for transparency and reuse.

Who We Need:

• Bachelor's Degree in Computer Science, Information Systems, Data Engineering, or related field.
• Post-graduate training or certification in Big Data, Cloud Technologies, or Machine Learning Infrastructure is a plus.
• Minimum 2–5 years of experience in data engineering, data pipeline development, or backend data.
• Experience in supporting data science or analytics teams is preferred.
• Experience building pipelines using Apache Airflow, Kafka, Spark, or similar tools.
• Strong awareness of Indonesian data compliance frameworks and methodologies.
• Familiarity with data security, data classification, and secure processing principles in accordance with ISO 27001 or similar standards.
• Bahasa Indonesia proficiency and English proficiency (written and verbal) is required.
• Mastering other languages are a plus.
• Able to work with cross-functional teams and across cultures.
• Results-driven, proactive, and comfortable with ambiguity.
• Robust nature.
• Mentor, motivator, communicator.
• Energy & dynamism.
• Results-oriented mindset with a "can-do" attitude.
• Able to handle competing objectives and multiple priorities.
• Clear thinker.
  

Technical Skills Needed:

• Deep familiarity with MongoDB, PostgreSQL, and Elasticsearch for data warehousing and indexing.
• Strong programming skills in Python and/or Golang.
• Strong understanding of REST APIs, web scraping techniques, and asynchronous job queues (Celery, etc.)
• Proficiency with containerization and orchestration tools like Docker and Kubernetes.
• Experience with web scraping or crawler engine development (e.g., using Scrapy, Selenium, or custom tools).
• Knowledge of data modeling, data warehousing, and data lake architectures.
  
  

If you meet these requirements and you are looking for a new, exciting opportunity within a successful, growing company, then this role is for you!

Integrity Indonesia is an equal opportunity employer.

* * *

Integrity Indonesia is a foreign investment company that has operated since 2001 in Indonesia, and with established affiliates in Malaysia and Thailand. Our business is to provide services related to the Prevention, Detection, and Investigation of risk.

As a rapidly expanding company with a robust business plan, Integrity Indonesia is looking for self-motivated and results-oriented individuals with solid qualifications who enjoy the challenge to grow and succeed.