Data Engineer | Remote

Crossing Hurdles; Indonesia; On-site; IDR 180,000,000 - 320,000,000; Full time

Job summary

Crossing Hurdles is seeking a data engineer to design, build, and maintain scalable data pipelines across multiple sources. You will collaborate with AI researchers to structure datasets for experimentation and model training, while optimizing data models and storage for large-scale datasets.

The role requires strong Python and SQL skills, experience with ETL/ELT pipelines, and familiarity with PostgreSQL or MySQL.

Qualifications

• Strong proficiency in Python and SQL.
• Experience designing and maintaining ETL / ELT pipelines.
• Solid experience with data manipulation libraries such as Pandas and NumPy.
• Experience working with structured and semi-structured datasets.
• Familiar with relational databases such as PostgreSQL or MySQL.

Responsibilities

• Design, build, and maintain scalable data pipelines to ingest, process, and transform data from multiple sources.
• Collaborate with AI researchers and data scientists to structure and prepare datasets for experimentation and model training.
• Develop and maintain data models, schemas, and storage systems optimized for large-scale datasets.
• Write efficient SQL queries and Python scripts to extract, transform, and analyze data.
• Ensure data quality, integrity, and reliability across data pipelines and storage levels.
• Implement data validation, monitoring, and automation workflows that support iterative research cycles.

Skills

Python

SQL

Pandas

NumPy

Tools

PostgreSQL

MySQL

Job description

• Design, build, and maintain scalable data pipelines to ingest, process, and transform data from multiple sources.
• Collaborate with AI researchers and data scientists to structure and prepare datasets for experimentation and model training.
• Develop and maintain data models, schemas, and storage systems optimized for large-scale datasets.
• Write efficient SQL queries and Python scripts to extract, transform, and analyze data.
• Ensure data quality, integrity, and reliability across data pipelines and storage layers.
• Implement data validation, monitoring, and automation workflows that support iterative research cycles.

Requirements:

• Have strong proficiency in Python and SQL.
• Have experience designing and maintaining ETL / ELT pipelines.
• Have solid experience with data manipulation libraries such as Pandas and NumPy.
• Have experience working with structured and semi-structured datasets.
• Be familiar with relational databases such as PostgreSQL or MySQL.