Data Engineer Staff

nolimit; Kota Bandung; On-site; IDR 180,000,000 - 340,000,000; Full time

Job summary

nolimit is building a robust, large-scale social media crawler and maintaining daily operations with the data quality team to ensure timely data delivery. The role requires strong SQL, Scala experience, and familiarity with distributed data technologies like Kafka, Redis, Elasticsearch, and relational databases.

Fresh graduates welcome; work with cross-functional stakeholders to clarify data requirements. You will optimize data pipelines, monitor system health, and contribute to scalable

Qualifications

• Strong SQL proficiency with data quality mindset.
• Experience handling large volumes of data from social media.
• Proficient in production-level Scala (not Spark).
• Familiar with MariaDB, Postgres, Redis, Kafka, Elasticsearch.
• Comfort with Linux-based systems and web data acquisition.

Responsibilities

• Build robust large-scale social media crawlers.
• Maintain day-to-day crawler operations with data quality team.
• Ensure completeness and timeliness of incoming data.
• Collaborate with stakeholders to clarify data requirements.

Skills

SQL

Scala

Linux

Data processing

API usage

Education

Fresh Graduates Welcome

Tools

MariaDB

PostgreSQL

Redis

Kafka

Elasticsearch

Job description

Building robust large-scale social media crawler for various social media platforms. Maintain day-to-day operation of said crawlers alongside our data quality team to make sure the completeness and timeliness of incoming data. Work in team along with several stakeholders to better understand and communicate about data requirements and issues.

Key responsibilities

• Building robust large-scale social media crawler for various social media platforms
• Maintain day-to-day operation of crawlers alongside data quality team
• Ensure completeness and timeliness of incoming data
• Work with stakeholders to understand and communicate data requirements and issues

About you

• Intermediate or advanced understanding of SQL
• Experience in processing large volumes of data from social media and online media platforms
• Good experience in using Scala for production (not related to Spark)
• Intermediate or advanced understanding of several data-related technologies such as MariaDB, Postgres, Redis, Kafka, and Elasticsearch
• Ample experience using Linux-based systems
• Understanding of several data acquisition techniques such as webscraping, API-based data consumption
• Fresh Graduates Welcome