Staff Data Engineer - Large-Scale Social Crawler

nolimit; Kota Bandung; On-site; IDR 180,000,000 - 340,000,000; Full time

Job summary

nolimit is building a robust, large-scale social media crawler and maintaining daily operations with the data quality team to ensure timely data delivery. The role requires strong SQL, Scala experience, and familiarity with distributed data technologies like Kafka, Redis, Elasticsearch, and relational databases.

Fresh graduates welcome; work with cross-functional stakeholders to clarify data requirements. You will optimize data pipelines, monitor system health, and contribute to scalable

Qualifications

• Strong SQL proficiency with data quality mindset.
• Experience handling large volumes of data from social media.
• Proficient in production-level Scala (not Spark).
• Familiar with MariaDB, Postgres, Redis, Kafka, Elasticsearch.
• Comfort with Linux-based systems and web data acquisition.

Responsibilities

• Build robust large-scale social media crawlers.
• Maintain day-to-day crawler operations with data quality team.
• Ensure completeness and timeliness of incoming data.
• Collaborate with stakeholders to clarify data requirements.

Skills

SQL

Scala

Linux

Data processing

API usage

Education

Fresh Graduates Welcome

Tools

MariaDB

PostgreSQL

Redis

Kafka

Elasticsearch

Job description

nolimit is building a robust, large-scale social media crawler and maintaining daily operations with the data quality team to ensure timely data delivery. The role requires strong SQL, Scala experience, and familiarity with distributed data technologies like Kafka, Redis, Elasticsearch, and relational databases.

Fresh graduates welcome; work with cross-functional stakeholders to clarify data requirements. You will optimize data pipelines, monitor system health, and contribute to scalable