Medior/Senior Data Engineer (Web Scrapping)

NucleusX B.V; Denpasar; On-site; IDR 1.001.669.000 - 1.502.505.000; Full time

Job summary

A leading data technology firm located in Bali is seeking a Medior/Senior Data Engineer to develop and maintain scalable web scrapers. The role involves mentoring engineers, optimizing cloud infrastructure, and ensuring coding standards are met. Candidates must have strong skills in Python, SQL, and experience with various scraping frameworks. Attractive benefits include relocation reimbursement for candidates moving from other parts of Indonesia.

Benefits

One-time reimbursement for flights and relocation expenses

Work directly on-site with a passionate team

Collaborate in a data-driven environment

Qualifications

• Strong proficiency in Python and SQL, with hands-on web scraping experience.
• Deep knowledge of frameworks like Scrapy, Playwright, Selenium, and anti-bot strategies.
• Experience designing reusable libraries and modular scraper frameworks.

Responsibilities

• Lead the development and maintenance of scalable, production-grade web scrapers.
• Implement robust monitoring and alerting systems to ensure reliability.
• Manage and optimize cloud infrastructure for large-scale scraping workloads.

Skills

Python

SQL

Scrapy

Playwright

Selenium

Cloud infrastructure (AWS/GCP/Azure)

Docker

Kubernetes

CI/CD Pipelines

Git workflows

Tools

Terraform

CloudFormation

Job description

About the Role

We are looking for a Medior/Senior Data Engineer to join our team in Bali. In this role, you will play a key part in building and scaling production-grade web scrapers and supporting the Lead Data Engineer in technical decisions, project planning, and team coordination. You will also mentor junior and medior engineers while ensuring best practices are followed.

Responsibilities

• Lead the development and maintenance of scalable, production-grade web scrapers.
• Design, build, and maintain reusable scraper libraries and frameworks.
• Define coding standards, workflows, and processes with the Lead Engineer.
• Implement robust monitoring, testing, and alerting systems to ensure reliability.
• Manage and optimize cloud infrastructure (AWS/GCP/Azure) for large-scale scraping workloads.
• Optimize compute, storage, and proxy usage for performance and cost efficiency.
• Mentor and review code of junior and medior engineers.
• Collaborate with stakeholders to prioritize backlog items and integrate scraper outputs into pipelines.
• Act as second in command to the Lead Data Engineer for technical and project support.

Requirements

Must-Have Skills:

• Strong proficiency in Python and SQL, with hands-on web scraping experience.
• Deep knowledge of frameworks like Scrapy, Playwright, Selenium, and anti-bot strategies.
• Experience designing reusable libraries, coding standards, and modular scraper frameworks.
• Proficiency with cloud infrastructure (AWS/GCP/Azure), including compute, storage, and security basics.
• Experience with Docker & Kubernetes for containerization and orchestration.
• Familiarity with CI/CD pipelines, Git workflows, and IaC tools (Terraform/CloudFormation).
• Strong debugging, performance tuning, and large-scale scraper optimization skills.

Nice-to-Have Skills:

• Workflow orchestration (Airflow, Prefect, Dagster).
• Observability/monitoring tools (Prometheus, Grafana, CloudWatch, ELK).
• Cost optimization in cloud and proxy-heavy workloads.
• Knowledge of data governance, RBAC, and compliance.

Why You’ll Love Working Here

Be part of a data-driven company that values curiosity, precision, and impact.

Collaborate with talented data engineers and analysts to solve real business challenges.

Based in beautiful Bali, working directly on-site with a passionate and talented team.

If relocating from other parts of Indonesia, we offer one-time reimbursement for flights and relocation expenses.