Backend Engineer — High-Scale Data Crawler (Rust)

Lighthouse; Oost-Vlaanderen; Hybrid; EUR 70,000 - 110,000; Full time

Benefits offered by this job

Flexible working environment

Global offices

Health insurance

Pension

Mobility budget

Lunch vouchers

Eco vouchers

Referral bonuses

Job summary

Lighthouse is seeking a Backend Engineer for the Crawler team in Ghent, Belgium. You will design and develop high-performance data acquisition systems, primarily in Rust, handling billions of data points from the web and integrating with our in-house proxy and browser tooling.

Join a collaborative team building scalable, secure infrastructure. You will own code quality, contribute to tests, observability, and work closely with DevOps, Security, and crawler engineers to deliver reliable data

Qualifications

• Production backend services in a cloud environment.
• Proficiency with systems programming languages and a drive to deep-dive in Rust.
• Solid grasp of HTTP, TCP/IP, TLS, and proxy behavior.

Responsibilities

• Design and develop high-performance data-acquisition backends, mostly in Rust.
• Build and improve the in-house proxy layer and browser tooling for data collection.
• Create robust, reusable systems by escaping blocked sources and testing alternatives.
• Define clear interfaces and guardrails for AI agents to use the infrastructure.
• Own code quality, tests, observability and maintainability, including AI-assisted code.
• Collaborate with crawler engineers, DevOps and Security to ship secure, scalable solutions.

Skills

Rust

C/C++

Backend development

English (Fluent)

Debugging & profiling

Tools

TypeScript/Node.js

Python

Google Cloud

Kubernetes

Prometheus

Grafana

Job description

Lighthouse is seeking a Backend Engineer for the Crawler team in Ghent, Belgium. You will design and develop high-performance data acquisition systems, primarily in Rust, handling billions of data points from the web and integrating with our in-house proxy and browser tooling.

Join a collaborative team building scalable, secure infrastructure. You will own code quality, contribute to tests, observability, and work closely with DevOps, Security, and crawler engineers to deliver reliable data