Data Engineer — Recommendation Engine

remotestar-team; Gurugram District; On-site; INR 1,500,000 - 2,300,000; Full time

Benefits offered by this job

Equity

Job summary

Remotestar-team in Gurgaon seeks a Data Engineer to own the data infrastructure for our real-time product recommendations. You will build pipelines that ingest external market signals for the ranking system from day one.

You will own external data pipelines, the Deal Quality Score, seasonal calendars, and the ClickHouse-based in-store event schema, while ensuring data quality and SLAs. This foundational role reports to the CTO.

Qualifications

• 3–5 years data engineering experience
• Strong Python for production-grade pipelines
• Experience building web scrapers or data collection pipelines at scale
• Hands-on with a workflow orchestrator (Airflow or Prefect)
• SQL proficiency; ClickHouse or OLAP DB a plus
• Familiarity with Redis as a serving layer and TTL design
• Comfort with AWS S3, Glue, ECS; infra setup without DevOps

Responsibilities

• Own external data pipelines—scrapers for marketplace signals
• Build the Deal Quality Score pipeline for daily cohort lookup in Redis
• Maintain a seasonal/festive calendar for trend overlays
• Design and own the in-store event schema in ClickHouse
• Build ETL infrastructure (S3 + Spark/Glue) for long-horizon data
• Own data quality and freshness SLAs for all signals

Skills

Data pipeline design

Python

SQL

Airflow/Prefect

ClickHouse

Redis serving layer

AWS

Tools

S3

Glue

ECS

Job description

The Role

We're building a recommendation engine that surfaces the right products to the right player at the right moment inside our in-game store. A core challenge: we have limited in-app behavioural data today, so the system must rely heavily on external market signals to make great recommendations from day one.

As our first Data Engineer, you will own the data infrastructure that makes this possible. You'll build the pipelines that collect, process, and serve these signals into our real-time ranking system.

This is a foundational hire. The entire recommendation engine—from Deal Quality Scores to seasonal trends—depends on the pipelines you build.

What You'll Do

• Own external data pipelines—scrapers for Flipkart/Amazon bestseller rankings, PriceHunt/Smartprix for price benchmarking, Google Trends API for brand and category demand signals
• Build the Deal Quality Score pipeline—a daily batch job that computes a competitiveness score for every product in our catalogue, stored in Redis for sub-millisecond lookup at serving time
• Maintain a seasonal and festive calendar—structured data store for trend overlays (IPL, Diwali, back-to-school, etc.)
• Design and own the in-store event schema in ClickHouse that will power behavioural cohorts as in-app data
• accumulates
• Build ETL infrastructure (S3 + Spark/Glue) for longer-horizon trend and market data
• Own data quality and freshness SLAs—you are responsible when a signal the reco engine depends on breaks silently

What We're Looking For

• 3-5 years of data engineering experience, ideally at a startup or product company
• Strong Python—you write clean, production-grade pipeline code, not just notebooks
• Experience building and maintaining web scrapers or data collection pipelines at scale
• Hands-on experience with a workflow orchestrator—Airflow, Prefect, or equivalent
• Solid SQL; experience with ClickHouse or another OLAP database is a strong plus
• Familiarity with Redis as a serving layer—you understand TTL, key design, and cache invalidation
• Comfortable with AWS-S3, Glue, ECS; you can set up infra without needing DevOps help
• You care about data quality—you monitor pipelines, set up alerts, and feel responsible when something breaks

Strong Plus (Nice to Have)

• Experience with e-commerce or marketplace data—price intelligence, product catalogues, category taxonomy
• Familiarity with recommender system data patterns
• Experience with Spark or distributed processing for larger datasets
• Prior work on gaming or consumer mobile products

Location: Gurgaon

Compensation: Competitive salary with an opportunity to get meaningful equity

Reports to: CTO