Sr. Data Engineer

M-League; Bengaluru; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

M-League in Bengaluru is seeking a Senior Data Engineer (SDE 2) to own the design and evolution of our data warehouse. You will architect the medallion layers (bronze → silver → gold) on BigQuery, drive data modeling and design decisions, and partner with analytics, product, and game teams to turn ambiguous requirements into reliable, well-modeled datasets.

You'll work as an AI-augmented engineer—modern AI tooling is part of how this team ships, with emphasis on scalable, observable data

Qualifications

• Strong expertise in SQL and BigQuery
• Experience with data modeling and medallion architecture
• Production-grade Python and PySpark development
• Exposure to batch and streaming ETL pipelines
• Proficiency with Kafka-based ingestion and data quality checks

Responsibilities

• Own design of silver/gold tables, partitioning and clustering decisions
• Build and maintain batch and streaming ETL pipelines
• Document designs and decisions for long-term durability
• Collaborate with analytics, product, and game teams to translate requirements

Skills

SQL

BigQuery

Python

PySpark

Kafka

Data Modeling

Education

Bachelor's/Master's in CS/Engineering

Tools

Dataproc

Databricks

GCP

Job description

We are looking for a Senior Data Engineer (SDE 2) to own the design and evolution of our data warehouse. You will architect and build the medallion layers (bronze → silver → gold) on BigQuery, drive data modeling and design decisions, and partner directly with analytics, product, and game teams to turn ambiguous requirements into reliable, well-modeled datasets. You'll work as an AI-augmented engineer—modern AI tooling is part of how this team ships.

The core responsibilities for the job include the following:

Data Warehousing and Modeling:

• Own the design of silver/gold layer tables: grain, join keys, schema, partitioning, and clustering decisions for large-scale datasets.
• Build and maintain batch and streaming ETL pipelines (using big data technologies) feeding the medallion architecture.
• Design idempotent, observable pipelines with QC gates, backfill strategies, and clear failure semantics.
• Standardize data models and conventions across multiple game products/projects.
• Deliver high-quality data pipelines with utmost importance to the correctness and availability of the data to different target audiences/systems.

Design Decisions and Technical Ownership:

• Lead source-system profiling and drive grain, deduplication, and enrichment decisions backed by data, not assumptions.
• Own trade-off calls: cost vs. freshness, load strategy, schema evolution, and query/slot optimization.
• Document designs and decisions so they survive beyond the author.

Stakeholder Management:

• Partner with analysts, PMs, and game teams to translate business questions into warehouse requirements and reconcile instrumentation gaps.
• Communicate design proposals and data profiling findings to both technical and leadership audiences.

Requirements:

• Experience: 2+ years in data engineering, with deep data warehousing exposure at scale.
• SQL and BigQuery: Expert-level SQL; hands-on BigQuery optimization (partitioning, clustering, slots, cost).
• Data Modeling: Strong command of medallion architecture, dimensional/fact modeling, grain definition, and schema design.
• Programming: Production-grade Python; PySpark on Dataproc (or Databricks/EMR equivalents).
• Streaming: Working experience with Kafka-based ingestion (Confluent preferred).
• Data Quality: Experience building validation, observability, and reconciliation into pipelines.
• Stakeholder Skills: Proven ability to gather requirements, challenge assumptions, and communicate trade-offs clearly.
• Education: Bachelor's/Master's in CS, engineering, or a related field (or equivalent experience).

AI-Augmented Engineering (Must-Have):

• We expect AI tooling to be part of your daily workflow, with sound judgment on when to trust and when to verify:
• Hands-on daily use of AI coding agents: Claude Code / CLI, Claude web app, or equivalents (Cursor, Copilot, Gemini CLI).
• Experience creating reusable AI assets: custom skills/commands, prompt libraries, or repo-level agent context.
• Familiarity with MCP or similar integrations connecting agents to warehouses and internal tools.
• Ability to demonstrate real examples of AI-accelerated work and where you drew the human-review line.

Preferred:

• LLM-in-pipeline experience: AI-driven data quality checks, anomaly triage, or text-to-SQL for self-service analytics.
• Broader GCP: Pub/Sub, Cloud Run, Cloud Functions, Cloud SQL.
• Gaming, fintech, or other high-volume consumer event domains.