ETL Data Engineer

Lumiq; Dadri; On-site; INR 2,400,000 - 4,200,000; Full time

Benefits offered by this job

Group Medical Policies

Maternity Leave

Upskilling opportunities

Certification sponsorship

Equal Employment Opportunity

Job summary

LUMIQ is seeking a highly skilled ETL Data Engineer to re-engineer data pipelines by migrating data extraction from Microsoft SQL Server to PostgreSQL/CURA, while preserving load patterns into Elasticsearch and MongoDB. You will ensure data quality and system performance with zero end-user impact.

The role requires expertise in Python, SQL, and distributed data systems, plus hands-on experience with Docker & Kubernetes in production environments.

Qualifications

• Strong experience building and maintaining large-scale distributed data systems.
• Proficiency in Python for data analysis and manipulation (pandas, NumPy, Polars).
• Advanced SQL development with performance optimization.
• Working knowledge of Docker and Kubernetes.
• Familiarity with Elasticsearch and MongoDB as data stores.

Responsibilities

• Analyze existing ETL pipelines and dependencies on Microsoft SQL Server.
• Replicate ETL extractions to PostgreSQL (CURA) while preserving transformations and loading to Elasticsearch and MongoDB.
• Ensure data integrity, quality, and freshness after source switch.
• Write optimized SQL queries to extract from new source.
• Conduct performance testing to confirm no throughput or latency degradation.
• Containerize, orchestrate, and deploy updated ETLs using Docker and Kubernetes.
• Monitor post-deployment performance and resolve issues proactively.
• Document design, code, data mappings and runbooks.

Skills

Python

SQL

Distributed data systems

pandas

NumPy

Polars

Tools

Docker

Kubernetes

Elasticsearch

MongoDB

PostgreSQL

Job description

Who we are:

LUMIQ is the leading Data and Analytics company in the Financial Services and Insurance (FSI) industry. We aretrusted by the world's largest FSIs, including insurers, banks, AMCs, and NBFCs, to address their datachallenges. Our clients include 40+ enterprises with over $10B in deposits/AUM, collectively representingabout 1B customers globally.Our expertise lies in creating next-gen data technology products to help FSI enterprises organize, manage, andeffectively use data. We have consistently challenged the status quo, introducing many industry-firsts like thefirst enterprise data platform in Asia on cloud for a regulated entity.Founded in 2013, LUMIQ has now completed a decade of innovation, backed by Info Edge Ventures (a JV between Temasek Holdings of Singapore and Naukri) and US-based Season 2 Ventures.

Our Culture:

At LUMIQ, we strive to create a community of passionate data professionals who aim to transcend the usualcorporate dynamics. We offer you the freedom to ideate, commit, and navigate your career trajectory at your own pace. Culture of ownership empowerment to drive outcomes. Our culture encourages 'Tech Poetry' combining creativity and technology to create solutions thatrevolutionize industry. We trust our people to manage their responsibilities with minimal policy constraints. Ourteam is composed of the industry's brightest minds, from PhDs and engineers to industry specialists fromBanking, Insurance, NBFCs, AMCs, who will challenge and inspire you to reach new heights.

Role:

We are seeking a highly skilled ETL Data Engineer to re-engineer our existing data pipelines to extract data from a new data source (PostgreSQL / CURA system) instead of the current Microsoft SQL Server (CRM persistence store), while preserving the existing load patterns to Elasticsearch and MongoDB. The engineer will ensure this migration has zero impact on data quality, system performance, or end-user experience.

Key Responsibilities

• Analyze existing ETL pipelines and their dependencies on Microsoft SQL Server as source systems.
• Design and implement modifications to repoint ETL extractions from PostgreSQL (CURA) while preserving the current transformations and load logic into Elasticsearch and MongoDB.
• Ensure end-to-end data integrity, quality, and freshness remain unaffected after the source switch.
• Write efficient and optimized SQL queries to extract data from the new source.
• Conduct performance testing to confirm no degradation of pipeline throughput or latency in production.
• Work closely with DevOps and platform teams to containerize, orchestrate, and deploy the updated ETLs using Docker and Kubernetes.
• Monitor post-deployment performance and handle any production issues proactively.
• Document design, code, data mappings, and operational runbooks.

Required Skills and Qualifications

• Strong experience building and maintaining large-scale distributed data systems.
• Expert-level proficiency in Python, especially data analysis/manipulation libraries like pandas, NumPy, and Polars.
• Advanced SQL development skills with proven experience in performance optimization.
• Working knowledge of Docker and Kubernetes.
• Familiarity with Elasticsearch and MongoDB as data stores.
• Experience working in production environments with mission-critical systems.

What Do You Get:

• Opportunity to contribute to an entrepreneurial culture and exposure to the startup hustler culture.
• Competitive Salary Packages.Group Medical Policies.
• Equal Employment Opportunity.
• Maternity Leave.
• Opportunities for upskilling and exposure to the latest technologies.
• 100% Sponsorship for certification