Lead Data Engineer

Lumiq; Dadri; On-site; INR 2,000,000 - 4,000,000; Full time

Benefits offered by this job

Competitive Salary Packages

Group Medical Policies

Maternity Leave

Upskilling opportunities

Job summary

LUMIQ is seeking a highly skilled Lead Data Engineer to join our BFSI-focused team. You will design and develop our cloud-based data infrastructure and lead a team of Data Engineers to deliver robust, scalable data solutions.

This Tech Lead role sits at the intersection of cloud technologies and data engineering, guiding architecture, data models, security, and performance for BFSI clients. Strong PySpark, Airflow, and AWS experience with leadership is required.

Qualifications

• 6+ years of data engineering experience required.
• Advanced SQL and PySpark proficiency essential.
• API development experience in Python preferred.
• Familiarity with Docker and Kubernetes is required.
• Experience with AWS and data pipelines in BFSI domain preferred.
• Strong leadership and people-management abilities.

Responsibilities

• Lead and mentor a team of Data Engineers to deliver data solutions.
• Design, develop, and optimize cloud-based data pipelines and ETL processes on AWS.
• Collaborate with cross-functional teams to model scalable data architectures for BFSI.
• Ensure data security, compliance, and cost-efficiency in cloud environments.
• Work with data scientists and analysts to enable access to high-quality data.
• Monitor, troubleshoot, and improve data availability and reliability.

Skills

PySpark

Airflow

AWS

SQL

Python

Docker

Kubernetes

Delta Lake

Education

Bachelor's degree in CS/Engineering

Master's preferred

Tools

Hudi/Iceberg/Delta Lake

Job description

LUMIQ is the leading Data and Analytics company in the Financial Services and Insurance (FSI) industry. We are trusted by the world's largest FSIs, including insurers, banks, AMCs, and NBFCs, to address their data challenges. Our clients include 40+ enterprises with over $10B in deposits/AUM, collectively representing about 1B customers globally.

Our expertise lies in creating next-gen data technology products to help FSI enterprises organize, manage, and effectively use data. We have consistently challenged the status quo, introducing many industry-firsts like the first enterprise data platform in Asia on cloud for a regulated entity.

Founded in 2013, LUMIQ has now completed a decade of innovation, backed by Info Edge Ventures (a JV between Temasek Holdings of Singapore and Naukri) and US-based Season 2 Ventures.

Our Culture:

At LUMIQ, we strive to create a community of passionate data professionals who aim to transcend the usual corporate dynamics. We offer you the freedom to ideate, commit, and navigate your career trajectory at your own pace.

Culture of ownership empowerment to drive outcomes.

Our culture encourages 'Tech Poetry' combining creativity and technology to create solutions that revolutionize industry. We trust our people to manage their responsibilities with minimal policy constraints. Our team is composed of the industry's brightest minds, from PhDs and engineers to industry specialists from Banking, Insurance, NBFCs, AMCs, who will challenge and inspire you to reach new heights.

Job Description:

We are seeking a highly skilled Lead Data Engineer to join our dynamic team. In this role, you will not only design and develop our cloud-based data infrastructure but also lead a team of Data Engineers to deliver robust and scalable data solutions for our BFSI customers. As a Tech Lead, you will work at the intersection of cloud technologies, data engineering, and the BFSI domain, while guiding the team to ensure project success.

Key Responsibilities:

• Lead, mentor, and manage a team of Data Engineers to deliver high-quality data solutions.
• Design, develop, and implement cloud-based data pipelines, ETL processes, and data integration solutions using platforms AWS.
• Collaborate with cross-functional teams to understand data requirements and design scalable data models and architectures that align with BFSI industry needs.
• Optimize data storage, processing, and retrieval for maximum performance and cost-efficiency in cloud environments.
• Implement data security and compliance measures to ensure the protection and integrity of sensitive BFSI data.
• Work closely with data scientists and analysts to enable seamless access to high-quality data for analytical purposes.
• Troubleshoot and resolve data-related issues, ensuring data availability and reliability for BFSI customers.
• Stay updated on industry best practices, emerging cloud technologies, and trends in the BFSI sector to drive continuous improvement and innovation.
• Manage project timelines, delegate tasks, and ensure the successful delivery of data engineering projects.

Qualifications:

• Minimum 6 years experience with good Hands on Pyspark, Airflow, AWS.
• Advanced SQL and PySpark expertise; strong understanding of Data Engineering concepts (DW, Data Lake, OLTP vs OLAP, etc.)
• API development experience (preferably in Python)
• Familiarity with Docker and Kubernetes
• Experience with Airflow
• Exposure to Hudi, Iceberg, or Delta Lake
• Strong AWS project experience(should also be adaptable to other cloud platforms if required)
• Proven experience in leading and managing a team of Data Engineers in previous roles.
• Bachelors degree in computer science, Engineering, or a related field. Master's degree preferred.
• Proficiency in programming languages like Python, Java, or spark for data processing and ETL operations.
• Experience with big data technologies (e.g., Hadoop, Spark) and data processing frameworks.
• Familiarity with BFSI industry data standards, compliance requirements, and security protocols.
• Excellent problem-solving skills and the ability to work in a collaborative, cross-functional team environment.

What do you get:

• Opportunity to contribute to an entrepreneurial culture and exposure to the startup hustler culture.
• Competitive Salary Packages.
• Group Medical Policies.
• Maternity Leave.
• Opportunities for upskilling and exposure to the latest technologies.