Lead Data Engineer

Indxx; Gurugram District; On-site; INR 3,600,000 - 5,200,000; Full time

Job summary

Indxx in Gurgaon is seeking a Lead Data Engineer to drive the development of data warehouses, data pipelines, ETL processes, and data integrations to support analytics and BI needs. You will design, build, and maintain efficient data pipelines, transforming raw data into actionable insights and ensuring data accessibility.

You will collaborate with developers, analysts, and business stakeholders to optimize data flow across the organization, leveraging cloud platforms (AWS, GCP, Azure) and

Qualifications

• Min. 4+ years of experience building data-intensive applications.
• Strong SQL writing and optimization; SQL Server Stored Procedures.
• Proficiency in Python or Scala for data processing.
• Experience with data warehousing solutions like Snowflake, Redshift, or BigQuery.
• Familiarity with cloud platforms and their data services.
• Experience with ETL orchestration tools like Apache Airflow and DBT.
• Knowledge of data governance, data quality, and security best practices.

Responsibilities

• Develop, optimize, and maintain scalable data pipelines and ETL processes for analytics and reporting.
• Build and manage data infrastructure for storage, transformation, and retrieval of diverse data sources.
• Collaborate with developers and analysts to ensure data accuracy, quality, and availability.
• Design, implement, and maintain data architecture, databases, and warehouses for BI and analytics.
• Monitor and troubleshoot data pipelines to ensure performance and reliability.
• Develop and enforce data quality standards, governance policies, and security protocols.

Skills

SQL proficiency & optimization

Python

Scala

Data modeling

Cloud data platforms

ETL/ELT development

Data governance & quality

CI/CD for data pipelines

Tools

Snowflake

Redshift

BigQuery

Airflow

DBT

SSIS

Power BI

MySQL

PostgreSQL

MongoDB

Cassandra

S3

EMR

Job description

Role Summary: We are looking for a Lead Data Engineer to join our team and drive the development of data warehouses, data pipelines, ETL processes, and data integrations to support our data analytics and business intelligence needs. In this role, you will design, build, and maintain efficient and reliable data pipelines, transforming raw data into actionable insights. You will work closely with developers, analysts, and business stakeholders to optimize data flow and ensure data accessibility across the organization.

Location: Gurgaon

• Develop, optimize, and maintain scalable data pipelines and ETL processes to support ongoing data analytics, and reporting.
• Build and manage data infrastructure, ensuring the efficient storage, transformation, and retrieval of structured and unstructured data from multiple sources.
• Collaborate with developers, analysts, and other engineers to ensure data accuracy, quality, and availability for analysis and reporting.
• Design, implement, and maintain data architecture, databases, and warehouses, supporting business intelligence and advanced analytics requirements.
• Monitor and troubleshoot data pipeline performance and resolve any issues to ensure data processing efficiency and reliability.
• Develop and enforce data quality standards, data governance policies, and data security protocols across the data lifecycle.
• Leverage cloud platforms (AWS, GCP, Azure) for data engineering tasks, implementing best practices for cloud-native data storage, processing, and pipeline automation.

Requirements

• Strong proficiency in writing and optimizing large, complicated SQL queries, and SQL Server Stored Procedures.
• Min. 4+ years of experience working and building Data-Intensive Applications.
• Proficiency in programming languages such as Python, or Scala for data processing and manipulation.
• Experience with data warehousing solutions like Snowflake, Redshift, or BigQuery.
• Familiarity with cloud platforms (AWS, GCP, or Azure) and their respective data services (e. g. S3 EMR, BigQuery).
• Strong understanding of relational and non-relational databases, including MySQL, PostgreSQL, MongoDB, and Cassandra.
• Experience in SSIS Packages and Power BI reports would be an added advantage.
• Solid understanding of data modeling, data warehousing concepts, and data governance best practices.
• Handling common database requirements such as upgrades, backup, recovery, migration, etc.
• Experience with data pipeline tools such as Apache Airflow, and DBT for ETL orchestration and automation would be an added advantage.
• Familiarity with CI/CD tools and practices for data engineering.
• Experience with data quality frameworks, monitoring, and validation techniques.