Lead Data Engineer

SourcingXPress; Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

SourcingXPress is seeking an exceptional Engineering Lead to join the Technology team in Bengaluru, India. The role involves designing and optimizing scalable data pipelines and collaborating with cross-functional teams. The ideal candidate has 6-8 years of experience in data engineering, strong skills in Python, SQL, and cloud technologies, and proven leadership abilities. Responsibilities include overseeing data workflows, contributing to enterprise-grade software, and engaging with various stakeholders to drive efficiency and solutions.

Qualifications

• 6-8 years of experience in data engineering, fintech domain is a plus.
• Strong hands-on experience in distributed data processing.
• Advanced proficiency in Python for building scalable data pipelines.

Responsibilities

• Architect, build and maintain data solutions to meet business requirements.
• Engage and lead a team of talented technologists.
• Oversee end-to-end data workflows from ingestion to storage.

Skills

Data engineering

ETL/ELT data pipelines

Cloud technologies

Python

SQL

Agile environments

CI/CD

Communication skills

Leadership

Education

Bachelor's degree in computer science or equivalent

Tools

Spark

Hadoop

Elasticsearch

Kafka

Airflow

ADF

Glue Workflow

Job description

We are looking for a bright and exceptional Engineering Lead to join our Technology team. In this role, you will work closely with Product Managers, Subject Matter Experts, Data Scientists and Software Engineers to design, build, and optimize scalable data pipelines for our Data Platform. The ideal candidate has strong hands‑on experience in distributed data processing, expertise in architecting data pipelines and operational skills to drive efficiency and speed.

Key Responsibilities

• Work closely with the product and other teams to understand business requirements and translate them into robust engineering solutions.
• Architect, build and maintain data solutions and pipelines to meet business requirements.
• Engage and lead a team of talented technologists and establish engineering best practices.
• Act as a point of contact for managing and driving production defects and stability improvements to resolution.
• Oversee end‑to‑end data workflows from ingestion, transformation, and storage to data lake in both batch and streaming environments.
• Drive improvements in performance, reliability, and scalability of data pipelines.

Required Skills & Qualifications

• A bachelor’s degree in computer science, math, or other technical fields with 6‑8 years of experience; fintech domain is a plus.
• Demonstrated contribution to end‑to‑end delivery of enterprise‑grade software.
• Experience in data engineering, data lakes, data warehousing, data pipelines, and data analytics tools.
• Experience with open‑source big data technologies such as Spark, Hadoop, Elasticsearch, Kafka.
• Hands‑on experience in enterprise application development.
• Strong SQL skills and experience working with relational and NoSQL databases.
• Experience in cloud technologies is a plus.
• Advanced proficiency in Python for building scalable data pipelines and automating workflows.
• Experience building scalable ETL/ELT data pipelines and data lakes/warehouses.
• Familiarity with workflow orchestration tools (Airflow, ADF, Glue Workflow, etc.).
• Solid understanding of CI/CD, version control, and DevOps practices.
• Experience working in agile, collaborative environments.
• Attention to detail and quality, and ability to work well in and across teams.
• Ability to advocate and influence multiple stakeholders.
• Excellent problem‑solving, communication, and leadership skills.
• Experience leading a team of highly skilled engineers.
• Strong communication skills.