Lead Data Engineer

Venpa Global Technologies; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

A leading technology firm is seeking a Lead Data Engineer to support and enhance enterprise-grade data products. This role requires expertise in AWS and strong data engineering skills, as well as team management experience. The ideal candidate will engage with clients, lead cross-functional teams, and implement best practices in data solutions. This position offers an opportunity to work in a dynamic environment and drive technical decisions.

Qualifications

• 8+ years of experience in Data Engineering/Application Support and Development.
• 4+ years of strong hands-on expertise in AWS ecosystem.
• Proficiency in PySpark, SQL, and Python.
• Strong application debugging skills.
• Excellent communication skills with proven client-facing capabilities.

Responsibilities

• Act as the primary technical lead for support and enhancement of assigned products.
• Design data engineering solutions end to end.
• Drive best practices in code performance tuning.
• Lead a cross-functional team of engineers.
• Implement engineering best practices including CI/CD.

Skills

Data Engineering

AWS expertise

PySpark

SQL

Python

Incident management

Leadership

Communication

Documentation

CI/CD

Tools

AWS Glue

AWS Lambda

AWS Redshift

AWS S3

JIRA

Confluence

Git

Jenkins

Job description

ROLE- Lead Data Engineer

LOCATION- Mumbai / Pune

Experience- 8+ yrs

Job Summary:

We are seeking a technically strong and delivery-focused Lead Engineer to support and enhance enterprise‑grade data and application products under the Durables model. The ideal candidate will act as the primary technical interface for the client, ensuring high system availability, performance, and continuous improvement. This role requires a hands‑on technologist with strong team management experience, cloud (AWS) expertise, and excellent communication skills to handle client interactions and drive technical decisions.

Key Responsibilities:

Support & Enhancement Leadership

• Act as the primary technical lead for support and enhancement of assigned products in the Durable portfolio.
• Ensure incident resolution, problem management, and enhancement delivery within agreed SLAs.
• Perform root cause analysis (RCA) and provide technical solutions to recurring issues.
• Should design data engineering solutions end to end. Ability to come up with scalable and modular solutions.
• Experience working in Agile implementations.

Technical Ownership

• Provide technical direction and architectural guidance for improvements, optimizations, and issue resolutions.
• Drive best practices in code performance tuning, ETL processing, and cloud‑native data management.
• Lead the modernization of legacy data pipelines and applications by leveraging AWS services (Glue, Lambda, Redshift, S3, EMR, Athena, etc.).
• Leverage PySpark, SQL, Python, and other tools to manage big data processing pipelines efficiently.

Client Engagement

• Maintain high visibility with client stakeholders, act as a trusted technical advisor.
• Proactively identify and suggest improvements or innovation areas to improve business outcomes.
• Participate in daily stand‑ups, retrospectives, and client presentations; communicate technical concepts clearly.

Team & Delivery Management

• Lead a cross‑functional team of engineers, ensuring effective task allocation, mentorship, and upskilling.
• Monitor team performance, support capacity planning, and ensure timely and high‑quality deliveries.
• Ensure adherence to governance, documentation, change management practices and High Availability.

Process & Quality Assurance

• Implement and ensure compliance with engineering best practices including CI/CD, version control, and automated testing.
• Define support procedures, documentation standards, and ensure knowledge transition and retention within the team.
• Identify risk areas, dependencies, and propose mitigation strategies.

Required Skills & Qualifications:

• 8+ years of experience in Data Engineering / Application Support and Development.
• 4+ Strong hands‑on expertise in AWS ecosystem (Glue, Lambda, Redshift, S3, Athena, EMR, CloudWatch).
• Proficiency in PySpark, SQL, Python, and handling big data pipelines.
• Strong application debugging skills across batch and near real‑time systems.
• Good knowledge of incident lifecycle management, RCA, and performance optimization.
• Proven experience leading engineering teams (3+ years), preferably in support/enhancement environments.
• Excellent communication skills with proven client‑facing capabilities.
• Strong documentation and process adherence mindset.
• Experience with tools like JIRA, Confluence, Git, Jenkins, or any CI/CD pipeline.

Good to Have:

• Experience in on‑prem to AWS migration projects.
• Familiarity with legacy tech and their interaction with modern cloud stacks.
  • Good knowledge of designing Hive tables with partitioning for performance.

Share profiles to [e-mail] / 9500272455 - Anushya Natarajan