Manager

Coforge Limited; Hyderabad; Hybrid; INR 2,500,000 - 3,800,000; Full time

Job summary

Coforge invites an experienced Data Engineer Lead in Hyderabad to design and implement scalable data pipelines using Snowflake, DBT, PySpark, and AWS. You will mentor a team of data engineers and collaborate with analytics and business stakeholders to set direction and deliver reliable data platforms.

You will own ETL/ELT development, enforce data quality and governance, optimize Snowflake performance, and drive CI/CD, monitoring, and deployment of cloud-based data solutions.

Qualifications

• Strong Snowflake experience.
• Hands-on DBT (Data Build Tool).
• Advanced PySpark development skills.
• Strong AWS experience (S3, Glue, IAM).
• Proficient SQL and data modelling.
• ETL/ELT pipeline development and Python programming.

Responsibilities

• Lead end-to-end design and development of enterprise-scale data solutions.
• Build and optimize ETL/ELT pipelines using PySpark, DBT, AWS Glue, and Snowflake.
• Design scalable data architectures on AWS cloud.
• Develop and maintain data transformation models using DBT.
• Implement data ingestion frameworks from multiple sources.
• Optimize Snowflake data warehouse performance and costs.
• Collaborate with stakeholders, architects, and analytics teams.
• Ensure data quality, governance, and security best practices.
• Lead code reviews, mentor junior engineers, and drive CI/CD adoption.

Skills

SQL proficiency

Data Modelling

Python programming

ETL/ELT development

CI/CD

Mentoring

Tools

Snowflake

DBT

PySpark

AWS S3

AWS Glue

IAM

Lambda

Step Functions

Job description

Role & responsibilities

Job Description - Data Engineer Lead

Position: Data Engineer Lead

Experience: 12-14 Years

Location: Hyderabad

Employment Type: Full Time

Job Summary

We are seeking an experienced Data Engineer Lead with strong expertise in DBT, Snowflake, AWS S3, AWS Glue, PySpark, and Modern Data Engineering practices. The ideal candidate will lead the design, development, optimization, and maintenance of scalable data pipelines and cloud-based data platforms while mentoring a team of data engineers.

Key Responsibilities

• Lead end-to-end design and development of enterprise-scale data solutions.
• Build and optimize ETL/ELT pipelines using PySpark, DBT, AWS Glue, and Snowflake.
• Design scalable data architectures on AWS cloud.
• Develop and maintain data transformation models using DBT.
• Implement data ingestion frameworks from multiple structured and unstructured sources.
• Optimize Snowflake data warehouse performance, storage, and query execution.
• Work closely with business stakeholders, architects, and analytics teams.
• Ensure data quality, governance, security, and best practices.
• Lead code reviews, technical discussions, and mentor junior team members.
• Drive automation, CI/CD, monitoring, and deployment of data pipelines.

Required Skills

Must Have

• Strong experience in Snowflake
• Hands-on expertise in DBT (Data Build Tool)
• Advanced PySpark development experience
• Strong AWS experience with:
• S3
• AWS Glue
• IAM
• Lambda (Good to Have)
• Step Functions (Good to Have)
• Strong SQL and Data Modelling skills
• ETL/ELT Pipeline Development
• Python Programming
• Performance Tuning and Troubleshooting

Good to Have

• Airflow
• Kafka
• Terraform
• Databricks
• Data Governance Tools
• CI/CD Implementation