Lead/Senior Data Engineer

Team Geek Solutions; Chennai District; On-site; INR 2,500,000 - 4,000,000; Full time

Benefits offered by this job

Onsite role in Chennai

Job summary

Team Geek Solutions in Chennai onsite seeks a Lead / Senior Data Engineer to design, develop, and deliver scalable data platforms. You will lead a team of engineers, shape architecture, mentor juniors, and collaborate with cross-functional teams to ensure robust data solutions.

Bring 8+ years of hands-on experience with Python, Scala, Spark, SQL, and modern data lake technologies. You will own end-to-end pipelines, enforce governance, and drive continuous improvement in a distributed global team.

Qualifications

• Minimum 8+ years hands-on experience designing and building scalable data pipelines.
• Proficient in Python, Scala, and Spark for large-scale data processing.
• Expertise with Spark, Data Lake, Delta Lake, and Hive.
• Strong SQL skills across large data volumes.
• Experience with MySQL and Snowflake.
• Familiarity with Agile methodologies.
• Bachelor’s degree in CS/Engineering/IS.

Responsibilities

• Lead end-to-end design, development, and delivery of data pipelines and components.
• Define technical strategy, architecture, and best practices for data engineering.
• Build optimized data pipelines in cloud environments and tune ETL processes.
• Architect and maintain data lake and pipeline architectures for AI/ML readiness.
• Mentor engineers, review code, and ensure high quality and performance.
• Collaborate with product, design, and QA to translate requirements into technical solutions.
• Drive continuous improvement of data platforms and tooling.

Skills

Python

Scala

Spark

SQL

Data Processing

Data Pipelines

ETL

Distributed Systems

Observability

Data Engineer

Education

Bachelor's degree in Computer Science / Engineering / Information Systems

Tools

Hive

Delta Lake

Snowflake

MySQL

Data Lake

Job description

• Shift: US Eastern Time ( 5:00 PM - 2:00 AM )
• Experience : 8+ years overall
• Interviews: Assessment || 2 Interview rounds.

• Position: Lead / Senior Data Engineer
• Location: Chennai-Onsite
• Shift: US Eastern Time ( 5:00 PM - 2:00 AM )
• Experience : 8+ years overall
• Interviews: Assessment || 2 Interview rounds.

Notice Period: Immediate Joiner/Serving Notice Period

Job Description

• Job title : Lead / Senior Data Engineer
• Experience: 8+ years

Roles And Responsibilities

Role Overview

• The Lead Data Engineer will be responsible for designing, developing, and delivering high-quality software and data solutions while leading a team of engineers. The role involves hands‑on technical work, architectural decision‑making, mentoring junior developers, and collaborating with cross‑functional teams to ensure successful delivery of scalable data platforms and analytical solutions.

Key Responsibilities

• Lead the end-to-end design, development, and delivery of software systems, data pipelines, and components.
• Define technical strategy, architecture, and best practices for development and data engineering.
• Design and build optimized data pipelines using cutting-edge technologies in a cloud environment.
• Construct infrastructure for efficient ETL processes from various sources and storage systems.
• Architect, design, and maintain database pipeline architectures, ensuring readiness for AI/ML transformations.
• Lead the implementation of algorithms and prototypes to transform raw data into useful information.
• Review code for quality, scalability, and performance.
• Collaborate with Product Managers, Business Managers, Designers, and QA teams to translate business requirements into technical solutions.
• Develop analytical tools, programs, and reporting mechanisms.
• Create data validation methods and data analysis tools.
• Interpret data trends and patterns to establish operational alerts.
• Conduct complex data analysis and present results effectively.
• Prepare data for prescriptive and predictive modeling.
• Ensure compliance with data governance and security policies.
• Troubleshoot, debug, and resolve complex technical issues.
• Drive continuous improvement in software and data development processes, tools, and methodologies.
• Mentor and guide engineers through code reviews, technical discussions, and training.
• Ensure timely delivery of projects while maintaining high engineering standards.
• Continuously explore opportunities to enhance data quality and reliability.
• Apply strong programming and problem-solving skills to develop scalable solutions.
• Demonstrate passion for testing strategy, problem-solving, and continuous learning.
• Willingness to acquire new skills and knowledge.
• Possess a product/engineering mindset to drive impactful data solutions.
• Experience working in distributed environments with global teams.
• Stay current with emerging technologies and industry trends to propose innovative solutions.

Technical Skills And Experience Requirements

• Minimum 8+ years of hands‑on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
• High proficiency in Python, Scala, and Spark for applied large-scale data processing.
• Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive.
• Solid understanding of batch and streaming data processing techniques.
• Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion.
• Expert-level ability to write complex, optimized SQL queries across extensive data volumes.
• Experience with RDBMS and OLAP databases such as MySQL and Snowflake.
• Familiarity with Agile methodologies.
• Obsession for service observability, instrumentation, monitoring, and alerting.
• Knowledge or experience in architectural best practices for building data lakes.
• Qualifications - Bachelor’s degree in computer science, Engineering, Information Systems, or related field.

Skills: sql,snowflake,delta lake,scala,python,data lake,spark,data engineer,mysql,hive