Lead Data Engineer

MADTECH.AI; Karnataka; Híbrido; INR 2.800.000 - 3.800.000; Jornada completa

Descripción de la vacante

MADTECH.AI is seeking a Lead Data Engineer to design, build, and optimize scalable data solutions powering BI and analytics. The role emphasizes SQL proficiency, modern data architectures, and hands-on cloud experience.

Expect collaboration across teams, leadership in data initiatives, and contributions to department goals. Location in India (Karnataka) with on-site work at Mysore or Bangalore.

Formación

• Bachelor's or master's degree in CS, engineering or related field.
• 7+ years of data engineering and analytics.
• 5+ years with MSSQL or MySQL (strong SQL).
• Experience with SQL and NoSQL databases.
• Production data pipelines using Python and AWS.
• On-premises to cloud data migration projects.
• Knowledge of AWS data services and dimensional modeling.
• AWS Certification in Data Analytics or ML is a plus.

Responsabilidades

• Design and develop robust data pipelines from ingestion to consumption.
• Build and optimize scalable data solutions using Python, AWS, and Snowflake.
• Redesign MSSQL codebases to improve performance and scalability.
• Lead on-premises to cloud data migration initiatives.
• Develop data platforms using AWS Redshift, S3, and Athena.
• Implement data ingestion, curation frameworks using AWS or custom solutions.
• Assess current data platforms and plan cloud transition paths.

Conocimientos

Structured thinker

Results-driven

Communication

Ownership & accountability

Time management

Collaboration

Leadership

Educación

Bachelor's or Master's in CS/Engineering

Herramientas

Python

AWS

MSSQL

MySQL

Snowflake

Redshift

S3

Athena

Glue

Descripción del empleo

Role Overview

We are seeking a highly skilled and performance-driven Lead Data Engineer to design, build, and optimize scalable data solutions that power business intelligence and advanced analytics. The ideal candidate will bring deep expertise in SQL, strong experience in modern data architectures, and hands-on exposure to cloud-based data ecosystems.

This role requires both strong technical capabilities and the ability to collaborate effectively across teams while contributing to department-level goals and initiatives.

Key Responsibilities

Data Engineering & Architecture

• Design and develop robust, production-grade data pipelines from ingestion to consumption within modern big data architectures.


• Build and optimize scalable data solutions using Python, AWS, and Snowflake.


• Redesign and enhance legacy MSSQL codebases to improve performance, scalability, and maintainability.


• Lead and execute on-premises to cloud data migration initiatives.


• Develop and operationalize enterprise-grade data platforms using AWS services such as Redshift, S3, and Athena (Glue and Snowflake exposure is an advantage).


• Implement data engineering, ingestion, and curation frameworks using AWS native services or custom solutions.


• Perform current-state assessments of data platforms and define structured transition paths to cloud environments.

Data Modeling & Quality

• Design data models and articulate trade-offs between different modeling approaches (preferably Kimball-based dimensional modeling).


• Build and maintain both ETL and ELT pipelines.


• Integrate data quality standards and validation processes into everyday workflows and delivery cycles.


• Work with structured and unstructured datasets to derive scalable patterns and solutions.

Engineering Excellence

• Write clean, test-driven, and iterative code following TDD practices.


• Leverage continuous integration and continuous delivery (CI/CD) practices to deploy and maintain data pipelines.


• Demonstrate strong understanding of distributed storage and computing technologies.

Collaboration & Leadership

• Contribute to departmental planning including goal setting, prioritization, and continuous improvement initiatives.


• Build service-oriented relationships with internal teams and external stakeholders.


• Support recruiting, mentoring, and fostering a strong engineering culture focused on quality and accountability.

Qualifications & Experience

• Bachelors or Masters degree in Computer Science, Engineering, or a related field.


• 7+ years of overall experience in Data Engineering and Analytics.


• 5+ years of hands-on experience with MSSQL or MySQL (strong SQL expertise is mandatory).


• Proven experience working with both SQL and NoSQL databases.


• Strong experience designing and building production data pipelines using Python and AWS.


• Experience with on-premises to cloud data migration projects.


• Hands-on expertise with AWS data services (Redshift, S3, Athena - mandatory; Glue and Snowflake - good to have).


• Strong knowledge of Data Warehousing concepts and Dimensional Modeling (Kimball preferred).


• Proficiency in ETL and ELT methodologies.


• AWS Certified Data Analytics or AWS Certified ML certification is an advantage.


• Exposure to Media & Marketing analytics domain is a plus.


• Familiarity with GCP or data visualization tools is desirable.

Behavioral Competencies

• Structured thinker with strong analytical orientation.


• Results-driven with a passion for data-led decision-making.


• Strong problem-solving skills with end-to-end ownership mindset.


• Excellent communication and stakeholder management skills.


• Ability to work effectively under time constraints and in late EMEA/ET hours.


• Willingness to work from Mysore or Bangalore location.