Data Engineering Manager

Good co India; India; On-site; INR 2,400,000 - 5,400,000; Full time

Job summary

Good co India is seeking a senior data engineering leader to build and scale data platforms across cloud environments. You will guide architecture, governance, and delivery for data pipelines, warehouses, lakes, and lakehouse solutions, partnering with Data Science, Analytics, Product and Platform teams.

You will own the technical direction on AWS/Azure/GCP, drive modernization, optimize costs, ensure quality and security, and mentor a high-performing engineering organization.

Qualifications

• 5 to 10 years of experience in Data Engineering or related roles with leadership.
• Proven experience managing Data Engineering teams and large-scale platforms.
• Strong expertise in Data Engineering, Architecture, Warehousing, Lakes, modeling, ETL/ELT, and pipelines.
• Hands-on with SQL and Python; Scala/Java is a plus.
• Extensive experience with Spark, PySpark, Kafka, Airflow, Databricks, Snowflake, Hadoop or equivalent.
• Cloud data platforms including AWS, Azure or GCP.
• Experience with AWS Glue, Redshift, EMR, Azure Data Factory, Azure Synapse, BigQuery, Dataflow, dbt.
• Knowledge of Data Warehouse/Lakehouse/Delta Lake, data governance and metadata.
• CI/CD, DevOps, IaC, data testing, monitoring, DataOps.
• Understanding data security, privacy, governance, lineage.
• Proven ability to optimize pipelines, distributed workloads, and cloud costs.
• Strong leadership: architecture decisions, reviews, mentoring.
• Ability to hire and develop Data Engineers and Leads.
• Strong stakeholder management and cross-functional collaboration.

Responsibilities

• Lead and manage the Data Engineering team building scalable data platforms and pipelines.
• Define and execute data engineering strategy, roadmap, architecture, and delivery plans.
• Design scalable data pipelines, warehouses, lakes, and lakehouse platforms.
• Drive batch and real-time data processing using Spark, Kafka, Airflow, Databricks, Snowflake, and cloud services.
• Establish ETL/ELT, data modeling, data quality, governance, security, and observability practices.
• Partner with Data Science, Analytics, Product, Engineering, BI, and Platform teams to deliver data solutions.
• Own architecture and direction for cloud data platforms across AWS, Azure, or GCP.
• Drive modernization: cloud migration, lakehouse adoption, pipeline automation, legacy transformation.
• Ensure platforms meet scalability, reliability, performance, security, availability, cost efficiency.
• Set standards for data architecture, coding, testing, CI/CD, deployment, monitoring, operational excellence.
• Identify and resolve data pipeline failures, bottlenecks, quality issues, and architectural challenges.
• Drive quality, lineage, governance, privacy, and compliance with security teams.
• Optimize data infrastructure and workloads for performance and cloud costs.
• Evaluate emerging data tech and recommend solutions.
• Manage engineering capacity, priorities, dependencies, and delivery commitments.
• Hire, mentor, coach Data Engineers and Technical Leads.
• Conduct performance reviews and career planning for team members.
• Communicate strategy, progress, risks, metrics, and business impact to senior leadership.

Skills

Data Engineering

Leadership

SQL

Python

Spark

Cloud Platforms

Data Modeling

Data Lakehouse

Education

Bachelor's degree in CS/IT or related

BE/B.Tech/MCA/M.Tech

Certifications: AWS/GCP/Azure Data Engineer

Tools

Spark

Databricks

Snowflake

Airflow

Kafka

BigQuery

Redshift

Azure Data Factory

CI/CD tools

Job description

Role & responsibilities

• Lead and manage the Data Engineering team responsible for building scalable, reliable, and high-performance data platforms and pipelines.
• Define and execute the data engineering strategy, technical roadmap, architecture, and delivery plans aligned with business objectives.
• Design and oversee scalable data pipelines, data warehouses, data lakes, lakehouse platforms, and data integration solutions.
• Drive development of batch and real-time data processing solutions using technologies such as Spark, Kafka, Airflow, Databricks, Snowflake, and cloud-native data services.
• Establish best practices for ETL/ELT, data modeling, data quality, data governance, data security, metadata management, and data observability.
• Partner with Data Science, Analytics, Product, Engineering, Business Intelligence, and Platform teams to deliver high-quality data solutions.
• Own the architecture and technical direction for cloud-based data platforms across AWS, Azure, or GCP.
• Drive modernization initiatives including cloud migration, lakehouse adoption, pipeline automation, and legacy data platform transformation.
• Ensure data platforms meet requirements for scalability, reliability, performance, security, availability, and cost efficiency.
• Establish standards for data architecture, coding practices, testing, CI/CD, deployment, monitoring, and operational excellence.
• Identify and resolve data pipeline failures, performance bottlenecks, data quality issues, and architectural challenges.
• Drive data quality, lineage, governance, privacy, and compliance initiatives in collaboration with security and governance teams.
• Optimize data infrastructure and workloads for performance and cloud cost efficiency.
• Evaluate emerging data technologies, tools, and frameworks and recommend solutions based on business and technical requirements.
• Manage engineering capacity, resource allocation, project priorities, technical dependencies, and delivery commitments.
• Hire, mentor, coach, and develop Data Engineers and Technical Leads to build a high-performing engineering organization.
• Conduct performance reviews, career planning, and technical mentoring for team members.
• Communicate technical strategy, project progress, risks, data platform metrics, and business impact to senior leadership.

Preferred candidate profile

• 5 to 10 years of experience in Data Engineering, Big Data, Data Platform Engineering, or related technology roles, with demonstrated leadership experience.
• Proven experience managing or leading Data Engineering teams and delivering large-scale data platforms.
• Strong expertise in Data Engineering, Data Architecture, Data Warehousing, Data Lakes, Data Modeling, ETL/ELT, and Data Pipelines.
• Strong hands‑on experience with SQL and Python; experience with Scala or Java is an added advantage.
• Strong experience with Apache Spark, PySpark, Kafka, Airflow, Databricks, Snowflake, Hadoop, or equivalent data technologies.
• Experience designing and implementing batch processing, real‑time/streaming data pipelines, distributed data processing, and data integration solutions.
• Strong knowledge of cloud data platforms such as AWS, Azure, or GCP.
• Experience with technologies such as AWS Glue, Redshift, EMR, Azure Data Factory, Azure Synapse, Google BigQuery, Dataflow, dbt, or equivalent platforms.
• Strong understanding of Data Warehouse, Data Lake, Data Lakehouse, Delta Lake, dimensional modeling, data governance, data quality, and metadata management.
• Experience implementing CI/CD, DevOps, Infrastructure as Code, data testing, monitoring, and DataOps practices.
• Strong understanding of data security, access control, privacy, compliance, data lineage, and governance.
• Proven experience optimizing data pipelines, distributed workloads, storage, query performance, and cloud infrastructure costs.
• Strong technical leadership skills with experience in architecture decisions, technical design reviews, code reviews, mentoring, and engineering best practices.
• Proven ability to hire, mentor, coach, and develop Data Engineers and Technical Leads.
• Strong stakeholder management, communication, problem-solving, decision-making, and cross-functional collaboration skills.
• Ability to work effectively with Data Scientists, BI/Analytics, Product, Engineering, Cloud, Security, and Business teams.
• Bachelor's degree in Computer Science, Information Technology, Data Engineering, Mathematics, Statistics, or a related technical discipline.
• B.E./B.Tech/MCA/M.Tech/M.Sc. or equivalent qualification preferred.
• Certifications such as AWS Data Engineer/Analytics, Azure Data Engineer, Google Professional Data Engineer, Databricks, or Snowflake are an added advantage.