Manager, Data Engineering

Vanguard Technologies, LTD.; Hyderabad; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Vanguard Technologies, LTD. in Hyderabad, Telangana seeks a Manager, Data Engineering to lead a team of data engineers and newer data analysts. You will design, develop and optimize scalable data pipelines and infrastructure to enable high-performance data workflows and analytics across the organization.

The role requires strong leadership, hands-on expertise in ETL, data warehousing and cloud platforms, and the ability to translate business needs into reliable data products and platforms.

Qualifications

• Minimum 8 years of relevant work experience, with at least 5 years in a leadership role managing data engineering teams and large-scale data infrastructure.
• Bachelor s degree (B.E./B.Tech) in Computer Science, IT, or a related field or Diploma in Data Science/Data Engineering.
• Strong expertise in ETL development, data warehousing and real-time data processing frameworks.
• Hands-on experience with SQL, Python, Spark, Kafka and cloud platforms (AWS, Azure, Google Cloud).
• Proficiency in database technologies (PostgreSQL, MySQL, MongoDB, Snowflake, Big Query) and distributed computing frameworks.
• Proven experience leading teams in a product-based environment with responsibility for delivery execution and stakeholder management.
• Strong understanding of Agile delivery practices, backlog management, sprint planning, and Jira.
• Experience leading data-focused technology initiatives, including data engineering, analytics, data quality, governance, or modern data platforms.
• Strong people leadership, coaching, and talent development skills.
• Excellent stakeholder management, communication, and influencing abilities.
• Demonstrated ability to drive execution, manage ambiguity, solve complex problems, and lead organisational change.
• Ability to understand technical concepts and effectively partner with engineering teams to deliver business outcomes.
• Experience in financial services or other highly regulated industries.
• Exposure to cloud-based data platforms and AI-enabled technology initiatives.
• Agile, Scrum, PMP, SAFe, or related certifications preferred.
• Experience leading cloud data platform, data modernization, or platform transformation initiatives.

Responsibilities

• Hire, mentor and manage a team of data engineers, fostering a culture of innovation, collaboration and continuous learning in data engineering best practices.
• Design, develop and maintain scalable ETL (Extract, Transform, Load) pipelines to efficiently process and integrate structured and unstructured data from multiple sources.
• Build and maintain scalable, fault-tolerant data architectures, ensuring efficient data storage, retrieval and processing across cloud and on-premises environments.
• Oversee the design, implementation and performance tuning of relational and NoSQL databases, ensuring data consistency, security and accessibility.
• Automate data ingestion, transformation and validation processes to enhance data pipeline efficiency and reduce manual intervention.
• Develop cloud-based data solutions using platforms like AWS, Azure, or Google Cloud, leveraging technologies such as Spark, Hadoop and Kafka for large-scale data processing.
• Work closely with data scientists, analysts and business intelligence teams to understand data requirements and develop scalable solutions for analytics and reporting.
• Proactively monitor data pipelines and systems for performance issues, troubleshoot failures and implement preventive measures for system reliability.
• Partner with business and IT leaders to align data engineering initiatives with company goals, ensuring seamless integration of data-driven solutions across departments.
• Lead, coach, and develop high-performing data and technology teams, including hiring, performance management, workforce planning, and talent development.
• Drive delivery of strategic initiatives across data engineering, data warehousing, data lakes / lakehouses, data integration, data governance, data quality, analytics, and business intelligence.
• Partner with business, product, architecture, and engineering stakeholders to translate priorities into product roadmaps, delivery plans, and measurable outcomes.
• Lead product-oriented teams with end-to-end accountability for the development, delivery, maintenance, production support, reliability, and continuous improvement of data products.
• Drive Agile practices, including intake management, backlog prioritization, estimation, sprint and release planning, and delivery governance.
• Manage capacity planning, resource forecasting, delivery commitments, risks, issues, and cross-team dependencies to enable predictable execution.
• Leverage AI-enabled tools and capabilities to improve personal and team productivity, automate and streamline day-to-day workflows, and accelerate responsible AI adoption across the team by identifying practical use cases, encouraging experimentation, and embedding effective AI practices into how the team works.
• Ensure data solutions are scalable, secure, reliable, reusable, and compliant with applicable technology, risk, privacy, and information security standards.
• Promote effective data governance, data quality, metadata management, lineage, ownership, and stewardship practices.
• Monitor delivery, quality, and operational health through relevant metrics, and provide clear, concise updates to stakeholders and leadership.
• Foster strong stakeholder relationships and communicate complex data and technology topics effectively to technical and non-technical audiences.
• Promote automation, standardization, engineering excellence, cost optimization, and modern data practices, including cloud and AI-enabled capabilities.
• Drive a culture of accountability, collaboration, innovation, continuous learning, and business-outcome ownership.
• Provide leadership during major incidents and critical production events, partnering with support and engineering teams to drive timely resolution, stakeholder communication, and service recovery while ensuring operational continuity.

Skills

SQL

Python

Spark

Kafka

ETL development

Data warehousing

Real-time processing

Jira

Agile practices

Leadership

Stakeholder management

Cloud data platforms

Education

Bachelor's degree in CS/IT

Diploma in Data Science/Data Engineering

Tools

PostgreSQL

MySQL

MongoDB

Snowflake

BigQuery

AWS

Azure

Google Cloud

Hadoop

Job description

Job Summary

Manages the activities of a team of data analysts and newer data engineers responsible for developing software code that formats, tags, processes, stores and serves data. Manages the development of data pipelines to ensure that they are scalable, repeatable, and secure and can serve multiple groups at Vanguard.

Role Summary

The Manager, Data Engineering is responsible for leading a team of data engineers to design, develop, and optimize scalable data pipelines and infrastructure. This role focuses on enabling high-performance data workflows, ensuring seamless data integration and driving automation to enhance data accessibility.

Responsibilities

• Hire, mentor and manage a team of data engineers, fostering a culture of innovation, collaboration and continuous learning in data engineering best practices.
• Design, develop and maintain scalable ETL (Extract, Transform, Load) pipelines to efficiently process and integrate structured and unstructured data from multiple sources.
• Build and maintain scalable, fault-tolerant data architectures, ensuring efficient data storage, retrieval and processing across cloud and on-premises environments.
• Oversee the design, implementation and performance tuning of relational and NoSQL databases, ensuring data consistency, security and accessibility.
• Automate data ingestion, transformation and validation processes to enhance data pipeline efficiency and reduce manual intervention.
• Develop cloud-based data solutions using platforms like AWS, Azure, or Google Cloud, leveraging technologies such as Spark, Hadoop and Kafka for large-scale data processing.
• Work closely with data scientists, analysts and business intelligence teams to understand data requirements and develop scalable solutions for analytics and reporting.
• Proactively monitor data pipelines and systems for performance issues, troubleshoot failures and implement preventive measures for system reliability.
• Partner with business and IT leaders to align data engineering initiatives with company goals, ensuring seamless integration of data-driven solutions across departments.
• Lead, coach, and develop high-performing data and technology teams, including hiring, performance management, workforce planning, and talent development.
• Drive delivery of strategic initiatives across data engineering, data warehousing, data lakes / lakehouses, data integration, data governance, data quality, analytics, and business intelligence.
• Partner with business, product, architecture, and engineering stakeholders to translate priorities into product roadmaps, delivery plans, and measurable outcomes.
• Lead product-oriented teams with end-to-end accountability for the development, delivery, maintenance, production support, reliability, and continuous improvement of data products.
• Drive Agile practices, including intake management, backlog prioritization, estimation, sprint and release planning, and delivery governance.
• Manage capacity planning, resource forecasting, delivery commitments, risks, issues, and cross-team dependencies to enable predictable execution.
• Leverage AI-enabled tools and capabilities to improve personal and team productivity, automate and streamline day-to-day workflows, and accelerate responsible AI adoption across the team by identifying practical use cases, encouraging experimentation, and embedding effective AI practices into how the team works.
• Ensure data solutions are scalable, secure, reliable, reusable, and compliant with applicable technology, risk, privacy, and information security standards.
• Promote effective data governance, data quality, metadata management, lineage, ownership, and stewardship practices.
• Monitor delivery, quality, and operational health through relevant metrics, and provide clear, concise updates to stakeholders and leadership.
• Foster strong stakeholder relationships and communicate complex data and technology topics effectively to technical and non-technical audiences.
• Promote automation, standardization, engineering excellence, cost optimization, and modern data practices, including cloud and AI-enabled capabilities.
• Drive a culture of accountability, collaboration, innovation, continuous learning, and business-outcome ownership.
• Provide leadership during major incidents and critical production events, partnering with support and engineering teams to drive timely resolution, stakeholder communication, and service recovery while ensuring operational continuity.
• The Manager, IT Delivery is responsible for leading high-performing data and technology teams and delivering strategic data initiatives that generate measurable business value. This role combines people leadership, product delivery, stakeholder management, and operational oversight. The manager will lead teams operating across data engineering, data platforms, data warehousing, data lakes, data governance, data quality, and analytics. The manager will have end-to-end accountability for the development, delivery, maintenance, production support, reliability, and continuous improvement of data products. The successful candidate will bring strong leadership and communication capabilities, experience managing product-oriented teams, and a sound understanding of modern data ecosystems. The role requires the ability to translate business priorities into executable delivery plans, develop engineering talent, manage delivery commitments, and ensure the scalability, quality, security, and reliability of data solutions.

Qualifications and Skills

• Minimum 8 years of relevant work experience, with at least 5 years in a leadership role managing data engineering teams and large-scale data infrastructure.
• Bachelor s degree (B.E./B.Tech) in Computer Science, IT, or a related field or Diploma in Data Science/Data Engineering.
• Strong expertise in ETL development, data warehousing and real-time data processing frameworks.
• Hands-on experience with SQL, Python, Spark, Kafka and cloud platforms (AWS, Azure, Google Cloud).
• Proficiency in database technologies (PostgreSQL, MySQL, MongoDB, Snowflake, Big Query) and distributed computing frameworks.
• Proven experience leading teams in a product-based environment with responsibility for delivery execution and stakeholder management.
• Strong understanding of Agile delivery practices, backlog management, sprint planning, and Jira.
• Experience leading data-focused technology initiatives, including data engineering, analytics, data quality, governance, or modern data platforms.
• Strong people leadership, coaching, and talent development skills.
• Excellent stakeholder management, communication, and influencing abilities.
• Demonstrated ability to drive execution, manage ambiguity, solve complex problems, and lead organisational change.
• Ability to understand technical concepts and effectively partner with engineering teams to deliver business outcomes.
• Experience in financial services or other highly regulated industries.
• Exposure to cloud-based data platforms and AI-enabled technology initiatives.
• Agile, Scrum, PMP, SAFe, or related certifications preferred.
• Experience leading cloud data platform, data modernization, or platform transformation initiatives.

Location

Hyderabad, Telangana