Group Data Engineer I

The Peninsular and Oriental Steam Navigation Company; Bengaluru; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

The Peninsular and Oriental Steam Navigation Company in Bengaluru seeks an experienced Data Platform Architect to lead end-to-end design of cloud-native data platforms. You will drive governance, security, and migration to modern architectures while mentoring a growing team.

Responsibilities include defining roadmaps, collaborating with data engineers and scientists, and ensuring scalable, cost-efficient solutions for real-time analytics and AI workloads.

Qualifications

• Bachelor’s or master’s degree in computer science, Engineering, Information Technology, or a related field.
• Minimum of 7+ years of experience in data architecture, data engineering, or a similar role, with a strong focus on designing large-scale data platforms.
• Proven experience in architecting cloud-based data solutions, including data lakes, data warehouses, ETL pipelines, and analytics platforms (Azure, AWS, GCP).
• Strong knowledge of data modelling, data governance, data security, and cloud-native data technologies.
• Experience with data integration techniques, including ETL/ELT processes, real-time data streaming, and batch processing.
• Expertise in big data tools (e.g., Hadoop, Spark), database systems (e.g., SQL, NoSQL), and data warehousing platforms.
• Strong understanding of data privacy, security, and compliance frameworks (e.g., GDPR, HIPAA).
• Experience in leading data migration projects and modernizing legacy data systems.

Responsibilities

• Lead end-to-end data platform design, ensuring alignment with business and technical requirements.
• Architect scalable, high-performance data platforms leveraging cloud technologies and real-time streaming.
• Design integrated solutions for diverse data sources to support analytics and AI applications.
• Define the long-term roadmap for data platform and migrate legacy systems to modern architectures.
• Collaborate with data engineers, scientists, and analysts to align architectures with business needs.
• Implement governance frameworks and data security policies across the organization.
• Drive cloud migration projects and optimize cost, storage, and compute.
• Provide technical leadership and mentor junior team members.
• Oversee implementation and delivery of data platform solutions, ensuring timely and within-budget delivery.
• Create comprehensive documentation and regular status updates to senior management.

Skills

Leadership

Communication

Cloud platforms

Apache Airflow

ETL/ELT

Data governance

Data security

Agile

Education

Bachelor's/Master's in CS or related field

Tools

Azure Data Factory

Apache Airflow

Terraform

CloudFormation

Power BI

Tableau

Hadoop

Spark

Job description

KEY ACCOUNTABILITIES

• Solution Design & Architecture:

• Lead the design of end-to-end data platform solutions, ensuring they meet both business and technical requirements.

• Architect scalable, high-performance data platforms leveraging technologies such as cloud-based solutions (AWS, Azure, GCP), data lakes, data warehouses, ETL/ELT pipelines, and real-time data streaming.

• Design integrated solutions that can handle diverse data sources (structured, semi-structured, unstructured) and support advanced analytics, machine learning, and AI applications.

• Strategic Data Platform Planning & Roadmap:

• Define the long-term vision and roadmap for the data platform, ensuring alignment with the organization’s data strategy and business goals.

• Develop migration and modernization strategies for legacy data systems to modern data platforms, including cloud adoption and hybrid architectures.

• Evaluate emerging technologies and industry trends to recommend innovative solutions that enhance the data platform’s capabilities.

• Cross-Functional Collaboration:

• Collaborate closely with data engineers, data scientists, business analysts, and other IT teams to ensure that data architectures align with business objectives and deliver the necessary insights for decision-making.

• Work with business stakeholders to understand data requirements, translating them into technical specifications and ensuring solutions meet business needs.

• Foster collaboration across technical teams to ensure the seamless integration of systems, data sources, and workflows.

• Data Governance, Security & Compliance:

• Implement data governance frameworks, ensuring that data is accurate, consistent, and accessible across the organization while maintaining the highest levels of security and compliance.

• Develop and enforce data security policies, ensuring that data is protected in compliance with regulatory standards such as GDPR, CCPA, HIPAA, etc.

• Establish data lineage and metadata management practices to support data integrity and transparency.

• Cloud Data Architecture & Migration:

• Lead the design and implementation of cloud-based data solutions, optimizing data storage, compute, and analytics services to ensure performance, scalability, and cost-efficiency.

• Drive cloud migration projects, working with engineering teams to move on-premises data solutions to cloud platforms (Azure, AWS, GCP).

• Architect solutions for data warehousing, data lakes, and analytics, ensuring that the architecture is resilient, flexible, and cost-optimized.

• Performance Optimization & Scalability:

• Ensure that the data platform is capable of handling large volumes of data and providing low-latency access for real-time analytics and reporting.

• Continuously assess and optimize the performance, scalability, and cost-efficiency of the data architecture.

• Lead the design of systems that scale efficiently, both vertically and horizontally, to accommodate growing data needs.

• Leadership & Mentoring:

• Provide technical leadership to the data engineering and architecture teams, ensuring best practices and high standards are maintained in solution design and implementation.

• Mentor junior team members, offering guidance on architectural design, data modelling, and the latest data technologies.

• Promote a culture of continuous learning, encouraging team members to stay up to date with industry trends and innovations.

• Solution Implementation & Delivery:

• Oversee the implementation and delivery of data platform solutions, ensuring they are deployed successfully and meet technical specifications.

• Troubleshoot and resolve any issues related to the data architecture and platform.

• Ensure solutions are delivered on time and within budget, meeting both functional and non-functional requirements.

• Documentation & Reporting:

• Create and maintain comprehensive documentation for data architecture, solution designs, and technical processes.

• Produce regular status reports and updates to senior management, highlighting key milestones, risks, and opportunities.

QUALIFICATIONS, EXPERIENCE AND SKILLS

Qualifications:

• Bachelor’s or master’s degree in computer science, Engineering, Information Technology, or a related field.
• Minimum of 7+ years of experience in data architecture, data engineering, or a similar role, with a strong focus on designing large-scale data platforms.
• Proven experience in architecting cloud-based data solutions, including data lakes, data warehouses, ETL pipelines, and analytics platforms (Azure, AWS, GCP).
• Strong knowledge of data modelling, data governance, data security, and cloud-native data technologies.
• Experience with data integration techniques, including ETL/ELT processes, real-time data streaming, and batch processing.
• Expertise in big data tools (e.g., Hadoop, Spark), database systems (e.g., SQL, NoSQL), and data warehousing platforms.
• Strong understanding of data privacy, security, and compliance frameworks (e.g., GDPR, HIPAA).
• Experience in leading data migration projects and modernizing legacy data systems.

Key Skills:

• Strong leadership, collaboration, and communication skills.
• Expertise in cloud platforms and services (Azure preferred).
• Proficiency in data pipeline orchestration tools (e.g., Apache Airflow, Azure Data Factory).
• Knowledge of containerization and microservices architecture.
• Familiarity with data visualization and BI tools (e.g., Power BI, Tableau).
• Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation).
• Ability to think strategically while balancing business needs and technical solutions.
• Experience with Agile methodologies and working in a fast-paced, collaborative environment.

Desirable Qualifications:

• Microsoft Certified: Azure Solutions Architect Expert or Google Cloud Professional Data Engineer.
• Experience with machine learning and AI workloads on data platforms.
• Knowledge of DevOps practices and CI/CD for data pipelines.

#LI-AA6