Cloud Platform Engineer

Northern Trust; Pune District; On-site; INR 3,500,000 - 7,000,000; Full time

Job summary

Northern Trust is seeking a senior Platform Engineer to design, implement, and operate cloud-native data platforms across AWS/Azure, focusing on reliability and scalability for enterprise data workloads.

You will lead a global team, build advanced data pipelines with Airflow, Databricks, and Snowflake, and partner with data science and BI teams to enable secure, cost-efficient data services.

Qualifications

• Bachelor's degree in Computer Science or Data Engineering; Masters preferred.
• 10+ years in data/ cloud/platform engineering with 15+ years total experience including Azure.
• 5+ years leading Platform Engineering, DevOps, Infra, or Operations teams.
• Proven experience with enterprise-scale cloud data platforms (Snowflake, Databricks).
• Experience leading ITIL service management (Incident, Change, etc.).
• Experience building executive dashboards and driving platform reliability and automation.
• Experience managing globally distributed teams in complex environments.
• Hands-on with Azure and cloud data services; familiar with Spark, Databricks, Flink.
• Expertise building data pipelines with Airflow, dbt, Kafka, etc.

Responsibilities

• Design, implement, and maintain cloud-native data platforms (AWS, Azure).
• Develop and optimize data pipelines for ingestion, transformation, and processing of structured, semi-structured, and unstructured data.
• Build reusable frameworks and automation for data workflows, monitoring, and scaling.
• Ensure data storage and compute solutions are optimized for performance, security, and cost.
• Integrate cloud data services with enterprise apps, APIs, and third-party tools.
• Partner with data scientists, BI teams, and developers to enable analytics and AI/ML workloads.
• Establish CI/CD pipelines for data services and platform components.
• Mentor junior engineers and contribute to best practices in cloud engineering.

Skills

Leadership

Cross-functional collaboration

Problem-solving

Mentoring

Education

Bachelor's in CS/Data Engineering

Masters preferred

Tools

Snowflake

Databricks

Apache Airflow

dbt

Kafka

Terraform

Docker

Kubernetes

Azure

Python/Java/Scala

Job description

Key Responsibilities -

Platform Engineering & Development

• Design, implement, and maintain cloud-native data platforms (e.g., AWS, Azure).
• Develop and optimize data pipelines for ingestion, transformation, and processing of structured, semi-structured, and unstructured data.
• Build reusable frameworks and automation for data workflows, monitoring, and scaling.
• Ensure data storage and compute solutions are optimized for performance, security, and cost efficiency.
• Integrate cloud data services with enterprise applications, APIs, and third-party tools.

Data Management & Governance

• Implement data quality, lineage, and cataloging solutions across the data platform.
• Enforce governance, security, and compliance standards (e.g., GDPR, HIPAA, SOC2).
• Work with architects and security teams to implement role-based access, encryption, and audit controls.
• Support metadata management, data discoverability, and enterprise-wide data democratization.

Operations & Reliability

• Manage the operational health of the cloud data platform, including monitoring, alerting, and troubleshooting.
• Establish CI/CD pipelines for data services and platform components.
• Implement disaster recovery, backup, and failover strategies.
• Continuously optimize platform performance, cost, and scalability.

Collaboration & Innovation

• Partner with data scientists, BI teams, and application developers to enable advanced analytics and AI/ML workloads.
• Evaluate emerging cloud data services and recommend adoption based on enterprise needs.
• Provide technical expertise in designing cloud-based data architectures and solutions.
• Mentor junior engineers and contribute to best practices in cloud engineering and data development.

Education & Experience -

• Bachelors degree in Computer Science, Data Engineering, or related field (Masters preferred).
• 10+ years of experience in data engineering, cloud engineering, or platform engineering with 15+ years of total work experience, including Azure environments.
• 5+ years of leadership experience managing Platform Engineering, DevOps, Infrastructure, or Operations Teams.
• Proven experience managing enterprise-scale cloud data platforms, specifically:
• Snowflake
• Databricks
• Other Data Platform services
• Strong experience leading ITIL service management functions, including Incident, Problem, Change, and Service Request Management.
• Experience building and presenting executive operational dashboards and service reviews.
• Proven track record of driving platform reliability, operational transformation, and automation initiatives.
• Experience managing globally distributed teams and complex enterprise environments.
• Hands-on experience with major cloud provider Azure and Cloud Data Services.
• Experience with big data and data processing frameworks (e.g., Spark, Databricks, Flink).
• Expertise in building data pipelines with tools like Apache Airflow, dbt, Kafka, or similar.

Skills & Competencies -

• In-depth knowledge and experience with Snowflake, Databricks, or similar cloud data platforms.
• Extensive knowledge and real hands-on experience on Github terraform pipelines and objects deployment management.
• Extensive knowledge and real hands-on experience of AI/ML data pipelines and feature engineering.
• Exposure to regulated environments (financial services, healthcare, etc.).
• Proficiency in terraform, SQL and one or more programming languages (Python, Java, or Scala).
• Deep understanding of cloud-native data services (e.g., AWS Redshift, Glue, S3, Athena; Azure Synapse, Data Factory; GCP BigQuery, Dataflow).
• Familiarity with containerization and orchestration (Docker, Kubernetes).
• Experience with infrastructure-as-code (Terraform, CloudFormation, Pulumi).
• Knowledge of FinOps principles for cost-efficient data platform design.
• Strong problem-solving, communication, and cross-functional collaboration skills.
• Excellent communication, mentorship, and ability to influence cross-functional teams.