About the Role
This role sits within the AI, Platform Engineering & DevX (APEX) area, supporting platform engineering for Near-Realtime (NRT) teams under the Data Streaming Platform area.
The role will play a key part in shaping and delivering the organisation's next-generation platforms. It involves building resilient, scalable solutions, managing platform performance, and ensuring high standards of code quality. The engineer will also drive the adoption of open-source and agile methodologies within the organisation.
Key Responsibilities
- Install, configure, and maintain Kubernetes clusters, and implement open-source projects in Kubernetes.
- Set up monitoring, logging, and alerting for Kubernetes clusters.
- Implement Kubernetes security best practices, including RBAC, network, and pod security policies.
- Automate infrastructure provisioning and configuration using Infrastructure as Code tools such as Terraform or CloudFormation.
- Design, implement, and maintain robust CI/CD pipelines using Jenkins.
- Manage and optimise Artifactory repositories for efficient artefact storage and distribution.
- Architect, deploy, and manage AWS EC2 instances, Lambda functions, Auto Scaling Groups (ASG), and Elastic Block Store (EBS) volumes.
- Monitor, troubleshoot, and optimise AWS resources to ensure high availability, scalability, and performance, and implement security best practices and compliance standards in the AWS environment.
- Develop and maintain scripts in Python, Groovy, and Shell for automation and core engineering tasks.
- Collaborate with cross-functional teams to integrate DevOps practices into the software development lifecycle, and with product managers to scope new features and capabilities.
- Drive the adoption of open-source and agile methodologies within the organisation.
- Build resilient, scalable solutions, manage platform performance, ensure high standards of code quality, and mentor team members through complex technical problems.
Required Experience & Qualifications
- Experience in data streaming and orchestration solutions such as Kafka, NiFi, and Airflow.
- Deep expertise in Kubernetes, AWS Services, full-stack development, and cloud-native architectures, with a strong understanding of AWS Cloud Services, Kubernetes, DevOps, Databricks, Java and/or Go-lang, and other cloud-native platforms.
- Working experience designing and building production-grade microservices in any programming language, with proficiency in Java and/or Go-lang, Node JS, React, SQL, and AWS.
- Experience building end-to-end CI/CD pipelines to build, test, and deploy to different AWS environments such as Lambda, EC2, ECS, and EKS, with good knowledge of CI/CD pipelines and DevOps skills including Jenkins, Docker, and Kubernetes.
- Familiarity with software engineering best practices, including unit tests, code review, version control, and production monitoring.
- Strong experience with Jenkins, Terraform, Docker, Kubernetes, and Argo CD, and a strong understanding of PaaS architecture and DevOps tools.
- Proficiency in managing cloud-native platforms, with a strong PaaS (Platform as a Service) focus.
- A proactive approach with the ability to work independently in a fast-paced, agile environment.
- Strong collaboration and problem-solving skills.
- Experience with container runtimes such as Docker.
- Working experience with Infrastructure as Code tools such as Helm, Kustomize, or Terraform.
- Deep expertise in at least one of Python, React, or NodeJS.
- 3-4 years of experience designing and building production-grade platforms.
- Familiarity with governance, security features, and performance optimisation.
- Keen attention to detail with a growth mindset and desire to explore new technologies.
Preferred Skills
- Experience designing and building production-grade microservices in Java and/or Go-lang is preferred.