About the Role
This role sits within AI, Platform Engineering & DevX (APEX) for platform engineering of Near-Realtime (NRT) teams under the Data Streaming Platform area. The Software Engineer III – Platforms will shape and deliver next-generation platforms, building resilient and scalable solutions while managing platform performance and ensuring high standards of code quality. The position also involves driving the adoption of open-source and agile methodologies within the organisation. The engineer will work in a collaborative, fast-paced environment and mentor the team through complex technical problems.
Key Responsibilities
- Build resilient, scalable solutions, manage platform performance, and ensure high standards of code quality.
- Drive the adoption of open-source and agile methodologies within the organisation.
- Install, configure, and maintain Kubernetes clusters, and implement open-source projects in Kubernetes.
- Set up monitoring, logging, and alerting for Kubernetes clusters, and implement Kubernetes security best practices such as RBAC, network, and pod security policies.
- Automate infrastructure provisioning and configuration using Infrastructure as Code tools such as Terraform or CloudFormation.
- Design, implement, and maintain robust CI/CD pipelines using Jenkins, and manage and optimise Artifactory repositories for artefact storage and distribution.
- Architect, deploy, and manage AWS EC2 instances, Lambda functions, Auto Scaling Groups, and Elastic Block Store volumes.
- Monitor, troubleshoot, and optimise AWS resources to ensure high availability, scalability, and performance, and implement security best practices and compliance standards in the AWS environment.
- Develop and maintain scripts in Python, Groovy, and Shell for automation and core engineering tasks.
- Collaborate with cross-functional teams to ensure seamless integration of DevOps practices into the software development lifecycle, and collaborate with product managers to scope new features and capabilities.
- Mentor the team through complex technical problems.
Required Experience & Qualifications
- 5-7 years of experience in designing and building production-grade platforms.
- Experience in data streaming and orchestration solutions such as Kafka, NiFi, and Airflow.
- Deep expertise in Kubernetes, AWS Services, full-stack development, cloud-native architectures, and other cloud-native platforms, with a strong understanding of AWS Cloud Services, DevOps, Databricks, and Python.
- Working experience designing and building production-grade microservices in any programming language.
- Experience building end-to-end CI/CD pipelines to build, test, and deploy to different AWS environments such as Lambda, EC2, ECS, and EKS.
- Familiarity with software engineering best practices, including unit tests, code review, version control, production monitoring, and unit testing.
- Strong experience with Jenkins, Terraform, Docker, Kubernetes, Argo CD, and CI/CD pipelines and DevOps skills, including Infrastructure as Code tools such as Helm, Kustomize, or Terraform.
- Proficiency in managing cloud-native platforms, with a strong PaaS (Platform as a Service) focus and understanding of PaaS architecture and DevOps tools such as Kubernetes, Jenkins, Terraform, and Docker.
- A proactive approach with the ability to work independently in a fast-paced, agile environment.
- Strong collaboration and problem-solving skills.
- Experience with container runtimes such as Docker.
- Deep expertise in at least one of the following technologies: Python, React, or NodeJS.
- Proficiency in Java and/or Go, NodeJS, React, SQL, and AWS.
- Familiarity with governance, security features, and performance optimisation.
- Keen attention to detail with a growth mindset and the desire to explore new technologies.
Preferred Skills
- Preference for Java and/or Go for microservices development.