AWS Integration – DevOps/Cloud Engineer

Job Description:

Location: Knutsford

Job Type: Contract

Industry: Cloud & Infrastructure

Job reference: BBBH399097_1741082578

Posted: about 1 hour ago

Key Skills/requirements:

• Deploy comprehensive cloud infrastructure for various products, including Astronomer Airflow and AccelData environments.
• Facilitate cross-functional integration between vendor products and other systems, such as data lakes, storage, and compute services.
• Establish best practices for cloud security, scalability, and performance.
• Manage and configure vendor product deployments, ensuring the setup and maintenance of environments.
• Ensure high availability, scalability, and fault tolerance of Airflow clusters.
• Implement monitoring, alerting, and logging for Airflow and related components.
• Perform upgrades and patches for platform-related components.
• Oversee capacity planning, resource allocation, and optimization of Airflow workers.
• Maintain and configure integrations with source control systems (e.g., GitHub, GitLab) for version control.
• Collaborate with cloud providers (e.g., AWS) for pipeline integration and scaling requirements.
• Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
• Develop infrastructure for optimal extraction, transformation, and loading of data from various sources using AWS and SQL technologies.
• Work with stakeholders, including design, product, and executive teams, to address platform-related technical issues.
• Build analytical tools to leverage the data pipeline, providing actionable insights into key business performance metrics, such as operational efficiency and customer acquisition.

Note: All profiles will be reviewed against the required skills and experience. Due to the high volume of applications, only successful applicants will be contacted. Thank you for your interest!