Senior Data Engineer (Python / AWS / ML Pipelines)

Vigil Global; Brasil; Teletrabalho; BRL 180 000 - 240 000; Tempo integral

Resumo da oferta

Vigil Global is seeking a Senior Data Engineer to join a UK-based client, focused on building and maintaining production-grade data and ML pipelines in a cloud environment. You will design, develop, and deploy data workflows using Python, Airflow, AWS Glue, and SageMaker, collaborating with Data Scientists and ML Engineers across distributed teams.

The role emphasizes productionising ML models, scalable pipelines, and strong communication in an agile setting, with remote or Brazil/Portugal-based

Qualificações

• Experience building data pipelines and ML pipelines in production.
• Hands-on experience with Airflow.
• Experience deploying ML models using AWS SageMaker.

Responsabilidades

• Build and maintain data and ML pipelines in production.
• Develop ETL and data processing workflows using Python.
• Orchestrate workflows with Airflow and AWS Step Functions.
• Work with AWS Glue for data processing and pipeline execution.
• Support deployment of ML models into production using SageMaker.
• Collaborate with Data Scientists and ML Engineers to productionise models.
• Monitor, troubleshoot and improve pipeline reliability and performance.
• Contribute to architecture and decisions around data platform and ML infra.
• Work in an agile environment with distributed teams.

Conhecimentos

Python

Production data pipelines

ML pipelines

Airflow

AWS Cloud

SageMaker

Cross-functional teamwork

English communication

Ferramentas

AWS Glue

AWS Step Functions

SageMaker

Descrição da oferta de emprego

SUMMARY:

We are looking for a Senior Data Engineer to join one of our UK-based clients, working on large-scale data and machine learning pipelines in a production environment.

This role is focused on building and maintaining ML pipelines used for forecasting and data-driven decision-making in a high-scale platform. You will be working in a cloud-based environment, helping design, build, and deploy data and ML workflows into production.

We are looking for someone with strong Python experience, cloud experience (AWS), and experience building production data pipelines and ML infrastructure.

WHAT WILL YOU BE DOING:

• Building and maintaining data and machine learning pipelines in production;
• Developing ETL and data processing workflows using Python;
• Orchestrating workflows using tools such as Airflow and AWS Step Functions;
• Working with AWS Glue for data processing and pipeline execution;
• Supporting the deployment of machine learning models into production using SageMaker;
• Working closely with Data Scientists and ML Engineers to productionise ML models;
• Monitoring, troubleshooting, and improving pipeline reliability and performance;
• Contributing to architecture and technical decisions around data platform and ML infrastructure;
• Working in an agile environment with distributed teams.

WHAT WE ARE LOOKING FOR:

• Strong experience with Python;
• Experience building data pipelines and/or ML pipelines in production;
• Hands-on experience with Airflow;
• Experience with AWS Step Functions and AWS Glue;
• Experience deploying ML models using AWS SageMaker;
• Strong experience with AWS cloud;
• Experience working with production systems at scale;
• Experience working in cross-functional teams with Data Scientists;
• Strong communication skills and ability to work in a distributed team environment.
• Strong English communication skills, both written and verbal

AWESOME BUT NOT REQUIRED:

• Experience with GCP;
• Experience with monitoring and observability tools.

WHAT’S IN IT FOR YOU?

• Be part of our collegial environment where responsibility and authority are shared equally amongst colleagues and help create our company culture
• A culture in which we don’t criticise failure but ensure we learn from our mistakes
• An Agile environment where your ideas are welcome
• The possibility to grow and experience different projects
• Ongoing Training & Mentoring
• The possibility of travel

- ATTENTION! THIS POSITION IS FOR PORTUGAL OR BRAZIL BASED ONLY