Data Engineer: Build Scalable Pipelines on AWS with Python

YellowIpe; Lisboa; On-site; EUR 48,000 - 80,000; Full time

Job summary

YellowIpe is seeking a Data Engineer to design and build scalable data solutions on AWS, using Python and SQL, in Lisbon or Porto.

You will work with cross-functional teams to shape data architectures, deploy ML models, and improve data pipelines with a focus on quality and performance.

The role offers growth, collaboration, and exposure to cutting-edge cloud technologies and data science initiatives.

Qualifications

• Bachelor's degree in CS/SE/CE or related field.
• Up to 5 years in Data or Software Engineering.
• Strong Python skills with Pandas and NumPy.
• SQL knowledge and ability to optimize complex queries.
• Experience with data pipelines and processing solutions.
• Experience with PostgreSQL/MySQL and NoSQL databases.
• ML model deployment and lifecycle management (MLflow/Kubeflow).
• English fluency required.

Responsibilities

• Design, develop, and maintain end-to-end data solutions within an AWS-based Big Data platform.
• Build data pipelines and processing applications using Python and SQL.
• Collaborate with cross-functional teams to define architectures and data flows.
• Develop solutions for data extraction, transformation, processing, and analysis.
• Work with relational and NoSQL databases for efficient data access.
• Support Data Science initiatives via exploration and data prep.
• Deploy and monitor ML models in production.
• Develop PoCs to assess new tools and approaches.
• Apply software engineering and DevOps best practices (CI/CD, testing, secure deployments).
• Monitor and improve existing data applications and production environments.
• Maintain technical documentation and contribute to debt management.

Skills

Python

Pandas

NumPy

SQL

Data pipelines

Machine Learning deployment

Jira

Confluence

Analytical skills

Communication skills

Education

Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, or related field

Tools

PostgreSQL

MySQL

MongoDB

DynamoDB

MLflow

Kubeflow

AWS

Git

Jupyter

Job description

YellowIpe is seeking a Data Engineer to design and build scalable data solutions on AWS, using Python and SQL, in Lisbon or Porto.

You will work with cross-functional teams to shape data architectures, deploy ML models, and improve data pipelines with a focus on quality and performance.

The role offers growth, collaboration, and exposure to cutting-edge cloud technologies and data science initiatives.