Data Engineer | Lisbon/Porto

YellowIpe; Lisboa; Presencial; EUR 40 000 - 64 000; Tempo integral

Resumo da oferta

YellowIpe in Lisbon is seeking a Data Engineer to build end-to-end data functionalities on an AWS Big Data platform, ensuring quality and scalable solutions.

You will participate in architecture discussions, create analytics applications, deploy models, and maintain documentation while collaborating with global teams and using SQL/NoSQL databases.

Qualificações

• 5 years or less in data/software engineering roles is acceptable.
• Bachelor’s degree in a related field; strong Python and SQL skills required.
• Experience with both relational and NoSQL databases and data processing.
• Familiarity with ML lifecycle tools and production deployments.
• Good communication and ability to work with global teams.

Responsabilidades

• Develop end-to-end data functionalities on an AWS Big Data platform.
• Participate in solution architecture discussions and data flows.
• Create and optimize data processing and analytics apps.
• Deploy production-ready solutions with automated testing.
• Monitor, debug, and drive improvements for performance and security.
• Work with SQL and NoSQL databases to optimize data access.
• Contribute to deploying models and managing their lifecycle.
• Maintain up-to-date technical documentation and manage debt backlog.
• Support infrastructure changes aligning data needs and architecture.

Conhecimentos

Python (Pandas, NumPy)

SQL

Data engineering

Debugging

Relational databases

NoSQL databases

MLflow / Kubeflow

Agile / Jira / Confluence

English (Advanced)

Formação académica

Bachelor’s degree in Software Engineering or Computer Engineering

Ferramentas

PostgreSQL

MySQL

MongoDB

DynamoDB

MLflow

Kubeflow

Jupyter Notebooks

Descrição da oferta de emprego

Our mission is to inspire the connection between technology and people. We nurture the best in our professionals through our expertise in finding and attracting top talent for the best projects. People Focus, Collaboration, and Commitment are the pillars that guide us on this journey.

Join the yellow team as our new Data Engineer!

Responsibilities

• Develop end-to-end data functionalities on an AWS Big Data platform, ensuring quality and scalability;
• Actively participate in strategic discussions around solution architecture, data flows, and required processes, contributing both technical and business insights;
• Create, maintain, and optimize data processing and analytics applications, integrating data engineering and data science methodologies;
• Create proofs of concept to validate innovative approaches and technologies that meet project requirements;
• Develop production-ready solutions following DevOps practices, including automated testing and secure deployments across development, quality assurance, and production environments;
• Monitor implemented solutions, supporting debugging activities and driving continuous improvements to ensure performance and security;
• Work with relational and NoSQL databases, implementing efficient queries to optimize data access;
• Support data extraction and perform exploratory data analysis to identify relevant patterns and insights;
• Contribute to deploying models into production environments and managing their lifecycle;
• Maintain accurate and up-to-date technical documentation covering workflows and solutions in use;
• Manage and track the technical debt backlog, prioritizing issues that impact operational efficiency;
• Support infrastructure change requests, ensuring alignment between data requirements and system architecture.

Requirements

• Up to 5 years of experience in a relevant role, preferably within data and software engineering environments;
• Bachelor’s degree in Software Engineering, Computer Engineering, or a similar field;
• Strong experience with Python, particularly for data manipulation and processing using libraries such as Pandas and NumPy;
• Strong understanding of SQL and the ability to write complex queries for data analysis and integration;
• Experience with programming and debugging in software development environments;
• Experience with relational databases such as PostgreSQL or MySQL and NoSQL databases such as MongoDB or DynamoDB;
• Experience deploying models into production environments and managing their lifecycle using tools such as MLflow or Kubeflow;
• Genuine interest in Data Science and its practical applications;
• Good written and verbal communication skills, with the ability to explain technical concepts to non-technical audiences;
• Ability to work independently and proactively, demonstrating initiative when solving problems;
• Ability to work effectively in collaborative environments and with global teams, adding value to technical discussions;
• Strong critical thinking and problem-solving skills to address complex challenges;
• Knowledge of Agile methodologies and tools such as Jira and Confluence;
• Strong commitment to the quality of delivered work;

Nice to Have

• Experience with AWS services such as S3, Lambda, Athena, EC2, Glue, CDK / CloudFormation, Step Functions, Batch, or Fargate, applying them to Big Data solutions;
• Practical knowledge of Spark / PySpark for large-scale data processing;
• Familiarity with version control systems such as Git for effective team collaboration;
• Experience with Jupyter Notebooks, supporting interactive development and code sharing;
• Experience developing applications or working in production environments, understanding the importance of collaboration between development and operations;
• Knowledge of the Energy and Utilities sector, bringing domain insights to the solutions developed;
• Understanding of GDPR regulations and their application to Data Science projects;
• Portuguese language skills are valued;
• Proactive and autonomous mindset, with a willingness to learn new skills and technologies;
• Strong team spirit and a collaborative approach focused on collective success.
• Contract type: Full-time
• Location: Portugal - Lisbon
• Language(s) required: English — Advanced (C1)