Data Engineer – Python & AWS

YellowIpe; Lisboa; Presencial; EUR 48 000 - 80 000; Tempo integral

Resumo da oferta

YellowIpe is seeking a Data Engineer to design and build scalable data solutions on AWS, using Python and SQL, in Lisbon or Porto.

You will work with cross-functional teams to shape data architectures, deploy ML models, and improve data pipelines with a focus on quality and performance.

The role offers growth, collaboration, and exposure to cutting-edge cloud technologies and data science initiatives.

Qualificações

• Bachelor's degree in CS/SE/CE or related field.
• Up to 5 years in Data or Software Engineering.
• Strong Python skills with Pandas and NumPy.
• SQL knowledge and ability to optimize complex queries.
• Experience with data pipelines and processing solutions.
• Experience with PostgreSQL/MySQL and NoSQL databases.
• ML model deployment and lifecycle management (MLflow/Kubeflow).
• English fluency required.

Responsabilidades

• Design, develop, and maintain end-to-end data solutions within an AWS-based Big Data platform.
• Build data pipelines and processing applications using Python and SQL.
• Collaborate with cross-functional teams to define architectures and data flows.
• Develop solutions for data extraction, transformation, processing, and analysis.
• Work with relational and NoSQL databases for efficient data access.
• Support Data Science initiatives via exploration and data prep.
• Deploy and monitor ML models in production.
• Develop PoCs to assess new tools and approaches.
• Apply software engineering and DevOps best practices (CI/CD, testing, secure deployments).
• Monitor and improve existing data applications and production environments.
• Maintain technical documentation and contribute to debt management.

Conhecimentos

Python

Pandas

NumPy

SQL

Data pipelines

Machine Learning deployment

Jira

Confluence

Analytical skills

Communication skills

Formação académica

Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, or related field

Ferramentas

PostgreSQL

MySQL

MongoDB

DynamoDB

MLflow

Kubeflow

AWS

Git

Jupyter

Descrição da oferta de emprego

Our mission is to inspire the connection between technology and people. We nurture the best in our professionals through our expertise in finding and attracting top talent for the best projects. People Focus, Collaboration, and Commitment are the pillars that guide us on this journey.

Join the yellow team as our new Data Engineer – Python & AWS!

Responsibilities

• Design, develop, and maintain end-to-end data solutions within an AWS-based Big Data platform, ensuring scalability, reliability, and performance.
• Build, optimize, and maintain data pipelines and processing applications using Python and SQL.
• Collaborate with cross-functional teams to define technical architectures, data flows, and integration strategies.
• Develop and implement efficient solutions for data extraction, transformation, processing, and analysis.
• Work with relational and NoSQL databases, ensuring efficient data access, querying, and management.
• Support Data Science initiatives through exploratory data analysis, data preparation, and the integration of Machine Learning models into production environments.
• Contribute to the deployment, monitoring, and lifecycle management of Machine Learning models.
• Develop Proofs of Concept (PoCs) to evaluate new technologies, tools, and technical approaches.
• Apply software engineering and DevOps best practices, including automated testing, secure deployments, and continuous integration and delivery.
• Monitor, troubleshoot, and continuously improve existing data applications and production environments.
• Maintain technical documentation, contribute to technical debt management, and support infrastructure improvement initiatives.

Requirements

• Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, or a related field.
• Up to 5 years of professional experience in Data Engineering, Software Engineering, or a similar technical role.
• Strong programming skills in Python, including hands-on experience with Pandas and NumPy.
• Solid knowledge of SQL, including the ability to develop and optimize complex queries.
• Experience designing, developing, or maintaining data pipelines and data processing solutions.
• Experience working with relational databases, such as PostgreSQL or MySQL, and NoSQL databases, such as MongoDB or DynamoDB.
• Good understanding of software development principles, debugging, testing, and code quality.
• Experience with Machine Learning model deployment and lifecycle management, preferably using tools such as MLflow or Kubeflow.
• Interest in Data Science, data analytics, and Machine Learning technologies.
• Familiarity with Agile methodologies and collaboration tools such as Jira and Confluence.
• Strong analytical and problem-solving skills, with the ability to work independently and collaborate with international teams.
• Excellent communication skills, including the ability to explain technical concepts to non-technical stakeholders.
• Fluent English is mandatory.

Nice to Have

• Hands-on experience with AWS services, including S3, Lambda, Glue, Athena, EC2, Step Functions, Batch, or Fargate.
• Experience with Infrastructure as Code (IaC) tools such as AWS CDK or CloudFormation.
• Knowledge of Apache Spark or PySpark for large-scale data processing.
• Experience with Git, Jupyter Notebooks, and collaborative development workflows.
• Experience developing, deploying, and maintaining production-grade applications.
• Previous experience in the Energy or Utilities sector.
• Knowledge of GDPR and data protection best practices.

What You Can Expect from Us

• Integration into a collaborative and people-focused team.
• Opportunities to work on challenging projects involving Data Engineering, Cloud Technologies, and Data Science.
• A professional environment that encourages continuous learning, innovation, and knowledge sharing.
• Opportunities for professional growth and technical development.

Location: Lisbon or Porto

If you're passionate about building scalable data solutions and want to make an impact through technology, we'd love to hear from you!

Join the yellow team and take the next step in your career with YellowIpe!