Data Engineer

AdvanceWorks; Lisboa; Híbrido; EUR 45 000 - 65 000; Tempo integral

Resumo da oferta

AdvanceWorks in Lisbon is seeking a Data Engineer to join a cloud-based Big Data platform initiative. You will develop data processing solutions, prototype new approaches, and deploy production-ready applications in a small, hands-on team.

You will work across the full lifecycle using Python, SQL, and AWS services, collaborating with a dynamic team to improve data pipelines and analytics capabilities in a hybrid Lisbon setup.

Qualificações

• Proven experience with Python and SQL for data processing.
• Experience building and maintaining data pipelines and analytics solutions.
• Familiarity with Big Data technologies and cloud-based platforms.
• Strong communication and teamwork skills.

Responsabilidades

• Develop end-to-end data processing and analytics functionalities on a cloud-based Big Data platform.
• Use Python and SQL to manipulate, process, analyze, and structure data.
• Collaborate on architecture, data flows, and processing techniques.
• Design and implement data processing solutions from requirements to production.
• Create proofs of concept for new approaches and technologies.
• Develop, test, deploy, and monitor production-ready data solutions across environments.
• Monitor, debug, troubleshoot, and evolve data pipelines and applications.
• Support exploratory data analysis and data extraction activities.
• Document solutions and promote continuous improvement of pipelines.

Conhecimentos

Python

SQL

Big Data

Data pipelines

Spark / PySpark

AWS

Git

Jupyter Notebooks

Communication skills

Autonomy

Ferramentas

Apache Spark

Git

Jupyter Notebooks

CloudFormation / CDK

Hadoop

Descrição da oferta de emprego

At AdvanceWorks, we learn together, grow together, and have fun together. And we do all this while creating great software solutions for clients across many industries and geographies.

We are constantly challenging ourselves and each other to use the latest technologies and the best methodologies and make sure we walk the talk.

About the project

You’ll join a data engineering team working on a large-scale cloud-based data platform for a leading organization in the energy sector.

The project focuses on developing and maintaining data-intensive applications and solutions in a Big Data environment running on AWS, supporting data processing, analytics, exploratory analysis, and data-driven initiatives across the organization.

You’ll be part of a small and dynamic team, contributing to solutions end-to-end, from exploring requirements and creating proof of concepts to developing, testing, deploying, monitoring, and continuously improving production solutions.

This is a great opportunity for someone who enjoys working hands-on with Python, SQL, AWS, and data, while having the opportunity to deepen their knowledge of Big Data and Data Science.

What will be your role

We are looking for a Data Engineer to join our team in Lisbon and contribute to the development of data and analytics solutions in a cloud-based Big Data environment.

In this role, you’ll work across the full development lifecycle, helping design and implement data processing solutions, build prototypes, deploy production-ready applications, and support their operation and continuous improvement.

You’ll collaborate closely with a small and dynamic team, taking ownership of your work and proactively looking for effective solutions to technical challenges.

Your responsibilities will include

• Developing end-to-end data processing and analytics functionalities on a cloud-based Big Data platform.
• Working with Python and SQL to manipulate, process, analyze, and structure data.
• Participating actively in technical discussions around architecture, data flows, and processes.
• Designing and implementing data processing solutions according to business and technical requirements.
• Creating proofs of concept to validate new approaches and technologies.
• Developing, testing, and deploying production-ready solutions across development, quality, and production environments.
• Monitoring newly deployed solutions and performing debugging, troubleshooting, and evolutionary maintenance.
• Supporting data extraction activities and exploratory data analysis.
• Documenting the solutions and functionalities developed.
• Contributing to the continuous improvement of data pipelines, processes, and applications.
• Working closely with the team to identify problems, investigate solutions, and proactively improve the platform.

What should you bring to the team

• Professional experience with Python, particularly for data manipulation and processing.
• Solid experience with SQL.
• Experience with programming, debugging, and troubleshooting.
• Interest in Data Science, Data Analytics, or Big Data.
• Experience working with data pipelines or data processing solutions.
• Ability to work autonomously and proactively when investigating and solving technical problems.
• Good communication and teamwork skills.
• Willingness and motivation to learn new technologies and continuously develop your technical skills.
• Availability to work in a hybrid model in Lisbon, with 2 to 3 days per week onsite.

Nice to have

• Experience with AWS, particularly services such as S3, Lambda, Athena, EC2, Glue, Step Functions, Batch, Fargate, CDK / CloudFormation
• Experience with Apache Spark / PySpark.
• Experience with Git and version control practices.
• Experience using Jupyter Notebooks.
• Previous experience developing and operating applications in production environments.
• Experience working with Big Data platforms or cloud-native data solutions.
• Exposure to data science, statistical analysis, or machine learning concepts.

What is in it for you

• An amazing informal culture of smart, hardworking, and friendly people who support and care about each other.
• A mentorship program from day one.
• The opportunity to work on large-scale data and cloud projects using modern technologies.
• Formal training and certifications in data, cloud, and engineering excellence.
• Access to cutting-edge tools and technologies.
• A dynamic company where your ideas matter more than job titles.
• Flexibility with responsibility.
• A Rubber Duck to help you debug those tricky data pipelines.