Full Stack Data Engineer

Caixa Mágica Software; Lisboa; On-site; EUR 45,000 - 65,000; Full time

Benefits offered by this job

Seguro de saúde

Seguro de vida

Eventos sociais

Job summary

Caixa Mágica Software procura um Full Stack Data Engineer para desenvolver features e soluções de dados na AWS. O candidato ideal tem experiência em Python e SQL, conhecimento de Spark/PySpark e capacidade de trabalhar de forma independente dentro de uma equipa dinâmica.

Oferecemos contrato estável para projeto de longo prazo, equipamento de trabalho, seguro de saúde e eventos de equipa. Valoriza-se proatividade, curiosidade e vontade de aprender novas competências.

Qualifications

• Experiência com Python, especialmente em manipulação e processamento de dados.
• Experiência com SQL.
• Experiência em programação e depuração.
• Interesse em ciência de dados.
• Experiência com serviços AWS (S3, Lambda, Athena, EC2, Glue, CDK/CloudFormation, Step Functions, Batch, Fargate).
• Experiência com Spark/PySpark.
• Experiência com Git.
• Experiência com Jupyter Notebooks.
• Experiência desenvolvendo aplicações e/ou ambientes de produção.
• Proatividade e iniciativa.
• Boa capacidade de trabalho em equipe.
• Capacidade de resolver problemas de forma independente.
• Interesse em aprender novas habilidades.

Responsibilities

• Desenvolver features End-to-End em plataforma de big data na AWS, com foco em análise e processamento de dados.
• Contribuir para arquitetura de features e definição de fluxos de dados.
• Criar provas de conceito.
• Desenvolver versões prontas para produção, testar, fazer deploy em ambientes de desenvolvimento, QA e produção, monitorar desempenho inicial e realizar debugging.
• Documentar as features usando Confluence.
• Apoiar extração de dados e atividades de análise exploratória de dados.

Skills

Python

SQL

Git

Jupyter Notebooks

Spark/PySpark

AWS

S3

Lambda

Athena

EC2

Glue

CDK/CloudFormation

Step Functions

Fargate

Data analysis

Autonomia

Job description

Full Stack Data Engineer

Empresa : Caixa Mágica Software Tipo de Emprego : Periodo Integral Lisbon, Portugal

Descrição do Emprego - Full Stack Data Engineer

Overview:

The ideal candidate has experience with Python and SQL and is capable of developing, operating, and maintaining data analytics applications on an AWS cloud-based big data platform. They have a strong interest in data science, can work independently and proactively, and integrate well into a small, dynamic team.

What will you do?

• Develop end-to-end features on an AWS cloud-based big data platform, focused on data analysis and processing.
• Actively contribute to discussions on feature architecture and the definition of the required data flows and processes.
• Create proofs of concept.
• Develop production-ready versions, perform testing, deploy to development, quality assurance, and production cloud environments, monitor initial performance, and carry out debugging and ongoing maintenance.
• Document the developed features using Confluence.
• Support data extraction and exploratory data analysis activities.

What are we looking for?

• Experience with Python, particularly in data manipulation and processing;
• Experience with SQL;
• Experience in programming and debugging;
• Interest in data science;
• Experience with AWS cloud services, such as S3, Lambda, Athena, EC2, Glue, AWS CDK/CloudFormation, Step Functions, Batch, and Fargate;
• Experience with Spark/PySpark;
• Experience with Git;
• Experience with Jupyter Notebooks;
• Experience developing applications and/or working in production environments;
• Proactivity and initiative;
• Strong teamwork skills;
• Ability and motivation to independently find solutions to problems;
• Interest in learning new skills.

What can you expect from us?

• A permanent job contract for a long term project;
• Tech equipment + SIM Card + personal smartphone;
• Health and Life Insurance;
• Social events and team buildings;
• The commitment of letting you grow with us, and be rewarded accordingly;
• A dynamic and young team that will be always there to support you;
• Training in the latest technologies;
• Coffee, fruits, snacks and a warm welcoming when you pass by the office.