Oferta de emprego: Data Engineer (GCP)

KCS iT; Porto; Presencial; EUR 50 000 - 80 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Desenvolvimento de carreira

Contrato permanente

Formação e certificação contínuas

Salário competitivo com benefícios

Resumo da oferta

A KCS IT procura um Engenheiro de Dados com GCP para atuar em Porto, desenvolvendo pipelines de dados robustos, em equipa multidisciplinar, com foco em soluções de dados e IA. O profissional irá colaborar com Data Scientists e Analysts para desenhar e avaliar modelos, utilizando BigQuery, Spark e Hadoop.

A função envolve construção de código para processar grandes volumes de dados e monitorização de serviços na infraestrutura, com oportunidades de formação contínua e certificações internas.

Qualificações

• 4+ anos de experiência em GCP e em implementar serviços de dados escaláveis que processam grandes datasets.
• Conhecimento aprofundado em tecnologias de Big Data e armazenamentos de dados e linguagens como Scala/Java/Python para processamento.
• Experiência em analisar grandes volumes de dados com SQL, Spark, Python e Hive.
• Familiar com plataformas de workflow (Airflow) e ambientes colaborativos (Jupyter/Zeppelin).
• Conhecimento de YAML (serialização de dados).
• Azure e PowerBI são um plus.
• Experiência com metodologias ágeis (Scrum, Kanban).

Responsabilidades

• Projetar e implementar fluxos de processamento de dados em lote ou em tempo real, escaláveis e com uso eficiente de dados para funcionalidades orientadas por dados.
• Colaborar com Data Scientists e Analysts no design, implementação e avaliação de modelos e recursos orientados a dados.
• Utilizar GCP: BigQuery, Python, Scala e Hadoop (Spark/Hive) conforme necessidade.
• Desenvolver código para processar e analisar grandes volumes de dados (dados não estruturados, grandes conjuntos de texto, grafos, etc.).
• Construir, implantar, monitorizar e operar serviços, melhorando qualidade de dados e automatizando processos.

Conhecimentos

GCP

Big Data

Python

Scala

SQL

Ferramentas

Hadoop

Spark

Hive

Airflow

Descrição da oferta de emprego

Oferta de emprego: Data Engineer (GCP): KCS IT

Ref: 7111930

28-12-2024

Porto

Informática ( Programação )

KCS IT – WE MAKE IT

Founded in 2008, KCS IT is a consultancy company in the field of Information Technology and Services, focused on creating value for its clients through three main areas: Consulting, Technology and Training.

We are looking for a Data Engineer (GCP)

Based in: Porto

Keywords: Google Cloud Platform, Scala, Python, Hadoop

Years of Experience: 2+

What You’ll Do

• Design and implement highly scalable batch or real-time data processing workflows/components and products that make smart use of data to provide data-driven functionality (e.g., search services, recommender services, classification services, etc.), leveraging machine learning, Big Data technology, and distributed systems.
• Work in a cross-functional team and collaborate closely with Data Scientists and Analysts to support the (technical) design, implementation and evaluation of new algorithms, machine learning models and other data-driven features and services.
• Use technologies like GCP (Google Cloud Platform), Google BigQuery, Scala, Python and Hadoop technologies such as Spark or Hive.
• Develop code for processing and analyzing large amounts of data (e.g., unstructured tracking data, large full-text data sets, graph data, etc.).
• Build, deploy, monitor and operate services delivery within the infrastructure, while enhancing data quality/utility and automating processes.

Your Skills & Qualifications

• 4+ years of work experience in GCP as well as in implementing scalable software and data-driven services that process large datasets.
• In-depth knowledge in Big Data technologies, data storages (e.g., relational databases, key-value stores) and related data processing technologies, as well as programming languages such as Scala / Java / Python (for real-time and batch processing).
• Proven experience in analyzing large datasets (e.g., with SQL, Spark, Python, Hive etc.).
• Familiar with workflow management platforms (like Airflow) and collaborative platforms (like Jupyter or Zeppelin).
• Knowledge of YAML (data serialization).
• Microsoft Azure and PowerBI is a plus.
• Some experience in the domain of information retrieval, information extraction or machine learning is a plus.
• Worked before with Agile methodologies (Scrum, Kanban).

We Offer

• Personal career management.
• Permanent Contract.
• Continuous and personalized Training and Certification (Salesforce, Microsoft, Oracle, and more through an in-house PearsonVue center).
• Competitive salary package with added benefits.

Do you have this profile?

Send your application to *** Candidatura Online *** , including the job reference [CBB/NE/1020/DataEngineer].

Please check to see all opportunities we currently have open or feel free to send us a spontaneous application.