Data Engineer

Pyramid Consulting, Inc; Vitória; Presencial; BRL 80.000 - 120.000; Tempo integral

Resumo da oferta

A leading consulting firm in Espírito Santo is looking for a skilled data engineer with expertise in cloud platforms like AWS and modern data architectures. The ideal candidate will have proficiency in ingestion tools like Kafka and AWS Glue, along with a strong background in SQL and DevOps practices. This role demands familiarity with streaming architectures and tools, as well as transformation processes using Spark. Join a dynamic team focused on innovation and data excellence.

Qualificações

• Experience with modern data architectures including data lakes and lakehouses.
• Knowledge of streaming architectures and tools like Flink.
• Demonstrated experience in DevOps practices including CI/CD deployment.

Conhecimentos

Familiarity with AWS

Proficient in Python

Expert-level SQL

Ferramentas

Kafka

AWS Glue

Spark

Descrição da oferta de emprego

Familiarity with major cloud platforms (AWS)Experience with modern data architectures (data lakes, lakehouses, hubs)Proficiency in ingestion tools (Kafka, AWS Glue) and formats (Iceberg, Parquet)Knowledge of streaming architectures and tools (Kafka, Flink)Strong background in SQL-based frameworks and orchestration tools (dbt, Glue, Airflow).

Experience with SCD and schema evolutionFamiliarity with Spark for transformation, streaming, tuning, debugging with Spark UIProficient in Python.

Expert-level SQLDemonstrated experience in DevOps practices (code management, CI / CD, deployment)Understanding of governance principles (quality, privacy, security)