Data Engineer

Pyramid Consulting, Inc; Parnamirim; Presencial; BRL 80.000 - 120.000; Tempo integral

Resumo da oferta

A leading technology consulting firm in Brazil is seeking a data engineer with extensive expertise in cloud platforms like AWS and modern data architectures such as data lakes. The ideal candidate will have a strong background in SQL frameworks and tools, proficiency in ingestion tools like Kafka, and familiarity with streaming architectures. This role demands proficiency in Python and experience in DevOps practices, understanding principles of governance related to data quality and security.

Qualificações

• Familiarity with major cloud platforms like AWS.
• Experience with modern data architectures including data lakes and hubs.
• Proficiency in ingestion tools such as Kafka and AWS Glue.

Conhecimentos

Cloud platforms (AWS)

Data architectures

Ingestion tools (Kafka, AWS Glue)

Streaming architectures and tools (Kafka, Flink)

SQL-based frameworks

Python

Descrição da oferta de emprego

Familiarity with major cloud platforms (AWS)Experience with modern data architectures (data lakes, lakehouses, hubs)Proficiency in ingestion tools (Kafka, AWS Glue) and formats (Iceberg, Parquet)Knowledge of streaming architectures and tools (Kafka, Flink)Strong background in SQL-based frameworks and orchestration tools (dbt, Glue, Airflow).

Experience with SCD and schema evolutionFamiliarity with Spark for transformation, streaming, tuning, debugging with Spark UIProficient in Python.

Expert-level SQLDemonstrated experience in DevOps practices (code management, CI / CD, deployment)Understanding of governance principles (quality, privacy, security)