Data Engineer

Pyramid Consulting, Inc; Teletrabalho; BRL 80.000 - 120.000; Tempo integral

Resumo da oferta

A leading consulting firm in Brazil is seeking a candidate experienced with cloud platforms, data architectures, and SQL. The role requires proficiency in Python and knowledge of streaming tools like Kafka and Flink. Candidates should also have a solid background in DevOps practices and governance principles. This opportunity is ideal for someone looking to work in a dynamic environment focusing on modern data solutions.

Qualificações

• Experience with data lakes, lakehouses, and modern data architectures.
• Demonstrated experience in DevOps practices including CI/CD.
• Understanding of governance principles related to data quality, privacy, and security.

Conhecimentos

Familiarity with major cloud platforms (AWS)

Proficiency in ingestion tools (Kafka, AWS Glue)

Expert-level SQL

Proficient in Python

Knowledge of streaming architectures and tools (Kafka, Flink)

Strong background in SQL-based frameworks (dbt, Glue, Airflow)

Descrição da oferta de emprego

Familiarity with major cloud platforms (AWS)Experience with modern data architectures (data lakes, lakehouses, hubs)Proficiency in ingestion tools (Kafka, AWS Glue) and formats (Iceberg, Parquet)Knowledge of streaming architectures and tools (Kafka, Flink)Strong background in SQL-based frameworks and orchestration tools (dbt, Glue, Airflow).

Experience with SCD and schema evolutionFamiliarity with Spark for transformation, streaming, tuning, debugging with Spark UIProficient in Python.

Expert-level SQLDemonstrated experience in DevOps practices (code management, CI / CD, deployment)Understanding of governance principles (quality, privacy, security)