Mid-level Data Engineer

Jobtailor; São Paulo; Presencial; BRL 180 000 - 260 000; Tempo integral

Resumo da oferta

Jobtailor in São Paulo, Brazil, seeks a Data Engineer to build and maintain data ingestion and transformation pipelines across Lakehouse architectures and cloud-native tools. You will work with BigQuery, Delta Lake and collaborate with data teams to support analytics.

You will ensure data quality, security, and cost optimization while applying CI/CD practices with Git and orchestrating pipelines using Airflow/Composer in a Databricks/GCP environment.

Qualificações

• Experience as a Data Engineer or Analytics Engineer.
• Strong SQL skills.
• Experience with Python and Spark/PySpark.
• Experience with GCP (BigQuery, GCS, Pub/Sub, Dataflow or Dataproc).
• Hands-on experience with Databricks.
• Knowledge of orchestration and pipeline automation (Airflow/Composer).
• Familiarity with CI/CD and source control (Git).

Responsabilidades

• Develop and maintain data ingestion and transformation pipelines
• Work with Lakehouse architectures
• Operate in GCP and Databricks environments
• Design and evolve data structures in BigQuery and Delta Lake
• Support the definition and implementation of dimensional and relational modeling (Star Schema, Snowflake, etc.)
• Ensure data quality, consistency, and security
• Support performance and cloud cost monitoring activities
• Collaborate with business stakeholders and data teams

Conhecimentos

Data Engineer

SQL

Python & Spark

GCP (BigQuery, GCS, Pub/Sub, Dataflow/

Databricks

Airflow/Composer

Git (CI/CD)

Ferramentas

Databricks

Airflow

Git

BigQuery

GCS

Pub/Sub

Dataflow

Dataproc

Delta Lake

Spark

Descrição da oferta de emprego

Responsibilities

• Develop and maintain data ingestion and transformation pipelines
• Work with Lakehouse architectures
• Operate in GCP and Databricks environments
• Design and evolve data structures in BigQuery and Delta Lake
• Support the definition and implementation of dimensional and relational modeling (Star Schema, Snowflake, etc.)
• Ensure data quality, consistency, and security
• Support performance and cloud cost monitoring activities
• Collaborate with business stakeholders and data teams

Requirements

• Experience as a Data Engineer or Analytics Engineer
• Strong SQL skills
• Experience with Python and Spark/PySpark
• Experience with GCP (BigQuery, GCS, Pub/Sub, Dataflow or Dataproc)
• Hands‑on experience with Databricks
• Knowledge of orchestration and pipeline automation (Airflow/Composer)
• Familiarity with CI/CD and source control (Git)