Data Engineer

PALO IT; São Paulo; Presencial; BRL 180 000 - 240 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Stimulating working environments

Unique career path

International mobility

Internal R&D projects (including Gen-e

Resumo da oferta

PALO IT in São Paulo, Brazil, is seeking a Data Engineer for a leading insurance client. You will design scalable data pipelines, ensure data quality, and enable analytics for underwriting, risk, fraud, and customer experience.

You will work with Spark, dbt, Kafka, Airflow on cloud platforms (AWS/GCP/Azure), build data lakes/warehouses, and collaborate with data scientists and stakeholders to democratize trusted data.

Qualificações

• Experience with modern data processing frameworks: Spark, databricks, dbt, Kafka, Apache Beam.
• Strong command of Python and advanced SQL.
• Hands‑on experience with Azure cloud platforms (Data Factory, Synapse).
• Knowledge of modern Data Lake / Data Warehouse design (Snowflake, Redshift, BigQuery).
• Familiarity with DataOps, testing, and CI/CD practices in data pipelines.
• Experience with workflow orchestration systems such as Airflow or Dagster.
• Understanding of data quality and governance frameworks: Great Expectations, Deequ, DataHub, OpenLineage.
• Upper-intermediate English level (B2 or higher) for global collaboration.

Responsabilidades

• Design, build, and maintain scalable and secure data pipelines in cloud environments (AWS, GCP, or Azure).
• Integrate large volumes of structured and unstructured data from core insurance systems (policies, claims, CRM, ERP).
• Automate data ingestion, transformation, and quality processes using tools such as Apache Spark, dbt, Kafka, and Airflow.
• Build and maintain modern data lakes and warehouses (Snowflake, BigQuery, Redshift).
• Implement data quality, lineage, and governance validations (Great Expectations, Deequ, OpenMetadata).
• Ensure compliance with data regulations (GDPR, SOC2) and internal security policies.
• Design optimized datasets to enable machine learning models, predictive analytics, and dashboards (Power BI, Looker, Tableau).
• Collaborate with data scientists, architects, and business stakeholders to democratize access to trusted data.
• Document data architectures, pipelines, transformation standards, and lineage.

Conhecimentos

Spark

Databricks

dbt

Kafka

Apache Beam

Python

SQL

Azure Data Factory

Synapse

Snowflake

Redshift

BigQuery

DataOps

CI/CD

Airflow

Dagster

Great Expectations

Deequ

DataHub

OpenLineage

Power BI

Looker

Tableau

English (B2+)

Ferramentas

Azure Data Factory

Synapse

Snowflake

Redshift

BigQuery

Airflow

Dagster

Power BI

Looker

Tableau

Descrição da oferta de emprego

Who We Are

Building the AI-first frontier enterprise.

We are a global technology consultancy with a trademarked, AI-first approach—Gen-e2™. It redefines how enterprises build digital products and transform their organizations with AI. We do the right thing, and we do it right. We're proud to be a World Economic Forum New Champion, and a B Corp-certified company.

• We are small enough to care locally, big enough to deliver globally (10 countries, 450+ experts from 50+ nationalities)

• We are becoming an agentic organization, adopting the AI-native operating model we bring to our clients.

• We are robust and resilient (100% independent, 0 debt, founded 2009)

• We are AI-native professionals who invest in what we believe and work as a collective intelligence

We are positive, courageous and deliver at the leading edge.

Your Role

As a Data Engineer for a leading insurance company, you will play a key role in building a data-driven decision-making culture. You will design resilient and secure solutions that transform raw data into strategic insights for underwriting, risk management, fraud prevention, and customer experience.

• Design, build, and maintain scalable and secure data pipelines in cloud environments (AWS, GCP, or Azure).

• Integrate large volumes of structured and unstructured data from core insurance systems (such as policies, claims, CRM, ERP).

• Automate data ingestion, transformation, and quality processes using tools such as Apache Spark, dbt, Kafka, and Airflow.

• Build and maintain modern data lakes and warehouses (Snowflake, BigQuery, Redshift).

• Implement data quality, lineage, and governance validations (Great Expectations, Deequ, OpenMetadata).

• Ensure compliance with data regulations (GDPR, SOC2, etc.) and internal security policies.

• Design optimized datasets to enable machine learning models, predictive analytics, and dashboards (Power BI, Looker, Tableau).

• Collaborate with data scientists, architects, and business stakeholders to democratize access to trusted data.

• Document data architectures, pipelines, transformation standards, and lineage.

Who You Are

• Experience with modern data processing frameworks: Spark, databricks, dbt, Kafka, Apache Beam.

• Strong command of Python and advanced SQL.

• Hands‑on experience with Azure cloud platforms (Data Factory, Synapse).

• Knowledge of modern Data Lake / Data Warehouse design (Snowflake, Redshift, BigQuery).

• Familiarity with DataOps, testing, and CI/CD practices in data pipelines.

• Experience with workflow orchestration systems such as Airflow or Dagster.

• Understanding of data quality and governance frameworks: Great Expectations, Deequ, DataHub, OpenLineage.

• Upper-intermediate English level (B2 or higher) for global collaboration.

Nice to Have

• Knowledge of streaming technologies: Kafka, Pub/Sub, Kinesis.

• Familiarity with event-driven architectures and microservices.

• Previous experience in the financial or insurance sector.

• Certifications such as Azure Data Engineer or Fabric Data Engineer.

• Experience with MLOps solutions (Vertex AI, SageMaker, MLFlow).

More About PALO IT

Our clients include some of the world’s most successful companies. We collaborate with leading enterprises, next-generation businesses and frontier partners, shaping what comes next, helping them scale AI and solve complex business and technology challenges.

What We Offer

• Stimulating working environments

• Unique career path

• International mobility

• Internal R&D projects (including Gen-e2™)

• Knowledge sharing

• Personalized training via PALO IT Academy

• Entrepreneurship & intrapreneurship

For more on our team culture and benefits, check out our careers page.