AI Native Data Engineer

Palo IT; Ciudad de México; Hybrid; MXN 600,000 - 900,000; Full time

Job summary

PALO IT is seeking a Data Engineer in CDMX to design and implement scalable data solutions for an insurance client. You will build resilient data pipelines, enable underwriting analytics, and support fraud prevention with robust data governance.

You will work with Spark, Databricks, dbt, Kafka, and Airflow in a hybrid setup across AWS/GCP/Azure, collaborating with data scientists and business stakeholders. Expect growth opportunities within a global, AI-native consulting environment.

Qualifications

• Experience with modern data processing frameworks: Spark, Databricks, dbt, Kafka, Apache Beam.
• Strong command of Python and advanced SQL.
• Hands-on experience with Azure Data Factory and Synapse.
• Knowledge of Data Lake/Data Warehouse design (Snowflake, Redshift, BigQuery).
• Familiarity with DataOps, testing, and CI/CD in data pipelines.
• Experience with Airflow or Dagster for orchestration.
• Understanding data quality and governance frameworks: Great Expectations, Deequ, DataHub, OpenLineage.
• Knowledge of streaming technologies: Kafka, Pub/Sub, Kinesis.
• Upper-intermediate English (B2+).

Responsibilities

• Design, build, and maintain scalable data pipelines in cloud environments.
• Integrate large volumes of structured and unstructured data from core systems.
• Automate data ingestion, transformation, and quality processes.
• Build and manage data lakes and warehouses (Snowflake, BigQuery, Redshift).
• Implement data quality, lineage, and governance validations.
• Ensure compliance with data regulations and security policies.
• Design datasets to support ML, analytics, and dashboards (Power BI, Looker, Tableau).
• Collaborate with data scientists, architects, and business stakeholders.
• Document data architectures, pipelines, and lineage.

Skills

Spark

Databricks

dbt

Kafka

Apache Beam

Python

SQL

Azure Data Factory

Azure Synapse

Snowflake

Redshift

BigQuery

DataOps

CI/CD

Airflow

Dagster

Great Expectations

Deequ

DataHub

OpenLineage

Pub/Sub

Kinesis

English

Tools

Snowflake

BigQuery

Power BI

Looker

Tableau

Job description

Quick answer: what is this job?

PALO IT is hiring a Data Engineer in CDMX to design and implement scalable data solutions for a leading insurance company. This role focuses on building resilient data pipelines and creating strategic insights for underwriting, risk management, fraud prevention, and customer experience.

Who We Are

Building the AI-first frontier enterprise.

We are a global technology consultancy with a trademarked, AI-first approach, Gen-e2™. It redefines how enterprises build digital products and transform their organizations with AI. We do the right thing, and we do it right. We’re proud to be a World Economic Forum New Champion and a B Corp-certified company.

• We are small enough to care locally, big enough to deliver globally ( 9 countries, 400+ experts from 50+ nationalities ).

• We are becoming an agentic organization, adopting the AI-native operating model we bring to our clients.

• We are robust and resilient (100% independent, 0 debt, founded 2009).

• We are AI-native professionals who invest in what we believe and work as collective intelligence.

• We are positive, courageous and deliver at the leading edge.

What does a Data Engineer do at PALO IT?

As a Data Engineer at PALO IT, you will lead and accelerate the next wave of software innovation through tasks that include:

• Designing, building, and maintaining scalable and secure data pipelines in cloud environments (AWS, GCP, or Azure).

• Integrating large volumes of structured and unstructured data from core insurance systems (such as policies, claims, CRM, ERP).

• Automating data ingestion, transformation, and quality processes using tools such as Apache Spark, dbt, Kafka, and Airflow.

• Building and maintaining modern data lakes and warehouses (Snowflake, BigQuery, Redshift).

• Implementing data quality, lineage, and governance validations (Great Expectations, Deequ, OpenMetadata).

• Ensuring compliance with data regulations (GDPR, SOC2, etc.) and internal security policies.

• Designing optimized datasets to enable machine learning models, predictive analytics, and dashboards (Power BI, Looker, Tableau).

• Collaborating with data scientists, architects, and business stakeholders to democratize access to trusted data.

• Documenting data architectures, pipelines, transformation standards, and lineage.

What experience do you need for this role?

You’re a strong fit for this Data Engineer role if you have:

Required / Must-have

• Experience with modern data processing frameworks: Spark, Databricks, dbt, Kafka, Apache Beam.

• Strong command of Python and advanced SQL.

• Hands‑on experience with Azure cloud platforms (Data Factory, Synapse).

• Knowledge of modern Data Lake / Data Warehouse design (Snowflake, Redshift, BigQuery).

• Familiarity with DataOps, testing, and CI/CD practices in data pipelines.

• Experience with workflow orchestration systems such as Airflow or Dagster.

• Understanding of data quality and governance frameworks: Great Expectations, Deequ, DataHub, OpenLineage.

• Knowledge of streaming technologies: Kafka, Pub/Sub, Kinesis.

• Upper‑intermediate English level (B2 or higher) for global collaboration.

Nice to Have

• Familiarity with event‑driven architectures and microservices.

• Previous experience in the financial or insurance sector.

• Certifications such as Azure Data Engineer or Fabric Data Engineer.

• Experience with MLOps solutions (Vertex AI, SageMaker, MLFlow).

More About PALO IT

Joining PALO IT as a Data Engineer means being part of a forward‑thinking, collaborative environment. You will join a passionate international team pushing the boundaries of AI‑native development. Enjoy continuous personal and professional growth through PALO IT Academy, specialized training, R&D projects, and a knowledge‑sharing culture.

You will have global opportunities for international mobility and collaboration with visionary clients and frontier tech partners. Expect to be empowered to innovate, shape, and implement next‑level AI methodologies as part of a collective intelligence, along with competitive compensation and benefits reflecting your skills, impact, and leadership.

Diversity, Equity and Inclusion

As a B Corp, PALO IT provides equal opportunities and assesses candidates solely on skills, potential, and motivation. PALO IT welcomes applications regardless of gender identity or expression, sexual orientation, age, background, ethnicity, religion, disability, or any other protected attribute.

Frequently asked questions about this role

Q: What work model is available for this position?
A: Hybrid

Q: What is the expected contract type for the role?
A: 100% salary