Senior Data Engineer

Deel; México; Presencial; MXN 2.171.000 - 3.257.000; Jornada completa

Descripción de la vacante

Deel is transforming how global data is turned into actionable insights. Our Data Platform team builds streaming pipelines using Spark Structured Streaming and Apache Iceberg to deliver near-real-time models.

You’ll own the platform end-to-end, from design to deployment, driving cost efficiency and scalability across domains. You’ll collaborate with partners to onboard new data sources, optimize lakehouse performance, and enhance observability with modern tooling.

Formación

• 7+ years of data engineering experience, with at least 3 years running streaming pipelines in production.
• Deep hands-on experience with Apache Spark, especially Structured Streaming, state stores, and performance tuning.
• Production experience with a lakehouse table format, ideally Apache Iceberg; familiarity with partitioning strategies.

Responsabilidades

• Design, build, and harden Spark Structured Streaming applications that process CDC events into near-real-time data models.
• Run and scale AWS EMR clusters, manage sizing, multi-cluster deployment, and cost optimization.
• Own health of Apache Iceberg tables with partition specs, compaction, and cleanup at scale.
• Onboard new business domains to the streaming platform and coordinate with source-database owners.
• Debug production issues, identify root causes, and implement durable fixes with data-backed validation.
• Build observability tooling and runbooks to speed up diagnosis and recovery.
• Improve deployment and developer experience via CI/CD pipelines and guardrails.
• Handle ad hoc ingestion and backfill requests from partner teams

Conocimientos

Spark Structured Streaming

Python

SQL

Scala/Java

CDC/Kafka semantics

Cloud/AWS

Herramientas

Apache Iceberg

Snowflake

Estuary Flow

Kafka

Airflow

Grafana Loki

Kubernetes (K8s)

Descripción del empleo

Who We Are Is What We Do.

Deel is the all-in-one payroll and HR platform for global teams. Our vision is to unlock global opportunity for every person, team, and business. Built for the way the world works today, Deel combines HRIS, payroll, compliance, benefits, performance, and equipment management into one seamless platform. With AI-powered tools and a fully owned payroll infrastructure, Deel supports every worker type in 150+ countries—helping businesses scale smarter, faster, and more compliantly.

Who We Are Is What We Do.

Deel is the all-in-one payroll and HR platform for global teams. Our vision is to unlock global opportunity for every person, team, and business. Built for the way the world works today, Deel combines HRIS, payroll, compliance, benefits, performance, and equipment management into one seamless platform. With AI-powered tools and a fully owned payroll infrastructure, Deel supports every worker type in 150+ countries—helping businesses scale smarter, faster, and more compliantly.

Among the largest globally distributed companies in the world, our team of 7,000 spans more than 100 countries, speaks 74 languages, and brings a connected and dynamic culture that drives continuous learning and innovation for our customers.

Why should you be part of our success story?

As the fastest-growing Software as a Service (SaaS) company in history, Deel is transforming how global talent connects with world-class companies – breaking down borders that have traditionally limited both hiring and career opportunities. We're not just building software; we're creating the infrastructure for the future of work, enabling a more diverse and inclusive global economy. In 2024 alone, we paid $11.2 billion to workers in nearly 100 currencies and provided healthcare and benefits to workers in 109 countries—ensuring people get paid and protected, no matter where they are.

Our momentum is reflected in our achievements and customer satisfaction: CNBC Disruptor 50, Forbes Cloud 100, Deloitte Fast 500, and repeated recognition on Y Combinator's top companies list—all while maintaining a 4.83 average rating from 15,000 reviews across G2, Trustpilot, Captera, Apple and Google.

Your experience at Deel will be a career accelerator. At the forefront of the global work revolution, you'll tackle complex challenges that impact millions of people's working lives. With our momentum—backed by a $17.3 billion valuation and $1 B in Annual Recurring Revenue (ARR) in just over five years—you'll drive meaningful impact while building expertise that makes you a sought-after leader in the transformation of global work.

Our Data Platform team is rebuilding how Deel turns operational data into data products. We're moving from a batch, trigger-based system to a streaming architecture built on change data capture (CDC), Spark Structured Streaming, and Apache Iceberg.

The goal is to deliver business-critical data models and transformations in under 5 minutes, make their cost visible, and let engineering teams onboard their own models. This platform is the foundation for Deel's customer analytics and audit log features.

You’ll own this platform end to end. You'll design the streaming jobs, run the compute clusters they live on, keep the lakehouse tables healthy, and make the whole system easier and cheaper to operate every week.

What You’ll Do

• Design, build, and harden Spark Structured Streaming applications that process CDC events into near-real-time data models. This includes stateful processing, slowly changing dimensions (SCD), late and partial records, and safe restarts.
• Run and scale our AWS EMR clusters. You'll own instance sizing, managed scaling, multi-cluster deployment, driver and executor memory tuning, and consolidation of low-traffic workloads to cut cost.
• Own the health of our Apache Iceberg tables. You'll design partition specs and run compaction, snapshot expiry, and delete-file cleanup at scale, including fast-lane maintenance for high-churn tables.
• Onboard new business domains to the streaming platform. You'll configure CDC sources and destinations in Estuary Flow, set up Kafka-compatible topics, verify backfills in Snowflake, and coordinate with source-database owners.
• Debug production issues across the stack and fix the root cause, not the symptom. Examples include connection drops, offset regressions, token expiry, schema mismatches, and write conflicts.
• Build observability and tooling that shorten time to diagnosis. Examples include shipping Spark driver and executor logs to Grafana Loki, watchdog and recovery steps, and AI-assisted runbooks.
• Improve deployment and developer experience through CI/CD pipelines, configuration structure, and guardrails that let other teams ship data models safely on their own.
• Handle ad hoc ingestion/backfill requests and escalations from partner teams

What You’ll Bring

• 7+ years of data engineering experience, with at least 3 years running streaming pipelines in production
• Deep hands-on experience with Apache Spark, especially Structured Streaming, state stores (RocksDB), checkpointing, and performance tuning
• Production experience with a lakehouse table format, ideally Apache Iceberg (Delta Lake or Hudi is also relevant), including table maintenance and partitioning strategy
• Experience operating Spark on AWS EMR, EKS, or similar, and comfort with cluster capacity planning, autoscaling, and cost trade-offs
• Solid understanding of CDC and Kafka semantics, including offsets, consumer configuration, and delivery guarantees
• Strong Python and SQL skills; Scala or Java is a plus
• Experience with Airflow or a similar orchestrator, plus CI/CD and infrastructure-as-code practices
• A methodical debugging style: you read logs, form hypotheses, and prove the fix with data

Nice to have

• Experience with Snowflake, especially querying external or Iceberg tables
• Observability experience with Grafana, Loki, or Alloy
• Experience enabling other teams to self-serve on a data platform
• Experience with K8S

Total Rewards

Our workforce deserves fair and competitive pay that meets them where they are. With scalable benefits, rewards, and perks, our total rewards programs reflect our commitment to inclusivity and access for all.

• Our salary range reflects gross base salary. For commercial roles with commission eligibility, this figure represents On-Target Earnings (OTE), inclusive of base salary and target commission.
• Salary ranges are quoted in USD as a consistent global reference. Your offer will be localized to your country's currency using a market-aligned conversion.
• Final pay is based on objective, job-related criteria including experience, skills, and location.
• We don't ask about salary history. Offers are based on the role and what you bring to it.

Some things you’ll enjoy

• Stock grant opportunities dependent on your role, employment status and location
• Additional perks and benefits based on your employment status and country
• Optional flexible working office membership, with IWG

At Deel, we're an equal-opportunity employer that values diversity and positively encourage applications from suitably qualified and eligible candidates regardless of race, religion, sex, national origin, gender, sexual orientation, age, marital status, veteran status, disability status, pregnancy or maternity or other applicable legally protected characteristics.

Unless otherwise agreed, we will communicate with job applicants using Deel-specific emails, which include @deel.com and other acquired company emails like @payspace.com and @paygroup.com. You can view the most up-to-date job listings at Deel by visiting our careers page.

We welcome persons with disabilities to apply to any of our open roles. We will provide application and/or interview accommodations on request throughout the recruitment, selection and assessment process for applicants with disabilities or other needs. If you require application and/or interview accommodations, please inform our Talent Acquisition Team via email ([e-mail]) and a team member will be in touch to ensure your equal participation.

As part of our hiring process, we primarily rely on interviews and role-related assessments. In limited cases, we may also consider informal background information relevant to the role, in line with our privacy and fairness obligations.

This application process may utilise Automated Employment Decision Tools (AEDT) and AI systems to assist in evaluating candidates based on experience level, technical skills and qualifications. This processing is conducted in compliance with applicable Data Protection, AI Governance and Labour Laws. We ensure human oversight is maintained in all final hiring decisions. Your personal data is not used to train AI models. For more information on how we process your personal data, please see our Recruitment Privacy Policy.

• For NYC Residents: In accordance with NYC Local Law 144, an independent bias audit has been conducted on AEDT