Data Engineer Spark

PALO IT; México; Presencial; MXN 400,000 - 600,000; Jornada completa

Descripción de la vacante

Join PALO IT as a Data Engineer - Apache Spark, where you will design and implement robust data pipelines. This role requires a strong focus on big data technologies and collaboration with teams to drive actionable insights. Be part of a global technology consultancy committed to sustainable development and continuous improvement.

Servicios

Stimulating working environments

Unique career path

International mobility

Personalized training

Formación

• 4+ years of experience in data engineering.
• Proficiency in Apache Spark and Python, Scala, or Java.
• Experience with CI/CD pipelines and Agile environments.

Responsabilidades

• Design and implement large-scale data pipelines.
• Collaborate with data scientists and analysts to deliver insights.
• Monitor and troubleshoot production pipelines.

Conocimientos

Big Data Technologies

Data Governance

Problem Solving

Herramientas

Apache Spark

AWS

Azure

Google Cloud

Apache Airflow

Git

Descripción del empleo

PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We do the right thing, and we do it right. We're proud to be a World Economic Forum New Champion, and a B Corp-certified company.

• We are small enough to care locally, big enough to deliver globally (5 continents, 18 offices, +650 experts from +50 nationalities)
• We are robust and resilient (100% independent and 0 debt)
• We are entrepreneurs and passionate experts : We invest in what we believe genuinely and work as a collective intelligence
• We are positive, courageous, caring, doers and committed to excellence

Your R ole

As a Data Engineer - Apache Spark, you will be responsible for designing and implementing robust, scalable data pipelines and ensuring data is ready for analytics, machine learning, and operational purposes. Working with stakeholders across teams, you will build solutions that leverage big data technologies to process and manage complex datasets.

Who You Are

• Design, develop, and optimize large-scale data pipelines using Apache Spark.
• Implement ETL (Extract, Transform, Load) processes to support data transformation and integration.
• Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver actionable insights.
• Experience modeling and managing data in various formats (e.g., Parquet, Avro, ORC, csv, etc.) across structured and unstructured environments.
• Optimize data processing for performance, scalability, and cost-effectiveness.
• Work with cloud platforms (AWS, Azure, or Google Cloud) to manage and deploy data workflows.
• Ensure data security, quality, and governance are integrated into all data workflows.
• Monitor and troubleshoot production pipelines, ensuring reliability and uptime.
• Stay updated on advancements in big data technologies and recommend innovations to improve the data ecosystem.
• Enable reliable, scalable access to data for analytics and ML.
• Strong understanding of software design patterns.

What You Need for This Role

• 4+ years of experience in data engineering, with a strong focus on big data technologies.
• Proficiency in Apache Spark and distributed data processing frameworks.
• Experience in programming languages such as Python, Scala, or Java.
• Strong understanding of data lakes, warehouses, and databases (e.g., Snowflake, Redshift, BigQuery).
• Hands-on experience with cloud platforms such as AWS (Glue, EMR, S3), Azure (Databricks, Data Factory), or Google Cloud (BigQuery, Dataflow).
• Familiarity with workflow orchestration tools like Apache Airflow or similar.
• Deep knowledge of data formats such as Parquet, Avro, and ORC.
• Experience with CI/CD pipelines and version control (e.g., Git).
• Solid understanding of data governance and security principles.
• Excellent problem-solving skills and the ability to work in an Agile environment.

You're aligned with our value by:

• Your willingness to do the right thing even when facing adversity
• You care about the well-being of others and the world at large
• You strive to approach things in a optimistic way
• You nail the fundamentals, sweat the details
• You understand the whole is more than the sum of its parts and actively work towards continuous improvement of the group
  

More About PALO IT

We’re eager to adapt to change, learn from our experiences and move to meet our planet’s urgent needs. We are continuously taking action to:

• Deliver projects with a positive impact
• Train 100% of our workforce on impact
• Achieve B Corp certification among all our offices across the globe
• Continuously measure & improve employee happiness

Our clients are amongst the world's most successful companies. We innovate with both established Fortune 1000s, SMEs and start-ups who aim to make an impact, become global leaders and address the world's most complex challenges.

What We Offer

• Stimulating working environments
• Unique career path
• International mobility
• Internal R&D projects
• Knowledge sharing
• Personalized training

For more on our team culture and benefits, check out our careers page .

*

Adjuntar

Adjuntar

LinkedIn Profile *

Do you speak english fluently? *

Are you aligned with Hybrid Work? * Select...

What are your salary expectations in Mexican pesos? Its negotiable? *