IC3 - Data Engineer

spin; Ciudad de México; On-site; MXN 600,000 - 900,000; Full time

Job summary

Spin is seeking a Data Engineer to design, develop, and maintain robust data pipelines and architectures that power its data-driven initiatives in a scalable and reliable way.

You will lead data engineering efforts across ingestion, processing, storage, and retrieval, collaborating with data scientists and analysts to deliver impactful data solutions.

Qualifications

• 3–4 years of experience as a Data Engineer or similar role.
• Strong Python and SQL/NoSQL proficiency.
• Experience with ETL, data pipelines, and data warehousing concepts.
• Familiarity with Spark, Kafka, Flink for batch and real-time processing.
• Experience with cloud data architectures (data lakes, data warehouses).
• Knowledge of data governance, data quality, lineage and security.

Responsibilities

• Design scalable data pipelines for ingestion, processing, and storage of structured and semi-structured data.
• Collaborate with stakeholders to gather requirements and define feasible solutions.
• Develop and optimize ETL processes ensuring data quality and integrity.
• Create and maintain data models using best practices for normalization and performance.
• Build and manage cloud-based architectures including data lakes, warehouses, and streaming solutions.
• Implement monitoring and alerting for production data pipelines.
• Contribute to data governance with data quality, lineage, and security measures.
• Utilize Spark, Kafka, and Flink for batch and streaming workloads.
• Maintain CI/CD pipelines to automate data workflows and deployments.
• Review code and enforce standards for quality and maintainability.
• Document architectures and workflows for knowledge sharing.
• Mentor junior engineers and promote autonomous team culture.
• Identify automation opportunities to improve efficiency.

Skills

Python

PySpark

Pandas

NumPy

Dask

SQL

NoSQL

Data modeling

ETL

Data warehousing

Cloud

Tools

Spark

Kafka

Flink

Git

Terraform

CI/CD

Job description

Objective of the Role Responsible for designing, developing, and maintaining robust data pipelines and architectures that support the organization's data-driven initiatives. This role contributes to all phases of the data engineering lifecycle, ensuring scalability, reliability, and performance. Demonstrates technical leadership by solving complex problems, mentoring junior engineers, and actively improving data engineering practices.

Main Responsibilities

• Design and implement scalable and efficient data pipelines for ingesting, processing, and storing structured and semi-structured data from diverse sources.
• Collaborate with stakeholders to gather requirements, identify data sources, and define the feasibility of requested solutions.
• Develop and optimize ETL processes, ensuring the accuracy, consistency, and integrity of data throughout its lifecycle.
• Create and maintain data models, adhering to best practices for normalization and performance optimization.
• Build and manage cloud-based data architectures, including data lakes, data warehouses, and real-time streaming solutions.
• Implement monitoring and alerting systems to ensure the reliability and performance of data pipelines in production.
• Contribute to data governance initiatives by ensuring data quality, security, lineage, and compliance with relevant regulations.
• Utilize advanced data processing frameworks like Apache Spark, Apache Kafka, and Flink for batch and real-time data processing.
• Maintain and enhance CI/CD pipelines to automate data engineering workflows and ensure seamless deployment.
• Perform code reviews and enforce coding standards to ensure quality and maintainability.
• Document processes, architectures, and technical workflows to support knowledge sharing and operational continuity.
• Mentor junior engineers, sharing expertise and fostering a collaborative team environment.
• Identify opportunities for process optimization and automation to improve efficiency and reduce manual effort.
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to deliver impactful data solutions.
• Promote an autonomous work culture by encouraging self-management, accountability, and proactive problem-solving among team members.
• Serve as a Spin Culture Ambassador to foster and maintain a positive, inclusive, and dynamic work environment that aligns with the company's values and culture.

Required Knowledge and Experience

• Minimum 3 - 4 years of experience as a Data Engineer
• In-depth understanding of core data engineering concepts and principles, including complex ETL (Extract, Transform, Load) processes, scalable data pipelines, and advanced data warehousing techniques.
• Advanced proficiency in Python, including writing efficient and optimized code, using advanced features like decorators, generators, and context managers.
• Extensive experience with Python libraries and frameworks commonly used in data engineering, such as Pandas, NumPy, PySpark, and Dask.
• Strong knowledge of both SQL and NoSQL databases, including advanced querying, indexing, and optimization techniques.
• Experience with database design, normalization, and performance tuning.
• Advanced understanding of data modeling concepts and techniques, including star schema, snowflake schema, and dimensional modeling.
• Experience with data modeling tools and best practices.
• Proficient in various data processing methods, including batch processing, stream processing, and realtime data processing.
• Extensive experience with data processing frameworks like Apache Spark, Apache Kafka, and Apache Flink.
• Advanced knowledge of file processing concepts, including handling large datasets and working with different file formats (e.g., CSV, JSON, Parquet, Avro).
• Strong understanding of data governance principles, including data quality management, data lineage, data security, and data privacy.
• Comprehensive understanding of the end-to-end data engineering lifecycle, including data ingestion, transformation, storage, and retrieval.
• Experience with CI/CD pipelines and automation for data engineering workflows.
• Advanced understanding of various data architectures, including data lakes, data warehouses, data marts, and data mesh.
• Experience designing and implementing scalable and robust data architectures.
• Proficient with version control systems (e.g., Git) and experience managing code repositories on platforms like GitHub or GitLab.
• Advanced understanding of data visualization tools (e.g., Tableau, Power BI) and reporting techniques.
• Ability to create insightful and impactful visualizations and dashboards.
• Strong understanding of cloud computing in AWS and GCP stacks.
• Basic experience with Infrastructure as Code (IaC) tools like Terraform or CloudFormation.
• Proven experience leading projects with Objectives and Key Results (OKRs), identifying risks, and delivering significant business value.
• Ability to