Data Engineer

Luxoft; Mexico; On-site; PHP 5,511,329 - 8,573,178; Full time

Job summary

Luxoft seeks a Senior Data Engineer to design and implement scalable data platforms for a major retail client. You will work across the full data pipeline, from ingestion to analytics, coordinating with data scientists, product teams, and IT to deliver reliable, governed solutions.

You will apply deep expertise in Azure Data Factory, Synapse, Databricks, and data modeling to build robust ETL/ELT pipelines and ensure data quality and governance across the enterprise.

Qualifications

• 8+ years of overall data engineering experience.
• Hands-on expertise with Azure Data Factory and Synapse (3+ years).
• Strong data modeling skills (conceptual, logical, physical).
• Extensive experience building ETL/ELT pipelines for data lakes/warehouses.
• Proficiency in SQL and data warehousing concepts.
• Experience with metadata governance and cloud analytics platforms.

Responsibilities

• Lead cross-functional engineering teams to define technical strategy and architecture.
• Drive delivery of complex data platforms and mentor engineers.
• Design and implement scalable data pipelines and data models.
• Integrate data from APIs, databases, and external sources.
• Collaborate with stakeholders across IT and business units to ensure alignment and governance.

Skills

Azure Data Factory

Azure Synapse Analytics

Data modeling

SQL

Python

PySpark

Azure Data Lake Storage

Azure Databricks

ETL/ELT pipelines

API integration

Data Vault

Data Mesh

Leadership

Cross-functional collaboration

Mentorship

Stakeholder communication

Tools

Docker

Kubernetes

Azure Databricks

Azure Analysis Services

PostgreSQL

Informix

Bash scripting

Job description

We are looking for Senior Data Engineer to implement a solution for a big retail company. The main focus is to support processing of big data volumes and integrate solution to current architecture.

Skills - Must have

• Overall years of experience required 8+
• Strong, recent hands-on expertise with Azure Data Factory and Synapse is a must (3+ years).
• Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, to support efficient data storage and retrieval.
• Strong knowledge of Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks, pySpark for building scalable and reliable data solutions.
• Extensive experience with building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehouses.
• Ability to integrate data from disparate sources, including databases, APIs, and external data providers, using appropriate techniques such as API integration or message queuing.
• Proficiency in designing and implementing data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Vault)
• Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data storages.
• Experience integrating metadata and governance processes into cloud-based data platforms
• Certification in Azure, Databricks, or other relevant technologies is an added advantage
• Experience with cloud-based analytical databases.
• Experience with Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Informix.
• Experience with Python and Python-based ETL tools.
• Experience with shell scripting in Bash, Unix or windows shell is preferable.
• Demonstrated ability to lead cross-functional engineering teams, define technical strategy and architecture, drive delivery of complex data platforms, mentor engineers, and effectively communicate with stakeholders at all organizational levels.

Nice to have

• Experience with Elasticsearch
• Familiarity with containerization and orchestration technologies (Docker, Kubernetes).
• Troubleshooting and Performance Tuning: Ability to identify and resolve performance bottlenecks in data processing workflows and optimize data pipelines for efficient data ingestion and analysis.
• Collaboration and Communication: Strong interpersonal skills to collaborate effectively with stakeholders, data engineers, data scientists, and other cross-functional teams.
• Ability to plan, estimate and track progress of implementing features
• Computer Science and data science academic and education credentials