Azure Senior Data Engineer

Luxoft; Mexico; On-site; PHP 7,524,000 - 11,285,000; Full time

Job summary

Luxoft is modernizing and maintaining an eCommerce platform for a large US-based retailer, handling millions of omnichannel customers weekly. The team covers Customer, Loyalty, Search & Browse, and Data Integration, with migration to a unified cloud-native solution and new brands onboarding.

The Senior Data Engineer will focus on big data processing, data integration, and aligning with the current architecture while mentoring engineers and guiding technical strategy.

Qualifications

• 8+ years of data engineering experience.
• Strong hands-on expertise with Azure Data Factory and Synapse (3+ years).
• Experience designing data models (conceptual, logical, physical).
• Knowledge of Azure services: Data Lake Storage, Synapse, Data Factory, Databricks.
• Extensive ETL/ELT pipeline experience.
• Ability to integrate data from databases, APIs, and external sources.
• Proficiency in SQL for complex queries and optimization.
• Experience with metadata governance on cloud platforms.
• Azure/Databricks certifications are a plus.
• Experience with cloud-based analytical databases.
• Python and Python-based ETL tools; Bash scripting preferred.
• Ability to lead cross-functional teams and communicate with stakeholders.

Responsibilities

• Implement a data solution for a big retail company.
• Support processing of large data volumes and integrate with current architecture.

Skills

8+ years experience

Data engineering

Cross-functional leadership

Mentoring engineers

Architectural strategy

Tools

Azure Data Factory

Azure Synapse Analytics

Azure Data Lake Storage

Azure Databricks

pySpark

SQL

ETL/ELT pipelines

APIs integration

Data warehousing

Python ETL tools

Shell scripting

Job description

Before you apply, please get familiar with Luxoft:

• Luxoft locations: https://career.luxoft.com/locations/
• Logeek Magazine: https://career.luxoft.com/logeek-magazine/
• Luxoft Alumni Club: https://career.luxoft.com/alumni

Project Description:

• The primary goal of the project is the modernization, maintenance and development of an eCommerce platform for a big US-based retail company, serving millions of omnichannel customers each week.
• Solutions are delivered by several Product Teams focused on different domains - Customer, Loyalty, Search and Browse, Data Integration, Cart.
• Current overriding priorities are new brands onboarding, re-architecture, database migrations, migration of microservices to a unified cloud-native solution without any disruption to business.

Responsibilities:

• We are looking for Senior Data Engineer to implement a solution for a big retail company. The main focus is to support processing of big data volumes and integrate solution to current architecture.

Mandatory Skills Description:

• Overall years of experience required 8+
• Strong, recent hands-on expertise with Azure Data Factory and Synapse is a must (3+ years).
• Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, to support efficient data storage and retrieval.
• Strong knowledge of Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks, pySpark for building scalable and reliable data solutions.
• Extensive experience with building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehouses.
• Ability to integrate data from disparate sources, including databases, APIs, and external data providers, using appropriate techniques such as API integration or message queuing.
• Proficiency in designing and implementing data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Vault)
• Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data storages.
• Experience integrating metadata and governance processes into cloud-based data platforms
• Certification in Azure, Databricks, or other relevant technologies is an added advantage
• Experience with cloud-based analytical databases.
• Experience with Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Informix.
• Experience with Python and Python-based ETL tools.
• Experience with shell scripting in Bash, Unix or windows shell is preferable.
• Demonstrated ability to lead cross-functional engineering teams, define technical strategy and architecture, drive delivery of complex data platforms, mentor engineers, and effectively communicate with stakeholders at all organizational levels.