Senior Data Engineer

Luxoft Poland; Town of Poland (NY); On-site; USD 56,000 - 86,000; Full time

Benefits offered by this job

Private Medical & Dental care & Life保险

MyBenefit program

Internal Mobility program

Job summary

Luxoft Poland is seeking a Senior Data Engineer to modernize and maintain an eCommerce data platform for a major US retail client. You will design scalable data models and build robust ETL/ELT pipelines using Azure data services.

You will lead data architecture efforts, mentor engineers, and ensure governance across cloud-based analytics environments while integrating data from diverse sources and APIs.

Qualifications

• 8+ years of experience in data engineering or a related field.
• Hands-on expertise with Azure Data Factory and Synapse (3+ years).
• Experience designing conceptual, logical and physical data models.
• Strong Azure experience including Data Lake Storage, Synapse Analytics, Databricks.
• Extensive ETL/ELT pipelines building experience.
• Ability to integrate data from databases, APIs and external providers.
• Proficiency in dimensional modeling, star schemas, Data Vault and related approaches.
• Strong SQL skills for complex queries and performance tuning.
• Experience with metadata and governance in cloud data platforms.
• Azure or Databricks certification is a plus.
• Experience with Python-based ETL tools and Bash scripting.

Responsibilities

• Implement data engineering solutions to support processing of large data volumes.
• Integrate the solution with current cloud-native architecture.
• Lead cross-functional engineering teams and drive technical strategy.
• Mentor engineers and communicate with stakeholders at all levels.

Skills

Azure Data Factory

Azure Synapse

Data modeling

Azure Data Lake Storage

Azure Databricks

pySpark

ETL/ELT pipelines

SQL

Data governance

Cloud analytics

Python

Shell scripting

Leadership

Education

Bachelor’s degree in a relevant field

Job description

The primary goal of the project is the modernization, maintenance and development of an eCommerce platform for a big US-based retail company, serving millions of omnichannel customers each week.

Solutions are delivered by several Product Teams focused on different domains - Customer, Loyalty, Search and Browse, Data Integration, Cart.

Current overriding priorities are new brands onboarding, re-architecture, database migrations, migration of microservices to a unified cloud-native solution without any disruption to business.

Responsibilities:

We are looking for Senior Data Engineer to implement a solution for a big retail company. The main focus is to support processing of big data volumes and integrate solution to current architecture.

Mandatory Skills Description:

• Overall years of experience required 8+
• Strong, recent hands-on expertise with Azure Data Factory and Synapse is a must (3+ years).
• Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, to support efficient data storage and retrieval.
• Strong knowledge of Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks, pySpark for building scalable and reliable data solutions.
• Extensive experience with building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehouses.
• Ability to integrate data from disparate sources, including databases, APIs, and external data providers, using appropriate techniques such as API integration or message queuing.
• Proficiency in designing and implementing data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Vault)
• Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data storages.
• Experience integrating metadata and governance processes into cloud-based data platforms
• Certification in Azure, Databricks, or other relevant technologies is an added advantage
• Experience with cloud-based analytical databases.
• Experience with Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Informix.
• Experience with Python and Python-based ETL tools.
• Experience with shell scripting in Bash, Unix or windows shell is preferable
• Demonstrated ability to lead cross-functional engineering teams, define technical strategy and architecture, drive delivery of complex data platforms, mentor engineers, and effectively communicate with stakeholders at all organizational levels.

(Opportunity for Poland-based candidates)

• Tax-deductible costs on a contract of employment for all development roles
• Stable employment based on an employment contract
• Private Medical & Dental care & Life Insurance
• MyBenefit program (sports card, well-being program etc.)
• Internal Mobility program - possibility of rotation between projects, locations, accounts