Cloud Data Engineer: Azure Data Lake & Spark

Sii Poland; United States; Remote; USD 46,000 - 61,000; Full time

Benefits offered by this job

Profit sharing

Internal training center

Regular integration events and trips

Medical care

Job summary

Компания Sii Poland ищет специалиста по обработке данных для работы в центра компетенции Data & Analytics. Ваша задача — развивать высокопроизводительные Data Lake/Data Platform решения, проектировать и внедрять ETL/ELT пайплайны и моделировать данные под аналитические потребности.

Требуется опыт работы с Azure Databricks, Azure Data Factory/Synapse/Fabric и Spark/PySpark, знание английского и польского языков.

Qualifications

• At least 4 years of experience in projects involving processing and integrating large volumes of data.
• Strong knowledge of SQL and Python.
• Experience with data processing frameworks such as Apache Spark / PySpark.
• Practical experience designing and building Data Lake and Data Warehouse solutions.
• Ability to build and maintain ETL/ELT pipelines and work with data orchestration and integration tools.
• Commercial experience in cloud-based Microsoft Azure data processing projects (Databricks, Azure Data Factory, Synapse, Fabric).
• Knowledge of distributed data processing concepts and data pipeline optimization.
• Fluent English.
• Fluent Polish.
• Must be located in Poland.

Responsibilities

• Design, build, and develop scalable, high-performance data processing platforms using Data Lake / Data Platform architecture
• Create and maintain ETL/ELT pipelines to integrate data from various sources and process it in Azure Databricks
• Model data and prepare structures for analytical and business needs
• Implement batch and/or near-real-time data processing solutions
• Monitor, optimize, and develop existing pipelines and data processing processes
• Collaborate with data analysts, BI teams, and Data Science teams on data preparation and sharing
• Gather business requirements and propose optimal technological solutions
• Create and maintain technical documentation for developed solutions
• Support and mentor less experienced team members, sharing knowledge and best practices

Skills

SQL

Python

Spark/PySpark

Data processing

ETL/ELT

Data orchestration

English

Polish

Tools

Azure Databricks

Azure Data Factory

Azure Synapse

Microsoft Fabric

Job description

Компания Sii Poland ищет специалиста по обработке данных для работы в центра компетенции Data & Analytics. Ваша задача — развивать высокопроизводительные Data Lake/Data Platform решения, проектировать и внедрять ETL/ELT пайплайны и моделировать данные под аналитические потребности.

Требуется опыт работы с Azure Databricks, Azure Data Factory/Synapse/Fabric и Spark/PySpark, знание английского и польского языков.