Senior Data Engineer (Healthcare domain)

Sigmasoftware2; Poland; On-site; PLN 180,000 - 240,000; Full time

Job summary

Sigmasoftware2 is seeking a data engineering specialist to design and manage large-scale cloud data infrastructure and pipelines. You will work with PySpark, Python, and SQL to optimize data processing across cloud platforms such as Azure, Snowflake, and Databricks.

Docker and Kubernetes experience is valued for scalable microservices, with Terraform for infrastructure automation. The role emphasizes building self-service tooling, data governance, and ETL/Data Lake principles while handling

Qualifications

• Hands-on experience with cloud computing services in data and analytics.
• Experience with data modeling, reporting tools, data governance, and data warehousing.
• Proficiency in Python and PySpark for distributed data processing.
• Experience with Azure, Snowflake, and Databricks.
• Experience with Docker and Kubernetes.
• Knowledge of infrastructure as code (Terraform).
• Advanced SQL skills and familiarity with big data databases such as Snowflake, Redshift, etc.
• Experience with stream processing technologies such as Kafka, Spark Structured Streaming.
• At least an upper-intermediate level of English.

Responsibilities

• Collaborate with the Product Owner and team leads to define and design efficient pipelines and data schemas
• Build and maintain infrastructure using Terraform for cloud platforms
• Design and implement large-scale cloud data infrastructure, self-service tooling, and microservices
• Work with large datasets to optimize performance and ensure seamless data integration
• Develop and maintain squad-specific data architectures and pipelines following ETL and Data Lake principles
• Discover, analyze, and organize disparate data sources into clean, understandable schemas

Skills

Cloud computing

Python

PySpark

Data modeling

Data warehousing

SQL

English proficiency

Tools

Azure

Snowflake

Databricks

Docker

Kubernetes

Terraform

Kafka

Spark Structured Streaming

Job description

• Collaborate with the Product Owner and team leads to define and design efficient pipelines and data schemas
• Build and maintain infrastructure using Terraform for cloud platforms
• Design and implement large-scale cloud data infrastructure, self-service tooling, and microservices
• Work with large datasets to optimize performance and ensure seamless data integration
• Develop and maintain squad-specific data architectures and pipelines following ETL and Data Lake principles
• Discover, analyze, and organize disparate data sources into clean, understandable schemas

Hands-on experience with cloud computing services in data and analytics

• Experience with data modeling, reporting tools, data governance, and data warehousing
• Proficiency in Python and PySpark for distributed data processing
• Experience with Azure, Snowflake, and Databricks
• Experience with Docker and Kubernetes
• Knowledge of infrastructure as code (Terraform)
• Advanced SQL skills and familiarity with big data databases such as Snowflake, Redshift, etc.
• Experience with stream processing technologies such as Kafka, Spark Structured Streaming
• At least an Upper-Intermediate level of English