Senior Data Engineer

Sigmasoftware2; Poland; On-site; PLN 180,000 - 280,000; Full time

Job summary

Sigmasoftware2 is seeking a seasoned Data Engineer to design and implement cloud-native data platforms, enabling scalable analytics and real-time ingestion in a modern data stack.

You will optimize Spark/PySpark jobs, work with Databricks or Snowflake, and shape data models (Dimensional, Data Vault) while enforcing data governance and CI/CD practices. English proficiency is required; familiarity with AWS/Azure/GCP and orchestration tools like Airflow is a plus.

Qualifications

• 5+ years of professional experience in Data Engineering.
• Strong Python and SQL development skills for pipeline development and optimisation.
• Proficiency in Apache Spark / PySpark, including query optimisation and performance tuning.
• Hands-on experience with Databricks (preferred) or Snowflake.
• Experience with at least one major cloud provider: Azure (preferred), AWS, or GCP.
• Experience with stream processing technologies (Kafka, Spark Structured Streaming).
• Solid understanding of ETL/ELT patterns, data modelling (dimensional, Data Vault), and data warehousing.
• Experience with orchestration tools (Apache Airflow, Azure Data Factory, or equivalent).
• Knowledge of Infrastructure as Code (Terraform or equivalent).
• Understanding of production-grade system requirements: reliability, scalability, observability, and performance.
• Upper-Intermediate English level.
• Familiarity with RAG pipeline design and LLM integration patterns.
• Knowledge of data governance frameworks and tools (Unity Catalog, Apache Atlas, or similar).
• Experience with dbt for data transformation and modelling.
• Familiarity with MLflow, Feature Stores, or ML platform integration.

Responsibilities

• Design and build scalable, cloud-native data platforms from greenfield to production
• Implement near-real-time ingestion pipelines using event-driven patterns
• Define and enforce platform standards, including Data Lake / Lakehouse principles, medallion architecture, and data contracts
• Refactor and optimise existing Spark and PySpark scripts for performance and maintainability
• Introduce best practices for code quality, testing, and CI/CD across data pipelines
• Drive adoption of AI tooling and agentic workflows within the data engineering team
• Ensure data quality, observability, and reliability across all pipelines and platforms
• Develop self-service tooling and microservices to simplify platform usage for other teams

Skills

Python

SQL

Apache Spark / PySpark

Databricks

Cloud platforms

Kafka

Airflow

Terraform

Data modelling

Data governance

英文沟通

Tools

dbt

MLflow

Unity Catalog

Apache Atlas

Apache Airflow

Job description

• Design and build scalable, cloud-native data platforms from greenfield to production


• Implement near-real-time ingestion pipelines using event-driven patterns


• Define and enforce platform standards, including Data Lake / Lakehouse principles, medallion architecture, and data contracts


• Refactor and optimise existing Spark and PySpark scripts for performance and maintainability


• Introduce best practices for code quality, testing, and CI/CD across data pipelines


• Drive adoption of AI tooling and agentic workflows within the data engineering team


• Ensure data quality, observability, and reliability across all pipelines and platforms


• Develop self-service tooling and microservices to simplify platform usage for other teams

• 5+ years of professional experience in Data Engineering


• Strong Python and SQL development skills for pipeline development and optimisation


• Proficiency in Apache Spark / PySpark, including query optimisation and performance tuning


• Hands-on experience with Databricks (preferred) or Snowflake


• Experience with at least one major cloud provider: Azure (preferred), AWS, or GCP


• Experience with stream processing technologies (Kafka, Spark Structured Streaming)


• Solid understanding of ETL/ELT patterns, data modelling (dimensional, Data Vault), and data warehousing


• Experience with orchestration tools (Apache Airflow, Azure Data Factory, or equivalent)


• Knowledge of Infrastructure as Code (Terraform or equivalent)


• Understanding of production-grade system requirements: reliability, scalability, observability, and performance


• Upper-Intermediate English level


• WILL BE A PLUS


• Familiarity with RAG pipeline design and LLM integration patterns


• Knowledge of data governance frameworks and tools (Unity Catalog, Apache Atlas, or similar)


• Experience with dbt for data transformation and modelling


• Familiarity with MLflow, Feature Stores, or ML platform integration

PERSONAL PROFILE

• Self-driven and proactive in identifying improvements


• Comfortable working in a fast-paced, innovative environment


• Strong problem-solving mindset with attention to detail


• Open to experimenting with emerging technologies and approaches