Senior Data Engineer (Databricks & PySpark)

ITConnect; Warszawa; Hybride; PLN 180 000 - 280 000; Plein temps

Résumé du poste

ITConnect is seeking an experienced Senior Data Engineer to design, build, and operate our Sustainability Data Foundation (SDF) on a Databricks platform. The role emphasizes data products, pipelines, governance, and auditable reporting for sustainability initiatives.

You will own end-to-end delivery—from design through production support—working with a Sustainability Data & AI team to ensure data accuracy, quality, and scalable solutions across ERP, procurement, and external systems.

Qualifications

• 5+ years of experience designing, developing, and operating data platforms and pipelines.
• Hands-on experience with Databricks in production environments.
• Proficiency in PySpark and SQL for data transformation.
• Strong knowledge of Git-based development and version control practices.
• Experience with automated testing, deployment pipelines, and release management.
• Understanding of data lineage, metadata management, governance, and auditing.

Responsabilités

• Design and build scalable data products using Databricks, Delta Lake, and PySpark.
• Develop data models, transformation frameworks, and reusable engineering patterns.
• Create and maintain data pipelines from ERP, procurement, sustainability, and external systems.
• Implement automated data quality controls and CI/CD pipelines.
• Support platform governance, security controls, and operational standards.

Connaissances

Ownership & accountability

Proactive mindset

Strong troubleshooting

Delivery-focused

Collaboration

Outils

Databricks

Delta Lake

PySpark

Git

CI/CD pipelines

Infrastructure as Code (IaC)

Dashboarding

Data governance tooling

Testing frameworks

Description du poste

We are seeking an experienced Senior Data Engineer to design, build, and operate our Sustainability Data Foundation (SDF). The SDF supports trusted, scalable, and auditable sustainability reporting and analytics through a modern data platform built on Databricks.

The focus of this role is to deliver data products and pipelines that enable sustainability reporting, regulatory compliance, and advanced analytics rather than enterprise-wide IT solutions. This position requires a combination of deep technical expertise, strong delivery ownership, and a pragmatic approach to engineering.

You will join a Sustainability Data & AI team that values reliability, operational excellence, and practical solutions over unnecessary complexity. We are looking for someone who is proactive, solution-oriented, and comfortable taking ownership from design through production support. The ideal candidate independently validates outputs, identifies root causes, and ensures high-quality delivery.

• Design and implement scalable data products using Databricks, Delta Lake, and PySpark.
• Develop data models, transformation frameworks, and reusable engineering patterns.
• Build and maintain data pipelines processing complex datasets from ERP, procurement, sustainability, and external systems.
• Optimize workloads for performance, scalability, and cost efficiency.
• Design, develop, deploy, and support production-grade data solutions.
• Build Databricks-based tools, dashboards, and applications that enhance user interaction with data and streamline reporting processes.
• Implement automated data quality controls throughout the data lifecycle.
• Proactively identify and resolve data issues before they impact reporting.
• Ensure data accuracy, completeness, and trustworthiness.
• Take ownership of data quality, not just technical delivery.
• Implement CI/CD pipelines, automated deployments, testing frameworks, and Infrastructure as Code (IaC).
• Support platform governance, security controls, access management, and operational standards.
• Contribute to platform reliability, monitoring, and continuous improvement initiatives.
• Partner with sustainability experts, business analysts, product owners, and reporting teams.
• Support requirements gathering, solution design, testing, and production releases.
• Communicate risks, dependencies, and delivery status clearly and proactively.
• Ensure solutions meet business, performance, and quality expectations.
• Take full ownership of deliverables and outcomes.
• 5+ years of experience designing, developing, and operating data platforms and pipelines.
• Extensive hands-on experience with Databricks in production environments.
• PySpark
• SQL
• Strong knowledge of Git-based development and version control practices.
• Experience with automated testing, deployment pipelines, and release management.
• Solid understanding of:
• Data lineage and metadata management
• Data governance and auditing
• Data validation and reconciliation frameworks
• Strong troubleshooting and root-cause analysis skills.
• Ability to balance engineering excellence with pragmatic delivery.
• Demonstrated ownership and accountability for production services and data products.
• Proactive, self-driven, and solution-oriented mindset.
• Preferred Qualifications
• Experience with sustainability reporting, ESG data, and regulatory reporting requirements.
• Experience developing Databricks applications, dashboards, or user-facing data tools.
• Knowledge of procurement, supplier, finance, and ERP data domains.
• Experience supporting executive reporting, disclosures, or audit-related data processes.