Data Engineer: Databricks Pipelines & Data Quality

Erste Digital; Wien; Vor Ort; EUR 60.000 - 95.000; Vollzeit

Benefits dieser Stelle

Health center

Employee benefits

Referral program

Zusammenfassung

Erste Digital, part of Erste Group, is looking for a data engineer to build and optimize data solutions on Databricks using PySpark and Delta Lake. You will manage a centralized ingestion framework, design end-to-end pipelines, and partner with stakeholders to ensure data quality and robust diagnostics.

You will contribute to IaC practices, CI/CD improvements, and mentor junior team members, while leveraging AWS/Azure cloud familiarity and strong communication across technical and business teams.

Qualifikationen

• 2–5 years of experience in data engineering or a similar role.
• Recent direct work experience with SQL and Python for data processing, transformation, and analysis & with Databricks and/or PySpark (Delta Lake, Spark SQL, structured streaming)
• Good understanding and expertise in data ingestion patterns, ETL/ELT processes, and data modeling fundamentals
• Experience with software engineering best practices: version control (Git), code reviews, CI/CD pipelines
• Familiarity with cloud platforms, preferably AWS and/or MS Azure
• Strong analytical and problem-solving skills
• Clear verbal and written communication; ability to interact with both technical and business stakeholders
• Systematic, collaborative approach with a continuous learning mindset
• Solid documentation practices
• Not mandatory but beneficial: Terraform, DataOps/MLOps, Unity Catalog, workflow orchestration tools
• Language skills: English professional proficiency; German (e.g. B2) is a plus

Aufgaben

• Work hands-on with the Databricks platform, PySpark, and Delta Lake to build and optimize data solutions
• Manage, operate, and continuously improve a centralized data ingestion framework
• Design, build, and maintain data ingestion pipelines end-to-end
• Troubleshoot and resolve data quality incidents in collaboration with data consumers and stakeholders
• Develop and implement data quality frameworks and monitoring solutions
• Expand platform capabilities by evaluating, testing, and adopting new Databricks features
• Contribute to infrastructure-as-code practices and CI/CD pipeline improvements
• Mentor junior team members and support knowledge sharing within the team

Kenntnisse

SQL

Python

Data ingestion

ETL/ELT

Data modeling

Communication

Problem-solving

Documentation

Continuous learning

Tools

Databricks

PySpark

Delta Lake

Git

CI/CD

Terraform

Unity Catalog

Workflow orchestration

Jobbeschreibung

Erste Digital, part of Erste Group, is looking for a data engineer to build and optimize data solutions on Databricks using PySpark and Delta Lake. You will manage a centralized ingestion framework, design end-to-end pipelines, and partner with stakeholders to ensure data quality and robust diagnostics.

You will contribute to IaC practices, CI/CD improvements, and mentor junior team members, while leveraging AWS/Azure cloud familiarity and strong communication across technical and business teams.