Databricks Data Engineer: Build Scalable Pipelines

Erste Digital GmbH; Wien; Vor Ort; EUR 47.000 - 57.000; Vollzeit

Benefits dieser Stelle

Health Centre

Employee benefits

Employee Referral Program

Zusammenfassung

Erste Digital GmbH is seeking a data engineer to build and optimize data solutions on Databricks with PySpark and Delta Lake. You will design end-to-end data ingestion pipelines and maintain centralized ingestion frameworks.

You will troubleshoot data quality issues, implement data quality monitoring, and contribute to CI/CD and IaC initiatives while mentoring junior teammates. English proficiency is required; German is a plus.

Qualifikationen

• 2–5 years of experience in data engineering or a similar role
• Proficient in SQL and Python for data processing, transformation, and analysis
• Experience with Databricks and/or PySpark (Delta Lake, Spark SQL, structured streaming)
• Understanding of data ingestion patterns, ETL/ELT, and data modeling fundamentals
• Experience with software engineering best practices: Git, code reviews, CI/CD pipelines
• Familiarity with cloud platforms, preferably AWS and/or MS Azure
• Strong analytical and problem-solving skills; good communication with technical and business stakeholders
• Not mandatory but beneficial: Terraform, DataOps/MLOps, Unity Catalog, orchestration tools
• English professional proficiency; German (e.g. B2) a plus

Aufgaben

• Work hands-on with Databricks platform, PySpark, and Delta Lake to build and optimize data solutions
• Manage, operate, and improve a centralized data ingestion framework
• Design, build, and maintain data ingestion pipelines end-to-end
• Troubleshoot and resolve data quality incidents with data consumers and stakeholders
• Develop and implement data quality frameworks and monitoring solutions
• Expand platform capabilities by evaluating and adopting new Databricks features
• Contribute to infrastructure-as-code practices and CI/CD pipeline improvements
• Mentor junior team members and support knowledge sharing within the team

Kenntnisse

SQL

Python

Data modeling

Tools

Databricks

PySpark

Delta Lake

Spark SQL

Structured streaming

Git

CI/CD

AWS

Azure

Terraform

Jobbeschreibung

Erste Digital GmbH is seeking a data engineer to build and optimize data solutions on Databricks with PySpark and Delta Lake. You will design end-to-end data ingestion pipelines and maintain centralized ingestion frameworks.

You will troubleshoot data quality issues, implement data quality monitoring, and contribute to CI/CD and IaC initiatives while mentoring junior teammates. English proficiency is required; German is a plus.