Data DevOps Engineer

PTT Consulting; Deutschland; Vor Ort; EUR 70.000 - 95.000; Vollzeit

Zusammenfassung

PTT Consulting is seeking a Data DevOps Engineer with 3-5 years of experience to design, develop, automate, test, support, and administer the Enterprise Infrastructure for Big Data and fast data processing. The role requires strong Python/SQL skills, system administration experience, and the ability to communicate with diverse stakeholders.

The position offers full-time, hybrid work (remote options with Warsaw collaboration).

Qualifikationen

• Background in computer science, engineering, physics, mathematics or equivalent
• Strong background in system administration and shell scripting
• Strong hands on experience with Python, SQL
• Strong problem-solving skills
• Eagerness to learn new approaches and technologies

Aufgaben

• Communicate effectively with technical and non-technical stakeholders.
• Support and host discussions within a multidisciplinary team, with potentially difficult dynamics.
• Be an advocate for the team externally, and can manage differing perspectives.
• Understand and help teams to apply a range of techniques for data profiling.
• Source system analysis from a different sources.
• Bring multiple data sources together in a conformed model for analysis.
• Establish enterprise-scale data integration procedures across the data development life cycle, and ensure that teams adhere to them.
• Manage resources to ensure that data services work effectively at an enterprise level.
• Identify areas of innovation in data tools and techniques, and recognise appropriate timing for adoption.
• Establish standards, keep them up to date and ensure adherence to them.
• Keep abreast of best practice in industry.
• Understand the concepts and principles of data modelling and can produce relevant data models.
• Recognise opportunities for the reuse and alignment of data models in different organisations.
• Design the method to categorise data models within an organisation.
• Design an appropriate metadata repository and present changes to existing metadata repositories.
• Understand a range of tools for storing and working with metadata.
• Provide oversight and advice to more inexperienced members of the team.
• Ensure that the most appropriate actions are taken to resolve problems as they occur.
• Co-ordinate teams to resolve problems and to implement solutions and preventative measures.
• Use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations.
• Collaborate with others to review specifications where appropriate.
• Show a thorough understanding of the technical concepts required for the role, and can explain how these fit into the wider technical landscape.
• Review requirements and specifications, and define test conditions.
• Identify issues and risks associated with work.
• Analyse and report test activities and results.

Kenntnisse

System administration

Shell scripting

Python

SQL

Problem solving

Willingness to learn

Ausbildung

CS/Engineering/Physics/Math

Tools

Azure cloud

Databricks

Spark

Cluster administration

Private endpoint configuration

Virtualization

Docker

Kubernetes

CI/CD tools

Infrastructure as code

Configuration/infrastructure management tools

Jobbeschreibung

Level: 3-5 years’ experience

Language: English

Type of work: Full-time; Remote/hybrid (Warsaw)

As a Data DevOps Engineer, you will serve as a platform expert responsibile for the design, development, automation, testing, support and administration of the Enterprise Infrastructure for Big Data and Fast Data processing.

Requirements

• Background in computer science, engineering, physics, mathematics or equivalent
• Strong background in system administration and shell scripting
• Strong hands on experience with Python, SQL
• Strong problem-solving skills
• Eagerness to learn new approaches and technologies

Nice to have

• Experience with Azure cloud services
• Experience with Databricks and Spark.
• Experience with administration of large clusters
• Experience with Private endpoint configuration
• Understanding of modern Big Data technologies (Hadoop, Spark, Kafka, NoSQL)
• Experience with virtualization
• Experience with containerization (Docker, Kubernetes)
• Experience with CI/CD tools
• Experience with Infrastructure as code
• Experience with configuration or infrastructure management tools

Responsibilities

• Communicate effectively with technical and non-technical stakeholders.
• Support and host discussions within a multidisciplinary team, with potentially difficult dynamics.
• Be an advocate for the team externally, and can manage differing perspectives.
• Understand and help teams to apply a range of techniques for data profiling.
• Source system analysis from a different sources.
• Bring multiple data sources together in a conformed model for analysis.
• Establish enterprise-scale data integration procedures across the data development life cycle, and ensure that teams adhere to them.
• Manage resources to ensure that data services work effectively at an enterprise level.
• Identify areas of innovation in data tools and techniques, and recognise appropriate timing for adoption.
• Establish standards, keep them up to date and ensure adherence to them.
• Keep abreast of best practice in industry.
• Understand the concepts and principles of data modelling and can produce relevant data models.
• Recognise opportunities for the reuse and alignment of data models in different organisations.
• Design the method to categorise data models within an organisation.
• Design an appropriate metadata repository and present changes to existing metadata repositories.
• Understand a range of tools for storing and working with metadata.
• Provide oversight and advice to more inexperienced members of the team.
• Ensure that the most appropriate actions are taken to resolve problems as they occur.
• Co-ordinate teams to resolve problems and to implement solutions and preventative measures.
• Use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations.
• Collaborate with others to review specifications where appropriate.
• Show a thorough understanding of the technical concepts required for the role, and can explain how these fit into the wider technical landscape.
• Review requirements and specifications, and define test conditions.
• Identify issues and risks associated with work.
• Analyse and report test activities and results.

Client

A global leader with a sharp focus on lottery solutions. A confident step forward building on a long history of delivering safe and secure technology, demonstrating strong commitment to customers as a dedicated lottery service provider. Leveraging collective insight, experience, and expertise to create reliable and engaging solutions that help lottery clients achieve objectives, meet player needs, and deliver meaningful benefits to communities.