Senior Professional Data-Engineer
GoHiring GmbH
Erhalte mehr Antworten von Arbeitgebern
Versende in nur wenigen Minuten einen passgenauen Lebenslauf.
Zusammenfassung
adorsys is seeking a Data Engineer to design, develop, and optimize scalable data pipelines, lakehouse architectures, and streaming solutions in a finance-focused platform. The role emphasizes data quality, governance, and security, with migration to modern Kubernetes/Iceberg-based infrastructure by 2026.
You will collaborate with data scientists, platform engineers, and business stakeholders to deliver data-driven solutions and ensure robust performance across distributed systems.
Qualifikationen
- Proven experience with Apache Iceberg, Apache NiFi, and Trino.
- Understanding and ability to work with Apache Spark code.
- Experience with ETL/ELT processes and integrating APIs.
- Hands-on expertise in real-time streaming with Apache Kafka.
- Strong background in large-scale data processing and distributed systems.
- Cloud platforms (AWS, Azure, or GCP), especially S3, IAM, Glue.
- Programming languages: Python, Java, or Scala.
- Containerization and orchestration: Docker, Kubernetes.
- CI/CD & DevOps tools: Git, Jenkins, Terraform, GitHub Actions.
- Monitoring & Logging: Prometheus, Grafana, ELK Stack, Splunk.
- Strong knowledge of SQL and performance optimization techniques.
Aufgaben
- Design, build, and maintain scalable ETL/ELT pipelines.
- Implement and manage data lakehouse structures based on Apache Iceberg.
- Develop and optimize SQL queries and data access layers with Trino.
- Integrate diverse data sources into unified pipelines.
- Ensure data quality, security, and governance across the data infrastructure.
- Perform performance tuning, monitoring, and troubleshooting of data pipelines.
- Collaborate with cross-functional teams to deliver data-driven solutions.
- Document technical designs, concepts, and processes.
Kenntnisse
Tools
Jobbeschreibung
At adorsys, we are not just developers and analysts; we are pioneers shaping the digital future for banking and insurance sectors. For over a decade, we have crafted innovative software solutions that grant our clients a decisive edge in their business. We foster an international and agile work environment that thrives on diversity, creativity, and collaboration.
We are looking for experienced Data Engineers to join an international platform team responsible for the infrastructure and data management of intelligent automation services in the finance domain. The services leverage artificial intelligence and machine learning models to generate automated booking proposals from various financial data sources.
The team is currently modernizing its platform to ensure long-term scalability, stability, and performance. By 2026, all core components and data pipelines will be migrated to a modern infrastructure stack based on Kubernetes, Apache Iceberg, and other state-of-the-art technologies.
As a Data Engineer, you will play a key role in designing, developing, and optimizing scalable data pipelines, lakehouse architectures, and streaming solutions, ensuring high data quality, governance, and security standards.
Main Tasks & Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines using Apache NiFi.
- Implement and manage data lakehouse structures based on Apache Iceberg, including schema evolution, partitioning, and time travel features.
- Develop and optimize SQL queries and data access layers with Trino, ensuring high performance and reliability.
- Ensure data quality, security, and governance across the data infrastructure.
- Perform performance tuning, monitoring, and troubleshooting of data pipelines and distributed systems.
- Collaborate with cross-functional teams (data scientists, platform engineers, business stakeholders) to deliver data-driven solutions.
- Document technical designs, concepts, and processes.
At adorsys, we are not just developers and analysts; we are pioneers shaping the digital future for banking and insurance sectors. For over a decade, we have crafted innovative software solutions that grant our clients a decisive edge in their business. We foster an international and agile work environment that thrives on diversity, creativity, and collaboration.
We are looking for experienced Data Engineers to join an international platform team responsible for the infrastructure and data management of intelligent automation services in the finance domain. The services leverage artificial intelligence and machine learning models to generate automated booking proposals from various financial data sources.
The team is currently modernizing its platform to ensure long-term scalability, stability, and performance. By 2026, all core components and data pipelines will be migrated to a modern infrastructure stack based on Kubernetes, Apache Iceberg, and other state-of-the-art technologies.
As a Data Engineer, you will play a key role in designing, developing, and optimizing scalable data pipelines, lakehouse architectures, and streaming solutions, ensuring high data quality, governance, and security standards.
Main Tasks & Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines using Apache NiFi.
- Implement and manage data lakehouse structures based on Apache Iceberg, including schema evolution, partitioning, and time travel features.
- Develop and optimize SQL queries and data access layers with Trino, ensuring high performance and reliability.
- Integrate diverse data sources (Kafka, S3, RDBMS, REST APIs, etc.) into unified pipelines.
- Ensure data quality, security, and governance across the data infrastructure.
- Perform performance tuning, monitoring, and troubleshooting of data pipelines and distributed systems.
- Collaborate with cross-functional teams (data scientists, platform engineers, business stakeholders) to deliver data-driven solutions.
- Document technical designs, concepts, and processes.
- Proven experience with Apache Iceberg, Apache NiFi, and Trino.
- Understanding and ability to work with Apache Spark code.
- Experience with ETL/ELT processes and integrating APIs.
- Hands-on expertise in real-time streaming with Apache Kafka.
- Strong background in large-scale data processing and distributed systems.
- Cloud platforms (AWS, Azure, or GCP), especially S3, IAM, Glue.
- Programming languages: Python, Java, or Scala.
- Containerization and orchestration: Docker, Kubernetes.
- CI/CD & DevOps tools: Git, Jenkins, Terraform, GitHub Actions.
- Monitoring & Logging: Prometheus, Grafana, ELK Stack, Splunk.
- Strong knowledge of SQL and performance optimization techniques.
- Experience with Azure cloud services.
- Familiarity with CI/CD automation and Infrastructure as Code.
- Practical knowledge of security configurations in data processing.
- Structured and solution-oriented working style.
- Strong communication and collaboration skills within interdisciplinary teams.
- Ability to document and explain complex technical processes in a clear manner.
The platform team develops and maintains the data infrastructure for automation services in the accounting and finance sector. These services use artificial intelligence to continuously improve booking suggestions based on historical cloud data, ensuring seamless integration into familiar workflows.
To ensure scalability and long‑term stability, the platform is being migrated to a new infrastructure (Kubernetes, Apache Iceberg, etc.) by 2026. This transformation secures performance, scalability, and future‑proof operations.
We're not just offering a job; we're inviting you to be part of a movement. With us, you'll navigate the exciting terrain of finance and banking and contribute to sustainable financial living. Benefit from a German firm's structure and security, coupled with competitive remuneration and a culture that nurtures growth.
Lucian Mihaila
We have been developing individual software for banks and insurance companies for more than 10 years. We do this to give our customers a real competitive advantage. Our teams accompany the clients from the first idea to the finished product. Regardless of whether an existing service needs to be modernized or the solution for a new business model is missing: we develop software for a digital future.
Weitere Details und Bewerbung
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
