data engineer in fintech

payabl.; United States; Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Provident Fund

Annual learning budget

€150 Monthly Wolt allowance

SportsBenefits membership

Company car after one year

Complimentary parking space

25 Days vacation plus holidays and 10+

Job summary

payabl. provides global financial services, payments innovation, and banking services for businesses through its payabl.one platform. The role focuses on designing scalable data lakehouse solutions on AWS, enabling real-time data ingestion and trusted analytics layers across the bronze-to-gold medallion architecture.

Collaborate with analysts and engineers to model data, optimize PySpark jobs on EMR, and implement governance, quality checks, and cost-efficient processes.

Qualifications

• 3+ years of experience in data engineering or related roles.
• Strong SQL and data modeling skills for analytics and reporting use cases.
• Strong Python programming experience, ideally including PySpark.
• Experience designing and maintaining ETL/ELT pipelines in production.
• Experience with real-time or near-real-time data ingestion.
• Experience with Kafka or similar streaming technologies.
• Experience with CDC concepts and tools such as Debezium.
• Experience with cloud data lake or lakehouse architectures.
• Hands-on experience with AWS data services such as S3, Glue Catalog, Glue Jobs, EMR, and IAM.
• Experience with Apache Iceberg, Delta Lake, or similar open table formats.
• Experience designing curated analytics layers, including silver and gold layers in a medallion architecture.
• Experience with Apache Airflow, Dagster, or similar orchestration tools.
• Experience with relational databases such as MySQL, MariaDB, or PostgreSQL.
• Working knowledge of Unix/Linux environments and shell scripting.
• Understanding of data quality, governance, lineage, and production monitoring concepts.

Responsibilities

• Design, build, and maintain scalable data lakehouse solutions on AWS using S3, Iceberg, and Glue Catalog.
• Contribute to the evolution of the medallion architecture across bronze, silver, and gold layers.
• Build and support real-time and near-real-time ingestion pipelines using Debezium, Kafka, Kafka Connect, and Iceberg sinks.
• Monitor and troubleshoot streaming pipelines, connector issues, schema changes, data consistency, and ingestion reliability.
• Design and implement silver and gold datasets as trusted, business-ready data products.
• Work with analysts, analytics engineers, and business stakeholders to understand requirements and create reusable data models.
• Develop and maintain batch and distributed processing jobs using PySpark on AWS EMR and AWS Glue.
• Optimize data transformation jobs for performance, scalability, reliability, and cost efficiency.
• Build and maintain Apache Airflow workflows for API ingestion, batch processing, and orchestration.
• Support integrations from external and third-party systems using tools such as Airbyte.
• Implement data quality checks, validation processes, and reconciliation logic.
• Contribute to data governance practices covering documentation, ownership, lineage, access, and compliance.
• Collaborate on AWS infrastructure and deployment practices using S3, Glue, EMR, IAM, EKS, and related services.
• Support infrastructure-as-code workflows using Terraform and Terragrunt.

Skills

SQL

Data modeling

Python

PySpark

ETL/ELT

Kafka

Debezium

AWS

Airflow

Airbyte

dbt

Docker

Kubernetes

Tools

Airflow

Terraform

Terragrunt

Apache Iceberg

Delta Lake

Snowflake

Databricks

Airbyte

dbt

Docker

Kubernetes

Job description

Описание:

payabl. provides global financial services, payments innovation, and banking services for businesses through its payabl.one platform. As a licensed financial company with principal membership in card schemes, it specializes in global payments and multi-currency accounts.

Задачи:

• Design, build, and maintain scalable data lakehouse solutions on AWS using S3, Apache Iceberg, and AWS Glue Catalog
• Contribute to the evolution of the medallion architecture across bronze, silver, and gold layers
• Build and support real-time and near-real-time ingestion pipelines using Debezium, Kafka, Kafka Connect, and Iceberg sinks
• Monitor and troubleshoot streaming pipelines, connector issues, schema changes, data consistency, and ingestion reliability
• Design and implement silver and gold datasets as trusted, business-ready data products
• Work with analysts, analytics engineers, and business stakeholders to understand requirements and create reusable data models
• Develop and maintain batch and distributed processing jobs using PySpark on AWS EMR and AWS Glue
• Optimize data transformation jobs for performance, scalability, reliability, and cost efficiency
• Build and maintain Apache Airflow workflows for API ingestion, batch processing, and orchestration
• Support integrations from external and third-party systems using tools such as Airbyte
• Implement data quality checks, validation processes, and reconciliation logic
• Contribute to data governance practices covering documentation, ownership, lineage, access, and compliance
• Collaborate on AWS infrastructure and deployment practices using S3, Glue, EMR, IAM, EKS, and related services
• Support infrastructure-as-code workflows using Terraform and Terragrunt

Требования:

• 3+ Years of experience in data engineering or related roles
• Strong SQL and data modeling skills for analytics and reporting use cases
• Strong Python programming experience, ideally including PySpark
• Experience designing and maintaining ETL/ELT pipelines in production
• Experience with real-time or near-real-time data ingestion
• Experience with Kafka or similar streaming technologies
• Experience with CDC concepts and tools such as Debezium
• Experience with cloud data lake or lakehouse architectures
• Hands-on experience with AWS data services such as S3, Glue Catalog, Glue Jobs, EMR, and IAM
• Experience with Apache Iceberg, Delta Lake, or similar open table formats
• Experience designing curated analytics layers, including silver and gold layers in a medallion architecture
• Experience with Apache Airflow, Dagster, or similar orchestration tools
• Experience with relational databases such as MySQL, MariaDB, or PostgreSQL
• Working knowledge of Unix/Linux environments and shell scripting
• Understanding of data quality, governance, lineage, and production monitoring concepts
• Nice to have: Apache Druid
• Nice to have: ClickHouse
• Nice to have: Snowflake
• Nice to have: Databricks, or other analytical databases and warehouse platforms
• Nice to have: Airbyte or similar data integration tools
• Nice to have: dbt or collaboration with analytics engineering teams
• Nice to have: Terraform and Terragrunt
• Nice to have: Docker and Kubernetes
• Nice to have: Spark job optimization on EMR or AWS Glue
• Nice to have: Tableau, Power BI, Superset, or AWS QuickSight
• Nice to have: data observability, alerting, and monitoring tools

Условия:

• Remote work is available from Poland or Portugal, or on-site from Cyprus
• Provident Fund is available after passing probation
• Annual learning budget is available after probation
• €150 Monthly Wolt allowance
• SportsBenefits membership
• Company car may be available after one year, subject to performance and availability
• Complimentary parking space
• 25 Days of vacation plus public holidays and 10 additional sick days
• Local discount card and event tickets
• Free Greek language classes twice a week
• Opportunities to participate in international company events and initiatives
• Hiring includes a short technical screening and a practical assessment, which may involve live coding or a real-world scenario