Senior Data Engineer

Jobgether; United States; Remote; USD 130,000 - 200,000; Full time

Benefits offered by this job

Fully remote

International team

Competitive salary

Job summary

Jobgether, via a partner company, is seeking a Senior Data Engineer based in the United States to build, operate, and optimize data infrastructure powering analytics and BI across global markets. In this role you will own CDC pipelines, ensure timely data delivery into Snowflake, and collaborate with Data Science, Analytics, and Product teams to provide reliable data access.

You will focus on CI/CD, IaC, monitoring, and data quality checks while contributing to scalable, distributed data systems

Qualifications

• 4+ years in backend or data engineering for consumer-scale apps.
• 2+ years Python development.
• Proficient in CDC and data pipelines with streaming tech (Kafka/MSK/DMS).
• Strong Spark expertise, especially Structured Streaming.
• Experience with lakehouse formats (Iceberg/Delta/Hudi) and Parquet.
• DB knowledge across MySQL, PostgreSQL, RDS, Snowflake.
• AWS cloud infra experience (EMR/S3/MSK/DMS/RDS/EKS).
• Infrastructure-as-code and CI/CD (Terraform, Terragrunt, Helm, Jenkins).
• Airflow or Kubernetes-based job scheduling.
• Strong understanding of distributed systems and large-scale data processing.
• Excellent problem-solving and analytical skills; strong communication.
• Ability to work independently and collaborate across teams.

Responsibilities

• Own, maintain, and improve CDC and data synchronization pipelines into Snowflake.
• Optimize pipeline performance, scalability, and cost across multiple markets.
• Design and onboard new data sources, tables, and markets while handling schema changes.
• Implement monitoring, SLOs, and automated data quality checks for reliability.
• Develop tests, CI/CD, IaC, and safe deployment strategies (dry runs, blue-green).
• Troubleshoot incidents, conduct root-cause analyses, and update runbooks and docs.
• Contribute to architectural design discussions and peer reviews for code quality.
• Collaborate with Data Science, Analytics, Data Platform, Product, and Business teams.
• Take end-to-end ownership of production data systems from dev to operations.

Skills

Problem solving

Analytical thinking

Written communication

Verbal communication

Independence

Cross-team collaboration

Tools

Kafka

Spark

Apache Iceberg

Delta Lake

Apache Hudi

Parquet

MySQL

PostgreSQL

Snowflake

AWS EMR

S3

MSK

DMS

RDS

EKS

Terraform

Terragrunt

Helm

Jenkins

Airflow

Kubernetes

Debezium

Kinesis

Flink

Trino

Presto

Job description

This position is listed on behalf of a partner company that manages all applications and next steps. Our partner is looking for a Senior Data Engineer based in the United States. We are seeking an experienced Senior Data Engineer to help build, operate, and optimize the data infrastructure powering analytics and business intelligence across global markets. In this individual contributor role, you will take ownership of critical data synchronization pipelines, ensuring reliable and timely data delivery from market-level databases into Snowflake.

Accountabilities

• Own, maintain, and continuously improve change data capture (CDC) and data synchronization pipelines, ensuring accurate, reliable, and timely data delivery from market-level databases into Snowflake.
• Optimize pipeline performance, scalability, and infrastructure efficiency by reducing processing latency, controlling operational costs, and supporting increasing data volumes across multiple markets.
• Design, develop, and evolve data infrastructure by onboarding new data sources, tables, and markets while managing schema changes and ensuring compatibility across systems.
• Establish and maintain service level objectives (SLOs), implement monitoring and alerting mechanisms, and develop automated data quality checks to ensure data accuracy, availability, and reliability.
• Strengthen engineering standards by developing unit and integration tests, maintaining CI/CD workflows, implementing infrastructure-as-code, and introducing safe deployment strategies such as dry runs and blue-green deployments.
• Troubleshoot production incidents, lead root cause analyses, and translate findings into improvements in monitoring, testing, technical documentation, and operational runbooks.
• Participate in architectural and technical design discussions, prepare and review technical design documents, and contribute to code quality through constructive peer reviews.
• Collaborate with Data Science, Analytics, Data Platform, Product, Credit, and Business Development teams to understand data requirements and provide dependable access to critical information.
• Take end-to-end ownership of production data systems, from development and deployment through ongoing maintenance and operational improvement.

Requirements

• At least 4 years of professional experience in backend engineering or data engineering, ideally supporting consumer-scale applications.
• Minimum 2 years of hands-on professional experience developing with Python.
• Proven ability to independently build, operate, troubleshoot, and optimize production-grade data pipelines.
• Practical experience with change data capture (CDC) and stream processing technologies such as Kafka, Amazon MSK, AWS Database Migration Service (DMS), or comparable solutions.
• Strong proficiency in Apache Spark, particularly Spark Structured Streaming.
• Experience with modern data lakehouse architectures and table formats, including Apache Iceberg, Delta Lake, or Apache Hudi, as well as Parquet.
• Strong database knowledge across relational and analytical platforms, including MySQL, PostgreSQL, Amazon RDS, and Snowflake.
• Hands‑on experience with AWS cloud infrastructure, ideally including EMR, S3, MSK, DMS, RDS, and EKS.
• Experience with infrastructure-as-code and CI/CD technologies such as Terraform, Terragrunt, Helm, and Jenkins.
• Familiarity with data pipeline orchestration and job scheduling tools such as Apache Airflow or Python-based services running on Kubernetes.
• Strong understanding of distributed systems, scalable algorithms, and large-scale data processing architectures.
• Excellent problem-solving and analytical skills, with the ability to investigate complex technical challenges and deliver reliable solutions.
• Strong written and verbal communication skills, including the ability to explain complex technical concepts to engineering teams and business stakeholders.
• Ability to work independently, take ownership of technical initiatives, and collaborate effectively across multidisciplinary teams.

Nice To Have

• Experience with Trino or Presto query engines.
• Practical knowledge of Kubernetes and containerized applications.
• Familiarity with AWS Glue.
• Experience with additional stream processing technologies such as Debezium, Amazon Kinesis, or Apache Flink.
• Exposure to modern data lakehouse technologies, including Delta Lake and Apache Hudi.
• Advanced degree in Computer Science, Mathematics, or another relevant technical discipline.

Benefits

• Annual salary: $130,000–$200,000, depending on qualifications, experience, and other relevant factors.
• Fully remote, remote-first working environment.
• Opportunity to collaborate with an international team across multiple global markets.
• Exposure to modern cloud infrastructure, AI-driven technology, and large-scale distributed data systems.
• Opportunity to contribute to innovative financial technology solutions with meaningful global impact.
• Inclusive and diverse working environment that values different backgrounds, perspectives, and professional experiences.
• Opportunities to influence technical architecture, engineering standards, and the evolution of critical data infrastructure.