Senior Data Engineer (AWS) – Real-Time Data Pipelines

Poland and Middle-Eastern Europe; United States; Remote; USD 100,000 - 150,000; Full time

Job summary

Xebia is a global AI-first, engineering partner seeking a Data Engineer to design and optimize scalable batch and real-time data pipelines in a remote US/EU context. You will implement ELT workflows, build cloud infrastructure on AWS, and collaborate with cross-functional teams to deliver robust data solutions.

The role requires 3+ years of data engineering, strong SQL and Python, and experience with Airflow, Redshift, Kafka, and Spark.

Qualifications

• 3+ years of professional experience in data engineering or a related technical role.
• Hands-on experience with AWS and cloud-based data platforms.
• Strong SQL and data modelling skills.
• Experience with production-scale data warehouses such as Amazon Redshift, Snowflake, or BigQuery.
• Proven experience with Python, dbt, and Apache Airflow.
• Familiarity with Kafka, Spark, and Apache NiFi.
• Experience with Docker and Kubernetes.
• Strong analytical, problem-solving, communication, and collaboration skills.
• Comfortable working with large and complex datasets in production environments.
• Excellent English skills and effective communication with international teams.

Responsibilities

• Designing, developing, and maintaining scalable batch and real-time data pipelines.
• Building ELT workflows and data transformations using Python, SQL, and dbt.
• Developing and maintaining cloud-based data infrastructure on AWS, including S3, Glue, Lambda, and Redshift.
• Implementing data orchestration and scheduling using Apache Airflow or Amazon MWAA.
• Working with streaming and big data technologies such as Kafka, Spark, and Apache NiFi.
• Ensuring the reliability, performance, scalability, and quality of production data pipelines.
• Collaborating with data architects, product teams, analysts, IT teams, and business stakeholders to deliver scalable data solutions.
• Contributing to data security, governance, monitoring, automation, and continuous improvement.
• Participating in a scheduled on-call rotation to support the reliability of the data platform.

Skills

SQL

Python

Data modeling

Cloud platforms

Communication

Tools

AWS

dbt

Airflow

Kafka

Spark

NiFi

Redshift

Snowflake

BigQuery

Job description

Xebia is a global AI-first, engineering partner seeking a Data Engineer to design and optimize scalable batch and real-time data pipelines in a remote US/EU context. You will implement ELT workflows, build cloud infrastructure on AWS, and collaborate with cross-functional teams to deliver robust data solutions.

The role requires 3+ years of data engineering, strong SQL and Python, and experience with Airflow, Redshift, Kafka, and Spark.