Cloud Data Engineer: Real-Time Streaming & Lakehouse

Motion Recruitment; Phoenix (AZ); On-site; USD 120,000 - 180,000; Full time

Job summary

Motion Recruitment seeks a Software Engineer focused on data engineering to support a large-scale cloud data platform in Phoenix. You will work with GCP, Hadoop, Spark, Kafka, Flink, PySpark, Airflow, BigQuery, and related technologies to build scalable pipelines and lakehouse architectures.

The role emphasizes real-time data processing, cloud modernization, and collaboration with an experienced engineering team, with a 12-month contract and W2 terms in the United States.

Qualifications

• 2+ years Software Engineering experience.
• 5+ years data engineering with Hadoop and Google Cloud data solutions.
• Experience creating and supporting Spark-based processing and Kafka streaming solutions.
• 2+ years developing data flows using Kafka, Flink, and Spark Streaming.
• 3+ years experience with data lakehouse architecture and design.
• Hands-on experience with Python, PySpark, Apache Kafka, Airflow, and SQL.
• Experience with GCP Cloud Storage, BigQuery, Dataproc, Cloud Composer.
• 2+ years experience with NoSQL databases (columnar, graph, document, key-value).
• Public cloud certification such as GCP Professional Data Engineer, Azure Data Engineer, AWS Specialty Data Analytics.

Responsibilities

• GCP and cloud-native data engineering tasks.
• Hadoop and data lakehouse technologies support.
• Spark, Kafka, and Flink streaming and processing work.
• Python, PySpark, Airflow, and SQL development.
• BigQuery, Dataproc, Cloud Composer, and Cloud Storage usage.
• NoSQL databases and modern data formats management.
• CI/CD, containerization, and DevOps tooling involvement.
• GenAI frameworks and agent-based data capabilities exploration.

Skills

Software Engineering

Data engineering

Spark

Kafka

Flink

Python

PySpark

Airflow

SQL

GCP

BigQuery

Dataproc

Cloud Composer

NoSQL

GCP Certification

Tools

Hadoop

Hive

HDFS

Parquet

Iceberg

Delta Tables

Jenkins

Docker

Kubernetes

Job description

Motion Recruitment seeks a Software Engineer focused on data engineering to support a large-scale cloud data platform in Phoenix. You will work with GCP, Hadoop, Spark, Kafka, Flink, PySpark, Airflow, BigQuery, and related technologies to build scalable pipelines and lakehouse architectures.

The role emphasizes real-time data processing, cloud modernization, and collaboration with an experienced engineering team, with a 12-month contract and W2 terms in the United States.