Senior Backend Data Engineer: Spark, Python, Cloud & Streaming

Quantum Technologies. LLC; Elk Grove (CA); On-site; Full time

Job summary

A technology firm is seeking a skilled Back-End Engineer with a strong background in data engineering to design, develop, and maintain scalable data pipelines and systems. Responsibilities include using Spark, Python, and SQL for ETL processes, collaborating with data scientists, and ensuring data quality. The ideal candidate has at least 5 years of experience in back-end development and proficiency in cloud technologies like AWS or GCP. This position is based in Elk Grove, CA and offers a competitive hourly rate based on experience.

Qualifications

• 5+ years of experience in back-end development with a focus on data engineering.

Responsibilities

• Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java.
• Write efficient and optimized SQL queries for ETL processes.
• Work with DataFrames to manipulate and analyze large datasets.
• Implement data storage solutions using AWS or GCP.
• Build and maintain real-time data streaming pipelines.
• Ensure data quality, integrity, and security.

Skills

Spark

Python

Scala

Java

SQL

AWS

GCP

DataFrames

MSK/Kafka

S3

Iceberg

Education

Bachelor’s degree in computer science or a related field

Job description

A technology firm is seeking a skilled Back-End Engineer with a strong background in data engineering to design, develop, and maintain scalable data pipelines and systems. Responsibilities include using Spark, Python, and SQL for ETL processes, collaborating with data scientists, and ensuring data quality. The ideal candidate has at least 5 years of experience in back-end development and proficiency in cloud technologies like AWS or GCP. This position is based in Elk Grove, CA and offers a competitive hourly rate based on experience.