Senior Data Backend Engineer - Spark, Kafka, Cloud

Cloud Analytics Technologies, LLC; Sunnyvale (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

CLOUD ANALYTICS TECHNOLOGIES, LLC in Sunnyvale, CA invites a Back-End Engineer with data engineering depth to design and maintain scalable data pipelines and back-end systems. You will work with Spark, Python, Scala, Java, SQL, DataFrames, and cloud storage (S3) in AWS/GCP to deliver reliable data solutions.

Join a collaborative team including data scientists to ensure data quality and security. Requirements include a BS in CS or related field and 5+ years in back-end data engineering.

Qualifications

• Bachelor's degree in computer science or related field or equivalent experience.
• 5+ years of back-end development with a focus on data engineering.

Responsibilities

• Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java.
• Write efficient and optimized SQL queries for ETL processes.
• Work with DataFrames to manipulate and analyze large datasets.
• Implement data storage and processing solutions using cloud technologies (AWS/GCP).
• Build and maintain real-time data streaming pipelines using MSK/Kafka.
• Utilize S3 for data storage and retrieval.
• Work with data lake technologies like Iceberg.
• Ensure data quality, integrity, and security.
• Collaborate with data scientists and other engineers to understand data requirements.
• Participate in code reviews and contribute to improving development processes.
• Troubleshoot and resolve issues in data pipelines and back-end systems.

Skills

Spark

Python

Scala

Java

SQL

DataFrames

AWS

GCP

Kafka

S3

Iceberg

Data warehousing

Communication

Education

Bachelor's degree in computer science or related field

Tools

Cloud platforms (AWS/GCP)

Job description

CLOUD ANALYTICS TECHNOLOGIES, LLC in Sunnyvale, CA invites a Back-End Engineer with data engineering depth to design and maintain scalable data pipelines and back-end systems. You will work with Spark, Python, Scala, Java, SQL, DataFrames, and cloud storage (S3) in AWS/GCP to deliver reliable data solutions.

Join a collaborative team including data scientists to ensure data quality and security. Requirements include a BS in CS or related field and 5+ years in back-end data engineering.