Senior Backend Engineer: Spark/Scala/Python on AWS/GCP

MACHINE LEARNING TECHNOLOGIES LLC; Elk Grove (CA); On-site; USD 83,000 - 124,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC is seeking a Senior Backend Engineer with strong data engineering background to design, develop, and maintain scalable data pipelines and back-end systems. You will work with large datasets, leverage cloud technologies, and collaborate with data scientists to deliver data-driven solutions.

This role is available in the United States and India. Responsibilities include building data pipelines with Spark, Python, Scala, and Java, writing optimized SQL, and

Qualifications

• Bachelor’s degree in computer science or a related field (or equivalent experience).
• 5+ years of experience in back-end development with a focus on data engineering.

Responsibilities

• Design, develop, and maintain data pipelines using Spark, Python, Scala, and Java.
• Write efficient and optimized SQL queries for ETL processes.
• Work with DataFrames to manipulate and analyze large datasets.
• Implement data storage and processing solutions using cloud technologies (AWS or GCP).
• Build and maintain real-time data streaming pipelines using MSK/Kafka.
• Utilize S3 for data storage and retrieval.
• Work with data lake technologies like Iceberg.
• Ensure data quality, integrity, and security.
• Collaborate with data scientists and other engineers to understand data requirements and deliver solutions.
• Participate in code reviews and contribute to improving development processes.
• Troubleshoot and resolve issues in data pipelines and back-end systems.

Skills

Spark

Python

Scala

Java

SQL

DataFrames

Education

Bachelor’s degree in computer science or related field

Tools

AWS

GCP

MSK/Kafka

S3

Iceberg

Job description

MACHINE LEARNING TECHNOLOGIES LLC is seeking a Senior Backend Engineer with strong data engineering background to design, develop, and maintain scalable data pipelines and back-end systems. You will work with large datasets, leverage cloud technologies, and collaborate with data scientists to deliver data-driven solutions.

This role is available in the United States and India. Responsibilities include building data pipelines with Spark, Python, Scala, and Java, writing optimized SQL, and