Senior Real-Time Data Engineer - Spark, Kafka, GCP

UST; Sunnyvale (CA); On-site; USD 72,000 - 108,000; Full time

Benefits offered by this job

401(k) with company match

Medical, dental, vision insurance

Paid vacation & sick leave

Job summary

UST is hiring a Data Engineer in Sunnyvale, CA to design, develop, and optimize real-time data pipelines using Spark, Kafka, and cloud services like GCP. You will build scalable architectures, ensure data quality, and collaborate with analysts and data scientists worldwide.

The role emphasizes PySpark/Scala, SQL, Python, and experience with data lakes/data warehouses (BigQuery). Expected salary range aligns with regional market standards.

Qualifications

• Bachelor's degree in Computer Science or equivalent.
• Mandatory: BigQuery, Apache Airflow, Spark/Java/PySpark.
• Nice to have: GCP, GCS, Relational DB Data Modeling, Kafka, Terraform.

Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines for batch and real-time ingestion using Spark (PySpark/Scala) and Kafka/Flink.
• Build scalable data architectures including data lakes and data warehouses (BigQuery).
• Tune Spark jobs, SQL queries, and data workflows for speed and cost efficiency.
• Implement data quality checks, monitoring, and governance.

Skills

BigQuery

Apache Airflow

Spark/PySpark/Java

Kafka

Python

SQL

GCP

Databricks

Docker

Kubernetes

Education

Bachelor's degree in Computer Science

Tools

Databricks

Docker

Kubernetes

Terraform

Job description

UST is hiring a Data Engineer in Sunnyvale, CA to design, develop, and optimize real-time data pipelines using Spark, Kafka, and cloud services like GCP. You will build scalable architectures, ensure data quality, and collaborate with analysts and data scientists worldwide.

The role emphasizes PySpark/Scala, SQL, Python, and experience with data lakes/data warehouses (BigQuery). Expected salary range aligns with regional market standards.