Data Platform Reliability Software Engineer

AppLab Systems, Inc; Sunnyvale (CA); On-site; USD 130,000 - 170,000; Full time

Job summary

engineeringjobs.net, Inc. in Sunnyvale, CA seeks engineers to build batch processing with Spark and streaming with Flink, while maintaining data platform infrastructure.

The role collaborates with teams to manage Airflow, EKS, Superset, and Trino, ensuring reliability and scalability of data workflows.

Qualifications

• Experience with Spark and Flink batch/stream processing.
• Experience managing data platform infrastructure.

Responsibilities

• Create and maintain batch processing jobs using Spark and stream processing jobs using Flink.
• Support data platform infrastructure, including Airflow, EKS, Superset and Trino.

Skills

Apache Spark

Apache Flink

Kafka

Data Platform Management

Tools

Airflow

Amazon EKS

Superset

Trino

Iceberg

Cassandra

Job description

Create and maintain batch processing jobs using Spark and stream processing jobs using Flink. Manage data platform infrastructure, including Airflow, EKS, Superset, and Trino.

Requirements: The role involves working with Spark and Flink jobs and managing data platform infrastructure. The posting does not specify education, years of experience, or other formal requirements.

Key Skills: Apache Spark, Apache Flink, Kafka, Apache Iceberg, Cassandra, Airflow, Amazon EKS, Superset, Trino, Batch Processing, Stream Processing, Data Platform Infrastructure