Data Platform Engineer - ETL, Spark & Airflow

Socket.dev; Irvine (CA); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Free snacks & drinks

Full medical/dental/vision insurance

401K contributions

Job summary

TP-Link Systems Inc. seeks a data platform engineer to own end-to-end data pipelines and quality across analytics workloads. You will design ETL on EMR using Spark and Airflow, ingesting from OLTP sources with DataX and CDC, and model data with dimensional schemas for real analytical use cases.

The role requires production experience with Spark, Airflow, and data ingestion tools, plus AWS, Linux, and Git. You’ll collaborate with analysts to turn requirements into reusable datasets and scalable

Qualifications

• Bachelor's degree or equivalent practical experience.
• 2+ years hands-on data development in production environments.
• Experience owning a pipeline or component in production and being accountable.
• Strong SQL with analytics functions and multi-table joins.
• Production-grade Python with ETL tooling (PySpark / pandas / boto3).
• Experience with Spark on EMR and/or Hive, partitioning and memory tuning.
• Airflow DAG design, retries, and SLA alerting.
• Experience with data ingestion tools (DataX, Sqoop, Debezium).
• Exposure to StarRocks, Doris, or ClickHouse and data warehouse modeling.
• Comfort with AWS, Linux, and Git.
• Ability to learn new engines quickly and use AI to assist data problems.

Responsibilities

• Own a data platform component end to end including pipelines and monitoring.
• Design and build ETL on EMR (Spark) and Airflow; ingest from OLTP sources with DataX/CDC.
• Model data using dimensional schemas for analytical use cases.
• Ensure data quality and write monitoring checks; resolve issues upstream.
• Tune runtime and infrastructure costs; choose appropriate engines.
• Create reusable components and automate repetitive tasks.
• Collaborate with analysts to translate requirements into usable datasets.

Skills

SQL

Python

Spark EMR

Airflow

ETL

Amazon Web Services

Linux

Git

Pandas

boto3

Data ingestion

CDC

Education

Bachelor's degree in CS/IS or equivalent

Tools

Spark on EMR

DataX

Sqoop

Debezium

StarRocks

Doris

ClickHouse

dbt

Glue Data Catalog

QuickSight

Job description

TP-Link Systems Inc. seeks a data platform engineer to own end-to-end data pipelines and quality across analytics workloads. You will design ETL on EMR using Spark and Airflow, ingesting from OLTP sources with DataX and CDC, and model data with dimensional schemas for real analytical use cases.

The role requires production experience with Spark, Airflow, and data ingestion tools, plus AWS, Linux, and Git. You’ll collaborate with analysts to turn requirements into reusable datasets and scalable