Senior Data Engineer: Spark Pipelines & Production APIs

Keka Technologies Private Limited; United States; On-site; USD 140,000 - 190,000; Full time

Job summary

Keka Technologies Private Limited is seeking a senior data engineer to build, optimize, and operate production Spark pipelines that generate attribution feeds from large-scale data. You will design data processing across modern data lake and warehouse tech, orchestrated with AWS workflows.

The role involves building partner integrations, maintaining REST APIs (PHP/Symfony), diagnosing failures, backfills, and delivering automated tests.

Qualifications

• 7+ years of professional software or data engineering experience.
• Advanced SQL, relational data modeling, and large-scale data processing experience.

Responsibilities

• Build, optimize, and operate production Apache Spark pipelines for attributed conversion feeds.
• Design and evolve data processing across data lake/warehouse with AWS-based orchestration.
• Develop and operate REST APIs, including PHP-based APIs, with secure, backward-compatible changes.
• Diagnose production failures, backfills, and manage staged releases and rollbacks.
• Write unit and integration tests for pipelines, APIs, and feeds.

Skills

Java

Python

SQL

Spark

Tools

Snowflake

MySQL

Iceberg

Airflow

Symfony

PHP

Job description

Keka Technologies Private Limited is seeking a senior data engineer to build, optimize, and operate production Spark pipelines that generate attribution feeds from large-scale data. You will design data processing across modern data lake and warehouse tech, orchestrated with AWS workflows.

The role involves building partner integrations, maintaining REST APIs (PHP/Symfony), diagnosing failures, backfills, and delivering automated tests.