Senior Software Engineer

Keka Inc.; United States; On-site; USD 120,000 - 160,000; Full time

Job summary

Keka Inc. is seeking a senior data/software engineer to build and optimize Spark pipelines and data models across data lake and warehouse stacks on AWS.

You will own production REST APIs (including PHP/Symfony) and collaborate on AI-assisted development, ensuring secure, backward-compatible changes and robust testing. Strong focus on data quality, reliability, and partner integrations across complex data pipelines.

Qualifications

• 7+ years of professional software or data engineering experience.
• 4+ years building, optimizing, and maintaining production Apache Spark pipelines, with strong Java and Python skills across JVM Spark and PySpark.
• 3+ years operating AWS data workloads using S3 and Airflow; container orchestration and data-catalog experience a plus.
• 3+ years building, changing, and operating production REST APIs, including PHP APIs built with Symfony or a closely equivalent PHP MVC framework.
• Demonstrated proficiency using AI development tools to deliver production-quality work efficiently.
• Ability to independently deliver secure, backward-compatible API changes, including implementation, validation, authorization, data access, documentation, automated testing, and production diagnostics.
• Experience implementing external provider or client integrations involving schemas, APIs, file delivery, identity fields, reconciliation, privacy-sensitive data, and SLAs.
• Experience writing unit and integration tests for data pipelines, APIs, and web applications.
• Hands-on experience building and running big data systems, with a focus on performance, reliability, and data quality.
• Experience in ad tech or advertising measurement, such as attribution, conversions, audience and impression data, identity matching, or partner optimization feeds.

Responsibilities

• Build, optimize, and operate production Apache Spark pipelines that generate attributed conversion and optimization feeds from large-scale impression and conversion data.
• Design and evolve data processing and models across modern data lake and warehouse technologies, orchestrated with a production workflow scheduler on AWS.
• Build and operate partner optimization feeds and integrations with external partners, including schemas, identity fields and unique IDs, file delivery, reconciliation, and SLAs.
• Build, change, and operate production REST APIs, including PHP APIs, delivering secure, backward-compatible changes end to end (implementation, validation, authorization, data access, documentation, automated testing, and production diagnostics).
• Diagnose production failures, reconcile partner-facing data, execute backfills, and manage staged releases and rollbacks.
• Write and maintain unit and integration tests across pipelines, APIs, and feeds.

Skills

Spark pipelines

Java

Python

SQL

AI development tools

Data processing

API development

Tools

Snowflake

MySQL

Iceberg

S3

Airflow

PHP

Symfony

Job description

• Build, optimize, and operate production Apache Spark pipelines that generate attributed conversion and optimization feeds from large-scale impression and conversion data.
• Design and evolve data processing and models across modern data lake and warehouse technologies, orchestrated with a production workflow scheduler on AWS.
• Build and operate partner optimization feeds and integrations with external partners, including schemas, identity fields and unique IDs, file delivery, reconciliation, and SLAs.
• Build, change, and operate production REST APIs, including PHP APIs, delivering secure, backward-compatible changes end to end (implementation, validation, authorization, data access, documentation, automated testing, and production diagnostics).
• Diagnose production failures, reconcile partner-facing data, execute backfills, and manage staged releases and rollbacks.
• Write and maintain unit and integration tests across pipelines, APIs, and feeds.

Qualifications and Education Requirements:

• 7+ years of professional software or data engineering experience.
• 4+ years building, optimizing, and maintaining production Apache Spark pipelines, with strong Java and Python skills across JVM Spark and PySpark.
• Advanced SQL, relational data modeling, and large-scale data processing experience, including production work with Snowflake, MySQL, and Iceberg.
• 3+ years operating AWS data workloads using S3 and Airflow (or equivalent production workflow orchestration); container orchestration and data-catalog experience a plus.
• 3+ years building, changing, and operating production REST APIs, including PHP APIs built with Symfony or a closely equivalent PHP MVC framework.
• Demonstrated proficiency using AI development tools to deliver production-quality work efficiently — including AI-assisted coding, code review, documentation, and investigation, working with agentic code harnesses, and spec-based (spec-driven) AI development — with the judgment to know when to rely on AI output and when not to.
• Ability to independently deliver secure, backward-compatible API changes, including implementation, validation, authorization, data access, documentation, automated testing, and production diagnostics.
• Experience implementing external provider or client integrations involving schemas, APIs, file delivery, identity fields, reconciliation, privacy-sensitive data, and SLAs.
• Experience writing unit and integration tests for data pipelines, APIs, and web applications.
• Ability to diagnose production failures, reconcile data, execute backfills, manage staged releases and rollbacks, and support delivery commitments.
• Hands-on experience building and running big data systems, with a focus on performance, reliability, and data quality.
• Experience in ad tech or advertising measurement, such as attribution, conversions, audience and impression data, identity matching, or partner optimization feeds.
• Ability to ramp quickly and deliver with minimal onboarding in an existing, complex codebase.

Preferred Skills:

• Direct experience building partner optimization or advertising feeds with external platforms (DSPs, publishers, or measurement partners).
• Experience with deterministic and probabilistic identity matching — unique IDs, device IDs, IP-based matching, and identity graphs.
• Familiarity with edge/log delivery infrastructure and pixel/impression tracking.
• Experience with data lake table formats and query engines at large scale.
• Experience with privacy and compliance-driven data workflows (deletion, opt-out, suppression, data retention).
• Comfort operating in uncharted territory — turning ambiguity into production systems without a detailed guide.