Lead Data Engineer - Experimentation Platform

aKUBE; Santa Monica (CA); Hybrid; USD 110,208 - 137,760; Full time

Job summary

aKUBE is seeking an experienced Data Engineer in Santa Monica for a 6-month hybrid assignment. Design scalable data platforms, build batch and streaming pipelines, and deliver analytics-ready data products to support experimentation and ML workloads.

Requirements include 7+ years in data engineering, Python, SQL, Spark, Databricks, Snowflake, Kafka, and Airflow, plus CI/CD and data governance expertise. Hybrid, onsite-heavy role with GC/USC work authorization preferred.

Qualifications

• Bachelor's degree in a technical field.
• 7+ years of experience in data engineering or large-scale data platforms.
• Strong experience with distributed data processing and cloud-based data architectures.
• Hands-on experience with Python, SQL, Spark, Databricks, Snowflake, Kafka, and Airflow.
• Strong understanding of data modeling, ETL/ELT, streaming architectures, and lakehouse concepts.
• Experience building experimentation, analytics, personalization, or ML data platforms.
• Experience implementing CI/CD, automated testing, monitoring, and data governance.
• Strong system design and architecture experience.
• Experience mentoring engineers and leading technical initiatives.

Responsibilities

• Design and build scalable data platforms supporting experimentation and A/B testing.
• Develop batch and streaming data pipelines for large-scale user and product datasets.
• Build reusable datasets and frameworks for experimentation, analytics, and product measurement.
• Design dimensional data models and analytics-ready data products.
• Implement automated data quality, validation, monitoring, lineage, and governance.
• Build production-grade deployment pipelines with CI/CD and observability.
• Partner with Product, Engineering, Data Science, and Analytics teams to deliver scalable data solutions.
• Optimize data infrastructure supporting experimentation, personalization, and machine learning workloads.
• Mentor engineers and establish best practices for large-scale data engineering.

Skills

Python

SQL

ETL / ELT

Databricks

Snowflake

Data Modeling

Data Warehousing / Lakehouse

CI/CD for Data Pipelines

Data Quality & Data Governance

Education

Bachelor's degree in Computer Science, Engineering, Mathematics, Statistics, or a related technical field

Tools

Spark

Kafka

Airflow

Job description

Onsite/ Hybrid/ Remote: Hybrid (4 days onsite per week, no flexibility)

Duration: 6Months

Rate Range: Upto $100/hr on W2

Work Authorization: GC, USC, All valid EADs except OPT, CPT, H1B

Must Have:

• Python
• SQL
• ETL / ELT
• Databricks
• Snowflake
• Data Modeling
• Data Warehousing / Lakehouse
• CI/CD for Data Pipelines
• Data Quality & Data Governance

Responsibilities:

• Design and build scalable data platforms supporting experimentation and A/B testing.
• Develop batch and streaming data pipelines for large-scale user and product datasets.
• Build reusable datasets and frameworks for experimentation, analytics, and product measurement.
• Design dimensional data models and analytics-ready data products.
• Implement automated data quality, validation, monitoring, lineage, and governance.
• Build production-grade deployment pipelines with CI/CD and observability.
• Partner with Product, Engineering, Data Science, and Analytics teams to deliver scalable data solutions.
• Optimize data infrastructure supporting experimentation, personalization, and machine learning workloads.
• Mentor engineers and establish best practices for large-scale data engineering.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, Mathematics, Statistics, or a related technical field.
• 7+ years of experience in data engineering or large-scale data platforms.
• Strong experience with distributed data processing and cloud-based data architectures.
• Hands-on experience with Python, SQL, Spark, Databricks, Snowflake, Kafka, and Airflow.
• Strong understanding of data modeling, ETL/ELT, streaming architectures, and lakehouse concepts.
• Experience building experimentation, analytics, personalization, or ML data platforms.
• Experience implementing CI/CD, automated testing, monitoring, and data governance.
• Strong system design and architecture experience.
• Experience mentoring engineers and leading technical initiatives.

Nice to Have:

• Experimentation platforms or A/B testing infrastructure.
• Causal inference or product analytics experience.
• ML feature engineering and model lifecycle pipelines.
• Infrastructure automation and observability.
• Subscription, streaming media, advertising, or consumer product experience.