Staff Engineer, Machine Learning & Data Infrastructure Staff

D8 LooP

San Jose (CA)

On-site

USD 180,000 - 270,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

OKX in San Jose is seeking a Staff Engineer to design and optimize offline data pipelines for ML infrastructure, working with Databricks, Spark, and warehouse technologies. You will implement scalable data foundations for training, evaluation, and real-time decisions, with a focus on cost and performance.

Collaborate across teams to move models from experimentation to production, ensure reproducibility with feature backfills, and build reusable infrastructure for packaging, versioning,

Qualifications

  • Proficient in building incremental offline data pipelines with Databricks/Spark and warehousing concepts.
  • Strong Python and SQL, plus at least one backend language (Java/Scala/Go/C++).
  • Deep knowledge of Databricks/Spark, including execution plans and cost management.
  • Experience building reusable model infrastructure across cloud platforms and regions.
  • Knowledge of model lifecycle, including training/serving, validation, observability, and rollback.
  • Experience connecting offline and online feature pipelines with correct backfills.
  • Focus on cost and performance optimization.

Responsibilities

  • Design and optimize data pipelines for ML model training and deployment.
  • Improve model inference latency and throughput while managing resource costs.
  • Build and maintain data foundations for training, evaluation, and real-time decisions.
  • Collaborate to move models from experimentation to production.
  • Ensure data correctness, lineage, and backfill strategies.

Skills

Python
SQL
Spark
Databricks
Java/Scala/Go/C++
Cloud platforms (AWS/GCP)
Model lifecycle
Streaming
Backfills & idempotency
Warehousing

Tools

Databricks
Spark
AWS
GCP

Job description

Staff Engineer, Machine Learning & Data Infrastructure

San Jose, California, United States Staff Data Platform posted 1 d ago

At OKX, we believe that the future will be reshaped by crypto, and ultimately contribute to every individual's freedom. OKX is a leading crypto exchange, and the developer of OKX Wallet, giving millions access to crypto trading and decentralized crypto applications (dApps). OKX is also a trusted brand by hundreds of large institutions seeking access to crypto markets.

Skills, with evidence

Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost. must have

  • Incremental loads Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost. must have
  • Python Strong Python, SQL, and at least one backend or systems language such as Java, Scala, Go, or C++. must have
  • SQL Strong Python, SQL, and at least one backend or systems language such as Java, Scala, Go, or C++. must have
  • Spark Deep understanding of Databricks and Spark, or comparable offline data platforms, including execution plans, distributed query optimization, storage layouts, data correctness, and workload cost management. must have
  • AWS Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions. must have · not practised here
  • Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions. must have · not practised here
  • GCP Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions. must have · not practised here
  • Cost & performance Improve model inference latency, throughput, resource utilization, and cost.
  • Streaming You will help teams move models from experimentation into reliable production services, improve model execution performance, and build scalable data foundations for training, evaluation, and real-time decisions.
  • Experience with the model lifecycle, including training/serving integration, validation, deployment, observability, and rollback.
  • Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions.
  • Idempotency & backfills Connect offline and online feature pipelines, ensuring point-in-time correctness, consistent feature definitions, freshness, lineage, and reproducible backfills.
  • Warehousing Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Engineer, Machine Learning & Data Infrastructure
Staff Engineer, Machine Learning & Data Infrastructure

OKX • San Jose (CA)

Hybrid
USD 215,000 - 322,000
Performance bonus
Long-term incentives
Medical benefits
Sr. Software Engineer, Machine Learning & Data Infrastructure
Sr. Software Engineer, Machine Learning & Data Infrastructure

OKX • San Jose (CA)

On-site
USD 179,000 - 268,000
Staff Engineer: ML Data Infrastructure & Pipelines
Staff Engineer: ML Data Infrastructure & Pipelines

D8 LooP • San Jose (CA)

On-site
USD 180,000 - 270,000
Staff Engineer, Machine Learning & Data Infrastructure
Staff Engineer, Machine Learning & Data Infrastructure

Okx • San Jose (CA)

On-site
USD 215,000 - 322,000
Competitive compensation
L&D programs
Team building events
+2
Sr. Software Engineer, Machine Learning & Data Infrastructure
Sr. Software Engineer, Machine Learning & Data Infrastructure

Unchain Data • San Jose (CA)

On-site
USD 179,000 - 268,000
Competitive compensation package
Learning & Development programs
Team events & company gatherings
+2
Staff Engineer, Machine Learning & Data Infrastructure
Staff Engineer, Machine Learning & Data Infrastructure

Unchain Data • San Jose (CA)

On-site
USD 215,000 - 322,000
Competitive total compensation package
Wellness and meal allowances
Healthcare coverage
Staff Engineer, Machine Learning & Data Infrastructure OKX Est. $210,000 - $265,000 San Jose, California, United States Engineering Machine Learning Data San Jose, California, United States 17 hours ago
Staff Engineer, Machine Learning & Data Infrastructure OKX Est. $210,000 - $265,000 San Jose, California, United States Engineering Machine Learning Data San Jose, California, United States 17 hours ago

SportsTechJobs • San Jose (CA)

On-site
USD 215,000 - 322,000
Competitive total compensation
Wellness and meal allowances
Healthcare for employees and dependn/$
+1
Sr. Software Engineer, Machine Learning & Data Infrastructure OKX San Jose, California, United States $179–268K Engineering check more →
Sr. Software Engineer, Machine Learning & Data Infrastructure OKX San Jose, California, United States $179–268K Engineering check more →

Neobankbeat • San Jose (CA)

On-site
USD 179,000 - 268,000
Competitive compensation package
Wellness and meal allowances
Comprehensive healthcare for employees
Senior ML & Data Infrastructure Engineer
Senior ML & Data Infrastructure Engineer

OKX • San Jose (CA)

On-site
USD 179,000 - 268,000
Senior ML & Data Infrastructure Engineer
Senior ML & Data Infrastructure Engineer

SportsTechJobs • San Jose (CA)

On-site
USD 179,000 - 268,000
Competitive total compensation
Wellness and meal allowances
Healthcare schemes