Staff Engineer, Machine Learning & Data Infrastructure
San Jose, California, United States Staff Data Platform posted 1 d ago
At OKX, we believe that the future will be reshaped by crypto, and ultimately contribute to every individual's freedom. OKX is a leading crypto exchange, and the developer of OKX Wallet, giving millions access to crypto trading and decentralized crypto applications (dApps). OKX is also a trusted brand by hundreds of large institutions seeking access to crypto markets.
Skills, with evidence
Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost. must have
- Incremental loads Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost. must have
- Python Strong Python, SQL, and at least one backend or systems language such as Java, Scala, Go, or C++. must have
- SQL Strong Python, SQL, and at least one backend or systems language such as Java, Scala, Go, or C++. must have
- Spark Deep understanding of Databricks and Spark, or comparable offline data platforms, including execution plans, distributed query optimization, storage layouts, data correctness, and workload cost management. must have
- AWS Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions. must have · not practised here
- Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions. must have · not practised here
- GCP Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions. must have · not practised here
- Cost & performance Improve model inference latency, throughput, resource utilization, and cost.
- Streaming You will help teams move models from experimentation into reliable production services, improve model execution performance, and build scalable data foundations for training, evaluation, and real-time decisions.
- Experience with the model lifecycle, including training/serving integration, validation, deployment, observability, and rollback.
- Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions.
- Idempotency & backfills Connect offline and online feature pipelines, ensuring point-in-time correctness, consistent feature definitions, freshness, lineage, and reproducible backfills.
- Warehousing Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost.