Senior AI/ML Platform Engineer - Scalable Data Pipelines

Vi; United States; Hybrid; USD 130,000 - 190,000; Full time

Job summary

Vi is seeking an engineer with deep expertise in building and scaling big-data and ML pipelines. The role will be a hybrid between an integration engineer owning customer data integrations with Vi’s data lakehouse and a platform engineer packaging Vi’s products into repeatable, scalable, automated pipelines.

You will work with Apache Spark, PySpark, Python and AWS technologies to design and implement data and ML workflows and to prototype predictive insight products.

Qualifications

• Experience building large-scale data analytics pipelines with Spark.
• Strong Python and AWS data/ML tooling experience.
• Excellent communication across customer accounts and technical counterparts.

Responsibilities

• Own the design and implementation of data and ML pipelines based on first-party customer data and Vi’s data lakehouse.
• Prototype predictive insight products and run rapid feasibility studies for new commercial opportunities.
• Identify commonalities between customer engagements to synthesize reusable data capabilities.

Skills

Big-data pipelines

Python & AWS for data/ML

Communication across accounts

Tools

Apache Spark

Apache Iceberg

AWS EMR

Glue

S3

Python

PyTorch

sklearn

xgboost

catboost

mlflow

SageMaker

Airflow

Job description

Vi is seeking an engineer with deep expertise in building and scaling big-data and ML pipelines. The role will be a hybrid between an integration engineer owning customer data integrations with Vi’s data lakehouse and a platform engineer packaging Vi’s products into repeatable, scalable, automated pipelines.

You will work with Apache Spark, PySpark, Python and AWS technologies to design and implement data and ML workflows and to prototype predictive insight products.