Senior ML Data Engineer: Real-Time ML Pipelines & DataOps

Cedent; United States; Hybrid; USD 55,104 - 82,656; Full time

Benefits offered by this job

Health

Dental

Vision Insurance

Job summary

A leading data engineering company in the United States seeks a Data Engineer to develop and manage ML feature engineering pipelines using Databricks and Apache Spark. The role involves overseeing data integration and optimizing pipelines for both real-time and batch model serving. Candidates should have 7 years of data engineering experience along with proficiency in Python and SQL. The position offers competitive hourly compensation and various health benefits.

Qualifications

• 7 years in data engineering, with 4 years in ML feature engineering.
• Experience managing pipelines on Databricks using Apache Spark.
• Familiarity with ML lifecycle management and MLflow is a plus.

Responsibilities

• Develop and maintain feature engineering pipelines using Databricks.
• Integrate diverse data sources to create user behavior profiles.
• Design and implement ETL, ELT pipelines for medallion architecture.

Skills

Apache Spark

Databricks

Java

Python

Scala

SparkSQL

Job description

A leading data engineering company in the United States seeks a Data Engineer to develop and manage ML feature engineering pipelines using Databricks and Apache Spark. The role involves overseeing data integration and optimizing pipelines for both real-time and batch model serving. Candidates should have 7 years of data engineering experience along with proficiency in Python and SQL. The position offers competitive hourly compensation and various health benefits.