Onsite Irvine Data Engineer: MLOps & Spark Pipelines

RSM Solutions, Inc; Irvine (CA); On-site; USD 120,000 - 150,000; Full time

Job summary

RSM Solutions, Inc is seeking a Data Engineer in Irvine, California. The role involves designing data pipelines using Apache Spark, Kubernetes, and integrating various data quality measures. Ideal candidates will have extensive experience in developing data solutions and automating processes. The position emphasizes collaboration with data scientists for enhanced operational efficiency.

This onsite role is suitable for US Citizens or Green Card Holders, requiring strong skills in T-SQL, PowerShell, GitHub, and Azure DevOps.

Qualifications

• Minimum 6 years experience developing data pipelines in Spark.
• 2 years deploying workloads on Kubernetes/Kubeflow.
• Experience with MLflow or similar experiment tracking.

Responsibilities

• Design and implement batch and streaming pipelines in Spark.
• Build high throughput ETL/ELT jobs with SSIS, SSAS, and T-SQL.
• Automate model retraining and deployment triggers within Kubeflow.

Skills

Data pipeline development in Spark

Kubernetes

MLflow or similar tools

T-SQL

Python/Scala for Spark

PowerShell/.NET scripting

GitHub and Azure DevOps

Prometheus and Grafana

Education

6+ years of experience in data engineering

Tools

Apache Spark

Kubernetes

SSIS and SSAS

Azure Monitor

Job description

RSM Solutions, Inc is seeking a Data Engineer in Irvine, California. The role involves designing data pipelines using Apache Spark, Kubernetes, and integrating various data quality measures. Ideal candidates will have extensive experience in developing data solutions and automating processes. The position emphasizes collaboration with data scientists for enhanced operational efficiency.

This onsite role is suitable for US Citizens or Green Card Holders, requiring strong skills in T-SQL, PowerShell, GitHub, and Azure DevOps.