Senior Machine Learning Engineer, Accelerated Apache Spark

NVIDIA Corporation; Santa Clara (CA); On-site; USD 224,000 - 357,000; Full time

Job summary

NVIDIA Corporation is seeking a Machine Learning Engineer to join the GPU-accelerated Apache Spark team. You will design ML solutions for performance prediction and optimization of Spark workloads on GPUs, enabling enterprises to scale ML/DL pipelines.

You will collaborate with open-source communities and customers, mentor engineers, and stay current with ML methods including GenAI, reinforcement learning, and online learning.

Qualifications

• BS/MS/PhD or equivalent in ML/DS/CS or related field.
• 12+ years designing, implementing, and productionizing ML/DL solutions.
• 3+ years as technical lead in ML model development.
• 2+ years with large-scale data processing platforms (e.g., Spark).
• Proficient in Python and data science libs (numpy, pandas, scikit-learn, scipy, pytorch, tensorflow).
• Experience with GenAI, RL, and online ML systems.

Responsibilities

• Design ML solutions for performance prediction and optimization of GPU-accelerated Spark workloads.
• Develop advanced algorithms and adaptive systems to improve Spark performance on GPUs.
• Develop AI-based agents and tools to fix system issues and optimize applications.
• Collaborate with partners and customers on deploying ML solutions in various environments.
• Maintain deep domain expertise by following latest ML advances.
• Provide mentorship and leadership in data science and ML to engineers.

Skills

Machine Learning

Deep Learning

Python

Numpy

Pandas

Scikit-learn

Scipy

PyTorch

TensorFlow

GenAI

Reinforcement Learning

Feature engineering

XGBoost

Spark

Spark (Apache)

Education

Bachelor's/Master's/PhD or equivalent

Tools

Apache Spark

Job description

NVIDIA is looking for a Machine Learning Engineer to join the GPU accelerated Apache Spark team. Apache Spark is the most popular data processing engine in data centers for running massive scale workloads for ETL, SQL, and ML/DL model training and inference pipelines, spanning many domains and use cases. NVIDIA GPUs offer a promising avenue for significantly speeding up and/or lowering the cost of running very large Apache Spark applications.

You will work with the open source community to accelerate Apache Spark with GPUs. You will apply the latest ML/AI methods to empower enterprises to migrate Spark workloads onto GPUs at scale.

What you'll be doing:

• Design and implement machine learning solutions for performance prediction and optimization of GPU accelerated enterprise Apache Spark workloads.
• Develop advanced algorithms and adaptive systems to continuously improve the performance of Apache Spark workloads on GPUs.
• Develop AI-based agents and tools to assist with fixing system issues and application optimization.
• Collaborate with key partners and customers on the deployment of complex machine learning solutions in various environments.
• Maintain deep domain expertise by knowing the latest published advances in ML systems and algorithms.
• Provide technical mentorship and leadership in data science and machine learning to a team of engineers.

What we need to see:

• BS, MS, or PhD or equivalent experience in Machine Learning, Data Science, Computer Science or a closely related field.
• 12+ years of professional experience in designing, implementing, and productionizing high-quality ML/DL solutions.
• 3+ experience as technical lead in ML model development.
• Proven hands-on experience (2+ years) with large-scale data processing platforms, such as Apache Spark.
• Proven ability to employ modern tooling and sound techniques for all aspects of crafting, deploying, and maintaining machine learning models.
• Excellent programming skills in Python and Python data science related libraries like numpy, pandas, scikit-learn, scipy, pytorch, and tensorflow.
• Deep experience with sophisticated ML methodologies, including LLM/GenAI, reinforcement learning, and adaptive, on-line ML systems.
• Strong expertise in feature engineering, feature importance assessment, and developing boosted tree model solutions (e.g., XGBoost).

Ways to stand out from the crowd:

• Understanding of the internal workings and architecture related to Apache Spark.
• Familiarity with NVIDIA GPUs and CUDA.
• Experience coding in Scala, Java, and/or C++.

NVIDIA is widely considered to be one of the technology world's most desirable employers. We have some of the most experienced and dedicated people in the world working for us. If you are passionate about what you do, creative and autonomous, we want to hear from you!

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD. You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until September 5, 2026. This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

NVIDIA pioneered accelerated computing. Today, our AI infrastructure powers global intelligence, transforming every industry. Learn more about NVIDIA.