Senior Data Engineer I

LexisNexis South Africa Shared Services (Pty) Ltd Company; Durban; On-site; ZAR 60 000 - 100 000; Full time

Job summary

An innovative firm is seeking a Senior Machine Learning Operations Engineer to join their dynamic team. This role involves developing cutting-edge solutions for high-value customer challenges, leveraging advanced techniques in AI and data analysis. You will collaborate with data scientists and engineers to build scalable data ingestion and ML inference pipelines, ensuring optimal performance and cost efficiency. The position offers a unique blend of start-up culture and the resources of an established company, promoting a healthy work-life balance with flexible working options. If you are passionate about technology and eager to make a significant impact, this opportunity is perfect for you.

Qualifications

• 5+ years of experience in ML Ops and data-driven solution development.
• Strong expertise in Scala and Python for scalable data ingestion.

Responsibilities

• Develop and implement ML Ops strategies for continuous improvement.
• Collaborate with cross-functional teams to build scalable pipelines.

Skills

Scala

Python

Machine Learning Operations

Data Engineering

Cloud Platforms (AWS, GCP, Azure)

CI/CD Pipelines

Data Security and Privacy

Education

Masters degree in Software Engineering

Masters degree in Data Engineering

Masters degree in Computer Science

Tools

Apache Spark

Ray

ML Flow

DVC

Grafana

DataHub

Databricks

Docker

Kubernetes

Jenkins

GitHub Actions

Job description

At LexisNexis, we develop the legal profession’s most innovative products for data analysis, visualization, and research. We utilize the latest techniques in AI, machine learning, and data visualization to uncover insights about judges’ rulings, build forecasts of likely outcomes, and reveal critical connections in large datasets spanning law, news, and finance.

Responsibilities

As a senior machine learning operations engineer, you will work on new product development within a small team environment, writing production code in both runtime and build-time environments. Your role involves proposing and building data-driven solutions for high-value customer problems by discovering, extracting, and modeling knowledge from large-scale natural language datasets. You will prototype new ideas and collaborate with data scientists, product designers, data engineers, front-end developers, and legal data annotators. This role offers experience in a start-up culture combined with the resources of an established company.

Your responsibilities include:

1. Developing and implementing strategies for continuous improvement of ML Ops, including versioning, testing, automation, reproducibility, deployment, monitoring, and data privacy.
2. Reporting on ML Ops metrics such as deployment frequency, lead time for changes, mean time to restore, and change failure rate.
3. Collaborating with data scientists, data engineers, API engineers, and the dev ops team.
4. Building scalable data ingestion and machine learning inference pipelines.
5. Scaling production systems to support increased demand from new products, features, and users.
6. Providing visibility into data platform health, including data flow, resource usage, and data lineage, while optimizing cloud costs.
7. Automating lifecycle management of data processing systems and platforms.

Requirements

• Masters degree in Software Engineering, Data Engineering, Computer Science, or a related field.
• At least 5 years of relevant work experience.
• Strong background in Scala and Python.
• Experience with Apache Spark and/or Ray.
• Knowledge of cloud platforms such as AWS, GCP, or Azure.
• Understanding of current ML Ops principles and frameworks.
• Experience with ML Ops tools like ML Flow, DVC, Grafana, DataHub, Databricks.
• Experience with machine learning technologies such as PyTorch, TensorFlow, AWS Sagemaker.
• Experience with CI/CD pipelines, including Jenkins or GitHub Actions.
• Proficiency with Docker containerization and Kubernetes orchestration.
• Experience in enhancing data security and privacy, and managing cloud costs.
• Knowledge of API development and machine learning deployment.

Work in a way that works for you

We promote a healthy work/life balance with flexible and remote working options.

Working with Us

LexisNexis Legal & Professional is proud to be an equal-opportunity employer committed to diversity and inclusion.

Working for you

We value your well-being and happiness, believing they are key to a successful career.