AI/ML Data Engineer

Onyx-Conseil; Swindon; On-site; GBP 75,000 - 115,000; Full time

Job summary

LA International is seeking an experienced BI Engineer to design and implement data platforms, ETL pipelines and AI/ML production solutions for a government client. The role involves infrastructure management with Terraform, AWS data services, and close collaboration with data scientists and engineers.

The ideal candidate has 5+ years of experience, proven production AI/ML integration, and strong Python and SQL skills, with willingness to travel monthly to Swindon and work in a mostly remote

Qualifications

• Minimum 5 years of overall experience.
• Production AI/ML implementation experience required.
• Recent experience with AWS data services, Terraform, CI/CD, Python and SQL.
• Experience with ETL/data platform design and data warehouses is preferred.

Responsibilities

• Design and implement data pipelines, data warehouses and ETL processes.
• Manage data infrastructure and cloud resources using Terraform.
• Develop AI/ML models and productionize them with MLOps workflows.
• Collaborate with data scientists and engineers to translate requirements into technical solutions.
• Ensure data integrity, security and availability of databases.

Skills

Python

SQL

AWS data services

Terraform

CI/CD

Data pipelines

AI/ML

PostgreSQL/MySQL/Oracle

Tools

LangChain / LlamaIndex

Vector databases (Pinecone, Weaviate, FAISS)

Job description

Our government client is looking for candidates who have at least one production AI/ML implementation, recent experience with AWS data services, Terraform, CI/CD, Python, and SQL. Candidates should have more than five years of overall experience and meet the other requirements outlined in the attached JD. Inside IR35. Monthly travel to Swindon, mostly remote. Initial 6 months + extensions.

Job Purpose:

The BI Engineer will be responsible for the requirements analysis and design of solutions for data platform, ETL, integration and analysis solutions. Our toolset includes AWS hosted databases (postgres, S3 based data lake, MySQL, Athena), integration services (AWS API gateway, lambda functions), ETL services (AWS Glue, Step functions, AWS Batch), infrastructure a management tool such as Terraform and extensive use of SQLs and python.

Key Responsibilities and Accountabilities:

1. Infrastructure Management:
   • Set up and manage data infrastructure, including clusters, servers, and cloud-based resources using Terraform.
   • Monitor and optimize system performance, troubleshoot issues, and ensure system availability.
2. Data Architecture and Design:
   • Design and implement scalable and efficient data pipelines, databases, and data warehouses.
   • Collaborate with data visualization and analysts' teams to understand data requirements and translate them into technical specifications.
3. Data Processing:
   • Develop and maintain ETL (Extract, Transform, Load) processes for ingesting data from various sources into the data infrastructure.
   • Optimize data processing and storage for performance and cost-effectiveness.
4. AI:
   • Develop and deploy AI/ML models into production environments.
   • Build scalable inference pipelines and AI APIs.
   • Implement MLOps workflows for model versioning, monitoring, and retraining.
   • Collaborate with data scientists to productionize machine learning solutions.
   • Evaluate and optimize AI model performance and operational efficiency.
5. Database Management:
   • Manage and maintain databases, ensuring data integrity, security, and availability.
   • Implement database schema changes and optimizations as needed.
6. Collaboration:
   • Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to meet data requirements.
   • Communicate effectively with stakeholders to gather requirements and provide updates on data engineering projects.

Skills:

• Comfortable working in an Agile rapidly changing environment
• You will be an accomplished developer and BI Engineer, with excellent python skills.
• Experience with the infrastructure deployment tool such as terraform, CDK or cloud formation.
• Experience developing API based data integration.
• Excellent analytic skills associated with working on structured and unstructured datasets.
• Excellent SQL experience on various platforms (SQL, PostgresSQL, PL/SQL etc)
• Experience of several of MySQL, Oracle, SQL, Postgres, RDS, Aurora, Athena or other similar large scale database technologies.
• Experience working with AI framework such as LangChain / LlamaIndex
• Experience with Vector databases (Pinecone, Weaviate, FAISS)

This is an outline description of the key responsibilities and accountabilities involved in the job. This is not an exhaustive list and the post-holder might be expected to undertake any other duties across the wider directorate, commensurate with the Band and level of responsibility of this post, for which the post holder has the necessary experience and/or training.

LA International is an award-winning partner of choice for many of the world's most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance.

A multiple award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over consecutive years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience.

Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.