MLOPS Architect

TechDigital Group; St. Louis (MO); On-site; USD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking a skilled AWS Cloud Architect to design and implement innovative cloud architectures. This role focuses on developing efficient data pipelines and MLOps practices, ensuring seamless scalability and resource optimization. You will collaborate with data scientists and engineers to deliver high-quality data products while adhering to security and compliance standards. If you are passionate about cloud technology and eager to make an impact in AI-driven projects, this opportunity is perfect for you. Join a dynamic team and contribute to cutting-edge solutions in a rapidly evolving field.

Qualifications

• Strong experience in Python and cloud computing, especially AWS.
• Experience with MLOps, CI/CD pipelines, and data management.

Responsibilities

• Design cloud architectures for MLOps implementations, ensuring scalability and efficiency.
• Develop data pipelines for machine learning models and collaborate with cross-functional teams.

Skills

AWS

Python

Airflow

Kedro

Luigi

Hadoop

Spark

Graph Databases

Tools

AWS CloudFormation

Terraform

Job description

Mandatory required skills - AWS, Python, Airflow, Kedro, or Luigi
Preferred/Desired skills - Hadoop, Spark, or similar frameworks. Experience with graph databases a plus.

1. Designing Cloud Architecture:
   • As an AWS Cloud Architect, you'll be responsible for designing cloud architectures, preferably on AWS, Azure, or multi-cloud environments.
   • Your architecture design should enable seamless scalability, flexibility, and efficient resource utilization for MLOps implementations.
2. Data Pipeline Design:
   • Develop data taxonomy and data pipeline designs to ensure efficient data management, processing, and utilization across the AI/Client platform.
   • These pipelines are critical for ingesting, transforming, and serving data to machine learning models.
3. MLOps Implementation:
   • Collaborate with data scientists, engineers, and DevOps teams to implement MLOps best practices.
   • This involves setting up continuous integration and continuous deployment (CI/CD) pipelines for model training, deployment, and monitoring.
4. Infrastructure as Code (IaC):
   • Use tools like AWS CloudFormation or Terraform to define and provision infrastructure resources.
   • Infrastructure as Code allows you to manage your cloud resources programmatically, ensuring consistency and reproducibility.
5. Security and Compliance:
   • Ensure that the MLOps architecture adheres to security best practices and compliance requirements.
   • Implement access controls, encryption, and monitoring to protect sensitive data and models.
6. Performance Optimization:
   • Optimize cloud resources for cost-effectiveness and performance.
   • Consider factors like auto-scaling, load balancing, and efficient use of compute resources.
7. Monitoring and Troubleshooting:
   • Set up monitoring and alerting for the MLOps infrastructure.
   • Be prepared to troubleshoot issues related to infrastructure, data pipelines, and model deployments.
8. Collaboration and Communication:
   • Work closely with cross-functional teams, including data scientists, software engineers, and business stakeholders.
   • Effective communication is essential to align technical decisions with business goals.

Activities –
• Strong experience in Python
• Experience in data product development, analytical models, and model governance
• Experience with AI workflow management tools such as Airflow, Kedro, or Luigi
• Exposure to statistical modeling, machine learning algorithms, and predictive analytics
• Highly structured and organized work planning skills
• Strong understanding of the AI development lifecycle and Agile practices
• Proficiency in big data technologies like Hadoop, Spark, or similar frameworks. Experience with graph databases a plus.
• Extensive experience in working with cloud computing platforms - AWS
• Proven track record of delivering data products in environments with strict adherence to security and model governance standards.