ML Architect

Sygna Solutions; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Sygna Solutions is seeking a senior ML Platform Engineer to define the target architecture for machine learning on AWS and Databricks, delivering scalable, secure, and production-ready solutions. You will drive end-to-end ML workflows, MLOps practices, and governance across teams to productionize ML use cases.

You will advise on Databricks features, integrate with AWS services, and mentor teams while optimizing workloads for performance and cost.

Qualifications

• Extensive experience designing and delivering enterprise ML platforms.
• Strong hands-on Databricks production experience.
• Deep knowledge of the Databricks Lakehouse architecture and its machine learning capabilities.
• Strong experience with MLflow, including experiment tracking, model lifecycle management, deployment, and governance.
• Extensive AWS architecture and implementation experience.
• Strong understanding of MLOps, CI/CD, infrastructure as code, automated testing, observability, and model monitoring.
• Experience deploying both batch and real-time inference workloads.
• Proficiency in Python and SQL, with experience using Apache Spark/PySpark.
• Experience with feature engineering, model evaluation, drift detection, explainability, retraining, and responsible AI practices.
• Strong knowledge of data security, IAM, encryption, networking, governance, and regulatory considerations in cloud environments.

Responsibilities

• Define the target architecture and roadmap for machine learning on AWS and Databricks.
• Design and implement scalable, secure, and production-ready ML solutions.
• Assess the current Databricks implementation and recommend improvements, integrations, and platform capabilities.
• Establish end-to-end ML workflows covering data preparation, feature engineering, experimentation, model training, validation, deployment, monitoring, and retraining.
• Implement MLOps practices using Databricks and MLflow, including experiment tracking, model management, automated deployment, governance, and monitoring.
• Design batch and real-time model-inference solutions based on business and technical requirements.
• Advise teams on the effective use of Databricks capabilities, including Unity Catalog, Feature Engineering, Model Serving, Lakehouse Monitoring, Delta Lake, Delta Live Tables/Lakeflow, and Databricks Workflows.
• Integrate Databricks with relevant AWS services, such as Amazon S3, IAM, KMS, SageMaker, Lambda, ECR, CloudWatch, and streaming services.
• Define reusable architecture patterns, development standards, and platform guardrails.
• Establish appropriate security, access-control, data-governance, model-governance, and compliance practices.
• Optimize workloads for scalability, reliability, performance, and cost.
• Partner with data scientists, data engineers, platform teams, security teams, and business stakeholders to move ML use cases from concept into production.
• Provide technical leadership, mentoring, documentation, and knowledge transfer to internal teams.

Skills

Enterprise ML platforms

Databricks production

MLOps

Python and SQL

CI/CD for ML

AWS cloud architecture

Tools

Databricks

MLflow

Apache Spark

PySpark

SageMaker

Lambda

ECR

CloudWatch

Delta Lake

Delta Live Tables

Job description

Key Responsibilities:

• Define the target architecture and roadmap for machine learning on AWS and Databricks.


• Design and implement scalable, secure, and production-ready ML solutions.


• Assess the current Databricks implementation and recommend improvements, integrations, and platform capabilities.


• Establish end-to-end ML workflows covering data preparation, feature engineering, experimentation, model training, validation, deployment, monitoring, and retraining.


• Implement MLOps practices using Databricks and MLflow, including experiment tracking, model management, automated deployment, governance, and monitoring.


• Design batch and real-time model-inference solutions based on business and technical requirements.


• Advise teams on the effective use of Databricks capabilities, including Unity Catalog, Feature Engineering, Model Serving, Lakehouse Monitoring, Delta Lake, Delta Live Tables/Lakeflow, and Databricks Workflows.


• Integrate Databricks with relevant AWS services, such as Amazon S3, IAM, KMS, SageMaker, Lambda, ECR, CloudWatch, and streaming services.


• Define reusable architecture patterns, development standards, and platform guardrails.


• Establish appropriate security, access-control, data-governance, model-governance, and compliance practices.


• Optimize workloads for scalability, reliability, performance, and cost.


• Partner with data scientists, data engineers, platform teams, security teams, and business stakeholders to move ML use cases from concept into production.


• Provide technical leadership, mentoring, documentation, and knowledge transfer to internal teams.

Required Qualifications:

• Extensive experience designing and delivering enterprise machine learning platforms and solutions.


• Strong hands-on experience with Databricks in a production environment.


• Deep knowledge of the Databricks Lakehouse architecture and its machine learning capabilities.


• Strong experience with MLflow, including experiment tracking, model lifecycle management, deployment, and governance.


• Extensive AWS architecture and implementation experience.


• Strong understanding of MLOps, CI/CD, infrastructure as code, automated testing, observability, and model monitoring.


• Experience deploying both batch and real-time inference workloads.


• Proficiency in Python and SQL, with experience using Apache Spark/PySpark.


• Experience with feature engineering, model evaluation, drift detection, explainability, retraining, and responsible AI practices.


• Strong knowledge of data security, IAM, encryption, networking, governance, and regulatory considerations in cloud environments.


• Ability to translate business requirements into practical architecture and implementation plans.


• Excellent communication skills and the ability to influence technical and non-technical stakeholders.