Senior Data/ML Engineer - Modernize a Live Forecasting Platform (Remote)

United States Digital Space LLC; United States; Remote; USD 150,000 - 190,000; Full time

Job summary

Senior Data/ML Engineer sought to refactor and operationalize a live Databricks-based forecasting platform. You will reduce technical debt, improve tests, and raise engineering standards for a system the business already relies on. The role emphasizes moving logic into reusable code and strengthening monitoring and reliability.

Flexible location: Toronto or St. Louis preferred, but remote consultants are welcome globally with North American working-hours overlap.

Qualifications

• Experience with a Databricks-based forecasting platform.
• Proficient Python engineering for modular, reusable code.
• Strong PySpark production data pipelines experience.
• Experience improving testing, monitoring and observability.

Responsibilities

• Review current forecasting platform architecture and identify improvements.
• Refactor Databricks, Python, and PySpark implementations.
• Move business logic from notebooks into reusable Python modules.
• Improve separation of orchestration and core business logic.
• Establish engineering standards and define 'good' for the platform.
• Improve automated testing and validation of forecasting workflows.
• Build monitoring/observability to increase visibility of predictions.
• Monitor model behavior and operational health over time.
• Improve reliability of scheduled training workflows and retries.
• Maintain and extend forecasting capabilities as needed.

Skills

Databricks

Python

PySpark

Code refactor

CI/CD

Observability

Testing

Tools

CI/CD pipelines

Job description

Senior Data/ML Engineer sought to refactor and operationalize a live Databricks-based forecasting platform. You will reduce technical debt, improve tests, and raise engineering standards for a system the business already relies on. The role emphasizes moving logic into reusable code and strengthening monitoring and reliability.

Flexible location: Toronto or St. Louis preferred, but remote consultants are welcome globally with North American working-hours overlap.