Senior Data/ML Engineer Python, Spark, AWS, Glue, SageMaker, Lambda, TensorFlow, PyTorch, SQL W[...]

Diverse CG Sp. z o.o. Sp.k.; Warszawa; Hybrid; PLN 180,000 - 240,000; Full time

Benefits offered by this job

Private medical care

Co-financing for the sports card

Dedicated consultant support

Job summary

DCG seeks an experienced ML Engineer to manage the lifecycle of ML models from development to deployment and monitoring. You will implement MLOps, work with Spark and Python for data ingestion, and build distributed pipelines for massive datasets.

The role involves collaboration with data scientists, DevOps, and IT to deliver high-quality ML solutions, plus engaging with analysts and stakeholders to shape analytics models. Strong English and AWS experience are essential.

Qualifications

• 5+ years of Python and Spark experience.
• Hands-on AWS ML/data processing with S3, Glue, SageMaker, Lambda, and Step Functions/Airflow/MWAA.
• Experience configuring AWS infrastructure with CLI, boto3 and IAM.
• Strong foundations in algorithms, data structures, statistics and linear algebra.
• Experience with TensorFlow or PyTorch for training and deploying ML models.
• Experience with Hadoop/Hive or other Big Data technologies.
• SQL proficiency (Spark/Hive SQL) and data flows.
• Version control with Bitbucket and Git; agile experience (SAFe).
• Excellent English communication skills.

Responsibilities

• Manage lifecycle of ML models from development to deployment and monitoring.
• Implement MLOps principles for CI/CD, testing, and monitoring.
• Work with Spark & Python for data ingestion and transformation (real-time and batch).
• Build distributed, parallel data processing pipelines for massive data sets.
• Use Spark to enrich data for searching, visualization, and analytics.
• Collaborate with data scientists, DevOps, and IT to deliver ML solutions.
• Engage with analysts and stakeholders to develop analytics models.
• Help optimize MLOps practices across wider use cases.

Skills

Python

Spark

AWS

SQL

Git

Machine Learning

Distributed systems

Statistics & linear algebra

English

Agile/SAFe

Tools

SageMaker

Airflow

MWAA

Hive

Spark SQL

Job description

As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:

Responsibilities:

• Managed the lifecycle of machine learning models from development to deployment and monitoring, ensuring optimal performance and reliability
• Implemented MLOps principles, including continuous integration, continuous delivery, testing, and monitoring, to streamline machine learning operations
• Working with Spark & Python to define and maintain data ingestions and transformation, handling both real-time and batch data processing
• Building distributed and highly parallelized big data processing pipeline which process massive amount of data (both structured and unstructured data) in near real-time
• Leverage Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics
• Collaborated with cross-functional teams, including data scientists, DevOps engineers, and IT, to deliver high-quality machine learning solutions
• Working closely with analysts and business stakeholders to develop analytics models
• Help in optimizing our current MLOps practices and libraries, make applicable for wider range of use cases outside propensity models
• Cloud solution exploration for AI/ML areas

Requirements:

• Proficiency in programming languages like Python & Spark - minimum 5 years
• Hands-on experience with AWS services for machine learning and data processing - particulary S3, Glue, SageMaker, Lambda and StepFunctions/Airflow/MWAA - covering data ingestion, model training, batch inference and automation pipelines
• Practical understanding of AWS Infrastracture setup and automation using AWS CLI, boto3 and IAM roles, ensuring reproducible and secure ML workflows
• Understanding of algorithms and data structures, knowledge of statistics and linear algebra
• Experience with machine learning frameworks (TensorFlow or PyTorch), including training new ML models, refining existing ones, and deploying these models into user-friendly applications
• Solid understanding of distributed systems with experience in Hadoop/Hive ecosystem and/or other Big Data Technologies
• Proficient in SQL (Spark/Hive SQL), previous experience in creating data flows
• Experience with BitBucket and GIT, code versioning and branching strategy
• Familiar with Agile/SAFe framework
• Very good command of English, both spoken and written

Nice to have:

• Experience in designing and implementing Machine Learning models (e.g., Propensity Models, Customer Lifetime Value, Micro-segmentation, and Channel Recommendation Models)
• Ability to support use case teams in model training and deployment

Offer:

• Private medical care
• Co-financing for the sports card
• Constant support of dedicated consultant

Personal Consulting Agency (License No. 4642)