Data engineer in machine learning

KCS iT; United States; Remote; USD 90,000 - 150,000; Full time

Benefits offered by this job

Free training programs

International projects in Benelux

Well-being programs (remote)

Job summary

KCS IT ищет Data Engineer с опытом ML для разработки и внедрения конвейеров данных в облаке Azure. Требуется глубокое владение Python, SQL и PySpark, а также опыт работы с Azure Databricks и хранилищами данных Azure.

Кандидат должен уметь работать над полным циклом разработки данных, включая подготовку данных, обучение моделей и пакетный скоринг. Вакансия предполагает работу с международными проектами и удаленными командами.

Qualifications

• 2–3 года опыта Data Engineering, участие в проектах с традиционным ML
• Узкие знания Azure Databricks
• Опыт Python, SQL и PySpark
• Опыт работы с Azure Data Lake Storage и форматами Parquet/Delta Lake
• Опыт полного цикла разработки данных: ingestion, transformation, modelling, delivery
• Практический опыт ML-библиотек
• Знание data preparation, feature engineering, model training/validation/evaluation
• Опыт интеграции моделей в конвейеры данных и пакетный скоринг
• Опыт работы с Git и процессами разработки
• Способность разрабатывать масштабируемые решения, фокус на производительности и затратах
• Свободное владение португальским и английским языками
• Дополнительный плюс: Azure ML, Azure Data Factory, мониторинг моделей, data drift, retraining
• Опыт публикации моделей через API/endpointы
• Знание Unity Catalog, управления данными и контроля доступа
• Опыт работы в Agile/Scrum, Jira, Confluence

Skills

Python

SQL

PySpark

Azure Databricks

ML-библиотеки

Tools

Azure Data Lake Storage

Parquet

Delta Lake

Git

Jira

Confluence

Azure Machine Learning

Job description

Описание: We’re looking for the special, unique and amazing YOU! @ KCS IT, we look for the ones that stands out, for those that always wants to be better and fight for it, and for those who has the same values that we do: dedication, energy, integrity, transparency, flexibility, trust, honesty, hard work, proactivity, team work. We are looking for an Amazing: Data engineer with Machine learning

Требования:

• 2–3 years of experience in Data Engineering, including involvement in projects involving traditional Machine Learning.
• Strong experience with Azure Databricks.
• Good knowledge of Python, SQL and PySpark.
• Experience with Azure Data Lake Storage and formats such as Parquet and Delta Lake.
• Experience across the full data solution development lifecycle: ingestion, transformation, modelling and data delivery.
• Hands‑on experience with Machine Learning libraries.
• Knowledge of data preparation, feature engineering, model training, validation and evaluation.
• Understanding of concepts such as overfitting, data leakage, train/test split and cross-validation.
• Experience integrating models into data processing pipelines and batch scoring workflows.
• Experience with Git and software development best practices.
• Ability to develop scalable solutions, with a focus on performance and execution costs.
• Fluent in Portuguese and English.
• Будет плюсом: Nice‑to‑have skills experience with Azure Machine Learning, including training jobs/pipelines, model registration and inference endpoints
• Experience with Azure Data Factory
• Knowledge of model monitoring, data drift detection and model retraining processes
• Experience deploying models through APIs or online endpoints
• Knowledge of Unity Catalog, data governance and access control
• Experience working with Agile methodologies, particularly Scrum, and using Jira and Confluence

Условия:

• You can develop a career that fits you: your career development is personalized, taking in consideration your needs and goals from a short to long term
• Interesting Challenges Ahead: you can work for several clients from different sectors of activity
• Free training programs: Our training and certification programs in languages, tech, behavior and business will help you to reach your full potential faster
• International projects in Benelux: you can gain international experience in Benelux and balance a new way of living with work
• Take care of your well-being: Enjoy our free nutrition, psychology, general medicine appointments and our yoga and personal training days… all remote