Data engineer produktów cyfrowych

RITS; United States; Remote; USD 49,000 - 58,000; Full time

Job summary

RITS Group ищет инженера по данным для разработки и поддержки конвейеров данных в гибкой удалённой среде. Требуется опыт с PySpark, dbt, SQL, Python и AWS. Участие в проектах MPP-хранилищ и интеграции с CDP, аналитическими инструментами и ML-модулями.

Кандидат будет работать над модернизацией архитектуры, обеспечением качества данных и оптимизацией затрат в AWS-экосистеме, включая SageMaker и Redshift. Работайте удалённо с международной командой.

Qualifications

• Опыт не менее 4 лет в Data Engineering и создании продакшн-хранилищ данных.
• Высокие знания SQL и Python (включая PySpark).
• Практический опыт с Amazon Redshift или аналогом и оптимизацией производительности.

Responsibilities

• Разработка и поддержка data pipelines и устранение инцидентов.
• Проектирование и разработка процессов загрузки данных, интеграции с Aurora PostgreSQL и SQS.
• Создание и оптимизация трансформаций данных с использованием PySpark и dbt.
• Обеспечение качества данных, мониторинг свежести данных и надёжности пайплайнов.

Skills

SQL

Python

PySpark

Data pipelines

CI/CD

AWS

Data modeling

Tools

Aurora PostgreSQL

SQS

dbt

Redshift

Looker

SageMaker

CloudFormation / Terraform

Job description

Описание:

RITS Group is an IT company that offers the highest quality software development services for both the Polish and international markets. We are proud to be among the fastest-growing technology companies. Our priority is the quality and innovation of the solutions we deliver, and our organizational culture is based on trust, collaboration, and continuous improvement.

Задачи:

• Developing and maintaining data pipelines and responding to issues related to their operation.


• Designing and developing data ingestion processes, including integrations with Aurora PostgreSQL databases and SQS.


• Building and optimizing data transformations using PySpark and dbt.


• Developing mechanisms for data quality control, monitoring data freshness, and ensuring pipeline reliability.


• Managing Segment configuration, designing event schemas, and collaborating with Product and Engineering teams.


• Ensuring data consistency between Redshift and Braze, Amplitude, AppsFlyer, and Looker.


• Preparing and sharing data with external tools, partners, and internal teams.


• Developing existing ML models, preparing feature pipelines and training datasets, and deploying models using AWS SageMaker.


• Supporting AI initiatives, including LLM-based solutions, preparing evaluation datasets, and optimizing costs.


• Optimizing Amazon Redshift performance and costs.


• Developing infrastructure as code using AWS SAM, CloudFormation, or similar solutions, and CI/CD.


• Modernizing the existing architecture and gradually migrating legacy solutions to the current technology stack.


• Collaborating with Analytics, Finance, and Product teams to define and standardize business metrics.


• Developing the dbt project, including incremental models, tests, and data quality controls.


• Conducting code reviews and providing subject matter support to less experienced team members.

Требования:

At least 4 years of experience in Data Engineering, including experience independently developing and maintaining a production data warehouse. Very good knowledge of SQL and Python, including PySpark. Practical experience with Amazon Redshift or a similar MPP solution, particularly in performance optimization, distribution keys, sort keys, and workload management. Knowledge of data modeling, including dimensional modeling and Slowly Changing Dimensions. Experience with dbt: incremental models, tests, and sources. Practical knowledge of Segment or a similar Customer Data Platform (CDP), as well as event schema design. Experience in an AWS environment: S3, IAM, Glue, Lambda, Step Functions, Spark on EMR, and RDS/Aurora. Knowledge of Infrastructure as Code, e.g. CloudFormation, AWS SAM, or Terraform. Experience working with CI/CD. Ability to design idempotent pipelines and processes that enable data to be reprocessed (backfilling). Practical experience preparing data for ML/AI solutions, e.g. feature engineering, training datasets, or model serving. Comprehensive management of the entire ML model lifecycle is not required. Very good command of English – B2, C1, or C2 level. Independence in solving technical problems. Ability to work with production data systems and make architectural decisions.

Условия:

100% remote B2B 140–165 PLN net + VAT / hour