data engineer for Oil & Gas products

EPAM; EE. UU.; A distancia; USD 120.000 - 180.000; Jornada completa

Descripción de la vacante

EPAM ищет инженера данных для команды, которая строит единую платформу данных для портфеля нефтегазовых продуктов. Вы будете проектировать канонические и семантические модели, устанавливать контракты на интеграцию данных и обеспечивать единообразие подходов к данным.

Вы будете настраивать и администрировать кластеры Apache Kafka, заниматься безопасностью, мониторингом и observability. Опыт работы с несколькими источниками данных, SQL и Python обязателен, владение контрактами данных и

Formación

• Опыт проектирования канонических и семантических моделей данных.
• Знание SQL и Python для production data engineering.
• Опыт работы с Apache Kafka на продакшн-кластерах.
• Умение интегрировать данные из нескольких источников.
• Навыки документирования контрактов данных и стандартов.

Responsabilidades

• Проектировать и владеть каноническими и семантическими моделями данных.
• Настраивать кластеры Apache Kafka и обеспечивать их безопасность и мониторинг.
• Интегрировать данные из разных продуктов и создавать единый доступ к ним.
• Разрабатывать и внедрять данные для аналитики, AI/ML и observability.
• Документировать контракты данных, словари и спецификации интеграций.

Conocimientos

Apache Kafka

Data Modeling

SQL

Python

Data Integration

Data Observability

ETL Pipelines

Herramientas

Schema Registry

Kafka Admin Tools

Descripción del empleo

Описание:

The team is building a next-generation data platform spanning a wide portfolio of Oil & Gas products. The platform unifies data across products and supports cross-product reporting, AI readiness, and data observability.

Задачи:

Design and own canonical and semantic data models that create a coherent, product-agnostic view of Oil & Gas data Define contracts and standards for reliably and consistently integrating data from different products Integrate data across a broad portfolio of O&G applications with different schemas, cadences, and business contexts Identify patterns, resolve conflicts, and build connections that make cross-product reporting meaningful Architect, deploy, and administer Apache Kafka clusters for real-time and event-driven data flows Own topic and partition design, security configuration, monitoring, capacity planning, and performance tuning across environments Build data foundations for machine learning, LLM-based features, and advanced analytics Instrument data flows to understand how clients produce, consume, and rely on data Design and implement usage tracking, lineage, and observability tools that provide visibility into data health and adoption Collaborate with product managers, data scientists, and application teams across multiple products Translate complex data challenges into clear models and communicate architectural decisions to technical and non-technical stakeholders Establish and document data standards, data dictionaries, model documentation, and integration specifications.

Требования:

Expertise in Apache Kafka, including setting up clusters from scratch and administering them in production Experience with broker configuration, topic and partition design, replication, SASL/SSL security, ACLs, schema registry, monitoring and alerting, and performance tuning at scale At least 5 years of data engineering experience, focused on data modeling and integration across complex, multi-source environments Expertise in canonical and semantic data modeling, including designing models for multiple consumers Proven ability to integrate data across heterogeneous systems, schemas, frequencies, and business contexts into coherent, queryable assets Proficiency in SQL and Python for production data engineering Experience designing data for downstream reporting, analytics, and AI/ML use Strong written communication skills and ability to produce operational documentation and data contracts Ability to work autonomously and drive work forward in ambiguous, evolving environments Будет плюсом: Oil & Gas, energy, or industrial/operational data experience; familiarity with data observability and telemetry patterns, including lineage tracking, usage instrumentation, and data quality monitoring.

Условия:

Remote hiring locations: Armenia, Georgia, Kazakhstan, Kyrgyzstan, Uzbekistan