Senior Data Engineer - BigData Development

DXC Luxoft; Kraków; Presencial; PLN 180.000 - 240.000; Jornada completa

Descripción de la vacante

Luxoft is seeking an experienced Senior Data Engineer to join a project focused on observability and embedded analytics for an automotive client. You will build production data pipelines, transform logs and telemetry, and create analysis-ready datasets for troubleshooting and product-quality insights.

The role involves hands-on data modeling, API integration and data quality monitoring across embedded, cloud and test sources, with a focus on high-volume time-series data.

Formación

• Proven track record building production data pipelines.
• Strong Python and SQL hands-on experience.
• Experience with cloud-based data solutions.
• Experience with data orchestration and automated workflows.
• Experience with APIs, event data and logs.
• Familiarity with high-volume time-series or telemetry data.

Responsabilidades

• Design, build and maintain production-grade data pipelines.
• Develop data models and curated datasets for different engineering use cases.
• Process logs, events, telemetry and time-series data.
• Integrate data from embedded, cloud, test and application sources via APIs.
• Implement automated data-quality checks and pipeline monitoring.
• Build analysis-ready datasets supporting troubleshooting and reliability analyses.
• Improve pipeline reliability, performance and maintainability.
• Support dashboards and analytical use cases.
• Document delivered solutions and support knowledge transfer.

Conocimientos

Python

SQL

Data pipelines

Data modeling

Cloud data solutions

APIs & data interfaces

Data quality & monitoring

Time-series data

Herramientas

Airflow

Descripción del empleo

Project description

We are looking for an experienced Senior Data Engineer to join a project within observability and embedded analytics for a leading automotive Client.The project focuses on transforming logs, events, measurements and telemetry from embedded, cloud, test and application systems into trusted and reusable datasets and engineering insights.You will have a hands-on role in building production data pipelines, transformations and analysis-ready datasets that support faster troubleshooting, reliability analysis, product-quality follow-up and improved understanding of platform performance in real-world conditions.

Responsibilities

• Design, build and maintain production-grade data pipelines and transformations.
• Develop data models, schemas and curated datasets for different engineering use cases.
• Process logs, events, telemetry and high-volume time-series data.
• Integrate data from embedded, cloud, test and application sources using APIs and other data interfaces.
• Implement automated data-quality checks and pipeline monitoring.
• Build analysis-ready datasets supporting troubleshooting, reliability and product-quality analysis.
• Improve pipeline reliability, performance and maintainability.
• Support dashboards and analytical use cases.
• Document delivered solutions and support knowledge transfer.

SKILLS

Must have

• Strong experience developing and operating production-grade data pipelines.
• Strong hands-on Python and SQL skills.
• Strong knowledge of data transformation and data modelling.
• Experience with cloud-based data solutions.
• Experience with data orchestration and automated workflows.
• Experience working with APIs, event data and logs.
• Experience with data quality and pipeline monitoring.
• Experience working with high-volume time-series and/or telemetry data.

Nice to have

• Automotive or connected-car experience.
• Experience working with data from embedded systems or devices.
• Experience supporting troubleshooting, reliability or product-quality analytics.
• Experience with data visualisation and dashboards.
• Mobile analytics experience.
• Knowledge of data lineage and data governance.