Investment Data Engineer: Scalable Pipelines & Lakehouse

Luxoft; Polska; On-site; PLN 180,000 - 260,000; Full time

Job summary

DXC Luxoft in Poland seeks an experienced Data Engineer to design, build, and operate data pipelines powering analytics for investment teams. You will own end-to-end data from ingestion to lakehouse modeling, ensuring discoverability, reliability and cost-efficiency.

Collaborate with BAs/PMs and quants, implement PySpark ETL pipelines, ensure testing and production readiness, and contribute to governance, CI/CD, and multi-source data mastering.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.
• 5+ years of data engineering experience.
• Strong hands-on Databricks production experience.
• Strong PySpark and SQL skills.
• Experience with Declarative Pipelines / pipeline orchestration on Databricks.
• Fundamentals of ingestion, transformation, incremental processing, testing, and performance tuning.
• Experience with production-ready datasets, monitoring, and reliability patterns.
• Exposure to data governance, cataloguing, glossary and lineage.
• Experience with CI/CD pipelines and enterprise release processes.
• Familiarity with vendor market data feeds or multi-source mastering patterns.

Responsibilities

• Participate in requirements clarification and sprint planning.
• Design and implement ETL pipelines and robust data solutions in PySpark.
• Optimize ETL processes for performance, scalability, and reliability.
• Write unit and integration tests; support QA in acceptance.
• Resolve PROD incidents as a 3rd line engineer.

Skills

PySpark

SQL

Databricks

Data modeling

Data governance

CI/CD

Testing

Education

Bachelor's degree (CS/Engineering/IS)

Tools

Databricks

Palantir Foundry

Dynatrace / Datadog

Job description

DXC Luxoft in Poland seeks an experienced Data Engineer to design, build, and operate data pipelines powering analytics for investment teams. You will own end-to-end data from ingestion to lakehouse modeling, ensuring discoverability, reliability and cost-efficiency.

Collaborate with BAs/PMs and quants, implement PySpark ETL pipelines, ensure testing and production readiness, and contribute to governance, CI/CD, and multi-source data mastering.