Senior Data Engineer (Investment Data)

Luxoft; Polska; On-site; PLN 180,000 - 260,000; Full time

Job summary

DXC Luxoft in Poland seeks an experienced Data Engineer to design, build, and operate data pipelines powering analytics for investment teams. You will own end-to-end data from ingestion to lakehouse modeling, ensuring discoverability, reliability and cost-efficiency.

Collaborate with BAs/PMs and quants, implement PySpark ETL pipelines, ensure testing and production readiness, and contribute to governance, CI/CD, and multi-source data mastering.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.
• 5+ years of data engineering experience.
• Strong hands-on Databricks production experience.
• Strong PySpark and SQL skills.
• Experience with Declarative Pipelines / pipeline orchestration on Databricks.
• Fundamentals of ingestion, transformation, incremental processing, testing, and performance tuning.
• Experience with production-ready datasets, monitoring, and reliability patterns.
• Exposure to data governance, cataloguing, glossary and lineage.
• Experience with CI/CD pipelines and enterprise release processes.
• Familiarity with vendor market data feeds or multi-source mastering patterns.

Responsibilities

• Participate in requirements clarification and sprint planning.
• Design and implement ETL pipelines and robust data solutions in PySpark.
• Optimize ETL processes for performance, scalability, and reliability.
• Write unit and integration tests; support QA in acceptance.
• Resolve PROD incidents as a 3rd line engineer.

Skills

PySpark

SQL

Databricks

Data modeling

Data governance

CI/CD

Testing

Education

Bachelor's degree (CS/Engineering/IS)

Tools

Databricks

Palantir Foundry

Dynatrace / Datadog

Job description

DXC Luxoft is a global IT services and consulting company, part of DXC Technology, delivering cutting-edge digital and software solutions worldwide. We partner with leading financial institutions and global enterprises, helping them transform and scale through modern technology.

Join a global community of engineers and be part of building impactful, modern technology solutions!

Project Description:

We're looking for an experienced, hands-on Data Engineer who is capable of designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back office. The ideal candidate has financial markets familiarity (securities, prices, corporate actions, positions/holdings) and thrives in ambiguous environments - proactively shaping solutions, not waiting for tickets. You'll own data end-to-end: from ingesting vendor and internal sources, to modeling in our lakehouse, to making data discoverable, reliable, and cost-efficient. You'll partner closely with BAs/PMs and quants, anticipate downstream needs, and propose pragmatic architectures that balance speed, governance, and scalability.

Responsibilities:

• Participate in requirements clarification and sprint planning sessions.
• Design technical solutions and implement them, inc ETL Pipelines - Build robust data pipelines in PySpark to extract, transform, using PySpark
• Optimize ETL Processes - Enhance and tune existing ETL processes for better performance, scalability, and reliability
• Writing unit and integration tests.
• Support QA teammates in the acceptance process.
• Resolving PROD incidents as a 3rd line engineer.

Mandatory Skills Description:

• Bachelor's degree (Computer Science, Engineering, Information Systems, or related discipline).
• 5+ years experience in data engineering roles (flexible based on depth of capability).
• Strong hands-on experience with Databricks in production environments (prerequisite).
• Strong programming experience with PySpark (must) and strong SQL (must).
• Proven experience with Declarative Pipelines / pipeline orchestration on Databricks (prerequisite).
• Strong understanding of data engineering fundamentals: ingestion patterns, transformation design, incremental processing, testing, performance tuning.
• Experience delivering production-ready datasets with appropriate operational controls (monitoring, troubleshooting, reliability patterns).
• Experience with modern Lakehouse concepts (Delta tables, optimization strategies, file skipping, metadata/statistics awareness).
• Exposure to data governance practices: cataloguing, documentation, business glossary/terms, lineage.
• Experience working in enterprise environments with CI/CD pipelines and structured release processes.
• Familiarity with vendor market data feeds (e.g., Bloomberg, Refinitiv, MSCI, FactSet) or similar multi-source mastering patterns.

Nice-to-Have Skills Description:

• Strong Hands-on Expertise in Palantir Foundry. Proven experience with Foundry pipelines, ontologies, data lineage, transformations, and platform governance.
• Proven Migration Experience from Palantir / to Databricks. Demonstrated experience leading or executing platform migrations, including pipeline conversion, data model redesign, and production cutover.
• Familiarity with Dynatrace or Datadog for system observability and monitoring.
• Databricks certification, cloud certifications (Azure/AWS), or enterprise data architecture certifications.