Senior Data Engineer - Spark & Python (Remote)

United States Digital Space LLC; Poland; Hybrid; PLN 388,000 - 603,000; Full time

Benefits offered by this job

International projects

In-office, hybrid, or remote flex

Medical healthcare

Recognition program

Ongoing learning & reimbursement

Well-being program

Team events & local benefits

Sports compensation

Referral bonuses

Top-tier equipment provision

Job summary

United States Digital Space LLC is seeking a Senior Data Engineer to bring data pipelines to life using Python and PySpark. You will design, develop, and optimize Spark-based solutions, collaborating with cross-functional teams to deliver scalable analytics platforms for enterprise clients.

You will leverage Databricks, Delta Tables, and cloud services to build robust data processing workflows, with a focus on performance, reliability, and clear documentation for stakeholders.

Qualifications

• Solid commercial experience in data engineering.
• Practical experience with SQL stored procedures and Spark SQL/PySpark.
• Expertise using PySpark on Databricks with Delta Tables, cluster mgmt, and automation.
• Proven track record of optimizing Spark performance in production.
• Strong Python skills for complex data manipulation (Polars/Pandas).
• Understanding of Parquet and Delta Tables for columnar storage.
• Experience with data processing, analysis, and transformation workflows.
• Analytical and problem-solving mindset; ability to balance process with flexibility.
• Solid understanding of microservices architecture and scalable systems.

Responsibilities

• Define and enforce coding standards across the project.
• Conduct thorough code reviews to maintain high quality.
• Collaborate with team members and work independently as needed.
• Communicate with the client regularly and manage day-to-day flow.
• Design, develop, and maintain robust Spark applications.
• Write clean, maintainable code following best practices.
• Optimize code for performance and scalability.
• Work with cross-functional teams to deliver high-quality software.
• Identify and resolve technical issues ensuring reliability.
• Create and maintain comprehensive documentation.

Skills

Python

SQL

PySpark

Databricks

Spark SQL

Polars

Pandas

Parquet

Microservices

Tools

Delta Tables

Azure Cloud

Kubernetes

Docker

FastAPI

Job description

United States Digital Space LLC is seeking a Senior Data Engineer to bring data pipelines to life using Python and PySpark. You will design, develop, and optimize Spark-based solutions, collaborating with cross-functional teams to deliver scalable analytics platforms for enterprise clients.

You will leverage Databricks, Delta Tables, and cloud services to build robust data processing workflows, with a focus on performance, reliability, and clear documentation for stakeholders.