Senior Data Engineer with Databricks

Bonapolia; Warszawa; On-site; PLN 180,000 - 260,000; Full time

Benefits offered by this job

Model: B2B contract with Bonapolia -一次

Rate negotiated per seniority

Monthly invoicing

Contract under local law

MSA, NDA, invoicing handled by us

Job summary

Bonapolia is seeking a senior software engineer with deep expertise in PySpark, Databricks, and Spark SQL to design, develop, and optimize large-scale data processing pipelines in a hybrid cloud environment. You will work on enterprise-scale tax-related data solutions, collaborate with cross-functional teams, and ensure scalable, high-performance systems.

The role requires 5+ years of hands-on software development, strong Python skills, and experience with Delta Tables, Parquet formats, and

Qualifications

• 5+ years of hands-on experience in software development.

Responsibilities

• Define and enforce coding standards across the project.
• Conduct thorough code reviews to ensure high quality.
• Work independently and with the team.
• Communicate with the client regularly.
• Design, develop, and maintain Spark applications.
• Write clean, maintainable code following best practices.
• Optimize code for performance and scalability.
• Collaborate with cross-functional teams to deliver solutions.
• Identify and resolve technical issues to ensure reliability.
• Create and maintain documentation for code and workflows.

Skills

5+ years experience

Analytical thinking

Python

Spark

PySpark

Polars

Pandas

SQL

Code reviews

Tools

Databricks

PySpark

Spark SQL

Delta Tables

Parquet

Azure

Docker

Kubernetes

Job description

About The Client

Our client is one of the Big Four accounting firms and the world’s largest professional services network. Headquartered in London, they operate in 150+ countries with 460,000+ professionals delivering excellence in audit, tax, consulting, and advisory. The team you will join designs, develops, and deploys innovative enterprise technology and AI-driven tools to support the delivery of tax services. It is a dynamic group combining expertise in tax, software engineering, change management, and project management, working on initiatives from tool design and deployment to training development and engagement management.

Project Tech Stack

Azure Cloud, Microservices Architecture, .NET 8, ASP.NET Core services, Python, MongoDB, Azure SQL, Angular 18, Kendo, GitHub Enterprise with Copilot, LangGraph, LangChain, RAG Pipelines, Multi-modal LLMs

About The Client

Our client is one of the Big Four accounting firms and the world’s largest professional services network. Headquartered in London, they operate in 150+ countries with 460,000+ professionals delivering excellence in audit, tax, consulting, and advisory. The team you will join designs, develops, and deploys innovative enterprise technology and AI-driven tools to support the delivery of tax services. It is a dynamic group combining expertise in tax, software engineering, change management, and project management, working on initiatives from tool design and deployment to training development and engagement management.

Project Tech Stack

Azure Cloud, Microservices Architecture, .NET 8, ASP.NET Core services, Python, MongoDB, Azure SQL, Angular 18, Kendo, GitHub Enterprise with Copilot, LangGraph, LangChain, RAG Pipelines, Multi-modal LLMs

What You’ll Do

Define and enforce best practices and coding standards across the project
Conduct thorough code reviews to ensure adherence to established guidelines and maintain high code quality
Working both independently and in close collaboration with others in the team
Communicating clear instructions to team members and helping manage the flow of day-to-day operations
Communicating with the client regularly
Design, develop, and maintain robust and scalable Spark applications
Write clean, maintainable, and efficient code following best practices and coding standards
Optimize code for performance and scalability, ensuring efficient data handling
Work closely with cross-functional teams to deliver high-quality software solutions
Identify and resolve technical issues, ensuring the reliability and performance of applications
Create and maintain comprehensive documentation for code, processes, and workflows

What You Bring

5+ years of hands-on experience in software development
Practical experience developing SQL stored procedures and migrating them into Spark SQL or PySpark environments
Extensive expertise using PySpark on Databricks, encompassing Delta tables, cluster management, and workflow automation
Proven track record of optimizing Spark performance within production-level projects
Strong proficiency in Python, specifically for complex data manipulation utilizing libraries like Polars or Pandas
Solid understanding of columnar data storage formats, particularly Parquet, with practical experience using Delta Tables
Proven expertise in data processing, analysis, and transformation workflows
Strong analytical and problem-solving abilities with a detail-oriented mindset
Practical and pragmatic approach to balancing standardized processes with flexibility to meet project goals effectively
Solid understanding of microservices architecture and its implementation in scalable systems

Nice to have

Experience working with Azure Cloud services (or other major cloud platforms),
including a range of SaaS offerings such as Service Bus, Data Lake, Blob Storage, Redis, and more Familiarity with FastAPI
Expertise in containerization and orchestration tools such as Docker and Kubernetes

English level

Advanced

What We Offer

• Model: B2B contract with Bonapolia - one contract, one monthly invoice
• Rate: negotiated per candidate seniority and client budget
• Payment: monthly invoice, paid on the same schedule the client pays us (usually 30 days)
• Contract written under your country's law
• We handle the client-side paperwork: MSA, NDA, invoicing

About Bonapolia

Bonapolia is a family-owned IT staff augmentation team based in Lisbon. We build engineering teams for product companies and IT consultancies across Europe and the US. More at bonapolia.com