Data Engineer

Two Circles; Greater London; On-site; GBP 70,000 - 110,000; Full time

Benefits offered by this job

Discretionary Bonus

Performance reviews every 6 months

Summer Away Days

Private healthcare

Gym membership

Cycle to work

Learning & Development opportunities

Job summary

Two Circles in London is seeking a Data Engineer to design, build, and optimise scalable data pipelines on Databricks, leveraging Delta Lake and Unity Catalog for robust data governance. You will write clean Python code and work in an Agile team to deliver data-driven insights for the sports industry.

The role requires strong Databricks production experience, PySpark and Spark SQL, plus SQL proficiency. A coachable mindset and willingness to learn AI-assisted development are essential.

Qualifications

• Hands-on production Databricks experience with Delta Lake and Unity Catalog.
• Strong Python (object-oriented) and clean, testable code.
• Solid SQL for data querying and analysis.
• Experience with Agile development and CI/CD practices.
• Willingness to learn AI-assisted development and coachable mindset.
• Experience in data lakehouse environments.

Responsibilities

• Design, build, and optimise scalable data pipelines on Databricks.
• Collaborate with product and marketing to deliver data-driven insights.
• Extend Azure data platform (ADF, Function Apps) and CI/CD.
• Actively improve the Databricks platform with modernization.
• Work within an Agile sprint framework.

Skills

Databricks

Python

SQL

Agile

Teamwork

Coachable

Education

Databricks Certification

Tools

Delta Lake

Unity Catalog

PySpark

Spark SQL

Azure DevOps

Azure Data Factory

Copilot

Job description

• We’re looking for a Data Engineer with strong, hands-on Databricks expertise to join our squad of >50 Data & BI Engineers, working on our Fan Intelligence Platform
• Databricks is a core focus for this role — it’s an area where we’re looking to deepen the team’s collective expertise
• Our team is also embracing AI-assisted development as a way of supercharging how we build, and you’ll be surrounded by engineers who are pushing this forward — so an openness to learning and evolving how you work is just as important as the technical experience you bring
• The data platform itself is growing fast, with an exciting suite of products being built on top of it — meaning real scope to shape its direction, not just maintain what’s there
• If you’re passionate about data engineering, enjoy collaborating in a fast-paced environment, and want to make an impact in the sports industry, we’d love to hear from you!
• Design, build, and optimise scalable data pipelines on Databricks — Delta Lake, Unity Catalog, Spark/PySpark — as the primary platform for this role
• Write clean, maintainable, object-oriented Python — solid class design, testable code, and adherence to good engineering practice
• Work within an Agile team — sprint planning, stand-ups, retros — estimating and delivering incremental value
• Collaborate closely with product and marketing teams to deliver data-driven insights and solutions that provide high business value
• Extend into the wider Azure data platform (ADF, Function Apps) and CI/CD practices as you grow in the role
• What success looks like
• Six to twelve months in, you’ll be actively improving our Databricks platform — not just maintaining it, but bringing forward-thinking modernisation improvements that help the team get more out of it

Benefits

• Renowned Team Days often throughout the year
• Discretionary Bonus based on company performance
• Performance Reviews every 6 months with discretionary salary increases
• Summer Away Days
• 23 standard days of holiday (+ 1 Birthday, +2 for a ‘Big Life Event’ and +1 Admin Day), closure of office over Christmas (plus Bank Holidays)
• Private healthcare (Vitality) and/or Health Care Plan (Medicash)
• Mobile phone contribution
• Sport Challenge contribution
• Gym membership contribution
• 2x annual kit drops
• Tickets to sporting events
• Lunch once a week, breakfast and continuous supply of snacks
• Private healthcare schemes
• Cycle to work scheme
• Learning and Development opportunities, including certification in certain areas

Strong teamwork skills and a collaborative approachCI/CD pipelines (Azure DevOps or similar)A proactive attitude with a passion for continuous learning and growthComfortable in a fast-paced, Agile environment — able to estimate and deliver incremental valueWhile the above are important, we believe attitude and ability to learn are the biggest predictors of success in this team. If you’re missing some of the experience above but this sounds like you, we’d love to hear from youObject-oriented Python — solid foundation in class design, best practices, and writing testable, maintainable codeDatabricks — strong, hands-on production experience: Delta Lake, Unity Catalog, PySpark/Spark SQL, performance tuning. This is the depth we most need in the team right nowStrong SQL for querying and managing dataAzure (ADF, Function Apps) alongside DatabricksAptitude for AI-assisted development — you don’t need existing mastery of tools like Copilot or Claude Code, but you do need to be genuinely coachable and excited to adopt this way of working, since you’ll be learning it directly from a mentor who lives it day to dayHigh attention to detail and commitment to qualityExperience in a data lakehouse environmentA genuinely coachable mindset, with a real desire to learn from a mentorDatabricks Certified Data Engineer Professional (or working towards it)