Senior Data Engineer

xdesign; United States; Hybrid; USD 140,000 - 190,000; Full time

Benefits offered by this job

Hybrid/Remote work

Office hubs and travel to client sites

Client-site travel

Impact-focused culture

Job summary

xdesign in the United States is seeking a senior data engineer to design, build, and maintain production data pipelines on Microsoft Fabric, Python, and PySpark. The role offers significant autonomy and mentorship of junior engineers, with collaboration across architects, product teams, and client stakeholders.

You will influence pipeline architecture, data modelling, and tooling decisions, while improving CI/CD for data workflows and translating ambiguous requirements into robust, scalable

Qualifications

• Extensive hands-on experience with modern cloud data platforms and production pipelines.
• Strong SQL expertise with data modelling patterns (Kimball, medallion, lakehouse).
• Proficient in Python and PySpark for batch data processing with production-grade code.
• Experience owning data engineering delivery with CI/CD and limited oversight.
• Comfort mentoring and supporting the development of less experienced engineers.

Responsibilities

• Design, build, and maintain scalable, production-ready data pipelines on Microsoft Fabric, Python, and PySpark.
• Make architectural decisions on pipeline design, data modelling, and tooling, balancing performance, cost, and maintainability.
• Mentor and review work from junior/mid-level data engineers.
• Continuously improve CI/CD pipelines for data engineering workflows.
• Act as escalation point for complex issues in live data pipelines.
• Translate ambiguous data requirements into robust solutions with stakeholders.

Skills

Mentoring

Stakeholder engagement

Architectural decision making

Problem solving

Tools

Microsoft Fabric

Databricks

Snowflake

Synapse

Python

PySpark

SQL

CI/CD

Job description

Role overview

A senior data engineering role focused on designing, delivering, and maintaining production data pipelines built on Microsoft Fabric, Python, and PySpark. You'll work with significant autonomy on complex and ambiguous problems, mentor less experienced engineers, and partner with architects, product teams, and client stakeholders to translate requirements into robust solutions. The team treats batch and ELT pipeline engineering as a separate discipline from real-time streaming.

Responsibilities

• Own the design, build, and maintenance of scalable, production-ready data pipelines on Microsoft Fabric, Python, and PySpark
• Make architectural decisions on pipeline design, data modelling, and tooling, weighing performance, cost, and maintainability
• Mentor and review work from junior and mid-level data engineers, raising the bar on delivery
• Continuously improve CI/CD pipelines for data engineering workflows
• Act as an escalation point for complex issues in live data pipelines
• Translate ambiguous data requirements into robust, maintainable technical solutions alongside stakeholders

Requirements

• Significant hands-on experience with a modern cloud data platform such as Microsoft Fabric, Databricks, Snowflake, or Synapse, including designing and operating production pipelines
• Deep SQL expertise with hands-on experience in data modelling patterns such as Kimball, medallion or lakehouse, or domain-driven modelling
• Strong proficiency in Python and PySpark for batch data processing, with a track record of production-grade, well-tested code
• Experience owning data engineering delivery with limited oversight, including CI/CD management and support
• Comfort mentoring and supporting the development of less experienced engineers
• Ability to engage directly with stakeholders to shape solutions from ambiguous requirements

Benefits and work setup

• Flexible working with hybrid and remote options
• Office hubs in Edinburgh, Leeds, Manchester, London, and Bulgaria
• Occasional travel to client sites or team offices when needed
• Supportive culture focused on impact and outcomes over activity