Data Engineer - Azure Databricks Platform

EnerMech; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

EnerMech is seeking a senior data platform engineer to own key Azure Databricks workstreams across the platform stack, from scoping to production. You will ensure medallion data models are correct and reliable for reporting and analytics.

This fully remote role supports EnerMech's Enterprise Business Systems with stakeholders across UK, Americas, Middle East and Asia Pacific, requiring strong DevOps, Terraform, and data governance expertise.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems or related discipline, or equivalent practical experience.
• 6+ years in data engineering, including 3+ years hands‑on with Azure Databricks in production.
• Ownership of medallion architecture design and data modelling.
• Implemented data quality frameworks and data contracts in production.
• PII protection across environments.
• Platform engineering experience: diagnosing infrastructure issues, not just consuming platform services.
• Track record of autonomous delivery, including direct stakeholder engagement.
• Databricks Certified Data Engineer Associate (minimum).
• Databricks Certified Data Engineer Professional.
• Microsoft AZ-400: Designing and Implementing Microsoft DevOps Solutions.
• Multi‑environment Azure networking (VNet peering, DNS zone design, NAT Gateway).
• Migrating or standing up CI/CD pipelines, incl. introducing pipeline templates/standards.
• Governance and identity patterns (e.g. Entra ID, RBAC scoping).
• Applying regulatory frameworks (e.g. GDPR) to data platform design.
• Shared service centre or global delivery model experience, ideally in energy or industrial services.

Responsibilities

• Design, build, quality and reliable operation of assigned workstreams on the Azure Databricks data platform, from scoping through to production deployment.
• Integrity of the medallion data model (bronze/silver/gold) for assigned data domains, ensuring data is modelled, validated, documented and certified for downstream reporting and analytics.
• Build and optimise Delta tables and materialized views, with attention to performance at scale.
• Design and implement data quality frameworks (preferably DQX or equivalent), including quarantine/error handling and validation against data contracts.
• Define job‑specific compute configurations appropriate to each workload, balancing performance and cost.
• Contribute to and troubleshoot Terraform‑managed infrastructure (networking, storage, Key Vaults, Managed DevOps Pools, Dev Centers).
• Own and maintain Azure DevOps build and release pipelines, variable groups/libraries, and repository security, permissions and branch policies.
• Support environment‑level Azure configuration, including subscription‑level access, RBAC for user and pipeline (service principal) identities, and Terraform state backend management across multiple storage accounts and environments.
• Operate autonomously across a workstream, from scoping through to production deployment.
• Use AI-assisted tooling to support day‑to‑day development, validating outputs against authoritative sources (architecture decisions, technical leads, documentation) before acting on them.
• Communicate clearly with technical and non‑technical stakeholders, including across time zones with colleagues in the UK, Americas, Middle East and Asia Pacific.

Skills

Azure Databricks

Data Engineering

Data Modeling

Terraform

Azure DevOps

CI/CD

RBAC

Security & GDPR

Education

Bachelor's degree in CS/Engineering or related

Tools

Azure

Databricks

Terraform

Azure DevOps

Job description

Описание:

We are EnerMech, a specialist service company that delivers safe, integrated solutions for complex energy projects. For over 50 years, we've been energizing our clients across the world with unrivalled expertise, equipment, and technology that supports the entire asset life-cycle - offshore and on land. We enable progress through our three global business lines: Energy Solutions, Infrastructure & Industrial Solutions, and Lifting Solutions. Within these areas, we offer a wide range of services, including cranes, lifting, fluid power, training, equipment rental, pipeline and subsea, process, and valves - all designed to help our clients optimize performance, enhance reliability, and improve efficiency. With a focus on operational excellence, we deliver value through our customized integrated solutions. This approach reduces risk and enhances efficiencies making us a leading and trusted partner every step of the way.

Задачи:

The role will be responsible for taking technical ownership of key workstreams on EnerMech's Azure Databricks data platform, operating across the full platform stack: data modelling and pipeline development in Databricks, the underlying Azure infrastructure, and the CI/CD, DevOps and DataOps practices that keep the platform reliable, secure and scalable. The role holder operates with a high degree of autonomy, makes and documents architectural decisions, resolves complex platform issues, and is a technical point of contact for the global EBS Team, and business stakeholders. This is a fully remote role in the UK,supporting our Enterprise Business Systems (EBS) function in the Mumbai Shared Service Centre. Key reponsibilities will include:

• Design, build, quality and reliable operation of assigned workstreams on the Azure Databricks data platform, from scoping through to production deployment.
• Integrity of the medallion data model (bronze/silver/gold) for assigned data domains, ensuring data is modelled, validated, documented and certified for downstream reporting and analytics.
• Build and optimise Delta tables and materialized views, with attention to performance at scale.
• Design and implement data quality frameworks (preferably DQX or equivalent), including quarantine/error handling and validation against data contracts.
• Define job‑specific compute configurations appropriate to each workload, balancing performance and cost.
• Contribute to and troubleshoot Terraform‑managed infrastructure (networking, storage, Key Vaults, Managed DevOps Pools, Dev Centers).
• Own and maintain Azure DevOps build and release pipelines, variable groups/libraries, and repository security, permissions and branch policies.
• Support environment‑level Azure configuration, including subscription‑level access, RBAC for user and pipeline (service principal) identities, and Terraform state backend management across multiple storage accounts and environments.
• Operate autonomously across a workstream, from scoping through to production deployment.
• Use AI-assisted tooling to support day‑to‑day development, validating outputs against authoritative sources (architecture decisions, technical leads, documentation) before acting on them.
• Communicate clearly with technical and non‑technical stakeholders, including across time zones with colleagues in the UK, Americas, Middle East and Asia Pacific.

Требования:

• Bachelor's degree in Computer Science, Engineering, Information Systems or a related discipline, or equivalent practical experience.
• Typically 6+ years in data engineering, including 3+ years hands‑on with Azure Databricks in production.
• Demonstrated ownership of medallion architecture design and data modelling.
• Implemented data quality frameworks and data contracts in production.
• Implemented PII protection (masking, tokenisation or access tiering) across environments.
• Platform engineering experience: diagnosing infrastructure issues, not just consuming platform services.
• Track record of autonomous delivery, including direct stakeholder engagement.
• Databricks Certified Data Engineer Associate (minimum).
• Databricks Certified Data Engineer Professional.
• Microsoft AZ-400: Designing and Implementing Microsoft DevOps Solutions.
• Multi‑environment Azure networking (VNet peering, DNS zone design, NAT Gateway).
• Migrating or standing up CI/CD pipelines, incl. introducing pipeline templates/standards.
• Governance and identity patterns (e.g. Entra ID, RBAC scoping).
• Applying regulatory frameworks (e.g. GDPR) to data platform design.
• Shared service centre or global delivery model experience, ideally in energy or industrial services.