Data Engineer – Azure Databricks & Scalable Pipelines

Cognita Schools; Greater London; On-site; GBP 70,000 - 105,000; Full time

Benefits offered by this job

Competitive salary based on experience

Private Medical Insurance & Healthcare

GPP Pension

Life Assurance

25 days annual leave

Employee Assistance Programme

Employee Discounts Site

Professional Subscriptions

Job summary

Cognita Schools is seeking a hands-on Data Engineer to build and maintain a modern cloud data platform in Azure Databricks. You will design robust data pipelines across CRM, MIS, Finance, and HR, ensuring quality, governance and scalable ingestion frameworks.

Working with analysts and engineers, you will translate business requirements into reliable data solutions, using CI/CD, and contribute to a medallion-architecture approach. London office with one day remote per week is offered.

Qualifications

• Degree in a relevant field (CS/IS/engineering).
• Proven experience designing and delivering production-grade data pipelines in cloud environments.
• Strong Azure Databricks, PySpark, Delta Lake, Databricks Jobs knowledge and SQL proficiency.

Responsibilities

• Build and maintain Cognita's modern data platform in Azure Databricks.
• Design, develop and optimise end-to-end data solutions from source systems (CRM, MIS, Finance, HR).
• Implement data quality, monitoring, governance and schema management; CI/CD practices.

Skills

PySpark

Delta Lake

SQL

Data pipelines

Medallion architecture

CI/CD practices

Collaboration

Education

Bachelor's degree in Computer Science, Data Engineering, Information Systems or related field

Tools

Azure Databricks

Git-based version control

APIs

Databricks Jobs

JSON/Parquet/CSV handling

Job description

Cognita Schools is seeking a hands-on Data Engineer to build and maintain a modern cloud data platform in Azure Databricks. You will design robust data pipelines across CRM, MIS, Finance, and HR, ensuring quality, governance and scalable ingestion frameworks.

Working with analysts and engineers, you will translate business requirements into reliable data solutions, using CI/CD, and contribute to a medallion-architecture approach. London office with one day remote per week is offered.