Director-Level Data Engineer for AI Platform & Pipelines

Morgan Stanley; Glasgow; On-site; GBP 68,000 - 108,000; Full time

Job summary

Morgan Stanley is seeking a Data Engineer in Scotland to design and maintain large-scale data pipelines, leveraging Snowflake, Databricks, and Spark. You will work with Python, SQL/NoSQL, and cloud services to ensure scalable data access and quality across teams.

The role requires 8+ years of experience, strong collaboration, and ability to contribute to architectural decisions while operating in a distributed team environment.

Qualifications

• Data Engineering experience with Snowflake and Databricks native solutions.
• Experience with Snowflake Cortex is required.
• Proficiency in Python.
• Experience with data processing frameworks like Apache Spark or Hadoop.
• Knowledge of database systems, including SQL and NoSQL.
• Familiarity with cloud platforms such as AWS and Azure, and their data services.
• Understanding of data modeling and data architecture principles.
• Experience with data warehousing concepts and technologies.
• Experience with message queues and streaming platforms, such as Kafka.
• Experience with version control systems, such as Git.
• Experience using Jupyter notebooks for data exploration, analysis, and visualization.
• Excellent communication and collaboration skills.
• Ability to work independently and as part of a geographically distributed team.
• 8+ years of experience as a practitioner in data engineering or a related field is generally expected.

Responsibilities

• Develop and maintain data pipelines and ETL processes.
• Work with structured and unstructured data to ensure it is accessible and usable.
• Optimize data systems for performance and scalability.
• Implement data quality and data governance standards.
• Collaborate with stakeholders across technology and business units to understand their data needs and translate them into technical solutions.
• Provide data-driven insights.
• Contribute to documentation and knowledge sharing within our team.
• Create and maintain technical documentation and training materials.
• Participate in code reviews and contribute to the improvement of development processes.
• Contribute to the broader data architecture community through knowledge sharing and presentations.

Skills

Data engineering

Python

SQL

NoSQL

Snowflake

Databricks

Kafka

Spark

Git

Jupyter

Tools

Snowflake

Databricks

Kafka

Git

Jupyter

Spark

Hadoop

Job description

Morgan Stanley is seeking a Data Engineer in Scotland to design and maintain large-scale data pipelines, leveraging Snowflake, Databricks, and Spark. You will work with Python, SQL/NoSQL, and cloud services to ensure scalable data access and quality across teams.

The role requires 8+ years of experience, strong collaboration, and ability to contribute to architectural decisions while operating in a distributed team environment.