Databricks Solution Engineer - Moody's

OpenTalent; Greater London; On-site; GBP 85,000 - 130,000; Full time

Job summary

OpenTalent is seeking a senior data/analytics engineer to design and support scalable data, analytics, and AI solutions on the Databricks Lakehouse Platform. You will build robust data pipelines using Spark, SQL, Python and Delta Lake, and develop governed data products to support reporting and AI initiatives.

You will collaborate with business stakeholders and tech teams to translate requirements into scalable, cost-efficient solutions, implementing best practices for performance, security,

Qualifications

• Experience designing and implementing scalable data and analytics solutions on the Databricks Lakehouse Platform.
• 6+ years in Data/Analytics/Data Engineering in enterprise environments.
• Strong expertise in Apache Spark, SQL, Python, Delta Lake.
• Experience building and optimizing data pipelines for analytics/AI use cases.
• Knowledge of data modeling, governance, quality, and enterprise data architecture.
• Experience integrating Salesforce, Power BI, Snowflake, and Microsoft Fabric.

Responsibilities

• Design, build, and support scalable data, analytics, and AI solutions on the Databricks Lakehouse Platform.
• Design and implement scalable data and analytics solutions using Databricks technologies.
• Build, maintain, and optimize data pipelines with Spark, SQL, Python, Delta Lake.
• Develop governed datasets and data products for reporting and AI initiatives.
• Translate requirements with stakeholders into scalable, cost-effective solutions.
• Implement best practices for performance, security, data quality, and governance.
• Support integration of enterprise platforms like Salesforce, Power BI, Snowflake, and Microsoft Fabric.
• Contribute to platform standards through automation, CI/CD, and DevOps.

Skills

Data engineering

Analytics engineering

Stakeholder management

Communication

AI in business

Education

Bachelor's degree in CS/IT/Data Engineering

Databricks Data Engineer Associate

Databricks Generative AI Engineer Associate

Tools

Databricks Lakehouse

Apache Spark

SQL

Python

Delta Lake

Git

CI/CD

DevOps practices

Salesforce

Snowflake

Microsoft Fabric

Job description

Skills and Competencies

• 6+ years of experience in Data Engineering, Analytics Engineering, Solution Engineering, or related disciplines within enterprise environments
• 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform
• Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices
• Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics and artificial intelligence use cases
• Proven knowledge of data modeling, data governance, data quality management, and enterprise data architecture principles
• Experience integrating cloud and software-as-a-service platforms such as Salesforce, Microsoft Power BI, Snowflake, and Microsoft Fabric
• Working knowledge of Git, continuous integration and continuous delivery pipelines, DevOps practices, and automation frameworks
• Demonstrated proficiency in artificial intelligence concepts, with hands-on experience using AI tools to streamline workflows and enhance operational efficiency. Proven ability to implement AI-powered solutions to solve business challenges. Demonstrates a growing awareness of AI risk management and a commitment to responsible and ethical AI use.
• Excellent communication, collaboration, and stakeholder management skills with the ability to translate business requirements into technical solutions

Education

• Bachelor's degree or equivalent qualification in Computer Science, Information Systems, Data Engineering, Software Engineering, or a related field
• Databricks Data Engineer Associate certification or higher preferred
• Databricks Generative AI Engineer Associate certification preferred

Responsibilities

Design, build, and support scalable data, analytics, and artificial intelligence solutions on the Databricks Lakehouse Platform.

• Design and implement scalable data and analytics solutions leveraging Databricks technologies
• Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake
• Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives
• Partner with business stakeholders and technical teams to translate requirements into cost-effective and scalable solutions
• Implement best practices for performance optimization, security, data quality, compliance, and governance
• Support integration of enterprise platforms including Salesforce, Microsoft Power BI, Snowflake, Microsoft Fabric, and other business applications
• Contribute to platform standards through automation, continuous integration and continuous delivery practices, and DevOps initiatives