Senior Big Data Engineer - AI Forecasting

ASOS; London; On-site; GBP 50,000 - 90,000; Full time

Job summary

Join a forward-thinking company as a Senior Big Data Engineer, where you'll collaborate with machine learning experts to drive innovative data solutions. This role offers the chance to work with cutting-edge technologies in a dynamic environment, ensuring data quality and optimizing data pipelines. You will have the opportunity to unleash your creativity while contributing to high-impact projects that support global growth. With a commitment to personal development and a culture that values diversity and inclusion, this position promises not just a job, but a chance to make a real difference in the tech landscape.

Benefits

Competitive salary

Pension scheme

Private medical care

Performance-related bonus

25 days paid annual leave

Employee discount

Tech development program

Opportunity to represent at events

Diversity initiatives

Cutting-edge technology projects

Qualifications

• Experienced Data Engineers with a strong grasp of data technologies.
• Proficient in Spark/PySpark, Azure services, and Python programming.

Responsibilities

• Develop scalable data solutions using Azure and Databricks.
• Optimize data pipelines and support production applications.

Skills

Data Engineering

Spark/PySpark

Azure Databricks

Python

SQL

CI/CD Workflows

Data Quality

Agile Methodologies

Problem-Solving

Communication Skills

Education

Bachelor's Degree in Computer Science or related field

Tools

Azure Data Factory

Azure Event Hubs

Azure Service Bus

Terraform

Git

pytest

Job description

Company Description

We're ASOS, the online retailer for fashion lovers all around the world.

We exist to give our customers the confidence to be whoever they want to be, and that goes for our people too. At ASOS, you're free to be your true self without judgement, and channel your creativity into a platform used by millions.

But how are we showing up? We're proud members of Inclusive Companies, are Disability Confident Committed and have signed the Business in the Community Race at Work Charter and we placed 8th in the Inclusive Top 50 Companies Employer list.

Everyone needs some help showing up as their best self. Let our Talent team know if you need any adjustments throughout the process in whatever way works best for you.

Job Description

We are looking for a Senior Big Data Engineer to join our cross-functional AI Forecasting team. You will work alongside machine learning engineers and scientists to solve problems and productionise interesting solutions that leverage cutting edge tech.

ASOS Technology is going through an exciting period of transition towards product operating model - this includes several strategic programmes to deliver the amazing technology and business solutions to support our ambitious global growth plans.

ASOS is a unique and forward-thinking company which understands the data opportunities and technology has on driving the business forward. The Digital Data Engineering team is responsible for providing high quality, scalable data solutions across the ASOS Digital Domain, advocating best practices, modern technologies and educating development teams.

As a Data Engineer, you'll be hands-on delivering business critical projects and processes, helping the digital platforms to fully take advantage of their data while ensuring continuous product improvement, best practices usage and ongoing development of engineers through pairing. From problem solving to new concept ideas, you have the genuine opportunity to unleash your full potential and creativity on a variety of ASOS Data engineering projects.

What you'll be doing:

1. Developing solutions using Microsoft Azure and Databricks to the highest standard in a way that favours a simple and maintainable approach over a complex one, is tested at every step on the path to production, using the appropriate tools and practices, is deployable using automated tooling only, and is suitably instrumented to meet core operational and business monitoring needs.
2. Optimising Data pipelines, frameworks, and systems to facilitate easier development of data artifacts.
3. Supporting production applications such as Data Pipelines, Core data infrastructure etc.
4. Developing proof of concepts by exploring innovative ways to utilise our data and presenting findings to Product teams. This also involves conducting feasibility tests such as performance evaluations, data modelling, cost analysis, etc.
5. Working with engineering team leads and Product managers to set and maintain standards and development practices that promote high delivery cadence without impeding robust, scalable, and quality solutions.
6. Working with an agile, cross functional team, taking responsibility for the engineering team deliverables and quality.
7. Keep up to date with emerging data technologies and industry trends with a view to bringing business value through early adoption.
8. Building strong and productive relationships across business, science, analytics, and engineering partners.
9. Implementing, contributing and automating Data Quality and Observability checks over key data and pipelines. Ensuring Data Quality results are proactively surfaced to source teams to act.

Qualifications

What you will need:

1. We're looking to meet experienced Data Engineers who live and breathe Data, who seek to take this best practice and use it in a dynamic environment while making waves in the industry.
2. In-depth knowledge in most of the following technologies:
   
   • Proficient in Spark/PySpark with hands-on experience.
   • Azure data related technologies such as Azure Databricks, Azure Data Factory, Azure Event Hubs and Azure Service Bus.
   • Solid programming skills in Python or Scala, and strong command of SQL.
   • Experience with testing frameworks like pytest, ScalaTest, or similar.
   • Open table formats such as Delta, Iceberg or Apache Hudi.
   • Experience in leveraging CI/CD workflows to automate the building, testing, publishing, releasing, and deployment of code using tools like Azure DevOps, GitHub Actions, and distributed version control systems such as GIT.
   • Understanding of cloud infrastructure and Infrastructure as Code (Terraform or Bicep).
3. Excellent communication skills.
4. Good analytical and problem-solving skills.
5. Broad expertise in the delivery of data product solutions.
6. Commitment to provide a high level of customer service to internal and external users.
7. Proven experience in the successful delivery and ongoing maintenance of complex, high volume, high performing, and high-quality software systems.
8. Experience of implementing/supporting cloud-based solutions.
9. Familiarity with agile methodologies, such as Scrum or Kanban.

Nice to have skills:

1. Experience in retail and/or e-commerce.
2. Understanding of Big Data and Distributed Computing.
3. Understanding of streaming technologies (Spark Structured streaming, Apache Flink, etc).
4. Other programming languages: PowerShell, Bash.
5. Understanding of Databricks Ecosystem (Unity Catalog, Workflows, Delta Live Tables).
6. Understanding of any Data Observability or Data Quality Framework.

Additional Information

What's in it for you?

• Competitive salary, pension, and private medical care scheme.
• Performance related bonus.
• Flex benefits allowance - which you can choose to take as extra cash, or use towards other benefits.
• 25 days paid annual leave + an extra day for your birthday.
• Employee discount (hello ASOS discount!).
• Tech Develops - our internal tech focused skills development programme to focus on your personal growth as a technologist.
• Opportunity to represent ASOS at industry leading events.
• Opportunity to help shape and drive our DE&I initiatives in Tech (like our WIT movement and Diversity mentoring in Tech).
• Opportunity to make an impact from day one and work with the latest in cutting edge technology.

Why take our word for it? Search #InsideASOS on our socials to see what life at ASOS is like.

Want to find out how we're tech powered? Check out the ASOS Tech Podcast here ASOS Tech Podcast. Prefer reading? Check out our ASOS Tech Blog.