Senior Data Engineer
Retail Insight
A complete application in a minute
Tailored resume and cover letter, ready to send.
Benefits offered by this job
Job summary
Retail Insight is seeking a Senior Data Engineer to build and support scalable data systems powering our US retail data solutions. You will work with a Chicago-based team to translate business needs into practical data engineering solutions, developing robust pipelines and data products with a focus on reliability and performance.
You’ll collaborate with data scientists, analysts, and product teams to improve platform quality, contribute to architecture decisions, and help drive best practices
Qualifications
- Databricks production pipelines expertise.
- Experience with Unity Catalog & Asset Bundles.
- Strong Spark, Delta Lake, and Databricks Workflows knowledge.
- Proficient in Python and SQL; data modelling and ETL/ELT patterns.
- Experience with cloud data platforms (Azure/AWS/GCP).
- Ability to implement data quality checks and monitor pipelines.
- Excellent communication and collaboration with technical and non-technical colleagues.
- CS/related degree or equivalent practical experience; willingness to travel.
Responsibilities
- Build and maintain scalable data pipelines (batch and streaming).
- Design and implement data architecture components.
- Implement data quality checks, monitoring, observability.
- Apply security, access control, and governance standards.
- Collaborate with data scientists and analysts to deliver data products.
- Work with Lead Data Engineer to translate business requirements into tech solutions.
- Participate in code reviews and uphold engineering standards.
- Identify opportunities to optimize cloud costs and performance.
- Contribute to platform documentation and tooling decisions.
Skills
Education
Tools
Job description
Senior Data Engineer
Salary - $125,000 (Chicago)
At Retail Insight, we're transforming the way some of the world's biggest retailers and CPG brands use data, analytics and technology to make smarter decisions.
Our people are at the heart of what we do. We bring together talented people with different experiences, perspectives and expertise to solve complex problems, challenge the status quo and make a real impact for our clients.
We're growing, and we're looking for a Senior Data Engineer to join our Engineering team.
We're looking for an experienced Senior Data Engineer to build and support the reliable, scalable data systems that power our solutions for US retail clients.
As our US client portfolio grows, you'll work with complex data environments, helping translate business requirements into practical technical solutions. Working closely with our Chicago-based Lead Data Engineer, alongside data scientists, analysts and product teams, you'll build data pipelines and products that help our clients make better decisions.
This is a hands-on role combining strong technical skills with collaboration and clear communication. You'll contribute to the evolution of our data platform, improve reliability and performance, and help maintain high engineering standards.
You’ll also work with colleagues and client stakeholders to understand their needs and support successful delivery.
- Building and maintaining scalable data pipelines, developing and optimising batch and streaming workflows for reliability and performance.
- Contributing to the design and implementation of data architecture.
- Implementing data quality checks, monitoring and observability across pipelines.
- Applying security, access control and governance standards established by our platform team.
- Collaborating with data scientists and analysts to deliver data products that support their work.
- Working with the Lead Data Engineer to translate business requirements into technical solutions.
- Participating in code reviews, setting a high bar for engineering quality and contributing to team standards.
- Helping identify opportunities to optimise cloud costs and platform performance.
- Contributing to platform documentation, tooling choices and continuous improvement.
- Solid, demonstrable experience in data engineering, including building and maintaining production-grade Databricks pipelines.
- Experience with Databricks Unity Catalog and Databricks Asset Bundles.
- Strong hands-on knowledge of Apache Spark, Delta Lake and Databricks Workflows.
- Proficiency in Python and SQL, with a good understanding of data modelling, ETL/ELT patterns and distributed data processing.
- Experience with Azure data platforms, including Azure Data Factory, ADLS Gen2 and Azure DevOps, or equivalent services on AWS or GCP.
- Experience implementing data quality checks and pipeline monitoring.
- Strong communication skills and the ability to work effectively with technical and non-technical colleagues.
- A collaborative approach, ownership of quality outcomes and a drive to learn and improve.
- A degree in computer science, data engineering or a related field, or equivalent practical experience.
- Willingness to travel as required.
- Familiarity with infrastructure-as-code tools such as Terraform or Bicep.
- Familiarity with dbt or similar data transformation frameworks.
- Exposure to real-time streaming technologies such as Kafka or Event Hubs.
- Exposure to retail or consumer packaged goods (CPG) data, such as point of sale, inventory or supply chain.
- Experience working directly with clients.
We believe in flexibility, growth, and taking care of our people, both in and out of the office.
Hybrid & Flexible Working: Work from anywhere benefits, plus summer hours and extra days off for life’s big moments (including your birthday and wedding).
Generous Time Off: 26 days PTO (accrued with service), paid volunteer days, and full support for your wellbeing, including mental health resources.
Career & Learning: Access to a top-tier learning platform, career progression plans, and support to create your ideal home workspace.
Health & Financial Benefits: Comprehensive medical, dental, and vision coverage, plus a 6% matched retirement plan.
Personal Perks: Annual phone reimbursement, onsite gym access, and customizable benefits via our online platform.
Plus much more!
Our interview process may vary slightly depending on the role. We’ll always let you know what to expect at each stage and give you plenty of opportunities to ask questions along the way.
An initial conversation with a member of our People Team to learn more about you, your experience and what you’re looking for in your next role. We’ll also introduce you to Retail Insight and explain the opportunity and interview process. During the call, we’ll ask interview questions based on our cultural drivers to understand how your values, behaviours and ways of working align with ours.
This stage explores your experience and the skills most relevant to the role, as well as how you approach your work. Depending on the position, it may include a practical task, presentation or technical assessment. We’ll always explain what’s involved in advance.
The final stage is an informal coffee chat with a member of our senior leadership team. It’s an opportunity to learn more about our business and culture, ask any remaining questions and make sure Retail Insight feels like the right fit for you.
Be your authentic self - Retail Insight is committed to promoting equal opportunities in employment. All employees and any job applicants will receive equal treatment. We actively seek to create an environment where everyone feels respected, supported, and encouraged to contribute their best work.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
