Lead Data Engineer

Experteer Italy; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

A top consulting firm is seeking a Lead Data Engineer to leverage their expertise in data engineering and software solutions to address complex challenges for clients. You will collaborate with multidisciplinary teams to develop innovative, data-driven strategies and engage in the entire engineering lifecycle. The ideal candidate will have a Master's degree and extensive experience in various programming languages and cloud technologies, along with a strong focus on collaboration and continuous learning.

Qualifications

• Minimum 5 years experience in data and software engineering.
• 3+ years in a Senior or Staff level position.
• Experience in deploying serverless data pipelines.

Responsibilities

• Develop data and software solutions for enterprise challenges.
• Collaborate with cross-functional teams to identify analytics solutions.
• Perform transformations on structured and unstructured data.

Skills

Python

Scala

C/C++

Java

C#

Go

SQL

NoSQL

ETL frameworks

Cloud platforms

DevOps

Education

Master's degree in Computer Science

Engineering or related technical field

Tools

Airflow

Spark

PostgreSQL

AWS

Azure

GCP

Terraform

Job description

Description & Requirements

WHAT MAKES US A GREAT PLACE TO WORK

We are proud to be consistently recognized as one of the world's best places to work. We are currently the #1 ranked consulting firm on Glassdoor's Best Places to Work list and have maintained a spot in the top four on Glassdoor's list since its founding in 2009. Extraordinary teams are at the heart of our business strategy, but these don't happen by chance. They require intentional focus on bringing together a broad set of backgrounds, cultures, experiences, perspectives, and skills in a supportive and inclusive work environment. We hire people with exceptional talent and create an environment in which every individual can thrive professionally and personally.

WHO YOU'LL WORK WITH

You'll join our Engineering Experts within the AI, Insights & Solutions team. This team is part of Bain's digital capabilities practice, which includes experts in analytics, engineering, product management, and design. In this multidisciplinary environment, you'll leverage deep technical expertise with business acumen to help clients tackle their most transformative challenges. You'll work on integrated teams alongside our general consultants and clients to develop data-driven strategies and innovative solutions. Together, we create human-centric solutions that harness the power of data and artificial intelligence to drive competitive advantage for our clients. Our collaborative and supportive work environment fosters creativity and continuous learning, enabling us to consistently deliver exceptional results.

WHERE YOU'LL FIT WITHIN THE TEAM

As a Lead, Data Engineering, you will leverage your experience to implement and refine technical solutions for a wide range of industries. Working with a diverse team, you will engage in the entire engineering life cycle, focusing on designing, developing, optimizing, and deploying sophisticated data engineering solutions and infrastructure at a production scale suitable for the world's largest companies.

WHAT YOU'LL DO:

1. Develop data and software solutions to address large-scale enterprise challenges for Bain's clients, serving as the data engineer and expert within a cross-functional team
2. Develop and maintain long-lasting products that support internal or client needs
3. Collaborate closely with and influence general consulting teams to identify analytics solutions for client business problems and to execute those solutions
4. Collaborate with data engineering leaders to develop and advocate for modern data engineering concepts to both technical audiences and business stakeholders
5. Enable data and technology for data science, analytics, and other application use cases via data engineering
6. Perform transformations at scale including cleaning, enriching, de-duping, joining, and correlating structured, semi-structured, or unstructured data
7. Define and implement new and innovative deployment techniques, tooling, and infrastructure automation within Bain, including designing, writing documentation, and conducting code reviews for data engineering solutions
8. Participate in infrastructure engineering for data ecosystem including development, testing, deployment, and release
9. Provide technical guidance to external clients and internal stakeholders in Bain
10. Contribute to industry-leading innovations that translate into great impact for clients in case work
11. Stay current with emerging trends and technologies in cloud computing, data analysis, and software engineering, and proactively identify opportunities to enhance the capabilities of the analytics platform
12. Travel is required (30%)

ABOUT YOU:

1. Proven experience in end-to-end data and software engineering within either product engineering or professional services organizations, including project setup, test cases, dependency, and build management
2. Master's degree in Computer Science, Engineering, or a related technical field
3. Minimum 5 years of experience
4. At least 3+ years at Senior or Staff level, or equivalent
5. Technical Skills and Knowledge:

• Working knowledge of (3+ years) Python, Scala, C/C++, Java, C#, Go, or similar programming language
• Experience in deploying serverless data pipelines through containerization and terraform orchestration
• Experience in data ingestion using modern ETL frameworks (Airflow, Beam, Luigi, Spark, Nifi, etc.)
• Experience (3+ years) with SQL or NoSQL databases: PostgreSQL, SQL Server, Oracle, MySQL, Redis, MongoDB, Elasticsearch, Hive, HBase, Teradata, Cassandra, Redshift, Snowflake
• Experience with Cloud platforms (AWS, Azure, GCP) or Kubernetes via Terraform, with understanding of failover, high-availability, and scalability
• Experience with DevOps, CI/CD, Github Actions, version control, and git workflows
• Experience in scaling and optimizing SQL and ETL pipelines in data lake and warehouse environments
• Strong fundamentals in data structures, algorithms, automated testing, object-oriented programming, performance analysis, and computer architecture implications
• Experience working with agile principles

Interpersonal Skills:

• Strong communication skills, ability to explain technical solutions to colleagues and clients
• Curiosity, proactivity, and critical thinking
• Ability to collaborate across levels and regions