Lead Data Engineer

Carex Consulting Group; Madison (WI); Hybrid; USD 140,000 - 180,000; Full time

Job summary

Carex Consulting Group is seeking a Lead Data Engineer to provide technical leadership and hands-on development of scalable, cloud-based data solutions. The role focuses on AI, marketing, and customer analytics while mentoring engineers and establishing data engineering best practices.

The ideal candidate has 8–10 years of experience with expert GCP, SQL, and Python skills, and can communicate with both technical teams and business stakeholders. Hybrid work is available in Madison or Boston.

Qualifications

• 8-10 years of relevant data engineering, software engineering, or related experience.
• Expert-level Google Cloud Platform (GCP) experience.
• Expert-level SQL and Python skills.
• Experience with distributed and big-data processing technologies such as Hadoop or Spark.
• Applied knowledge of data modeling concepts including dimensional modeling and star schemas.
• Experience with infrastructure-as-code and containerization technologies such as Terraform, CloudFormation, or Docker.
• Strong communication skills and ability to translate business needs into technical solutions.

Responsibilities

• Lead and mentor data engineers while ensuring project deadlines and quality standards.
• Design, develop, and maintain scalable data engineering solutions on GCP.
• Build maintainable Python code for ETL workflows and data-driven applications.
• Develop and optimize data loading and transformation in data warehouse environments.
• Provide technical guidance and establish engineering best practices for the team.
• Ensure data quality, security, privacy, and integrity across the lifecycle.
• Collaborate with architecture and development teams to promote reusable designs and operability.
• Support projects involving AI, marketing, and customer analytics.

Skills

GCP

SQL

Python

ETL

Data modeling

Big data

Spark

CI/CD

Terraform

Docker

Education

Bachelor’s degree or higher in a related field

Tools

Terraform

CloudFormation

Docker

Git

Jenkins

Job description

Carex is partnering with a Insurance industry partner to hire a Lead Data Engineer to provide technical leadership while remaining highly hands-on in the development of scalable, cloud-based data solutions. This senior-level role will focus on new initiatives across AI, marketing, and customer analytics while mentoring engineers and establishing data engineering best practices.

The ideal candidate brings 8-10 years of experience and expert-level knowledge of Google Cloud Platform (GCP), SQL, and Python. This role is approximately 70-75% hands-on development and 25-30% mentoring and technical leadership, requiring someone who can effectively communicate with both technical teams and business stakeholders.

Location

Hybrid - Madison, WI or Boston, MA, with approximately 2-3 days per week / 10 days per month in the office.

What You'll Do

• Lead and mentor data engineers while helping the team meet project deadlines, technical priorities, and quality standards.
• Design, develop, and maintain scalable data engineering solutions within Google Cloud Platform (GCP).
• Build maintainable, reusable Python code for ETL workflows, data processing, and data-driven applications.
• Develop and optimize solutions for loading, transforming, and manipulating data within data warehouse environments.
• Provide technical guidance and establish engineering best practices for other members of the team.
• Ensure data quality, completeness, security, privacy, and integrity throughout the data lifecycle.
• Partner with application architecture and development teams to promote reusable designs, scalability, stability, and operational efficiency.
• Support new projects involving AI, marketing, customer analytics, and data technology.
• Develop a deep understanding of data sources, including their granularity, availability, limitations, and appropriate use cases.
• Design and support distributed data-processing solutions using cloud and big-data technologies.
• Document critical workflows, technical designs, and operational support processes.
• Communicate technical concepts, recommendations, and project updates effectively across engineering teams and business stakeholders.

What You'll Bring

• 8-10 years of relevant data engineering, software engineering, or related experience.
• Expert-level Google Cloud Platform (GCP) experience.
• Expert-level SQL and Python skills.
• Extensive hands-on Python development experience, particularly building ETL workflows and data-driven solutions.
• In-depth knowledge of SQL and/or NoSQL technologies and experience with multiple data stores, including relational databases, analytical databases, and scalable document stores.
• Experience writing robust, high-quality, reusable, and maintainable production code.
• Ability to apply software design patterns and generalize solutions across common engineering use cases.
• Experience with distributed and big-data processing technologies such as Hadoop or Spark.
• Applied knowledge of data modeling concepts, including dimensional modeling and star schemas.
• Strong understanding of database internals, including indexes, transactions, and binary logging.
• Experience with infrastructure-as-code and containerization technologies such as Terraform, CloudFormation, or Docker.
• Experience with modern software engineering workflows and tools, including Git, Jenkins, and CI/CD.
• Practical experience developing and consuming web services.
• Strong communication skills with the ability to collaborate directly with business stakeholders and translate business needs into technical solutions.
• Experience mentoring engineers and providing technical guidance and engineering best practices.
• Working knowledge of distributed cloud systems and machine learning engineering is a plus.
• Familiarity with machine learning frameworks such as scikit-learn, Spark ML, or H2O is a plus.
• Experience working with data-intensive business domains such as insurance or financial services is a plus.

Carex Consulting Group is an equal opportunity employer. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender identity or Veteran status.