Lead Data Engineer

American Family Insurance; Madison (WI); On-site; USD 131,000 - 220,000; Full time

Job summary

American Family Insurance, based in Madison, WI, seeks a Lead Data Engineer to guide an engineering team in delivering scalable data solutions. You will oversee data pipelines, ensure quality and privacy, and collaborate with architecture and development teams to drive reusable, robust code.

Experts in GCP, SQL/NoSQL, and Python ETL are required, with strong knowledge of Hadoop/Spark and data modeling. Portfolios include security and operational excellence within insurance data contexts.

Qualifications

• Demonstrated customer-driven solutions or service experience.
• Must have GCP experience for this position.
• Strong SQL/NoSQL knowledge with various data stores.
• Extensive Python for ETL and data-driven solutions.
• Able to generalise code with design patterns.
• High-quality, reusable code and libraries.
• Experience with Hadoop or Spark for distributed processing.
• Knowledge of ML toolkits such as sklearn, SparkML, or H2O.
• Solid data understanding in data-rich industries like insurance/finance.
• Experience with dimensional modeling and star schemas.
• Strong DB internals understanding (indexes, logs, transactions).
• Experience with IaC tools like Docker, CloudFormation, Terraform.
• Experience with CI/CD workflows (Jenkins, CI/CD, Git).
• Experience with web services.

Responsibilities

• Lead an engineering team to meet project deadlines and priorities.
• Supervise data engineering team members and activities.
• Ensure data quality, security, privacy, and integrity across the lifecycle.
• Document critical workflows and operational aspects.
• Develop understanding of data sources, granularity, and availability.
• Provide technical oversight to architecture and development teams.
• Create scalable code to load and manipulate data in the warehouse.
• Facilitate communication with project teams and stakeholders.

Skills

GCP experience

SQL/NoSQL

Python ETL

Data modeling

Big data (Hadoop/Spark)

Open source ML toolkits

Data governance/security/privacy

Infrastructure as code

Docker

CloudFormation / Terraform

CI/CD (Jenkins, Git)

Web services consumption

Tools

Docker

CloudFormation

Terraform

Jenkins

Git

Job description

Position Compensation Range: $131,000.00 - $220,000.00

Pay Rate Type: Salary Compensation may vary based on the job level and your geographic work location.

Relocation support is offered for eligible candidates.

This position has approval to transfer a valid H1 visa.

Primary Accountabilities

• Leads an engineering team to meet project deadlines and priorities.
• Supervises assigned data engineering team members & activities.
• Ensures the quality, completeness, security, privacy, and integrity of data throughout the data lifecycle.
• Documents critical workflows and operational support aspects of team’s responsibilities
• Develops deep understanding of data sources, granularity, availability, and limitations.
• Provides proactive technical oversight and advice to application architecture and development teams fostering re-use, design for scale, stability, and operational efficiency of data/analytical solutions.
• Creates maintainable, scalable code to load and manipulate data in the data warehouse.
• Facilitates communication upward and across project teams and business stakeholders.

Specialized Knowledge & Skills Requirements

• Demonstrated experience providing customer-driven solutions, support or service.
• Must have GCP experience for this positon.
• In-depth knowledge of SQL or NoSQL and experience using a variety of data stores (e.g. RDBMS, analytic database, scalable document stores)
• Extensive hands‑on Python programming experience, with an emphasis towards building ETL workflows and data‑driven solutions.
• Able to employ design patterns and generalise code to address common use cases.
• Capable of authoring robust, high quality, reusable code and contributing to the division’s inventory of libraries.
• Expertise in big data batch computing tools (e.g. Hadoop or Spark), with demonstrated experience developing distributed data processing solutions.
• Knowledge of open source machine learning toolkits, such as sklearn, SparkML, or H2O.
• Solid data understanding and business acumen in the data rich industries like insurance or financial
• Applied knowledge of data modeling principles (e.g. dimensional modelling and star schemas).
• Strong understanding of database internals, such as indexes, binary logging, and transactions.
• Experience using tools for infrastructure-as-code (e.g. Docker, CloudFormation, Terraform, etc.)
• Experience with software engineering tools and workflows (i.e. Jenkins, CI/CD, git).
• Practical experience authoring and consuming web services.

Licenses / Travel / Physical / Working Conditions

• Licenses: Not Applicable.
• Travel Requirements: Up to 10%.
• Physical Requirements: Work that primarily involves sitting/standing.
• Working Conditions: Not Applicable.

Additional Information

To ensure a strong start, all employees participate in our New Employee Orientation during their first week. This experience is held in person at our Madison, WI Headquarters or one of our AmFam core locations to help you connect with our mission, meet key team members and build relationships that support your growth. At times, sessions may be delivered virtually based on scheduling and availability.

Offer to selected candidate will be made contingent on the results of applicable background checks Offering to selected candidate is contingent on signing a non‑disclosure agreement for proprietary information, trade secrets, and inventions Sponsorship will not be considered for this position unless specified in the posting.

We provide benefits that support your physical, emotional, and financial wellbeing. You will have access to comprehensive medical, dental, vision and wellbeing benefits that enable you to take care of your health. We also offer a competitive 401(k) contribution, a pension plan, an annual incentive, 9 paid holidays and a paid time off program (23 days accrued annually for full‑time employees). In addition, our student loan repayment program and paid‑family leave are available to support our employees and their families. Interns and contingent workers are not eligible for American Family Insurance Group benefits.

We are an equal opportunity employer. It is our policy to comply with all applicable federal, state and local laws pertaining to non‑discrimination, non‑harassment and equal opportunity. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. American Family Insurance is committed to the full inclusion of all qualified individuals. If a reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please email [e-mail] to request a reasonable accommodation.

About American Family Insurance

American Family Insurance, based in Madison, Wis., has been serving customers since 1927. We inspire, protect and restore dreams through our insurance products, exceptional service from our agency owners and employees, community investments and creative partnerships. American Family Insurance is the nation’s 12th‑largest property/casualty insurance company, ranking No. 202 on the Fortune 500 list. American Family has a current 'a+' credit rating through AM Best, and has distinction as a "Standing the Test of Time" company, which is represented by having an AM Best financial strength rating of 'A' or better for at least 75 years. The company offers American Family‑brand products through a network of exclusive agency owners as well as nationwide through key strategic partnerships. American Family Insurance also includes Homesite and Main Street America Insurance. The company has approximately 10,500 employees nationwide.