Senior Data Engineer - GCP, Spark & ETL Lead

American Family Insurance; Keene (NH); On-site; USD 131,000 - 220,000; Full time

Job summary

American Family Insurance in Keene, NH seeks a Senior Data Engineer to lead our data engineering efforts on a modern data platform. You will design, build, and maintain ETL pipelines, collaborate with application teams, and ensure data quality and security across the data lifecycle.

The role requires strong GCP, Python, SQL/NoSQL, and big data tooling experience, plus the ability to scale solutions and promote reusable code across projects.

Qualifications

• Demonstrated experience providing customer-driven solutions, support or service.
• Must have GCP experience for this positon.
• In-depth knowledge of SQL or NoSQL and experience using a variety of data stores (e.g. RDBMS, analytic database, scalable document stores)
• Extensive hands-on Python programming experience, with an emphasis towards building ETL workflows and data-driven solutions.
• Able to employ design patterns and generalize code to address common use cases.
• Capable of authoring robust, high quality, reusable code and contributing to the division's inventory of libraries.
• Expertise in big data batch computing tools (e.g. Hadoop or Spark), with demonstrated experience developing distributed data processing solutions.
• Knowledge of open source machine learning toolkits, such as sklearn, SparkML, or H2O.
• Solid data understanding and business acumen in the data rich industries like insurance or financial
• Applied knowledge of data modeling principles (e.g. dimensional modeling and star schemas).
• Strong understanding of database internals, such as indexes, binary logging, and transactions.
• Experience using tools for infrastructure-as-code (e.g. Docker, CloudFormation, Terraform, etc.)
• Experience with software engineering tools and workflows (i.e. Jenkins, CI/CD, git).
• Practical experience authoring and consuming web services.

Responsibilities

• Leads an engineering team to meet project deadlines and priorities.
• Supervises assigned data engineering team members & activities.
• Ensures the quality, completeness, security, privacy, and integrity of data throughout the data lifecycle.
• Documents critical workflows and operational support aspects of team's responsibilities
• Develops deep understanding of data sources, granularity, availability, and limitations.
• Provides proactive technical oversight and advice to application architecture and development teams fostering re-use, design for scale, stability, and operational efficiency of data/analytical solutions.
• Creates maintainable, scalable code to load and manipulate data in the data warehouse.
• Facilitates communication upward and across project teams and business stakeholders.

Skills

GCP

SQL/NoSQL

Python ETL

Hadoop/Spark

Design patterns

Data modeling

Database internals

CI/CD

Docker

IaC tools

Jenkins

Git

Spark

Web services

Tools

Docker

CloudFormation

Terraform

Jenkins

Git

Spark

Job description

American Family Insurance in Keene, NH seeks a Senior Data Engineer to lead our data engineering efforts on a modern data platform. You will design, build, and maintain ETL pipelines, collaborate with application teams, and ensure data quality and security across the data lifecycle.

The role requires strong GCP, Python, SQL/NoSQL, and big data tooling experience, plus the ability to scale solutions and promote reusable code across projects.