Remote Senior Data Engineer: Gen AI Powered Pipelines

Resonate; Northern (KY); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

401(k) match

Open PTO

Remote-first environment

Job summary

Resonate, a leading AI-powered consumer data company, seeks a senior data engineer to design and maintain terabyte-to-petabyte scale ETL/ELT pipelines on Spark/Scala in AWS. You will optimize performance and cost, work with Grafana monitoring, and apply Gen AI to accelerate development.

You will collaborate with senior engineers and product management to move requests from design to production, while supporting on-call operations and incident response.

Qualifications

• Around five years or more of professional experience in software engineering, data engineering, or a closely related field.
• At least three years hands-on with Spark and Scala, specifically the DataFrame and Dataset APIs.
• Proven experience tuning Spark applications at multi-terabyte or petabyte scale.
• Real debugging and problem-solving experience in the big data ecosystem, including production failures.
• Working knowledge of modern cloud stacks for processing big data - AWS (EMR, S3, Lambda), Kafka, Snowflake, Grafana, Hadoop, Elastic Stack, and Docker.
• Strong understanding of the full software development lifecycle and solution architecture.
• Interest in using generative AI to accelerate data engineering work.

Responsibilities

• Designing, developing, and maintaining ETL/ELT pipelines powering the business on Spark/Scala on AWS.
• Tuning Spark applications at multi-terabyte/petabyte scale for performance and cost.
• Collaborating with senior engineers and product management from design to production.
• Monitoring pipeline health, diagnosing data quality issues, and fixing root causes.
• Writing clean, testable code with unit and integration tests; leveraging Gen AI to speed up work.
• Participating in code reviews, technical design, and sprint planning; contributing architectural perspective.
• Supporting production ops including on-call rotation and incident response.

Skills

Spark

Scala

AWS

Data pipelines

Gen AI

Debugging

Cloud

Monitoring

Docker

Education

Bachelor's degree in CS/CE or equivalent

Tools

Grafana

Snowflake

EMR

Kafka

OpenAI tools

Job description

Resonate, a leading AI-powered consumer data company, seeks a senior data engineer to design and maintain terabyte-to-petabyte scale ETL/ELT pipelines on Spark/Scala in AWS. You will optimize performance and cost, work with Grafana monitoring, and apply Gen AI to accelerate development.

You will collaborate with senior engineers and product management to move requests from design to production, while supporting on-call operations and incident response.