Staff Data Engineer — Scale Real-Time Data Platforms

Intuit; Grand Rapids (MI); On-site; USD 189,000 - 274,000; Full time

Job summary

Intuit's Consumer Group is hiring a Staff Software Engineer - Data to build the data plane for AI-native experiences using Credit Karma and TurboTax data. You will design high-throughput data frameworks, streaming pipelines, and cloud-native services at scale.

The role emphasizes reliability, developer experience, and data governance, with contributions to Scala/Java services, gRPC, Akka, and Apache Beam on GCP.

Qualifications

• 7+ years of professional software engineering experience building backend services and data infrastructure in Scala, Java, or a similar JVM language.
• 7+ years of experience designing and operating high-throughput, low-latency distributed systems that process data at petabyte scale.
• 3+ years of experience with streaming and messaging platforms such as Apache Kafka, Google Pub/Sub, or equivalent.
• 3+ years of experience building data pipelines on a major cloud platform (GCP, AWS, or Azure), including services like Dataflow, BigQuery, Spanner, or their equivalents.
• Professional experience with RPC frameworks such as Finagle, gRPC, or Akka for building production-grade service-to-service communication.
• Strong understanding of software engineering best practices including CI/CD, version control (Git), code review, and automated testings.
• Experience building reusable frameworks, SDKs, or platform libraries consumed by other engineering teams — you think about developer experience as a product.
• Experience with Apache Beam (Dataflow) including custom transforms, side inputs, windowing strategies, and pipeline optimization.
• Experience with Change Data Capture (CDC) patterns, particularly MySQL binlog-based replication to analytical stores.
• Experience with data encryption at rest and in transit, including key management (KMS/GSM), SPIFFE/mTLS, and certificate authority integration.
• Experience with schema management, data governance, data catalog, data quality and lineage management frameworks in a large-scale production environment.
• Knowledge of AI/ML and GenAI technologies — LLMs, RAG, Semantic Search and Knowledge Graph to integrate our data with generative AI experiences.
• Familiarity with infrastructure-as-code, Kubernetes (GKE), and container-based deployment models.
• Track record of mentoring engineers and driving technical alignment across teams through design documents and architecture reviews.

Responsibilities

• Design, build, and maintain high-throughput, low-latency data frameworks used across Credit Karma's engineering organization, including ETL templates, persistence libraries, and streaming data pipelines
• Develop and extend Scala-based microservices and frameworks built on Finagle, Akka Streams, and gRPC that process petabytes of data daily
• Build and optimize cloud-native data pipelines on Google Cloud Platform using Dataflow (Apache Beam), Flink, Spark, Pub/Sub, BigQuery, and Spanner
• Own and evolve our Kafka-based streaming infrastructure — designing producers, consumers, and connectors that handle hundreds of terabytes of events per day with strict latency and durability guarantees
• Build persistence frameworks that provide a unified, type-safe API for reading and writing across Spanner, MySQL, and BigQuery
• Build agentic systems, semantic search, knowledge graph and personalization to disrupt how consumers do their taxes, money and personal finance using Consumer Data Plane
• Design and implement encryption, decryption, and fine-grained access control capabilities as reusable framework features, ensuring compliance with data governance requirements
• Create self-service developer tooling — CLI tools, templates, and onboarding automation — that reduces the time for other teams to adopt the data platform from weeks to hours
• Drive technical design through architecture reviews and Technical Design Documents (TDDs), influencing decisions across the broader Data organization
• Participate in on-call rotations and build observability (dashboards, alerting, metrics) into every system you ship
• You will own foundational infrastructure — the frameworks you build are the building blocks that every data pipeline and product feature at Credit Karma depends on
• You will work at real scale — hundreds of terabytes per day, millions of consumer profiles hydrated in real time, and strict SLAs that demand engineering rigor
• You will shape developer experience — designing the APIs, SDKs, and tooling that hundreds of engineers across the company interact with daily
• You will solve hard, novel problems — from building MySQL CDC pipelines with cloud-native encryption to designing Apache Beam framework SDKs that support both Java and Python
• You will be part of a high-impact, collaborative team with a strong culture of technical ownership and continuous learning

Skills

Scala/Java backend

Distributed systems design

Streaming platforms (Kafka, Pub/Sub)

Cloud data pipelines (GCP/AWS)

RPC frameworks (gRPC, Finagle)

CI/CD & version control

SDKs / framework development

Apache Beam / Dataflow

CDC patterns (MySQL binlog)

Data encryption at rest/in transit

Schema governance / data quality

AI/GenAI & knowledge graphs

Kubernetes / GKE

Mentoring engineers

Tools

Finagle

gRPC

Akka

Kafka

Pub/Sub

Dataflow

BigQuery

Spanner

Cloud Data services

Kubernetes

GKE

Job description

Intuit's Consumer Group is hiring a Staff Software Engineer - Data to build the data plane for AI-native experiences using Credit Karma and TurboTax data. You will design high-throughput data frameworks, streaming pipelines, and cloud-native services at scale.

The role emphasizes reliability, developer experience, and data governance, with contributions to Scala/Java services, gRPC, Akka, and Apache Beam on GCP.