Role overview
Build and support software for systems that identify, match, and link records across large, varied datasets. The work spans distributed batch processing and fast backend services for real-time identity lookups, with close collaboration across engineering, architecture, data, product, and security teams.
Responsibilities
- Design, implement, test, and maintain identity and entity resolution platforms and services.
- Develop batch and real-time workflows, including low-latency APIs and large-scale Spark applications.
- Create matching and record-linkage capabilities using normalization, fuzzy comparisons, confidence scoring, and identity consolidation.
- Improve performance and reliability across application code, SQL, data pipelines, and distributed workloads; investigate production issues.
- Add automated validation, monitoring, logging, and metrics to support accurate and dependable systems.
- Contribute to technical design and code reviews, guide other engineers, and coordinate implementation with partner teams.
Requirements
- At least five years of professional software development experience, including production systems.
- Strong Java, Python, and Scala skills, along with hands-on Apache Spark and SQL experience.
- Practical experience with AWS and Databricks for cloud-based data processing.
- Familiarity with identity or entity resolution, customer data consolidation, record linkage, or related matching systems.
- Ability to own complex work and collaborate with technical and product stakeholders; a relevant degree or equivalent experience.
Nice to have
- Experience building identity graphs or deterministic and probabilistic matching systems.
- Background with streaming architectures, graph data, containers, infrastructure as code, or observability tools.
- Experience handling large consumer, marketing, financial, advertising, or customer datasets.