Senior ML Engineer: Scalable Entity Matching

Gt Hq; Warszawa; Hybrid; PLN 240,000 - 360,000; Full time

Job summary

GT is seeking a Senior ML Engineer / Applied ML Engineer to build and evolve an internal entity resolution system handling hundreds of millions of records. You will develop ML models, matching logic, and scalable services using Python, SQL, Spark, and Airflow in production.

The role requires strong ML engineering experience, large-scale data pipelines, and expertise in algorithms and distributed systems to optimize accuracy, recall, and cost.

Qualifications

• 5+ years of software development with Python and strong CS fundamentals.
• Solid understanding of distributed systems and algorithms (partitioning, shuffles, joins).
• Experience building complex data pipelines or data systems.
• Strong SQL skills, ideally Snowflake or similar analytical DBs.
• Hands-on Spark – Databricks (PySpark/Scala) in production.
• Experience with AI/ML-assisted systems (embeddings, inference).

Responsibilities

• Design and improve entity matching, clustering, and deduplication algorithms at scale.
• Implement distributed matching approaches: blocking, multi-pass, nearest-neighbor, and similarity-based methods.
• Combine rule-based, statistical, and ML-assisted techniques, including embeddings.
• Optimize candidate generation and scoring for accuracy, recall, performance, and cost.
• Translate ideas into scalable Spark and SQL transformations for large data.
• Build and operate large-scale analytic systems processing hundreds of millions to billions of records.

Skills

Python

Distributed systems

SQL

Spark/Databricks

ML engineering

Complex data pipelines

Embeddings

Tools

Snowflake

Airflow

DBT

Job description

GT is seeking a Senior ML Engineer / Applied ML Engineer to build and evolve an internal entity resolution system handling hundreds of millions of records. You will develop ML models, matching logic, and scalable services using Python, SQL, Spark, and Airflow in production.

The role requires strong ML engineering experience, large-scale data pipelines, and expertise in algorithms and distributed systems to optimize accuracy, recall, and cost.