Senior AI Engineer, Unstructured AI

Collibra; New York (NY); Hybride; USD 204 000 - 255 000; Plein temps

Avantages offerts par ce poste

Hybrid work model

Equity

Bonus potential

Health coverage

401(k)/Pension

Résumé du poste

Collibra’s Unstructured AI team in New York is seeking a Senior AI Engineer to own end-to-end delivery of Unstructured AI systems, from prototype to production across enterprise environments. You will build full-stack pipelines that ingest and enrich large volumes of unstructured content (PDFs, contracts, reports), collaborate with xYC Founders, and advance context engineering and model-driven enrichment.

You will work in a hybrid NY office with a strong focus on governance, data quality, and

Qualifications

• Strong proficiency in Python for data processing, API development and integrations.
• Hands-on work with LLM-based and AI-driven enrichment models (e.g., classification, entity extraction, deduplication, PII detection).
• Production experience with Spark or comparable big data frameworks — tuned and debugged jobs at real scale.
• Experience shipping tested, reviewed production services rather than notebooks and enforcing gatekeeping.
• A track record of starting projects: define MVP, document assumptions, and ship without heavy PM intervention.
• Solid understanding of data pipelines, microservice architecture, and API design.
• Experience ingesting data from third-party enterprise sources (SharePoint/OneDrive, Salesforce, SaaS knowledge bases).
• Familiarity with metadata systems, data cataloging, or document AI workflows.
• Knowledge of model evaluation best practices.
• Experience with search relevance.
• A bachelor’s degree or equivalent related working experience is required.
• Agentic engineering in practice: guardrails, structured outputs, tests, and code-reviews.

Responsabilités

• Shipping complex systems under real ambiguity — often defining the scope and acceptance criteria yourself.
• Writing and reviewing production-grade backend code (Python, FastAPI).
• Building/deploying document-processing systems that handle large-scale, unstructured data environments.
• Integrating data from diverse enterprise data sources (SharePoint, Salesforce, internal APIs).
• Partnering across engineering, product, and sales teams to align from prototype to rollout.
• Occasionally working with modern frontend development.

Connaissances

Python

LLM enrichment

Spark

API design

Production services

Data pipelines

SharePoint integration

Salesforce integration

MVP shipping

Front-end work

Formation

Bachelor’s degree or equivalent

Description du poste

• Work at the forefront of context engineering - shaping how AI systems retrieve, structure, and leverage context to deliver accurate, high-quality results at scale.


• Own end-to-end technical delivery of Unstructured AI systems - from feature prototype to stable production across enterprise environments.


• Build and scale full-stack systems that ingest, process, and enrich large volumes of unstructured content from distributed enterprise silos (PDFs, contracts, reports, and other document types).


• Collaborate with the Best: Work closely with xYC Founders to understand complex business challenges and deliver Deasy to solve them. Be part of a dynamic team where ideas flow freely and creativity thrives.


• Learn and Lead: Stay ahead of the curve by engaging with the latest developments in machine learning and AI. Share knowledge and lead by example to maintain high building standards.

This is a hybrid role based in our New York office. Our hybrid model means you’ll work from the office at least two days each week. This setup helps us stay connected, work more closely together, and keep making progress as a team.

Joining Collibra’s Unstructured AI Team

• Work at the forefront of context engineering - shaping how AI systems retrieve, structure, and leverage context to deliver accurate, high-quality results at scale.


• Own end-to-end technical delivery of Unstructured AI systems - from feature prototype to stable production across enterprise environments.


• Build and scale full-stack systems that ingest, process, and enrich large volumes of unstructured content from distributed enterprise silos (PDFs, contracts, reports, and other document types).


• Collaborate with the Best: Work closely with xYC Founders to understand complex business challenges and deliver Deasy to solve them. Be part of a dynamic team where ideas flow freely and creativity thrives.


• Learn and Lead: Stay ahead of the curve by engaging with latest developments in machine learning and AI. Share knowledge and lead by example to maintain high building standards.

This is a hybrid role based in our New York office. Our hybrid model means you’ll work from the office at least two days each week. This setup helps us stay connected, work more closely together, and keep making progress as a team.

Senior AI Engineer at Collibra are responsible for

• Shipping complex systems under real ambiguity — often defining the scope and acceptance criteria yourself, not receiving them.


• Writing and reviewing production-grade backend code (Python, FastAPI) — you write the services you design, rather than handing implementation to someone else.


• Building/deploying document-processing systems that handle large-scale, unstructured data environments.


• Integrating data from diverse enterprise data sources (e.g., SharePoint, Salesforce, or internal APIs) to provide context for AI features.


• Partnering across engineering, product, and sales teams, ensuring alignment from prototype to rollout.


• Occasionally working with modern frontend development.

You have

• Strong proficiency in Python (data processing, API development, and integrations).


• Hands-on work with LLM-based and AI-driven enrichment models (e.g., classification, entity extraction, deduplication, PII detection).


• Production experience with Spark or comparable big data frameworks — you've tuned and debugged jobs at real scale, not just written ones that worked on sample data.


• Experience shipping tested, reviewed production services rather than notebooks — and the discipline to hold that line when a coding agent writes the first draft.


• A track record of starting things: a specific example where you took a vaguely-scoped problem, defined the MVP yourself, wrote down your assumptions, and shipped it — without a PM converting it into tickets first.


• Solid understanding of data pipelines, microservice architecture, and API design.


• Experience ingesting and processing data from third-party enterprise sources (e.g., SharePoint/OneDrive, Salesforce, and SaaS-based knowledge bases).


• Familiarity with metadata systems, data cataloging, or document AI workflows.


• Knowledge of model evaluation best practices.


• Experience with search relevance.


• A bachelor’s degree or equivalent related working experience is required.


• Agentic engineering in practice, not just tool usage: you can say where an agent loop earns its keep, what guardrails and structured outputs keep it in bounds, and how you manage its context. Same discipline when coding agents write for you — tests, review gates, repo conventions.


• This position is not eligible for visa sponsorship.

You Are

• Calm, structured decision-making under tight timelines or ambiguity.


• Capable of communicating clearly across engineering, product, and field teams, ensuring alignment from prototype to rollout.


• Experienced in spotting risks early and course-correcting without friction when delivery timelines are tight.


• Someone who cares deeply about data quality, precision, and governance.


• Strong communication and stakeholder-management skills across technical and business teams.

Measures of Success

• Within your first month, you will develop a deep understanding of our product vision and the unstructured data stack that powers it, shipping your first set of end-to-end features.


• Within your third month, you will take full ownership of technical delivery for key product areas, building robust capabilities that handle complex document processing and ensure a stable, high-performance experience for users interacting with diverse data sources.


• Within your sixth month, you will drive the development of ambitious, enterprise-grade AI product features that solve for data at scale, architecting the high-performance pipelines and advanced context engineering required to deliver accurate, reliable results.

Compensation for this role

The standard base salary range for this position is $204,000 - 255,000 per year. This position is not eligible for additional commission-based compensation. Salary offers are based on a combination of factors, including, but not limited to, experience, skills, and location.

In addition to base salary, we offer a competitive total rewards package, including bonus potential, equity for eligible roles, a Flex Fund monthly stipend, pension/401k plans, and more.

Benefits at Collibra

Collibra recognizes and values that everyone has different needs, interests, and life goals. We built our benefits program with flexibility in mind to support you and your loved ones through a diverse range of circumstances and life events. These flexible offerings sit on a foundation of competitive compensation, health coverage, and time off. Learn more about Collibra’s benefits.

We create inclusion and belonging through how we onboard, meet, connect, engage, and communicate. Learn more about diversity, equity, and inclusion at Collibra.

At Collibra, we’re proud to be an equal opportunity employer. We realize the key to creating a company with a world-class culture and employee experience comes from who we hire and creating a workplace that celebrates everyone.

With this, we proudly consider qualified applicants without regard to race, color, religion, creed, gender, national origin, age, disability, veteran status, sexual orientation, pregnancy, sex, gender identity, gender expression, genetic information, physical or mental disability, HIV status, registered domestic partner status, caregiver status, marital status, veteran or military status, citizenship status or any other legally protected category. If you have a need that requires accommodation, let us know by completing our Accommodations for Applicants form.