Data & AI Engineer

Solomon Page; New York (NY); Hybrid; USD 170,000 - 190,000; Full time

Job summary

Solomon Page is seeking a combined AI Engineer and Data Engineer to join a global investment firm to transform data management and analytics using AI. Hybrid schedule in New York City or Washington, DC, with residency required; visa sponsorship not available.

Compensation is $170K-$190K salary plus bonus. Report to the Senior AI & Data Architect and deliver AI-ready data pipelines and semantic models at enterprise scale.

Qualifications

• 6+ years of technical experience in data or AI engineering.
• Hands-on with generative AI or AI/ML in production.
• Experience in private equity, asset management, or financial services.
• Proven ability to implement RAG, vector stores, and semantic components for LLM- or agent-based applications.

Responsibilities

• Build and operate AI-ready data pipelines for LLMs and agents.
• Implement RAG components with vector stores and grounding.
• Develop interfaces for safe querying on enterprise data.
• Operationalize feature stores, evaluation datasets, and reusable AI data products.
• Design semantic models and data contracts for self-service analytics.
• Ensure scalable ELT, streaming, and transformation pipelines with dbt/Fivetran/Snowflake.
• Write clean Python and SQL with CI/CD and tests.
• Collaborate with architecture, data science, governance teams.

Skills

Python

SQL

Data engineering

AI engineering

Cloud

Communication

Education

Concentration in computer science / data engineering / information systems

Certifications in cloud / data engineering / AI/ML

Tools

dbt

Fivetran

Snowflake

OpenAI/Anthropic APIs

LangChain

LlamaIndex

PySpark

Databricks

Spark

Job description

We are seeking a combination AI Engineer and Data Engineer to join a talented team tasked with transforming how data is managed and interpreted throughout the company using the latest AI Technology. Join a global investment firm with $485 billion of assets under management, across 678 investment vehicles. One of the world's largest and most successful investment firms, with more than 2,500 professionals operating in 28 offices in North America, Europe, the Middle East, Asia and Australia. Investing primarily across Global Private Equity and Global Credit – and deep expertise across industries, markets, and geographies. The Data & AI Engineer is an experienced, hands-on engineer who turns data and AI architecture into working production systems. Reporting to the Senior AI & Data Architect, this role is responsible for building and operating the pipelines, semantic layers, retrieval systems, and AI-ready data products that power analytics, automation, LLMs, agents, and generative AI applications across the firm.

• Hybrid schedule either in New York City or Washington, DC. Local residency required.
• We are unable to transfer a visa or sponsor for permanent residency. Only Green Card holders or US Citizens will be considered.
• Compensation is a $170K-$190K salary plus bonus.

Department Description

The Data & AI Engineer sits within the Enterprise Technology & Data organization and supports firm-wide data and AI initiatives spanning investment platforms, portfolio operations, investor relations, and corporate functions. The role operates within a federated data operating model, partnering with domain engineering teams to implement shared platforms and reusable patterns for data and AI under the technical direction of the Senior AI & Data Architect.

Position Summary

The role requires deep, hands-on expertise across modern data engineering and applied AI engineering. The Data & AI Engineer will implement retrieval-augmented generation (RAG) patterns, embedding and indexing pipelines, vector stores, and semantic models alongside core ELT, streaming, and analytical pipelines — treating LLMs, agents, and copilots as first-class consumers of the data platform.

This is a senior individual-contributor engineering role that executes against architectural standards, contributes to their evolution through hands-on learning, and partners closely with data science, AI engineering, governance, and domain teams to deliver trusted, AI-consumable data at enterprise scale.

What Success Looks Like: In the first 12 months, this role will deliver foundational AI-ready data pipelines and retrieval components defined in the target-state architecture, productionize one or more priority RAG or agent-grounding use cases, and establish reusable engineering patterns that other domain teams can adopt across the federated data platform.

Responsibilities

• Build and operate AI-ready data pipelines — embedding generation, chunking, indexing, and refresh workflows — that make enterprise data reliably retrievable by LLMs, agents, and generative AI applications.
• Implement retrieval-augmented generation (RAG) components, including vector store integrations, hybrid search, re-ranking, and grounding logic, against architectural patterns defined by the Senior AI & Data Architect.
• Develop and maintain tool and function interfaces that allow agents and copilots to query and act on enterprise data safely, with appropriate guardrails, logging, and evaluation hooks.
• Partner with Data Science and AI Engineering teams to operationalize feature stores, evaluation datasets, and reusable AI data products.
• Contribute to semantic and context engineering work that powers natural-language analytics, conversational reporting, and AI-driven insights for business users.
• Design, build, and maintain production-grade ELT, streaming, and transformation pipelines using tools such as dbt, Fivetran and Snowflake.
• Implement ingestion, modeling, and consumption patterns that meet enterprise standards for scalability, performance, security, resiliency, and cost efficiency.
• Write clean, well-tested Python and SQL; apply software engineering best practices including version control, code review, CI/CD, modular design, and automated testing.
• Productionize new sources and domains under the federated operating model, partnering with domain data engineers to apply shared platform capabilities consistently.

Semantic Layer & Data Product Development (≈20%)

• Implement semantic models, data contracts, and analytical/dimensional models that enable trusted self-service analytics and reliable AI grounding.
• Build and maintain reusable data products with clear ownership, documented contracts, and contextual metadata suitable for both human and AI consumers.
• Collaborate with the Senior AI & Data Architect to refine and extend enterprise semantic standards based on what works in production.
• Support discovery and consumption tooling so that analysts, applications, and agents can find and use data products with minimal friction.

Data Quality, Observability & AI Trust (≈10%)

• Implement data quality checks, lineage capture, and pipeline observability across both data and AI workloads.
• Build logging, evaluation, and monitoring components for AI systems — including prompt and response capture, retrieval metrics, and model performance signals — in line with governance standards.
• Partner with Data Governance to operationalize metadata, stewardship, and access controls, ensuring AI systems consume enterprise data with the same rigor as human users.
• Surface issues early, propose remediations, and feed lessons learned back into architectural patterns.

Collaboration & Engineering Craft (≈5%)

• Participate in architectural design reviews and contribute hands-on engineering perspective to evolving patterns and standards.
• Mentor junior data engineers and analysts on modern data and AI engineering practices.
• Document patterns, write runbooks, and share knowledge across the federated organization to accelerate adoption of reusable platform capabilities.

Professional Experience

• 6+ years of overall relevant technical experience, required
• Experience in data engineering, analytics engineering, or platform engineering, with at least 1-2 years of direct, hands-on experience building generative AI or AI/ML systems in production.
• Experience in private equity, global credit, investment or asset management, or the financial industry is required.
• Proven experience implementing retrieval, grounding, and semantic components for LLM- or agent-based applications, including RAG pipelines, vector stores, embedding workflows, and structured tool use.
• Hands-on experience with one or more modern AI platforms and tooling categories (e.g., AWS Bedrock, Databricks ML, Snowflake Cortex, OpenAI/Anthropic APIs, LangChain/LlamaIndex or equivalents, MLflow, and vector databases such as Databricks Vector Search, pgvector, or Pinecone).
• Strong, demonstrable expertise in Python and SQL, with working knowledge of distributed processing frameworks (e.g., Spark).
• Deep, hands-on experience with modern data stacks - dbt, Fivetran, Snowflake - in AWS-based environments.
• Track record of building data pipelines and products whose consumers include AI systems, not only BI tools and human analysts.
• Experience operating within federated data operating models and complex, regulated enterprise environments; financial services experience preferred.
• Demonstrated AI-forward instinct: defaults to asking how AI changes what gets built, rather than whether AI can be added later.
• Fluency in current AI engineering patterns (RAG, agents, tool use, evaluations, guardrails, observability) and the practical trade-offs involved in shipping them.
• Strong engineering craft: clean code, automated testing, thoughtful design, and a bias toward production-quality systems over prototypes.
• Pragmatic, delivery-oriented mindset with strong attention to data quality, AI trust, and long-term maintainability; able to distinguish durable engineering decisions from AI hype.
• Collaborative partner to architects, data scientists, AI engineers, and domain teams; comfortable operating in a matrixed, federated organization and in high-visibility transformational initiatives.

Education & Certifications

• Concentration in computer science, data engineering, information systems, or a related field, preferred
• Relevant certifications in cloud, data engineering, analytics, or AI/ML are preferred

Who We Are

When people, ideas, and capital come together, opportunity expands across private markets. This belief has shaped how we invest for decades, fueling growth for companies and delivering performance for investors. Connecting people, ideas, and capital to fuel growth for companies and performance for investors, which range from public and private pension funds to wealthy individuals and families to sovereign wealth funds, unions and corporations.