Data & AI Engineer - Equities Technology

Williamblair

Chicago (IL)

Hybrid

USD 120,000 - 180,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

William Blair in Chicago seeks a Data & AI Engineer to design, build, and maintain scalable data and AI-enabled platforms across Trading, Research, and Sales. You will move data and AI solutions from concept to production, ensuring reliability, security, and compliance with firm standards, while collaborating with product and analytics teams.

This role focuses on Azure Databricks, Synapse, and LLM-based integrations to empower decision-making across the equities business within a hybrid Chicago

Qualifications

  • Bachelor's degree in information technology or related field.
  • 4-6+ years of hands-on experience with Databricks, Spark, Azure Data Factory, Azure Synapse, Python, ADLS, and Azure Functions.
  • Strong experience designing and managing Synapse/ADF pipelines, activities, and linked services.
  • Proven ability to build full and incremental data loads from Azure and on-prem data sources.
  • Experience implementing LLM-enabled or RAG-based solutions in production environments.
  • Proficiency with REST APIs, data gateways, and third-party system integrations.
  • Strong SQL skills with data modeling, analytical storage, and performance tuning.
  • Experience with Azure DevOps and YAML-based CI/CD pipelines.
  • Familiarity with Azure Key Vault, automation runbooks, Logic Apps, and cloud security best practices.
  • Strong communication, collaboration, and problem-solving skills.

Responsibilities

  • Design, build, and maintain scalable data pipelines using Databricks, Azure Data Factory, and Azure Synapse
  • Implement robust ETL/ELT workflows for structured and unstructured financial data across on-prem and cloud platforms
  • Ensure data quality, lineage, governance, security, and observability across pipelines and storage layers
  • Design and optimize data models and analytical schemas (star/snowflake, partitioning, distribution strategies)
  • Build reusable ingestion and transformation frameworks to support analytics and AI workloads
  • Build and deploy AI-enabled services, agents, and workflows supporting equity research, trading, sales, and client service use cases
  • Implement LLM-based and agentic patterns, including Retrieval-Augmented Generation (RAG), using proprietary firm data
  • Integrate AI capabilities into existing applications and platforms via APIs, batch jobs, and event-driven workflows
  • Partner with Data Science and business stakeholders to translate AI concepts into production-ready solutions
  • Productionalize AI and Data Science POCs into secure, scalable, and monitored services suitable for regulated environments

Job description

Solutions for Today's Challenges. Vision for Tomorrow's Opportunities.

Join William Blair, the Premier Global Partnership.

We're seeking a Data & AI Engineer to design, build, and maintain intelligent, scalable data and AI-enabled platforms supporting our Equities business across Trading, Research, and Sales. This role spans the full software development lifecycle and is responsible for moving data and AI solutions from concept through production, while ensuring reliability, security, and compliance with firm standards.

The ideal candidate brings strong data engineering fundamentals combined with hands-on experience integrating AI and LLM-based capabilities into production systems in a regulated environment.

The role will be based in Chicago with a hybrid work schedule.

Key Responsibilities
Data Engineering & Platform Foundations
  • Design, build, and maintain scalable data pipelines using Databricks, Azure Data Factory, and Azure Synapse
  • Implement robust ETL/ELT workflows for structured and unstructured financial data across on-prem and cloud platforms
  • Ensure data quality, lineage, governance, security, and observability across all pipelines and storage layers
  • Design and optimize data models and analytical schemas (star/snowflake, partitioning, distribution strategies)
  • Build reusable ingestion and transformation frameworks to support analytics and AI workloads
AI Integration & Agentic Workflows
  • Build and deploy AI-enabled services, agents, and workflows supporting equity research, trading, sales, and client service use cases
  • Implement LLM-based and agentic patterns, including Retrieval-Augmented Generation (RAG), using proprietary firm data
  • Integrate AI capabilities into existing applications and platforms via APIs, batch jobs, and event-driven workflows
  • Partner with Data Science and business stakeholders to translate AI concepts into production-ready solutions
AI Enablement & Productionization
  • Productionalize AI and Data Science POCs into secure, scalable, and monitored services suitable for regulated environments
  • Optimize prompts, embeddings, orchestration logic, and inference workflows for accuracy, performance, cost, and reliability
  • Ensure AI solutions meet firm standards for security, auditability, explainability, and compliance
  • Establish operational practices for AI solutions, including monitoring, alerting, lifecycle management, and runbooks
Cloud Modernization & DevOps
  • Support migration of legacy data and application solutions (SQL, SSIS, Synapse, custom ETLs) to modern Azure-native architectures
  • Implement CI/CD pipelines using Azure DevOps and YAML-based CI/CD pipelines, following infrastructure-as-code and automation best practices
  • Leverage Azure services (Functions, Key Vault, Logic Apps, Automation Runbooks) to build secure, reliable, and maintainable solutions
  • Develop operational dashboards to monitor pipeline health, SLAs, system performance, and cloud spend
Collaboration & Delivery
  • Work closely with Product Managers, Software Engineers, Data Scientists, and business stakeholders to define functional and technical requirements
  • Participate in Agile ceremonies, sprint planning, and retrospectives
  • Lead testing of new and modified software, analyze issues, and resolve defects efficiently
  • Document technical designs, integrations, and maintain operational playbooks and runbooks
  • Monitor industry trends in data engineering, cloud platforms, and AI, and recommend adoption where aligned with firm strategy
Essential Qualifications
  • Bachelor's degree in information technology or related field
  • 4-6+ years of hands-on experience with Databricks, Spark, Azure Data Factory, Azure Synapse, Python, ADLS, and Azure Functions
  • Strong experience designing and managing Synapse/ADF pipelines, activities, and linked services
  • Proven ability to build full and incremental data loads from Azure and on-prem data sources
  • Experience designing reusable ETL/ELT frameworks and orchestrating pipelines across ADF/Synapse/Databricks
  • Experience implementing LLM-enabled or RAG-based solutions in production environments
  • Proficiency with REST APIs, data gateways, and third-party system integrations
  • Strong SQL skills with experience in data modeling, analytical storage, and performance tuning
  • Experience with Azure DevOps and YAML-based CI/CD pipelines
  • Familiarity with Azure Key Vault, automation runbooks, Logic Apps, and cloud security best practices
  • Strong communication, collaboration, and problem-solving skills
Preferred Qualifications
  • Hands-on experience with Azure AI Services, including OpenAI, embeddings
  • Familiarity with Microsoft Fabric (future roadmap alignment)
  • Proficiency in ASP.NET, .NET Core, or C#
  • Background in financial services, capital markets, or other regulated environments

A reasonable estimate of the current base salary range at time of posting is below. Base salary does not include other forms of compensation or benefits. Actual base salary within the specified range is bas

#LI-CH

#LI-HYBRID

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Hybrid Data & AI Platform Engineer (Azure)
Hybrid Data & AI Platform Engineer (Azure)

Williamblair • Chicago (IL)

Hybrid
USD 120,000 - 180,000
Senior Data Scientist - AI Engineer
Senior Data Scientist - AI Engineer

Williamblair • Chicago (IL)

Hybrid
USD 150,000 - 170,000
Medical, dental and vision coverage
Employer paid short & long-term dis-
401(k)
+5
Data & Ai Engineer
Data & Ai Engineer

Tata Consultancy Services Limited • Boston (MA)

On-site
USD 100,000 - 120,000
Lead Data Scientist – AI Engineer
Lead Data Scientist – AI Engineer

William Blair • Chicago (IL)

On-site
USD 186,000 - 222,000
Senior Data Scientist – AI Engineer
Senior Data Scientist – AI Engineer

William Blair • Chicago (IL)

On-site
USD 157,000 - 187,000
Medical insurance
Dental insurance
Vision coverage
+1
Data & AI Engineer
Data & AI Engineer

1P284 THE CARLYLE GROUP EMPLOYEE CO., LLC • Ova (KY)

On-site
USD 160,000 - 180,000
Retirement benefits
Health insurance
Life and disability insurance
+4
Data Science and Analytics Engineer
Data Science and Analytics Engineer

Madison-Davis • New York (NY)

On-site
USD 150,000 - 230,000
Senior Data Engineer
Senior Data Engineer

Burtch Works • Chicago (IL)

Hybrid
USD 148,000 - 164,000
Data Engineering Senior Manager
Data Engineering Senior Manager

Sidley Austin • Chicago (IL)

Hybrid
USD 148,000 - 211,000
Lead Data Solution and AI Engineer
Lead Data Solution and AI Engineer

Gallagher • Chicago (IL)

Remote
Confidential
Competitive compensation
Comprehensive benefits programs
Career development opportunities
+1