Remote GenAI Data Engineer: Snowflake & AWS Pipelines

Insight Global; Chicago (IL); À distance; USD 180 000 - 240 000; Plein temps

Résumé du poste

Insight Global in Chicago is seeking a Senior AI Data Engineer to join the Data Engineering team and deliver data and AI capabilities for key initiatives. You will partner with Data Product Management, and Data Science to design, build, and operationalize scalable ingestion, transformation pipelines, GenAI-enabled pipelines in Snowflake and AWS.

You will lead ingestion and transformation, implement Snowflake Cortex AI workflows, develop prompt/versioning, and contribute to observability with

Qualifications

• 8+ years of total work experience with at least 5+ years in data engineering.
• 2+ years hands-on experience developing and deploying Generative AI applications.
• Strong Snowflake expertise including SQL, Tasks/Streams, stored procedures, and performance tuning.
• Hands-on experience with Snowflake Cortex AI (or equivalent production LLM platform).
• Experience developing Snowflake Native Apps using the Snowflake Native App Framework (Snowpark with Python/Scala, packaging, versioning, upgrades, secure distribution).
• Strong AWS experience (S3, Lambda, Glue, Step Functions, DynamoDB, EC2, CloudFormation, and Kinesis).
• Proficiency in Python (production-grade code, testing, packaging).
• Experience with CI/CD tools (GitHub Actions/Jenkins) and IaC (Terraform/CloudFormation).
• Experience building structured extraction pipelines from free-text inputs at scale.

Responsabilités

• Design and implement ingestion pipelines from multiple source systems into Snowflake using AWS services and Python.
• Build and optimize SQL-based transformation pipelines for text cleansing, normalization, and deduplication.
• Implement Snowflake Cortex AI workflows for classification, extraction, and structured data generation.
• Develop prompt versioning, response parsing, validation, and fallback logic.
• Snowflake Native App Development: build and deploy apps, manage objects, permissions, versioning, upgrades, and release workflows; integrate Snowpark logic and CI/CD.
• Define, instrument, and track data and model quality KPIs (precision, recall, coverage, drift, error rates).
• Integrate Arize (or comparable platform) for observability, monitoring, and root-cause analysis of model behavior.
• Support production rollout and operational readiness: deployment automation, runbooks, documentation, and handoff to internal teams.

Connaissances

GenAI applications

Data engineering

Python

SQL

CI/CD

Observability

Outils

Snowflake

Snowflake Cortex AI

Snowpark

AWS

GitHub Actions

Jenkins

Terraform

CloudFormation

Description du poste

Insight Global in Chicago is seeking a Senior AI Data Engineer to join the Data Engineering team and deliver data and AI capabilities for key initiatives. You will partner with Data Product Management, and Data Science to design, build, and operationalize scalable ingestion, transformation pipelines, GenAI-enabled pipelines in Snowflake and AWS.

You will lead ingestion and transformation, implement Snowflake Cortex AI workflows, develop prompt/versioning, and contribute to observability with