Lead Data Engineer - AWS

Tiger Analytics Inc.; Dallas (TX); On-site; USD 140,000 - 180,000; Full time

Job summary

Tiger Analytics Inc. is seeking an experienced Senior Data Engineer to join our team, focusing on building scalable Generative AI architectures within the AWS ecosystem. You will architect the data foundations that power LLMs and autonomous agents for our Fortune 500 partners.

The role centers on designing GenAI data pipelines, vector search, and modern data stack components while delivering scalable, production-grade infrastructure across AWS services.

Qualifications

• 8–12 years of Data Engineering experience with a heavy focus on the AWS Cloud stack.
• Deep hands-on experience with Glue, Athena, EMR, and Redshift.
• Proficiency in LangChain or LlamaIndex integrated with AWS services to handle unstructured data (text, images, PDFs).
• Experience deploying infrastructure using AWS CDK or Terraform.
• Advanced SQL, Python and PySpark skills tailored for distributed processing on AWS.

Responsibilities

• GenAI Infrastructure: Architect data pipelines using Amazon Bedrock and SageMaker to build, deploy, and scale Generative AI applications.
• Vector Foundations: Implement and optimize vector search capabilities using OpenSearch Serverless or vector engines for RAG.
• Serverless Data Engineering: Build scalable, event-driven ETL pipelines using AWS Lambda, AWS Glue, and Amazon Kinesis.
• Modern Data Stack: Manage large-scale data lakehouses leveraging S3, Lake Formation, and Redshift.
• LLM Ops: Automate fine-tuning and deployment of foundation models with Step Functions and SageMaker Pipelines.

Skills

Advanced SQL

Python

PySpark

Distributed processing on AWS

Tools

AWS Glue

Amazon Athena

Amazon EMR

Amazon Redshift

LangChain

LlamaIndex

AWS CDK

Terraform

Job description

Tiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for multiple Fortune 500 companies, enabling them to generate business value from data. Our business value and leadership has been recognized by various market research firms, including Forrester and Gartner. We are looking for top-notch talent as we continue to build the best global analytics consulting team in the world. Tiger Analytics is seeking an experienced Senior Data Engineer to join our team, specifically focused on building scalable Generative AI architectures within the AWS ecosystem. You will architect the data foundations that power LLMs and autonomous agents for our Fortune 500 partners.

Key Responsibilities:

• GenAI Infrastructure: Architect data pipelines using Amazon Bedrock and Amazon SageMaker to build, deploy, and scale Generative AI applications.
• Vector Foundations: Implement and optimize vector search capabilities using Amazon OpenSearch Serverless or specialized vector engines for RAG (Retrieval-Augmented Generation).
• Serverless Data Engineering: Build highly scalable, event-driven ETL pipelines using AWS Lambda, AWS Glue, and Amazon Kinesis.
• Modern Data Stack: Manage large-scale data lakehouses leveraging Amazon S3, AWS Lake Formation, and Amazon Redshift.
• LLM Ops: Integrate AWS Step Functions and SageMaker Pipelines to automate the fine-tuning and deployment of foundation models.

Requirements

• Experience: 8-12 years in Data Engineering with a heavy focus on the AWS Cloud stack.
• AWS Expertise: Deep hands-on experience with Glue, Athena, EMR, and Redshift.
• AI/ML Tools: Proficiency in LangChain or LlamaIndex integrated with AWS services to handle unstructured data (text, images, PDFs).
• DevOps & IAC: Experience deploying infrastructure using AWS CDK or Terraform.
• Core Skills: Advanced SQL, Python and PySpark skills tailored for distributed processing on AWS.

Benefits

This position offers an excellent opportunity for significant career development in a fast-growing and challenging entrepreneurial environment with a high degree of individual responsibility. Tiger Analytics provides equal employment opportunities to applicants and employees without regard to race, color, religion, age, sex, sexual orientation, gender identity/expression, pregnancy, national origin, ancestry, marital status, protected veteran status, disability status, or any other basis as protected by federal, state, or local law.