Senior Data Engineer: Azure Data Pipelines & GenAI

Tiger Analytics Inc.; United States; Remote; USD 110,000 - 150,000; Full time

Job summary

Tiger Analytics Inc. is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines and data infrastructure on AWS cloud. You will collaborate with data scientists, analysts, and AI teams to support analytics and Generative AI initiatives.

You will work with Azure data services and control data governance, security, and automated deployments, while optimizing performance and costs. Join a fast-growing analytics partner recognized by market leaders.

Qualifications

• Design, build, and maintain scalable ETL/ELT pipelines on Azure.
• Ingest and process high-volume batch and real-time streaming data.
• Collaborate with BI/analytics teams to model star/snowflake schemas for reporting.
• Implement data governance, automated data quality validation, and end-to-end lineage tracking.
• Secure data pipelines and storage using RBAC, Key Vaults, and encryption standards.
• Automate infrastructure deployments and code releases with Azure DevOps and GitHub Actions.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines usingAzure Data Factory,Azure Synapse, andAzure Databricks.
• Architect and manage secure enterprise cloud storage, includingADLSGen2, Delta Lake, and relational/NoSQL databases.
• Ingest and process both high-volume batch data and real-time streaming workloads.
• Optimize database queries, PySpark jobs, and data pipeline performance to minimize execution time and cloud costs.
• Collaborate with BI and analytics teams to model dimensional data marts (Star/Snowflake schemas) for reporting.
• Implement data governance, automated data quality validation, and end-to-end lineage tracking (e.g., viaMicrosoft Purview).
• Secure data pipelines and storage using Azure Role-Based Access Control (RBAC), Key Vaults, and encryption standards.
• Automate infrastructure deployments and code releases usingAzure DevOps, GitHub Actions, and CI/CD best practices

Tools

Azure Data Factory

Azure Synapse

Azure Databricks

ADLS Gen2

Delta Lake

Microsoft Purview

Azure DevOps

GitHub Actions

CI/CD

PySpark

Job description

Tiger Analytics Inc. is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines and data infrastructure on AWS cloud. You will collaborate with data scientists, analysts, and AI teams to support analytics and Generative AI initiatives.

You will work with Azure data services and control data governance, security, and automated deployments, while optimizing performance and costs. Join a fast-growing analytics partner recognized by market leaders.