Senior Data Engineer

TechDigital Group; United States; On-site; USD 100,000 - 140,000; Full time

Job summary

A leading technology firm in the United States is seeking an experienced Data Engineer to design, develop, and maintain scalable data solutions using Snowflake and Microsoft Azure services. The ideal candidate will have over 7 years of experience in data engineering, with strong skills in ETL processes, data quality, and governance. This role includes optimizing data pipelines and ensuring data integration across multiple sources.

Qualifications

• Minimum of 7 years of professional experience in data engineering.
• Deep expertise in Snowflake data warehousing.
• Extensive experience with ETL/ELT tools and techniques.

Responsibilities

• Design, develop, and maintain scalable data models within Snowflake.
• Build and optimize ETL/ELT pipelines using various tools.
• Implement data integration solutions connecting various data sources.

Skills

Data Warehouse Design and Development

ETL/ELT Pipeline Development

Azure Data Services integration

Data Quality and Governance

SQL Proficiency

Education

Bachelor's or Master's degree in Computer Science

Tools

Snowflake

Azure Data Factory

Azure Synapse Analytics

Azure Databricks

Job description

Responsibilities:
* Data Warehouse Design and Development (Snowflake): Design, develop, and maintain scalable and efficient data models and data warehouses within the Snowflake environment.
* ETL/ELT Pipeline Development: Design, build, and optimize robust ETL/ELT pipelines using various tools and technologies to ingest, transform, and load data from diverse sources into Snowflake and other data stores.
* Azure Data Services: Utilize and integrate various Azure data services, including but not limited to Azure Data Factory, Azure Synapse Analytics (SQL Data Warehouse, Data Lake), Azure Databricks, Azure Blob Storage, Azure SQL Database, and Azure Data Lake Storage (ADLS).
* Data Integration: Develop and implement data integration solutions to connect various internal and external data sources.
* Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and Snowflake queries.
* Data Quality and Governance: Implement and enforce data quality checks, data governance policies, and data security measures.
* Monitoring and Alerting: Design and implement monitoring and alerting systems for data pipelines and data infrastructure to ensure reliability and timely issueresolution.

Qualifications:
* Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
* Minimum of 7 years of professional experience in data engineering.
* Deep expertise in Snowflake data warehousing:
* Experience with Snowflake utilities and tools (e.g., SnowSQL, Snowpipe, Tasks).
* Strong experience with Microsoft Azure data services:
* Hands-on experience with Azure Data Factory for building ETL/ELT pipelines.
* Familiarity with other Azure data services such as Azure Synapse Analytics, Azure Databricks, Azure Data Lake Storage, and Azure SQL Database.
* Extensive experience with ETL/ELT tools and techniques:
* Proven ability to design, develop, and maintain complex ETL/ELT workflows.
* Experience with various data integration patterns and technologies.
* Strong proficiency in SQL and experience working with different database systems