Lead Data Engineer

Snowrelic Inc; Charlotte (NC); On-site; USD 75,000 - 234,000; Full time

Job summary

A leading IT Services company is seeking a Mid-Senior Level Data Engineer in Charlotte, NC. The ideal candidate will design and implement scalable data pipelines using Azure Databricks, optimizing ETL workflows and ensuring compliance with data governance regulations. This full-time position offers competitive compensation and opportunities for growth within the rapidly evolving data engineering field.

Responsibilities

• Design and implement scalable data pipelines on Azure Databricks.
• Optimize ETL/ELT workflows and troubleshoot Spark jobs.
• Lead migration of Azure SQL to Azure Databricks.

Skills

Data governance

Efficiency in data processing

Apache Spark

Azure services

Data compliance

Tools

Databricks

Azure SQL

Delta Lake

Job description

1 week ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

• Design and implement scalable data pipelines and architectures on Azure Databricks.
• Optimize ETL/ELT workflows, ensuring efficiency in data processing, storage, and retrieval.
• Leverage Apache Spark, Delta Lake, and Azure-native services to build high-performance data solutions.
• Ensure best practices in data governance, security, and compliance within Azure environments.
• Troubleshoot and fine-tune Spark jobs for optimal performance and cost efficiency.
• Lead the migration of Azure SQL to Azure Databricks, ensuring a seamless transition of data workloads.
• Design and implement scalable data pipelines to extract, transform, and load (ETL/ELT) data from Azure SQL into Databricks Delta Lake.
• Optimize Azure SQL queries and indexing strategies before migration to enhance performance in Databricks.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  IT Services and IT Consulting

Referrals increase your chances of interviewing at Snowrelic Inc by 2x

Get notified about new Data Engineer jobs in Charlotte, NC.

Charlotte, NC $124,800.00-$234,000.00 2 weeks ago

Charlotte, NC $123,494.00-$169,804.00 3 hours ago

Charlotte Metro $100,000.00-$180,000.00 1 day ago

Charlotte, NC $75,300.00-$143,100.00 5 days ago

Charlotte, NC $85,000.00-$150,000.00 5 days ago

Charlotte, NC $90,000.00-$115,000.00 13 hours ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.