Lead Data Engineers – Azure/ Databricks/ Snowflake

Exusia, Inc.; Snowflake (AZ); Remote; USD 80,000 - 130,000; Full time

Job summary

Join a cutting-edge digital transformation consultancy as a Lead Data Engineer, where you will harness the power of data to drive insights and solutions. This role offers the opportunity to design and develop robust ETL processes, optimize data storage solutions, and collaborate with a talented team of data professionals. With a focus on Azure technologies and a commitment to innovation, you'll play a key role in delivering high-quality data-driven insights that empower clients across various industries. If you are passionate about data analytics and thrive in a dynamic environment, this is the perfect opportunity to elevate your career and make a significant impact.

Qualifications

• 6+ years of experience in Azure Databricks and data analytics.
• Strong background in ETL processes and Python development.
• Experience with data governance and cloud technologies.

Responsibilities

• Design and maintain data pipelines using Azure Data Factory and Databricks.
• Collaborate with stakeholders to meet data requirements.
• Monitor and optimize data workflows for performance.

Skills

Azure Databricks

ETL/ELT pipelines

Python

Data Governance

Data Analysis

Agile Methodologies

Data Management

Problem-Solving

Education

Bachelor of Engineering

Bachelor of Science in Computer Science or Business Information Technology

Tools

Azure Data Factory

Azure Data Lake

Azure SQL Database

Azure Cosmos DB

Databricks

Snowflake

Job description

About The Position

Department: Sales and Delivery Team - Empower

Industry: Information Technology & Services, Computer Software, Management Consulting

Location: US Remote (US citizens, Candidates based in US with Valid Visa will be preferred)

Experience Range: 6 - 14 years

Basic Qualification: Bachelor of Engineering or Equivalent

Exusia, a cutting-edge digital transformation consultancy, is looking for top talent in Azure, Databricks, and Snowflake Data Warehouse Application Developers to join our global delivery team's Industry Analytics practice in India.

What’s the Role?

This is a full-time job to work with Exusia's clients to design and develop system components for the extraction and transformation of data from various source systems to target applications. Responsibilities include creating data structures or data models that support data across all stages of the extract, transform, and load process. Candidates should have good hands-on experience in implementing parallel and sequential jobs using Snowflake, Azure, Databricks, Spark, or Scala. This role involves analyzing, developing, monitoring, and administering the ETL process using any tool. Experience working with an on-shore/off-shore delivery model is a big plus.

The Lead Data Engineer will implement monitoring solutions, optimize ETL jobs for performance, and provide comprehensive support from data ingestion to final output. With in-depth knowledge of the Databricks platform and strong analytical skills, this position will significantly enhance the department's ability to deliver high-quality, data-driven insights and solutions.

Criteria for the Role!

1. Have a minimum of 6+ years’ experience working as an Azure Databricks Developer in a data analytics domain.
2. 4+ years of experience in Azure services and Databricks (ADLS, ADF, DevOps, etc.).
3. 10+ years of experience in designing and implementing ETL/ELT pipelines using ADF, integrating with other Azure services.
4. 7+ years of Python development experience, with the capability of developing own libraries.
5. 4+ years of experience in Snowflake or SQL experience (No-SQL experience is a plus).
6. Experience designing, building, and maintaining data processing systems.
7. Experience with or knowledge of Agile Software Development methodologies.
8. Education Qualification – Bachelor of Science (preferably in Computer and Information Sciences or Business Information Technology) or an Engineering degree in the above areas.

Responsibilities:

1. Design, develop, and maintain data pipelines and ETL processes using Azure Data Factory and Azure Databricks.
2. Build and optimize data storage solutions using Azure Data Lake, Azure SQL Database, and Azure Cosmos DB or Vector DB.
3. Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.
4. Implement data quality checks, data governance, and security best practices across data platforms.
5. Monitor, troubleshoot, and optimize data workflows for performance and scalability.
6. Automate data integration and transformation processes using Azure DevOps and CI/CD pipelines.
7. Stay up-to-date with emerging Azure technologies and data engineering trends.
8. Deep understanding of data management, data governance, cloud, and AI technologies, and their application in a business context.
9. Excellent problem-solving and troubleshooting skills.
10. Excellent oral and written communication skills.
11. Strong analytical and problem-solving skills.
12. Data experience working with multiple, large datasets / big data.
13. Demonstrated experience in effectively working with multiple projects and activities in a timely, organized manner.
14. Demonstrated experience in organizing and prioritizing work, meeting deadlines, and working independently as well as collaboratively with others.
15. Experience with Excel and PowerPoint.

Requirements

Nice-to-have Skills:

1. 2+ years of experience with Azure stack, e.g., Azure Kubernetes Service (AKS), Cosmos DB.
2. Understanding of cloud streaming techniques such as GCP, Snowpipe.
3. Experience in designing API development and integrating with React JS within the cloud platform.
4. Leverage modern technologies, such as AI and ML.
5. Must have strong DWH and ETL basics.
6. Basic cloud knowledge - Storage and Compute.
7. Work with business stakeholders to gather and analyze business requirements, building a solid understanding of the Data Analytics domain.
8. Document, discuss, and resolve business, data, and reporting issues within the team, across functional teams, and with business stakeholders.
9. Support all phases of the software development lifecycle (SDLC).
10. Present written and verbal data analysis findings to both the project team and business stakeholders.
11. Manage changing priorities and work on multiple projects concurrently.
12. Be self-motivated and proactive in a fast-paced environment and flexible to work.

About Exusia

Exusia (http://exusia.com/) is a global technology consulting company that empowers its clients to gain a competitive edge by accelerating business objectives and providing strategy and solutions in data management and analytics. The company has established its leadership position by solving some of the world's largest and most complex data problems in the financial, healthcare, telecommunications, and high technology industries.

Exusia’s mission is to transform the world through the innovative use of information.

Exusia was recognized by Inc. 5000 and by Crain’s publications as one of the fastest-growing privately held companies in the world. Since the company’s founding in 2012, Exusia has experienced impressive revenue growth and has expanded its operations in the Americas, Asia, Africa, and the UK. Exusia has recently also been recognized by publications such as the CIO Review, Industry Era, Insight Success, and the CIO Bulletin for the company’s innovation in IT Services, Telecommunications, and Healthcare industries and its entrepreneurship. The company is headquartered in Miami, Florida, United States, with development centers in Pune, Hyderabad, Bengaluru, and Gurugram, India.