Lead Azure Data Engineer

KPI Partners; United States; On-site; USD 140,000 - 180,000; Full time

Job summary

KPI Partners is looking for a Lead Data Engineer to spearhead data transformation projects. The role involves leading data engineering efforts using Azure technologies while mentoring a dynamic team. This is an excellent opportunity for experienced professionals passionate about large-scale data systems.

Qualifications

• 10+ years in IT Industry including 5+ years in data engineering
• Experience with large-scale data solutions using PySpark and SQL.
• Knowledge of ETL processes and data warehousing fundamentals.

Responsibilities

• Lead design and development of data pipelines.
• Collaborate with teams to meet business requirements.
• Mentor junior engineers and ensure best practices.

Skills

Data Engineering Leadership

Collaboration

Data Warehousing

Continuous Learning

Mentorship

SQL

PySpark

ETL processes

Education

BA/BS in Computer Science

MBA from a Top 50 business school

Tools

Azure Data Factory

Databricks

ADLS

Synapse/DW

Azure SQL DB

Job description

This range is provided by KPI Partners. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$140,000.00/yr - $180,000.00/yr

About KPI Partners

KPI Partners is a 5 times Gartner recognized data, analytics, and AI consulting company. We are leaders in data engineering on Azure, AWS, Google, Snowflake and Databricks. Founded in 2006, KPI has over 400 consultants and has successfully delivered over 1,000 projects to our clients. We are looking for skilled data engineers who want to work with the best team in data engineering.

Title: Lead Data Engineer

Work Hours: PST Zone (8 AM - 5 PM)

Job Type: Full-time/Hourly W2

About the Role:

We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team at KPI, working on challenging and multi-year data transformation projects. This is an excellent opportunity for a talented data engineer to play a key role in building innovative data solutions using Azure Native Services and related technologies. If you are passionate about working with large-scale data systems and enjoy solving complex engineering problems, this role is for you.

Key Responsibilities:

• Data Engineering Leadership: Lead the design, development, and implementation of data pipelines and solutions using PySpark, SQL, and related technologies.
• Collaboration: Work closely with cross-functional teams to understand business requirements and translate them into robust data solutions.
• Data Warehousing: Design and implement data warehousing solutions, ensuring scalability, performance, and reliability.
• Continuous Learning: Stay up to date with modern technologies and trends in data engineering and apply them to improve our data platform.
• Mentorship: Provide guidance and mentorship to junior data engineers, ensuring best practices in coding, design, and development.

Must-Have Skills & Qualifications:

• Minimum 10+ years of overall experience in IT Industry.
• 5+ years of experience in data engineering, with a strong background in building large-scale data solutions.
• 3+ years of hands-on experience developing and implementing data pipelines using PySpark, Azure Data Factory and Databricks
• Proven expertise in SQL for querying, manipulating, and analyzing large datasets.
• Strong knowledge of ETL processes and data warehousing fundamentals.
• Self-motivated and independent, with a “let’s get this done” mindset and the ability to thrive in a fast-paced and dynamic environment.

Good-to-Have Skills:

• Databricks Certification is a plus.
• Experience with Azure-native services such as ADLS, Synapse/DW, Azure SQL DB, and Microsoft Fabric.
• Familiarity with Event Hub, IoT Hub, Azure Stream Analytics, Azure Analysis Service, and Cosmos DB.
• Knowledge of SAP Hana.
• Familiarity with DevOps practices and CI/CD deployment pipelines.

Education:

BA/BS in Computer Science with an MBA from a Top 50 business school in the US.

Apply Now!

If you’re ready to take on a senior role and work on transformative projects with a talented team, we encourage you to apply today. Please send your resume and contact information to [e-mail]

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  Business Consulting and Services

Referrals increase your chances of interviewing at KPI Partners by 2x

Get notified about new Data Engineer jobs in United States.

United States $170,000.00-$720,000.00 2 weeks ago

United States $170,000.00-$720,000.00 2 weeks ago

United States $170,000.00-$720,000.00 1 week ago

United States $170,000.00-$720,000.00 3 days ago

San Francisco, CA $126,100.00-$186,800.00 2 weeks ago

New York, NY $140,000.00-$190,000.00 3 days ago

New York, NY $115,000.00-$145,000.00 3 hours ago

Austin, TX $140,960.00-$193,820.00 5 hours ago

United States $180,000.00-$220,000.00 2 weeks ago

United States $184,500.00-$217,100.00 2 weeks ago

New York, NY $139,500.00-$271,566.94 2 weeks ago

United States $145,000.00-$204,000.00 5 days ago

United States $97,482.00-$140,000.00 3 weeks ago

California, United States $139,500.00-$271,566.94 2 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.