Lead Data Engineer

TechDigital Group; Dallas (TX); On-site; USD 120,000 - 150,000; Full time

Job summary

A technology company in Dallas is seeking an experienced software developer with expertise in Python, PySpark, and Azure. The role requires 10+ years in software development, with proven experience in Databricks and Snowflake. You will mentor other engineers and contribute to solution architecture while working in a collaborative team environment. Competitive salary and opportunities for professional growth are offered.

Qualifications

• 10+ years experience in hands-on software development.
• 3+ years experience in Azure cloud platform.
• 3+ years experience in using tools like Databricks, Snowflake, python, pyspark, AKS.
• 2+ years experience in MySQL and NoSQL databases.
• 2+ years experience in tools such as Airflow, CI/CD using Jenkins.

Responsibilities

• Understanding containerization and performance tuning in Data Pipelines.
• Mentoring and leading other engineers from a technical capacity.
• Solution Architecting experience.

Skills

Python

PySpark

Snowflake

Databricks

Azure

MySQL

NoSQL

Airflow

CI/CD using Jenkins

Problem solving

API Development

Job description

Overview

Must Have - Python, PySpark, Snowflake & Databricks, Azure

• 10+ years experience in hands-on software development
• 3+ years experience in Azure cloud platform
• 3+ years experience in using tools like Databricks, Snowflake, python, pyspark, AKS
• 2+ years experience in MySQL and NoSQL databases
• 2+ years experience in tools such as Airflow, CI/CD using Jenkins
• Understanding containerization and performance tuning in Data Pipelines
• 3+ years experience in development using Python, Spark
• Python, PySpark, Snowflake & Databricks experience
• Excellent analysis, process, problem solving and critical thinking skills
• Ability to change and adapt quickly, mentoring and leading other engineers from a technical capacity
• Solution Architecting experience
• Excellent Team Player
• API Development (REST)