Lead Data Engineer (Python, AWS, SQL, Snowflake, Databricks)

Spectraforce Technologies; Plano (TX); Hybrid; USD 130,000 - 170,000; Full time

Job summary

Spectraforce Technologies is seeking a Lead Data Engineer to lead data engineering efforts on a critical project in Plano, TX. You will design and build scalable data pipelines, mentor teammates, and collaborate with agile teams across the organization.

Focus areas include Python and SQL development, cloud data warehousing with Snowflake/Databricks, and real-time data processing. Onsite three days per week with a hybrid work model.

Qualifications

• 5+ years of experience in Python and SQL development.
• 5+ years of public cloud experience (AWS, GCP, Azure).
• 5+ years with distributed data/compute tools (MapReduce, Hadoop, EMR, Kafka, Spark).
• 5+ years with data warehouses/services (Snowflake, Databricks, Redshift).
• 5+ years experience with data modeling for data warehousing.

Responsibilities

• Collaborate with Agile teams to design, develop, test, and support data solutions using data movement tools and technologies.
• Lead a team of developers experienced in data movement, distributed computing, and full-stack systems.
• Utilize Python, SQL, and Open Source RDBMS/NoSQL databases and cloud data warehousing services like Snowflake and Databricks.
• Optimize information systems for end-users and downstream consumers with solid data design practices.
• Mentor engineers and stay current with emerging tech through communities and experimentation.

Skills

Python

SQL

Data modeling

Data warehousing

Open source databases

Tools

AWS

GCP

Azure

MapReduce

Hadoop

EMR

Kafka

Spark

Snowflake

Databricks

Redshift

Job description

Title:

Lead Data Engineer (Python, AWS, SQL, Snowflake, Databricks)

Location:

Plano, TX | Onsite three days per week

Duration:

7 Months

Job Description:

At client, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about data, data modeling, data quality, governance, permissible use, and building data pipelines with emerging technologies.

Project:

As a lead data engineer, you will have the opportunity to work on the CODE Metrics team developing a developer productivity metrics platform used by engineers across the enterprise to measure developer effectiveness and happiness.

What You'll Do:

• Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies
• As a lead developer, work with a team of developers who have deep experience in data movement, distributed computing, and full-stack systems
• Utilize programming languages like Python, SQL, and Open Source RDBMS and NoSQL databases and cloud-based data warehousing services such as Snowflake and Databricks
• Optimize information systems for end-users and downstream application consumers by using sound data design practices
• Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community

Tech Stack: Data Engineering skillset

• 10+ years experience in application development using Python and SQL
• 5+ years experience with public cloud: AWS, GCP, Azure
• 5+ years experience with distributed data/computing tools (MapReduce, Hadoop, EMR, Kafka, Spark).
• 5+ years experience working with real-time data and streaming applications.
• 5+ years experience with data warehouses and services (Snowflake, Databricks, Redshift)
• 5+ years experience with data modeling for data warehousing
• Experience leveraging interactive AI tooling (Claude Code, Windsurf, GitHub Copilot) to accelerate software delivery

Nice to Have:

• Cube for semantic data layer
• Tableau/Quick for BI