Lead Data / AI Native Engineer

TechDoQuest; O’Fallon (MO); On-site; USD 150,000 - 210,000; Full time

Job summary

TechDoQuest seeks an senior data/softw are engineer to drive enterprise data platforms and scalable pipelines. You will lead cross-functional initiatives, shaping architecture, and mentoring teams to deliver cloud-native analytics and AI-powered solutions.

Expect hands-on leadership, deep experience with Spark/PySpark, Databricks, and AWS, plus a track record building data lakehouse architectures for enterprise-scale use cases.

Qualifications

• 8+ years of data engineering and software engineering experience.
• 3+ years leading complex data initiatives or engineering teams.
• Proven experience building enterprise scale data platforms and pipelines.
• Experience delivering cloud native analytics and AI solutions.

Responsibilities

• Lead the design and delivery of enterprise data platforms and pipelines.
• Mentor and guide data engineering teams on best practices.
• Deliver cloud-native analytics and AI-powered solutions.
• Build scalable ETL/ELT frameworks and data lakehouse architectures.

Skills

Data engineering experience

Software engineering experience

Leadership of data initiatives

Enterprise-scale data platforms

AI solutions experience

Tools

Spark

PySpark

Databricks

AWS EMR

GitHub Copilot

Claude Code CLI

Job description

• 8+ years of Data Engineering and Software Engineering experience.
• 3+ years leading complex data engineering initiatives or engineering teams.
• Proven experience building enterprise scale data platforms and pipelines.
• Experience delivering cloud native analytics and AI solutions.

Technical Skills

• Build highly available, secure, and scalable cloud native data solutions.
• Develop ETL/ELT frameworks for ingesting, transforming, and publishing enterprise data.
• Implement high volume distributed processing using Spark and PySpark.
• Leverage GitHub Copilot, Claude Code CLI, and AI assisted development tools to accelerate engineering productivity.
• Build Proof of Concepts (POCs) and MVPs using AI powered development approaches.
• Develop cloud native solutions leveraging AWS and Databricks ecosystems.
• Build scalable architectures for data ingestion, processing, storage, and analytics.
• Strong experience in building enterprise grade data pipelines, modern Lakehouse architectures, AI enabled data products, and cloud native platforms leveraging Databricks, AWS EMR, Spark, PySpark, and AI native engineering tools such as GitHub Copilot and Claude Code.
• The role aligns with enterprise initiatives around AI assisted engineering, intelligent automation, and AI powered development workflows