Lead Data Engineer

Strategic Staffing Solutions; Charlotte (NC); On-site; USD 124,000 - 138,000; Full time

Job summary

Strategic Staffing Solutions in Charlotte, NC is seeking a Lead Data Engineer to lead the design, architecture, and implementation of enterprise-scale data solutions on AWS. You will shape data pipelines, ensure governance, and partner with stakeholders to define scalable data strategies.

You will mentor engineers, perform code reviews, and drive Terraform-based IaC, as well as CI/CD workflows using GitHub Actions. This role emphasizes reliability, security, and AI-enabled data workflows.

Qualifications

• 8+ years in Data Engineering with 5+ years on AWS.
• Expertise building data lakes and data warehouses on AWS.
• Strong Python, PySpark, and SQL skills.
• Experience designing scalable, secure data architectures.
• Proven leadership, mentoring, and code-review capabilities.
• Familiarity with CI/CD and IaC practices in cloud environments.

Responsibilities

• Lead design and implementation of enterprise-scale data solutions on AWS.
• Develop scalable data pipelines, ETL, ingestion, and orchestration workflows.
• Architect and oversee data lake and data warehouse initiatives (Redshift, Lake Formation, S3).
• Provide hands-on technical leadership, reviews, and mentorship.
• Drive security, governance, observability, and operational excellence.
• Implement Terraform IaC across multiple AWS accounts and environments.
• Establish CI/CD using GitHub Actions and manage production support.

Skills

AWS

PySpark

Python

Data Modeling

Data Warehousing

ETL

Kafka

Terraform

GitHub Actions

Security & Governance

Tools

Confluent Kafka

AWS EMR

Redshift

Lake Formation

Apache Airflow

Databricks

Job description

Job Title: Lead Data Engineer

Location: Charlotte, NC

Duration: 12 months

Pay Rate: $90 - $100/HR (W2 Only)

Job/Role Description:

• Lead the design, architecture, and implementation of enterprise-scale data engineering solutions across AWS cloud environments.
• Design, develop, and optimize scalable, resilient data pipelines, ETL processes, data ingestion frameworks, and orchestration workflows.
• Architect and oversee enterprise data lake and data warehouse solutions using AWS Lake Formation, Amazon Redshift, Amazon Athena, S3, and related AWS technologies.
• Collaborate with Lead Developers, Data Scientists, Architects, Product Owners, and business stakeholders to define technical strategy and scalable data solutions.
• Provide hands-on technical leadership, engineering oversight, code reviews, and mentorship to Data Engineers and development teams.
• Drive architectural decisions in partnership with Data and Solution Architects to ensure scalability, security, reliability, performance, and maintainability.
• Design and support Kafka-based streaming and event-driven data architectures, preferably using Confluent Kafka.
• Develop distributed data processing solutions using Python, PySpark, AWS EMR, and other cloud-native technologies.
• Establish engineering standards and best practices for data modeling, ETL frameworks, pipeline reliability, monitoring, observability, and operational excellence.
• Lead end-to-end solution delivery while ensuring alignment with business requirements, enterprise architecture standards, security controls, and regulatory requirements.
• Design and implement Infrastructure as Code solutions using Terraform across multiple AWS accounts and environments.
• Develop and maintain CI/CD frameworks using GitHub and GitHub Actions.
• Oversee production support and operational management of AWS-based data platforms, including root-cause analysis, troubleshooting, and performance optimization.
• Champion data governance, metadata management, data quality, observability, and data stewardship practices across platforms and teams.
• Identify opportunities to modernize data architecture and improve operational efficiency through automation and cloud-native technologies.
• Support the development of AI-ready data pipelines and machine learning workflows, including feature engineering and MLOps.
• Design data solutions capable of supporting Generative AI applications, intelligent search, RAG architectures, LLMs, and vector databases.
• Drive technical decision-making and clearly communicate complex data architecture concepts to both technical and non-technical stakeholders.

Required Qualifications

• 8+ years of Data Engineering experience, including at least 5+ years of extensive experience working within AWS environments.
• Expert-level experience with AWS services including S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, and Step Functions.
• Strong hands-on experience designing and implementing enterprise-scale data lakes and data warehouses using AWS Lake Formation, Amazon Redshift, and Amazon Athena.
• Advanced Python development experience with extensive hands-on use of PySpark.
• Advanced SQL expertise, including query optimization, large-scale data processing, and enterprise data warehousing.
• Strong data modeling experience, including dimensional modeling, Data Vault, and other enterprise data modeling techniques.
• Deep experience designing, developing, and optimizing scalable and resilient data pipelines within AWS environments.
• Strong experience with distributed data processing frameworks, particularly PySpark and AWS EMR.
• Extensive knowledge of database management, performance tuning, and data architecture best practices.
• Experience with Kafka-based streaming architectures, preferably Confluent Kafka.
• Expertise with Infrastructure as Code using Terraform.
• Experience designing and implementing CI/CD frameworks using GitHub and GitHub Actions.
• Deep knowledge of AWS IAM roles, policies, governance, security controls, and cloud security best practices.
• Strong experience with workflow orchestration platforms such as AWS Step Functions, Apache Airflow, or equivalent technologies.
• Experience leading cloud migration, modernization, or enterprise data platform initiatives.
• Strong understanding of data governance, metadata management, data quality frameworks, observability, resiliency, and operational supportability.
• Experience creating AI applications using AWS Bedrock.
• Experience building AI-ready data pipelines and ML workflows, including feature engineering and MLOps.
• Knowledge of Generative AI technologies, LLMs, Retrieval-Augmented Generation (RAG), and vector databases.
• Experience working with cloud-based AI and data platforms such as AWS, Azure, or Databricks.
• Ability to lead hands-on development efforts while providing technical direction, engineering oversight, and code reviews across multiple initiatives.
• Experience establishing cloud development environments, infrastructure standards, security controls, and migration strategies across multiple accounts and environments.
• Proven ability to identify data gaps, develop strategic remediation plans, and implement scalable automation solutions.
• Experience designing highly reliable data pipelines with a strong emphasis on data quality, observability, resiliency, and operational support