Lead Consultant – Data On Cloud (AWS Data Lead)

Veriipro; Fort Mill (SC); On-site; USD 100,000 - 130,000; Full time

Job summary

Veriipro is looking for a highly experienced AWS Data Lead / Lead Consultant in Fort Mill, South Carolina. In this role, you will lead the design and development of scalable, cloud-based data engineering solutions, focusing on end-to-end ETL/ELT pipeline development using AWS services and modern data engineering tools.

You will be responsible for ensuring high performance, data quality, and governance of enterprise data platforms. The ideal candidate will provide technical leadership and mentorship to teams while collaborating with stakeholders to deliver reliable data solutions.

Qualifications

• Strong hands-on experience with AWS Glue and PySpark.
• Experience in building CI/CD pipelines and automation.
• Knowledge of data governance and performance optimization.

Responsibilities

• Lead design and development of scalable AWS-based ETL/ELT data pipelines.
• Build and manage scalable ETL/ELT workflows across cloud platforms.
• Collaborate with teams to deliver business-critical data solutions.

Skills

AWS Glue

PySpark

dbt (Data Build Tool)

Apache Airflow

Tools

AWS

Terraform

Job description

We are seeking a highly experienced AWS Data Lead / Lead Consultant with strong expertise in designing and implementing scalable cloud-based data engineering solutions. The ideal candidate will lead end-to-end ETL/ELT pipeline development using AWS services and modern data engineering tools, ensuring high performance, reliability, and data quality across enterprise data platforms.

Must-Have Skills

• AWS Glue (Strong hands-on experience)
• PySpark
• dbt (Data Build Tool)
• Apache Airflow

Detailed Responsibilities

• Design, develop, and optimize end-to-end data pipelines using AWS Glue with PySpark.
• Build and manage scalable ETL/ELT workflows and data integration solutions across cloud platforms.
• Develop orchestration workflows using Apache Airflow for scheduling and monitoring data pipelines.
• Design event-driven and scalable architectures using AWS services such as Lambda, EventBridge, S3, RDS (Aurora/PostgreSQL), and Kinesis/MSK.
• Work with relational and cloud-native databases including Snowflake, PostgreSQL, Aurora, and MS SQL Server.
• Implement robust data modeling practices and write optimized SQL queries for analytics and reporting.
• Build and maintain CI/CD pipelines and implement Infrastructure as Code (Terraform or similar tools).
• Ensure high data quality, governance, and performance optimization across all data workflows.
• Collaborate with cross-functional teams and stakeholders to deliver business-critical data solutions.
• Provide technical leadership and mentorship to engineering teams.

Top 3 Key Responsibilities

• Lead the design and development of scalable AWS-based ETL/ELT data pipelines using Glue, PySpark, and Airflow.
• Architect and implement cloud data solutions leveraging AWS services and modern data engineering practices.
• Drive CI/CD, automation, and best practices for reliable, high-quality data platform delivery.