Python Test Lead with AI – C2C jobs – Hybrid– San Francisco, CA

US staffing Inc; San Francisco (CA); On-site; USD 170,000 - 230,000; Full time

Job summary

US staffing Inc in San Francisco, CA seeks a Python Test Lead to design and oversee scalable cloud data architectures and automated testing pipelines. You will drive data ingestion, processing, and storage while mentoring the team and aligning with DevOps and data science stakeholders.

The role emphasizes hands-on Python, PyTest, and cloud platforms (AWS/Azure/GCP), IaC, and CI/CD automation, with a focus on data security, governance, and cost optimization in a fast-moving environment.

Qualifications

• Bachelor’s or Master’s degree in CS, IT, or a related field.
• 7+ years of experience in data architecture, cloud infrastructure, and automation.
• Hands-on experience with AWS, Azure, or Google Cloud plus services like AWS Lambda, S3, RDS, EMR, Azure Data Factory, Databricks.
• Hands-on experience in Python, PySpark, PyTest, Docker.
• Strong knowledge in Kafka and MongoDB.
• Proficiency with IaC tools such as Terraform, CloudFormation, or Ansible.
• Experience automating data pipelines using Apache Airflow, Jenkins, or similar CI/CD tools.
• Proficiency in scripting languages (Python, Bash, Java).
• Experience with Hadoop, Spark, or Kafka.
• Strong DevOps and containerization knowledge (Docker, Kubernetes).
• Deep understanding of data security, privacy, compliance in cloud environments.
• Excellent problem-solving and communication skills.

Responsibilities

• Design, build, and maintain scalable cloud data architectures and automated solutions for data ingestion, processing, and storage.
• Develop automation frameworks and tools for deploying and managing cloud infrastructure and data services.
• Collaborate with data engineers, DevOps, and data scientists to implement end-to-end data pipelines and workflows.
• Ensure automation of data quality, security, and governance across cloud environments.
• Drive cloud best practices including monitoring, disaster recovery, security, and cost optimization.
• Lead efforts in automating testing, deployment, and monitoring of data infrastructure.
• Evaluate and recommend tools and technologies to improve automation processes for cloud data platforms.
• Provide technical leadership, mentoring, and knowledge sharing within the team.

Skills

Python

PySpark

PyTest

Docker

Kafka

MongoDB

Terraform

CloudFormation

Ansible

Airflow

CI/CD

Java

Kubernetes

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

AWS

Azure

Google Cloud

AWS Lambda

S3

RDS

EMR

Databricks

Terraform

CloudFormation

Ansible

Airflow

Jenkins

Kubernetes

Job description

Python Test Lead

San Francisco, CA

Contract

Key Responsibilities

• Design, build, and maintain scalable cloud data architectures and automated solutions that support data ingestion, processing, and storage.
• Develop automation frameworks and tools for deploying and managing cloud infrastructure and data services.
• Collaborate with data engineers, DevOps teams, and data scientists to implement end-to-end data pipelines and workflows.
• Ensure automation of data quality, security, and governance processes, enabling seamless integration across cloud environments.
• Drive the implementation of cloud best practices, including monitoring, disaster recovery, security, and cost optimization.
• Lead efforts in automating testing, deployment, and monitoring of data infrastructure.
• Evaluate and recommend tools and technologies to improve automation processes for cloud data platforms.
• Provide technical leadership, mentoring, and knowledge sharing within the team.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• 7+ years of experience in data architecture, cloud infrastructure, and automation.
• Hands-on experience with cloud platforms like AWS, Azure, or Google Cloud, with deep expertise in services such as AWS Lambda, S3, RDS, EMR, Azure Data Factory, Databricks, etc.
• Hands-on experience in Python, PySpark, PyTest, Docker
• Strong knowledge in Kafka and MongoDB
• Strong knowledge of infrastructure-as-code (IaC) tools such as Terraform, CloudFormation, or Ansible.
• Proven expertise in automating data pipelines using tools like Apache Airflow, Jenkins, or similar CI/CD tools.
• Proficiency in scripting and programming languages (e.g., Python, Bash, Java).
• Experience with big data technologies such as Hadoop, Spark, or Kafka.
• Strong knowledge of DevOps methodologies and containerization tools (e.g., Docker, Kubernetes).
• Deep understanding of data security, privacy, and compliance in cloud environments.
• Excellent problem-solving and communication skills.