Senior Python Data Automation Lead

US staffing Inc; San Francisco (CA); On-site; USD 170,000 - 230,000; Full time

Job summary

US staffing Inc in San Francisco, CA seeks a Python Test Lead to design and oversee scalable cloud data architectures and automated testing pipelines. You will drive data ingestion, processing, and storage while mentoring the team and aligning with DevOps and data science stakeholders.

The role emphasizes hands-on Python, PyTest, and cloud platforms (AWS/Azure/GCP), IaC, and CI/CD automation, with a focus on data security, governance, and cost optimization in a fast-moving environment.

Qualifications

• Bachelor’s or Master’s degree in CS, IT, or a related field.
• 7+ years of experience in data architecture, cloud infrastructure, and automation.
• Hands-on experience with AWS, Azure, or Google Cloud plus services like AWS Lambda, S3, RDS, EMR, Azure Data Factory, Databricks.
• Hands-on experience in Python, PySpark, PyTest, Docker.
• Strong knowledge in Kafka and MongoDB.
• Proficiency with IaC tools such as Terraform, CloudFormation, or Ansible.
• Experience automating data pipelines using Apache Airflow, Jenkins, or similar CI/CD tools.
• Proficiency in scripting languages (Python, Bash, Java).
• Experience with Hadoop, Spark, or Kafka.
• Strong DevOps and containerization knowledge (Docker, Kubernetes).
• Deep understanding of data security, privacy, compliance in cloud environments.
• Excellent problem-solving and communication skills.

Responsibilities

• Design, build, and maintain scalable cloud data architectures and automated solutions for data ingestion, processing, and storage.
• Develop automation frameworks and tools for deploying and managing cloud infrastructure and data services.
• Collaborate with data engineers, DevOps, and data scientists to implement end-to-end data pipelines and workflows.
• Ensure automation of data quality, security, and governance across cloud environments.
• Drive cloud best practices including monitoring, disaster recovery, security, and cost optimization.
• Lead efforts in automating testing, deployment, and monitoring of data infrastructure.
• Evaluate and recommend tools and technologies to improve automation processes for cloud data platforms.
• Provide technical leadership, mentoring, and knowledge sharing within the team.

Skills

Python

PySpark

PyTest

Docker

Kafka

MongoDB

Terraform

CloudFormation

Ansible

Airflow

CI/CD

Java

Kubernetes

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

AWS

Azure

Google Cloud

AWS Lambda

S3

RDS

EMR

Databricks

Terraform

CloudFormation

Ansible

Airflow

Jenkins

Kubernetes

Job description

US staffing Inc in San Francisco, CA seeks a Python Test Lead to design and oversee scalable cloud data architectures and automated testing pipelines. You will drive data ingestion, processing, and storage while mentoring the team and aligning with DevOps and data science stakeholders.

The role emphasizes hands-on Python, PyTest, and cloud platforms (AWS/Azure/GCP), IaC, and CI/CD automation, with a focus on data security, governance, and cost optimization in a fast-moving environment.