Lead DevOps / AWS Architect

The Jupiter Group, Inc; Houston (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

The Jupiter Group, Inc invites a Senior DevOps Engineer to design, build, and run a scalable DevOps capability for AI and Data Engineering teams on AWS. You’ll define architecture, implement CI/CD pipelines, and ensure secure, compliant deployment across cloud and on-prem environments.

Join a Fortune-100-scale, Downtown Houston team to modernize data products, ML models and GenAI services with templated pipelines, AI-assisted tooling, and strong governance.

Qualifications

• 10+ years in DevOps, platform, infrastructure or SRE, including mentoring
• 5+ years in the AWS ecosystem as a DevOps/Platform engineer
• 5+ years with IaC on AWS (CloudFormation, Terraform/OpenTofu)
• 5+ years scripting in Python and Bash on Linux/Unix
• 5+ years supporting CI/CD tools (GitHub Actions, Harness, ArgoCD, Jenkins)
• 5+ years with Kubernetes and container orchestration
• Experience delivering CI/CD for data platforms (Snowflake, dbt, Airflow) and MLOps for ML/GenAI systems
• Experience in regulated environments with security/compliance governance
• Experience with AI-assisted development tools and reviewing AI-generated output
• Strong communication with executives and technical/non-technical stakeholders
• Open-minded, collaborative and adaptive, eager to learn

Responsibilities

• Initial focus on automating Security and Compliance in CI/CD pipelines using Terraform and Python/Bash
• Create reusable, templated CI/CD paths for data and AI workloads
• Take over DevOps work currently handled by Data Engineering
• Establish baseline observability and first runbooks for the data and AI platform

Skills

AWS expert

CI/CD

Kubernetes

IaC

Python scripting

Security & Compliance

AI-assisted DX

Observability

Tools

CloudFormation

Terraform/OpenTofu

GitHub Actions

Harness.io

ArgoCD

Jenkins

Snowflake/dbt/Airflow

Job description

We're working with a major oil / gas company in Downtown Houston, fully onsite 5x a week but you'll have a chance to make a real impact, be a difference maker with a Fortune 100 company. We know it's annoying an inconvenient for the people that still work hybrid or fully remote, but if you're looking for a chance to level up your skill set rolling out \"best practices\" and establishing AI based solutions in data engineering and data science teams....this is the one.

JOB DESCRIPTION

We are looking for a Senior (Level III) DevOps Engineer to design, build and run the DevOps capability for our AI and Data Engineering teams on AWS. You will collaborate directly with our customers to design and document the overall DevOps solution architecture, then create the CI/CD pipelines, automation and deployment paths that Data Engineers, ML Engineers and AI Engineers use to ship data products, ML models and GenAI applications, so those engineers can stay focused on data and AI.

You will know the data engineering space well: how Snowflake, dbt and Airflow workloads are built, tested, promoted and operated, and what it takes to take an ML or GenAI service to production. You will work under minimal supervision alongside Security, Compliance, Data Engineering, AI Engineering and the wider DevOps team to establish reusable, templated practices with the required gates and governance built in, and to make the compliant path the easiest path for developers.

You will be a daily, proficient user of AI-assisted engineering tools and will help shape how our DevOps team uses AI well. Everything you build is documented and handed over so that Enterprise DevOps staff can operate and extend it.

Design, build and automate

• Collaborate with customers (Data, ML and AI Engineering teams and their stakeholders) to understand their needs, then design and document the overall DevOps solution architecture: delivery paths, environments, gates, tooling and operating model. Record design decisions and their reasons so others can extend the design.
• Own CI/CD and deployment automation for data and AI workloads: Snowflake and dbt change promotion, Airflow/Astronomer DAG deployment, and ML model and GenAI service release, across AWS, containerized clusters and on-premises environments.
• Work with the Infrastructure Platform Engineering team on Infrastructure as Code (AWS CloudFormation, Terraform/OpenTofu, CDK) for our multi-account AWS environments, and incorporate it into our pipelines for data and AI platform services (S3, Glue, Athena, Lake Formation, SageMaker, Bedrock, Lambda, ECS, RDS, IAM, VPC). Automate CloudFormation stack/change set and Terraform plan/apply deployments, testing and evidence.
• Support MLOps practices: model registries, pipeline automation, promotion across environments, and cost and guardrail controls for LLM-based systems.
• Build reusable, versioned, templated pipelines and workflows that teams adopt rather than copy, with security and compliance controls built in by default.

Govern and enable

• Implement and automate required gates (approvals, segregation of duties, policy-as-code, scanning, change-ticket linkage) and produce auditable deployment evidence, working directly with Security and Compliance.
• Make developer experience a first-class goal: self-service tooling, fast feedback, clear errors and good documentation, so the governed route is also the easiest. Measure it and improve it.

Operate and improve

• Build observability (metrics, logs, traces, dashboards, actionable alerting) and apply site reliability practices (SLIs/SLOs, capacity and cost review, disaster recovery testing, incident response and blameless post-incident reviews).
• Write and maintain tested runbooks (daily operations, periodic maintenance, failure modes, recovery) that other team members can use, and turn them into automation wherever possible.
• Continuously reduce DevOps overhead: find manual, repetitive and high-volume work, then automate, simplify or eliminate it.

Work with AI, work with people

• Use AI-assisted development tools daily to design, build, test and refactor automation, and contribute to our standards for AI-assisted DevOps, including critical human review of generated output.
• Maintain collaborative working relationships with Data Development Engineers, Infrastructure Platform Engineers, and Security and Compliance team members, and mentor mid-level/junior engineers.

REQUIREMENTS

• 10+ years in DevOps, platform, infrastructure or site reliability engineering, including experience leading or mentoring other engineers.
• 5+ years in the AWS ecosystem as a DevOps (or platform) engineer, with deep AWS expertise, including cloud networking, security and architecture patterns.
• 5+ years with Infrastructure as Code on AWS, including AWS CloudFormation (AWS-native IaC) and Terraform or OpenTofu, and 5+ years scripting/programming in Python (building and testing tooling) and Bash/Shell on Linux/Unix.
• 5+ years supporting CI/CD tools (GitHub Actions, Harness.io, ArgoCD, Jenkins, etc), including reusable/templated workflow design, on GitHub, GitLab or ADO repositories.
• 5+ years supporting Kubernetes and containerized orchestration.
• Experience delivering CI/CD for data platforms (Snowflake, dbt, Airflow) and working knowledge of MLOps for classical ML and/or GenAI systems.
• Experience in change-controlled or regulated environments (approvals, segregation of duties, policy-as-code, auditable evidence), partnering with Security and Compliance teams.
• Experience with AI-assisted development tools such as Claude Code, Cursor or GitHub Copilot, including structuring work for AI coding agents (clear specifications, context, task breakdown) and reviewing generated output critically.
• Site reliability and observability experience (SLIs/SLOs, monitoring and alerting platforms such as Grafana, Prometheus, CloudWatch, Datadog or Elastic, incident management), with strong troubleshooting and incident-response skills.
• Experience designing and documenting DevOps or platform solution architectures in collaboration with customers or internal teams, including architecture diagrams and design decision records.
• A record of writing runbooks that others used successfully, automating away manual operational work, and building tooling or templates that other engineers chose to use. Be ready to show examples.
• Strong communication skills: will meet with executives regarding projects to which this position is assigned, and work with both technical and non-technical stakeholders.
• Open-minded, collaborative and adaptive, eager to learn.

PREFERRED

• Experience with S3 Tables, Glue, Lake Formation, SageMaker and Bedrock.
• Experience with Harness.io, including templated pipelines, governance policies (OPA) and approval/evidence features.
• Experience with Astronomer/Airflow operation and upgrades, and with change-data-capture pipelines.
• Experience with LLM observability, evaluation and cost controls; AI agents or MCP-style integrations in engineering workflows.
• Experience with OIDC/workload identity federation and enterprise secrets/credential management.
• Experience building internal developer platforms or golden-path tooling.

TOP DAILY PRIORITIES/RESPONSIBILITIES

• Initially will be tasked to help automate Security and Compliance processes into the CICD pipelines and our company compliance tool with Terraform and Python and Shell.
• Build the first reusable, templated CI/CD paths for data and AI workloads, with the required gates and evidence included.
• Take over DevOps work currently handled within the Data Engineering team.
• Establish baseline observability and the first runbooks for the data and AI platform, and identify the highest-toil processes to automate first.