Test Automation / CI Engineer

Codilime; Warszawa; Hybrid; PLN 180,000 - 240,000; Full time

Benefits offered by this job

Flexible hours

Remote or hybrid work

Training budget

Great team culture

Job summary

CodiLime is seeking a Python-based automation engineer to join a newly formed team focused on CI/CD test automation for a flagship routing product. You will design, implement, and optimize test pipelines, automate provisioning and diagnostics, and triage failures across large-scale test orchestrations.

The role emphasizes deep Python expertise, networking know-how, and the use of AI tools to speed scripting and root-cause analysis. Remote-friendly with hybrid options.

Qualifications

• Strong Python programming skills with hands-on pytest, PTF, sonic-mgmt, and Ansible experience.
• Extensive CI/CD engineering background focused on automating provisioning, traffic generation, diagnostics, and regression testing.
• Solid networking knowledge with routing (BGP, OSPF) and experience with SONiC routers.
• English at least B2+ level and ability to work with distributed teams.

Responsibilities

• Designing, building, maintaining robust CI/CD test automation pipelines and test suites in Python for pre-commit PR testing and weekly regression runs
• Automating environment provisioning, traffic generation, network diagnostics, and continuous regression testing across virtual and physical testbeds
• Debugging failed runs - analyzing device, system, and test logs, distinguishing product bugs from test-suite or environment issues, and raising well-documented defects
• Collaborating closely with development teams to understand the features they deliver and designing test coverage for those use cases
• Extending test automation as new features are delivered in the Routing area
• Leveraging AI tools (AI-assisted RCA, LLM coding assistants) to speed up scripting, log analysis, and root cause identification
• Coordinating issue resolution with suite owners and dev/test engineers through dedicated collaboration channels
• Managing physical and virtual testbed environments and provisioning resources for test execution
• Identifying, analyzing, and resolving flaky tests to maintain high pipeline reliability

Skills

Python

pytest

PTF

sonic-mgmt

Ansible

CI/CD engineering

Docker

Linux

Git

GitHub

Jenkins

Debugging

Networking basics

English (B2+)

Tools

Jenkins

GitHub Actions

Ansible

Docker

sonic-mgmt

pytest

Python tooling

Job description

Get to know us better

CodiLime is a software and network engineering industry expert and the first-choice service partner for top global networking hardware providers, software providers and telecoms. We create proofs-of-concept, help our clients build new products, nurture existing ones and provide services in production environments. Our clients include both tech startups and big players in various industries and geographic locations (US, Japan, Israel, Europe).

While no longer a startup - we have 250+ people on board and have been operating since 2011 we’ve kept our people-oriented culture. Our values are simple:

• Actto deliver.

• Disruptto grow.

• Team upto win.

The project and the team

You will join a newly formed team working for a globally recognized networking leader on their flagship router product. The team’s mission is to build and enhance robust test automation and CI/CD pipelines, keeping the routing area healthy through continuous pre-commit testing: every pull request automatically triggers automated test suites on a large-scale CI/CD test orchestration platform, catching bugs before they reach the codebase.

Your work will focus heavily on designing, developing, and optimizing CI/CD test automation frameworks and test suites. You will automate provisioning, traffic generation, diagnostics, and regression testing using Python, pytest, PTF, sonic-mgmt, and Ansible. Additionally, you will own weekly regression runs and triage failures - from reading device and test logs, through distinguishing genuine product bugs from pipeline or environment issues, to raising well-documented defects.

The scale is significant: the platform executes hundreds of test suites and roughly a thousand test runs a day across pre-commit, nightly, and weekly pipelines. To keep triage fast at this scale, the workflow is supported by modern AI tooling - an AI-powered root cause analysis agent that turns hours of log digging into minutes, and collaboration bots that automatically pull the right people into dedicated spaces when a suite fails.

What else you should know:

• The team is being formed right now - you will help shape how it works from day one

• Support is also available from the DevOps and the Network team, as CodiLime engineers are responsible for developing and maintaining the CI/CD test orchestration platform

• You will cooperate directly with the development and test engineers in a distributed, international environment (Europe, India, US)

• AI tools are part of the daily workflow, not an afterthought

We work on multiple interesting projects at a time, so it may happen that we’ll invite you to an interview for another project if we see that your competencies and profile are well suited for it.

Your role

As a part of the team, will be responsible for:

• Designing, building, and maintaining robust CI/CD test automation pipelines and test suites in Python for pre-commit PR testing and weekly regression runs

• Automating environment provisioning, traffic generation, network diagnostics, and continuous regression testing across virtual and physical testbeds

• Debugging failed runs - analyzing device, system, and test logs, distinguishing product bugs from test-suite or environment issues, and raising well-documented defects

• Collaborating closely with development teams to understand the features they deliver and designing test coverage for those use cases

• Extending test automation as new features are delivered in the Routing area

• Leveraging AI tools (AI-assisted RCA, LLM coding assistants) to speed up scripting, log analysis, and root cause identification

• Coordinating issue resolution with suite owners and dev/test engineers through dedicated collaboration channels

• Managing physical and virtual testbed environments and provisioning resources for test execution

• Identifying, analyzing, and resolving flaky tests to maintain high pipeline reliability

Do we have a match?

For success in this role, strong hands-on expertise in Python test automation, CI/CD pipelines, and network test tools is essential:

• Strong Python programming skills and deep hands-on experience with pytest, PTF (Packet Test Framework), sonic-mgmt, and Ansible

• Solid background in CI/CD engineering and automated test orchestration, focused on automating provisioning, traffic generation, diagnostics, and regression testing

• Solid networking knowledge with a focus on routing (BGP, OSPF, routing tables, VRFs, Layer 3 concepts) and working with SONiC routers

• Understanding of router concepts - overlay architecture, policies, IPsec tunnels (hands-on router experience is strongly preferred)

• Proven debugging and troubleshooting skills: reading logs, CLI-level investigation, isolating root causes across product, test, and environment layers

• Experience working closely with software development teams and translating delivered features into test scenarios

• Hands-on use of AI tools in everyday engineering work

• Experience with Docker

• Good Linux command-line skills (shell scripting, command-line tools)

• Good knowledge of English (B2+ level)

• Proficiency with Git and CI/CD platforms

Beyond the criteria above, we would appreciate the nice-to-haves:

• Hands-on experience with Golang, C/C++, Groovy

• Experience with SD-WAN

• Hands-on experience with router solutions

• Familiarity with Jenkins and GitHub-based CI/CD pipelines

• Experience with physical or virtual network testbeds and virtual routers

• Experience working with formal defect tracking systems

• Experience with traffic generators

More reasons to join us

• Flexible working hours and approach to work: fully remotely, in the office or hybrid

• Professional growth supported by internal training sessions and a training budget

• Solid onboarding with a hands-on approach to give you an easy start

• A great atmosphere among professionals who are passionate about their work

• The ability to change the project you work on