Rerun failed tests Robot Framework
Posted inRobot Framework

Rerun Failed Tests in Robot Framework

No Comments

When you execute a large test suite, some tests may fail because of temporary issues, incorrect locators, environment problems, or application instability. Running the complete test suite again just to execute one failed test can waste considerable time. We can only rerun failed tests in Robot Framework by using the --rerunfailed command-line.

With --rerunfailed, Robot Framework reads the previous output.xml file, identifies the failed test cases, and executes only those failed tests again.

We will be using the following command:

robot --rerunfailed log/output.xml --output log/output1.xml BrowserTest/RerunFailed.robot

What Is the Robot Framework --rerunfailed Option?

The Robot Framework --rerunfailed option selects failed test cases from a previous Robot Framework output file and executes them again.

The basic syntax is:

robot --rerunfailed <previous-output.xml> <test-suite>

For example:

robot --rerunfailed output.xml tests

Robot Framework uses the earlier output file to determine which test cases failed. It then runs only those failed test cases instead of executing the entire test suite again.

This is especially useful when:

  • The test suite contains hundreds or thousands of tests.
  • Only a few tests failed.
  • You want to verify whether a failure was temporary.
  • You fixed a locator or test-data issue and want to execute the failed test again.
  • You want to reduce execution time during debugging.

Example Robot Framework Test Suite

Consider the following test suite:

*** Settings ***
Library       Browser
Test Setup    Open Browser to Frame Page
Resource      ../Resource.robot
*** Variables ***
${expected text}       Frames
${selector}             //*[@id="app"]/header/a
*** Test Cases ***
TestA
    Log To Console    This is TestA
TestB
    Log To Console    This is TestB
TestC
    ${attr_name}    Get Attribute Names    ${selector}
    Log To Console    ${attr_name}

This suite contains three test cases:

Test CaseExpected Result
TestAPASS
TestBPASS
TestCFAIL

For this example, assume that the XPath stored in ${selector} is incorrect:

${selector}    //*[@id="app"]/header/a

Because the locator does not correctly identify the expected element, the following keyword fails:

${attr_name}    Get Attribute Names    ${selector}

As a result, TestC is marked as FAIL.

First Execution of the Test Suite

Initially, execute the complete test suite:

robot --output log/output.xml BrowserTest/RerunFailed.robot

The execution result is:

TestA    PASS
TestB    PASS
TestC    FAIL

The output.xml file contains the execution results of the test suite.

The output directory may look like this:

Project
│
├── BrowserTest
│   └── RerunFailed.robot
│
├── Resource.robot
│
└── log
    └── output.xml

The important point is that output.xml now contains the result of the first execution, including the failure of TestC.

Using --rerunfailed to Execute Only Failed Tests

Now execute the following command:

robot --rerunfailed log/output.xml --output log/output1.xml BrowserTest/RerunFailed.robot

Let’s break down this command:

robot

This starts the Robot Framework test execution.

--rerunfailed log/output.xml

This tells Robot Framework to read:

log/output.xml

and select the test cases that previously failed.

In our example, only the following test failed:

TestC

Therefore, Robot Framework executes only TestC.

--output log/output1.xml

This creates a new Robot Framework output file:

log/output1.xml

The original result remains unchanged in:

log/output.xml
BrowserTest/RerunFailed.robot

This is the test suite that Robot Framework uses for the rerun.

The complete command is:

robot --rerunfailed log/output.xml --output log/output1.xml BrowserTest/RerunFailed.robot

What Happens During the Rerun?

The original execution looked like this:

TestA    PASS
TestB    PASS
TestC    FAIL

When you execute:

robot --rerunfailed log/output.xml --output log/output1.xml BrowserTest/RerunFailed.robot

Robot Framework reads the previous result and identifies:

TestC = FAIL

Therefore, the rerun executes:

TestC

It does not need to execute:

TestA
TestB

again.

This is the main advantage of the --rerunfailed option.

Scenario 1: TestC Fails Again

Suppose the XPath is still incorrect:

${selector}    //*[@id="app"]/header/a

The rerun result is:

TestC    FAIL

The new output file contains the rerun result:

log/output1.xml

The failure may look similar to:

Element with selector '//*[@id="app"]/header/a' not found

In this case, the test failure is reproducible and the locator should be investigated.

Scenario 2: TestC Passes During the Rerun

Now suppose you fix the XPath:

${selector}    //header/a

or use the correct locator for the application.

The rerun command is executed again:

robot --rerunfailed log/output.xml --output log/output1.xml BrowserTest/RerunFailed.robot

The result may now be:

TestC    PASS

The rerun result is therefore:

TestC    PASS

This indicates that the test passed after the issue was corrected.

Original Output File vs Rerun Output File

It is important to understand the difference between the two output files.

Original Output

log/output.xml

This contains the results of the original execution:

TestA    PASS
TestB    PASS
TestC    FAIL

Rerun Output

log/output1.xml

This contains the results of the rerun:

TestC    PASS

The second execution does not automatically replace the original output.xml file because a different output file was specified:

--output log/output1.xml

This allows you to preserve both the original execution results and the rerun results.

Rerun Failed Tests and Merge the Results

In a real test execution pipeline, you may want to combine the original execution and rerun results into one final report.

The typical workflow is:

Step 1: Execute the Complete Suite

robot --output log/output.xml BrowserTest/RerunFailed.robot

Step 2: Rerun Failed Tests

robot --rerunfailed log/output.xml --output log/output1.xml BrowserTest/RerunFailed.robot

Step 3: Merge the Results

rebot --merge log/output.xml log/output1.xml

The merged result can represent the final status after the rerun.

For example:

Original Result:
TestA    PASS
TestB    PASS
TestC    FAIL
Rerun Result:
TestC    PASS
Final Merged Result:
TestA    PASS
TestB    PASS
TestC    PASS

This is useful in CI/CD pipelines where the final report should reflect the result after failed tests have been re-executed.

Important: --rerunfailed Does Not Automatically Fix Tests

The --rerunfailed option only executes previously failed tests again.

It does not:

  • Fix incorrect XPath expressions.
  • Change test data.
  • Wait automatically for a missing element.
  • Fix application defects.
  • Guarantee that a flaky test will pass.

For example, if the following XPath is incorrect:

${selector}    //*[@id="app"]/header/a

Robot Framework will simply execute the test again with the same locator.

The test will continue to fail unless the underlying problem is resolved.

--rerunfailed vs Running the Complete Suite

Suppose you have 500 tests:

Test001
Test002
Test003
...
Test500

Assume only three tests fail:

Test102
Test245
Test489

Running the entire suite again means:

500 tests executed again

Using:

robot --rerunfailed output.xml tests

means:

Only Test102, Test245, and Test489 are executed again

This can significantly reduce execution time.

Important Considerations

Use the Correct Previous Output File

The file specified with --rerunfailed must contain the results from a previous execution.

For example:

--rerunfailed log/output.xml

Robot Framework reads this file to identify failed tests.

Use the Same Test Suite

The previous output file should correspond to the tests you are executing during the rerun.

For example:

robot --rerunfailed log/output.xml \
      --output log/output1.xml \
      BrowserTest/RerunFailed.robot

The output.xml file should represent the relevant execution of:

BrowserTest/RerunFailed.robot

If There Are No Failed Tests

If the previous output file contains no failed tests, there is nothing to rerun.

The --rerunfailed option is intended specifically for selecting failed test cases from a previous execution.

Conclusion

Robot Framework’s --rerunfailed option is a simple and useful way to execute only the test cases that failed during a previous test run.

For example:

robot --rerunfailed log/output.xml --output log/output1.xml BrowserTest/RerunFailed.robot

In this example:

  • log/output.xml contains the original test results.
  • Robot Framework identifies the failed test cases from that file.
  • Only the failed tests are executed again.
  • log/output1.xml stores the results of the rerun.
  • rebot --merge can be used to combine the original and rerun results.

When a large test suite contains only a few failures, rerunning failed tests can save significant execution time and make debugging more efficient.

Run the complete suite first, rerun only failed tests using --rerunfailed, and merge the results when you need a final consolidated report.

Related Links

Discover more from Rotebit

Subscribe to get the latest posts sent to your email.

Tags:

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply