How can I repeat each test multiple times in a py.test run?

One possible strategy is parameterizing the test in question, but not explicitly using the parameter.

For example:

@pytest.mark.parametrize('execution_number', range(5))
def run_multiple_times(execution_number):
    assert True

The above test should run five times.

Check out the parametrization documentation: https://pytest.org/latest/parametrize.html