[pytest-dev] Keeping failing test ids for git bisect

[Daniel Hahler]

On 02.05.19 11:28, Florian Bruhin wrote:

>> Is there a way to run a bunch of tests like with --lf but don't clear
>> the cache when they pass?

> What would be nice is something like a --save-collected <name> and
> --load-collected <name>. Then you could do "--lf --save-collected bisect" once,
> and do "--load-collected bisect" to rerun exactly those tests.
>
> I wanted to write a plugin for that for months (if not years) now, but never
> got around to it. So far I've also done what you described.

See https://github.com/pytest-dev/pytest/issues/5073 for general
improvements for the last-failed plugin, and
https://github.com/pytest-dev/pytest/issues/1017 from 2016 for this
specific issue.

I think this should be addressed/fixed there, instead of creating a new plugin.

As for workarounds, I've used to do
`cp -a .pytest_cache/v/cache/lastfailed{,.stash}` (and the other way
around) also.

But a better workaround is using a wrapper that puts the cache in its
own Git directory, allowing for `p-cache commit -a -m saved`, `p-cache
checkout .` etc then.
(it also clearly showed a problem with the --newest-first plugin (#5206)).

```
#!/bin/sh
# Wrapper around git in .pytest_cache/v/cache.

set -e

# TODO: could use `m=((../)#.pytest_cache(N))` in Zsh for upward search.
root=.pytest_cache/v/cache

if ! [ -d "$root" ]; then
  error "Directory $root does not exist"
fi

cd "$root"

if ! [ -e ".git" ]; then
  git init .
  git commit --allow-empty -m "init"
  git add --all .
  git commit -m "initial state"
fi

git "$@"
```