@johnnadeau there is no built in way to have the runner pane close automatically on passing test. tpope’s dispatch.vim and will use tmux panes as needed.
With tmux-runner you can use the :VtrKillRunner command, which is by default bound to <leader>kr to manually close after a successful test run. I use this personally when I want more space (or I zoom the tmux pane wrapping Vim).
Hope that helps!