Have you tried using xvfb-run? I’ve had good luck with that. Looks like you already have xvfb installed, so you can do xvfb-run -a rake, for example. Check out the man page for more info.
If you run webkit_server directly, it may provide more information about what went wrong. You can use something like: docker-compose run web /usr/local/bundle/gems/capybara-webkit-1.6.0/bin/webkit_server.