The trick here was finding the right plugin. I'm sure the issues and problems were mine but one thing or another just did work on the various plugins I tried.
I went through the most downloaded Clojure plugins but finally ended up with Calva : Clojure for VS Code
You must first set up leiningen. It is a build automation and dependency management tool for Clojure. Clojure's npm/gulp if you will.
There are instructions to install leiningen. I most develop using the Windows Subsystem for Linux (WSL) so I followed the instructions. I put the script in
/usr/local/bin. Just call it
lein and update the permissions as per the instructions.
Then install the Calva : Clojure for VS Code plugin for VSC. In the terminal start the REPL with
Calva should then connect. If you stop the REPL you can use the
Calva: Reconnect VSC command to get Calva to connect again once you restart it.