Something you could try if you’re up for a bit of kludging is to add a diode and resistor immediately after U1B (before R11):
In my simulation, without the diode the voltage does tend to 0 V but very slowly; after 5 seconds (with both pots in the middle) it’s still about 100 mV. With the diode it’s about 2 mV.
