For a simple on off indicator, use a resistor + diode from the op amp output to ground, in parallel with the resistor + jack, not in series.
For something that responds kind of linearly to the voltage, use an op amp driver, see Vactrols are magic! - #10 by sebastian and further explanation in the North Coast Synthesis article linked from there.