The resonance control on a filter varies the amount of output signal that’s fed back to the input. When it’s raised high enough it “self oscillates”, which is a polite way of saying you’re getting feedback sound.
So yeah, you can get something like guitar/amp feedback in silicon. Of course a typical VCF does not give you anything very much like it, but it’s maybe a start. A VCF type circuit with resonance varying between large and ridiculous — voltage control over both cutoff frequency and resonance — maybe also an all pass filter phase shifter in the loop? Something like that… maybe…
I’m not sure you need time delay, but it might be worth experimenting with a PT2399 anyway. Or at least it’d be fun.
The speaker and contact mic idea would be sort of an acoustic version of an optoisolator, with the output being an approximation to the input. That would not be necessary for feedback, nor sufficient unless the speaker and mic had suitable resonance characteristics. It’s connecting the output of a resonant system back to the input that makes feedback, and that can be done just as well and more controllably with a filter and a piece of wire.