Assuming it has enough input voltage to work with, the LM317 wants 1.25 V on the ADJ pin between the OUT pin and ADJ when the OUT is what it’s supposed to be. R1 and R2 forms a voltage divider that lets you dial in that voltage. To hit 1.25 V, you need the trimmer’s wiper at around (220+5000)×1.25/10.6 = 615 ohm away from ground (~12% rotation), so lowering/removing the resistor at the other end won’t help. No, I used the wrong reference in that calculation, it’s more like 4600 ohm (see below), but changing R1 still won’t help. You cannot get around the voltage drop.
There are low dropout (LDO) regulators that might work here, but the TL431 circuit Sam ended up using is inexpensive and a lot better; roughly this, but perhaps with 220 ohm instead of 270 to provide a bit more headroom.
(more here)