After a challenging night (I've never been so close to believing in God as these nights where it seems he's against me every step of the way!) I've finally fixed the reset gremlins.
First of all, let's go into the details of why they happen:
I noticed that when hitting the reset switch (I've added a push button on the back that grounds the _reset line on the JTAG) the screen goes black, yet the modchip doesn't change to the yellow/orange LED (indicating a reset detection) for a few seconds. Further investigation shows the same thing on boot. So it seemed that the USB chip must be reset by the bootloader, since the reset detection LED change is instant.
This confused me a little, since if the bootloader is resetting it, the bootloader must have been loaded at that point, yet if you set the patch time to 0, it doesn't patch. Taking a peek at the modchip code I suddenly realized that the modchip starts up with the GFLAG_PATCHNEEDED set, so it straight away starts waiting, meaning that regardless of when the USB chip is reset it starts patching a certain time after boot, which is why it works on first boot.
So, how to fix this little problem. It was embarrassingly simple, simply comment out the line:
gflags |= GFLAG_PATCHNEEDED;// | GFLAG_FIRST_BOOT;
in main(), and set the PATCHWAITTIME to 0 in the x30Xtmodchip.h.
This way it always waits for the USB reset signal (which happens after the bootloader is loaded, but before the NK.BIN starts loading), and starts patching straight away, so it will always patch successfully.