However, NLR got their just desserts just before the break when a well-constructed move found Oli Nathan free on the right and his low cross was converted well at the near post by Nick Haringman.
NLR's hopes of building on their equaliser were dealt a blow less than a minute into the second half when a cross from the Rovers left was unfortunately turned into his own net by Liam Stein.
Raiders then again came back into the game and equalised on 60 minutes when the ball found its way to Haringman in the box and he put the ball home for his second of the game.
Goldin came to Rovers's rescue with a with a match-winning save to keep out a close-range header from Marc Woolfe.
That proved a turning point as within minutes, a break from Rovers lead to a fine finish from Zneimer putting Rovers 3-2 up.
Raiders pushed in an attempt to get the two goals they needed for victory but ultimately fell short.
Delighted Rovers boss Jeremy Brown told JC Sport: "It was a great cup tie. Raiders are a good bunch and it was end-to-end stuff.
"Again, I think our attacking quality was the difference today and in the last 15 minutes we must have missed around five one-on-ones.
"We're very exited to progress in this competition and let's see where it takes us."
