They also provided an early wedding present for chairman Lee Power, who was taking charge of the side for the last time before handing over to Quinn and taking a break to get married on Monday.
When Marcus Richardson produced a superb flick to put Cambridge ahead in the 32nd minute, it was the culmination of what was easily the team's best spell of their troubled start to the campaign.
They might easily have been three goals up by then against the shaky Stevenage side, who demonstrate exactly why they went into the match with the worst away goals record in the Conference, 11 conceded in four games.
The visitors had managed a couple of good attacking efforts of their own, however, in the opening half an hour and, when Cambridge midfielder Ritchie Hanlon was sent off on the stroke of half time, it looked like being a tough second half.
Hanlon, already with one red card previously this season, was dismissed after a challenge with Stevenage midfielder, Dwayne Lee. It looked like a case of a 50-50 encounter with the Cambridge man getting the ball, but referee, Russell Fletcher, appeared to indicate a two-footed tackle.
Hanlon protested as he walked off, and once the half-time whistle went, chairman Power ran on to the pitch to shout his protest at the official.
Obviously aiming to impress Quinn, who was watching from the grandstand, Cambridge could have raced into the lead in the first minute, but John Brady mis-kicked eight yards out with goalkeeper Alan Julian at his mercy.
As United kept Stevenage pinned in their penalty area, Richardson also failed to take advantage of a shooting opportunity. Then, in the sixth minute, there was a scare for the panicking visitors when Mark Peters hit a post from a header following a free kick.
It took Stevenage a quarter of an hour to give their big following, behind the goal they were attacking, anything to cheer about but they almost snatched the lead. George Boyd whipped in a ground shot as goalkeeper Paul Crichton failed to hold and the ball had to be scrambled away for a corner.
The derby clash threatened to boil over midway through the first half when Cambridge's Courtney Pitt was booked for diving in the penalty area, just before Borough skipper, Jason Goodliffe, got a yellow card for a bad tackle on Robbie Simpson.
Richardson had a lot of work to do in the second half as 10-man Cambridge regrouped, pulling men back into midfield to protect their lead.
They looked like losing it on the hour when Stevenage substitute Chris Sullivan ran on to a cross just inside the box, but defender Rob Gier made up yards to get across him just as he was set to make contact with the ball.
But sensibly Cambridge did not fall back on massed defence and Richardson went close to a second in the 65th minute when he turned expertly under pressure to force a diving save from Julian.
When the big centre forward was substituted close to exhaustion in the 81st minute, he received a standing ovation from the Abbey Stadium supporters.
It was a tribute to the home side's character that they denied Stevenage a really clear scoring chance in the last half an hour.