But it was a win which the visitors richly deserved as they launched a second-half fightback after being unlucky to go in at the break 2-0 down.
There was no sign of what was to come as Stevenage dominated the opening exchanges and took the lead on seven minutes.
A cleverly worked free-kick saw George Boyd release Adam Miller into space but, rather than shooting for goal, the Stevenage midfielder put in a cross that allowed Dino Maamria to slide the ball home.
Stevenage could have added to their lead a minute later when Ollie Berquez unleashed a drive which narrowly missed Saul Deeney's far post.
Burton came back into the game and enjoyed a bright spell which saw home keeper Alan Julian fully tested by Moore and Kris Taylor.
Albion had the best chance to pull level on the half-hour when Shaun Harrad put Jon Shaw clear on goal only for Julian to deny the striker with a full-length block.
Having ridden their luck, Stevenage looked to have clinched the points with a second on the stroke of half-time. Darryn Stamp was in the right place to force the ball home after he had been found by a neat cross from Simon Weatherstone.
Nigel Clough rallied Burton at the break and pulled Stevenage back from the outset before clawing their way back into the contest with a goal in the 63rd minute. Shaw got the decisive flick as he headed home at Julian's far post following a free-kick from Taylor.
Burton piled on the pressure with Harrad, Chris Hall and Darren Stride all forcing stunning saves from Julian before Burton pulled level on 78 minutes when Aaron Webster forced the ball in following another Taylor free-kick.
The equaliser looked to be a sickener for the home side but worse was to follow in added time when late substitute Moore chased down a hopeful ball and beat Luke Oliver before seeing his strike from a narrow angle deflecting to sink the home side.