Burton looked to be heading for defeat after having defender Aaron Webster sent off only two minutes after Jon Nurse had fired Boro into the lead.
But Mark Stimson's men were made to pay for their failure to capitalise on the extra man as Burton came out with all guns blazing in the second half.
Winger Keith Gilroy levelled the scores in the 55th minute and recent signing Paul Carden grabbed a dramatic winner in the 89th minute.
Boro should have gone in front in the tenth minute but Steve Morison dragged his shot wide after pouncing on a mistake by Darren Tinson.
But an uncertain Burton defence was punished in the 16th minute when Nurse drilled home an angled 12-yard shot after being found unmarked by George Boyd's cross.
Gilroy fired over from a Lee Fowler cross as Burton chased a quick equaliser, but they looked to be in deep trouble when Webster was sent off in the 18th minute.
The full-back was adjudged to be the last defender as he pulled back Nurse in a chase for John Nutter's long ball forward.
With Burton trying to reorganise, they could have gone further behind in the 22nd minute but Nurse's shot rebounded off the foot of the post after the home defence had again been exposed, this time by a long ball from Santos Gaia.
Gilroy was off target for a second time after more good work by Fowler, but it was Stevenage that always posed the greater threat in the first half.
And the two fine saves by Kevin Poole in quick succession denied first Boyd and then Morison a second goal for the visitors.
Stevenage seemed happy to play a patient game in the second half instead of pressing forward as they had done before the break, and it proved to be their undoing.
Burton drew level in the 55th minute when terrific work by Daryl Clare made the opening for Gilroy to race into the area and squeeze a low shot past Danny Potter after holding off Barry Fuller.
Boyd forced a smart save out of Poole, but it was a much more even game now and Potter was tested at the other end by a Clare shot.
Burton seemed happy with the point, but the visitors were left shell-shocked when Carden struck in the 89th minute.
The midfielder was on hand to drive home a 12-yard shot after first Gaia and then Potter had failed to deal with Gilroy's cross from the left.
Boro might have grabbed an equaliser in added time but Andy Corbett popped up on the line to head clear Morison's goalbound effort.