Referee Mark Tilling denied Stevenage an equaliser in the fourth minute of stoppage time as the home side clung on desperately to record a second straight win.
Cleveland-based official Tilling blew for a foul on Halifax keeper Mark Cartwright during a goalmouth scramble that ended with substitute Rocky Baptiste firing in what he believed to be the leveller.
Former Luton hitman Baptiste was distraught, however, when he realised the effort had been chalked off and bitterly continued protesting with Tilling after the referee had blown the final whistle.
Earlier, striker Christian Lee had haunted former manager Graham Westley by scoring what proved to be the winner on 50 minutes.
Lee, who played under Westley at Farnborough last season, latched on to a long punt from Cartwright and outpaced the Stevenage defence to fire home from 20 yards.
The front man, who was converted to centre-back by Westley during Boro's injury crisis last season, wheeled away in celebration and took off his shirt in front of the jubilant Halifax fans.
Halifax had taken an early lead when Andy Farrell blasted in after Stevenage had failed to clear a free kick on eight minutes.
Farrell should have got onto the scoresheet even sooner but a brilliant reflex save from former Sunderland and Newcastle keeper Lionel Perez on four minutes prevented the youngster from scoring from point-blank range after connecting with Steve Bushell's wicked free-kick.
But the home side wasted their early promise and probably should have gone in behind at the break.
Tony Battersby was left unmarked to head in the equaliser from a Simon Travis free-kick on 21 minutes.
And barely 90 seconds later the Town defence failed to pick up Matthew Langston from a Simon Wormull corner, but the defender blasted wide from close range when it looked easier to score.
Just after the break Stevenage created the move of the match, but Battersby ruined a glorious ten-pass move by shooting high and wide of the target on 48 minutes.
The move was started by Gary Holloway's cheeky backheel inside his own half to Travis with the right-back helping to build the move from which Lee Flynn supplied the cross for Battersby to shoot over.