The Sandgrounders seemed to be cruising to the three points when they surged into a 3-0 lead, but two goals inside two minutes by the visitors early in the second half set up a nervous finish.
Southport produced a fantastic first-half display as the visitors were only allowed two late chances through Ollie Berquez and Darryn Stamp. The latter snapped in a shot from a corner, only to see the ball cleared off the line by Dominic Krief.
Earlier Southport had dominated proceedings, with Carl Baker scoring from the penalty spot after Justin Gregory had fouled Neil Robinson in the box when the striker was through on goal.
Southport extended their lead when Neil Robinson played the ball to Steve Pickford who fired in a shot from just outside the area. The ball rebounded off the corner of the crossbar as far as Kevin Leadbetter whose attempted shot fell to Robinson who stabbed the ball home from three yards out.
In the first few minutes of the second half, Southport made the game look comfortable when Baker's free-kick came back from the Stevenage wall, as far as a cluster of players including Robinson who fired a shot home from 16 yards out.
However, Stevenage staged a fine comeback scoring twice within a couple of minutes.
A cross from Jon Nurse on the left wing found Anthony Elding who slotted the ball home from close range. Moments later the ball appeared to hit Earl Davis, which the referee adjudged had come back off his arm, and awarded a penalty to Borough, which was comfortably converted by Stamp.
However, Southport absorbed the pressure Stevenage applied to ensure they picked up three welcome points.