The Walsall forward created the Lambs' opening goal, forcing defender John Nutter to put through his own net, before adding a second after the break.
George Boyd pulled a goal back for the visitors and Alan Neilson was sent off for Tamworth but Mark Stimson's men were unable to find a way back into the clash and are still searching for their first win of the campaign.
Irish midfielder Graham Ward was replaced in the Lambs starting line-up by new loan signing Graham Deakin, who joined from League Two high-fliers Walsall on Thursday.
Stevenage gave a debut to former Kidderminster midfielder Dwayne Lee.
In windy conditions, Boyd curled a 20-yard shot wide on three minutes as the visitors looked to make early in-roads.
Stevenage continued to press and in-form Steve Morison went close to opening the scoring with a close-range header from a Steve Guppy corner on six minutes.
The Lambs found chances hard to come by in the opening stages. Bradley Thomas had their first shot on 14 minutes but blasted high and wide from inside the penalty area.
The visitors continued to have the better of a poor game and Adam Miller fired wide from outside the area.
Tamworth created a good chance on 27 minutes as John McGrath's free-kick found Atieno, whose header just evaded Matty Williams at the far post.
Tamworth took the lead on 32 minutes. McGrath fired in a vicious rising shot from the edge of the penalty area which was tipped onto the bar Alan Julian. The ball fell kindly for Atieno, whose low shot was put into is own net by Nutter.
The visitors almost responded immediately but Guppy saw his goalbound free-kick from the edge of the area deflected for a corner on 36 minutes.
The Lambs had keeper Sean Bowles to thank for holding their lead as he produced a fine one-handed save to deny Santo Gaia from the resultant corner.
The hardworking Williams almost gave Mark Cooper's men a dream end to the first half, firing just over the bar from 15 yards out.
Tamworth started the second half positively and Williams headed a Gavin McCallum cross wide on 47 minutes.
Stevenage went agonisingly close to levelling two minutes later as David Hicks saw his low drive cleared off the line by McCallum.
In a bright opening to the second half, Tamworth doubled their lead on 52 minutes as Atieno rose superbly to power home a McGrath free-kick from the left.
Stevenage forced their way back into contention on 57 minutes. Substitute Craig Dobson made an instant impact, whipping in a cross for Boyd, who controlled the ball and turned neatly in the box before coolly firing past Bowles.
A scrappy ten-minute period ensued with chances at a premium but McCallum did test Julian in the Stevenage goal with a well-struck shot from distance.
Stevenage had strong appeals for a penalty turned down when Miller went down under a challenge from McCallum.
Tamworth were controversially reduced to ten men on 72 minutes. Neilson, who had earlier been booked for a foul, was shown a second by referee David Coote for time-wasting.
The Lambs brought on midfielder Ward for striker Williams as they sought to hold on to their lead. They almost restored their two-goal advantage when Atieno managed to get his head to a McGrath free-kick but Julian was equal to the test.
Darryn Stamp also saw a late header deflected away to safety as Tamworth secured back-to-back home wins.
The visitors huffed and puffed to find an equaliser in the latter stages but there was little product to their endeavour until stoppage time when Dobson's right-footed effort took a deflection and hit the crossbar.