They will face Ireland in the Cup quarter-finals, to be played at 11h05 SA time.
The Blitzboks on Friday beat Kenya 31-5 in their pool opener, and earlier on Saturday outplayed Uruguay 28-12 after having to start the day without injured skipper, Siviwe Soyizwapi.
The dramatic finish to their match over New Zealand came while they were a man down after Impi Visser was deemed to deliberately knocked down a pass and sent to the sin-bin. The Blitzboks forced the All Blacks Sevens back into their own territory with dogged defence and when the Kiwis were penalised with 30 seconds left, Ricardo Duarttee stepped up to kick a penalty goal that sealed the win.
It was very even before that. New Zealand scored first after a flowing attack that started near their own line, but the Blitzboks came back just before the halftime buzzer when Shaun Williams broke down the blind side to send Shilton van Wyk over in the corner. Duarttee converted from the touchline to level scores at 7-7.
The second half did not start well for the Blitzboks. Jaiden Baron was yellow carded and New Zealand worked the numbers to send Roderick Solo over for a converted try.
That was soon equalled out when good defence by the Blitzboks forced a wild pass from New Zealand and Zain Davids intercepted to score. Duarttee's conversion made the score 14-14. Visser was then binned, but Duarttee's next kick proved the vital one.
Against Uruguay, the Blitzboks were caught napping early on when Diego Ardao scored two minutes in, but they soon stepped up the pace against the new kids on the block.
Duarttee opened the Blitzboks' scoring after five minutes before Dalvon Blood ran with great pace around his man to dot down just before the break.
The duo also catered for the scoring in the second half, with a second try each before Uruguay gave the crowd something to cheer about with a try in the final act of play.
Springbok Sevens 17 (7) New Zealand 14 (17)
Blitzboks - Tries: Shilton van Wyk, Zain Davids. Conversions: Ricardo Duarttee (2). Penalty goal: Duarttee.
New Zealand - Tries: Joe Webber, Roderick Solo. Conversions: Akuila Rokolisoa (2).
Springbok Sevens 28 (14) Uruguay 12 (5)
Blitzboks - Tries: Ricardo Duarttee (2), Dalvon Blood (2). Conversions: Duarttee (3), Darren Adonis.
Uruguay - Tries: Diego Ardao, Eugenio Plottier. Conversion: Guillermo Lijtenstein