Second-half tries from Louis Lynagh and Stephen Varney proved pivotal for the Azzurri, as a stellar showing after the break helped overhaul a six-point deficit at the half time.
Scotland looked in control, leading 22-10 at one stage in a first-half that saw the visitors largely on top, but the hosts roared back after the break and were full value for their first win under Gonzalo Quesada.
Sam Skinner scored late on to set up a nervous finish, but the Italian defence held strong against wave after wave of attack at the death to hold out for the win and send a sold-out Stadio Olimpico into raptures.
Scotland edge frenetic first half
It took just two minutes for Italy to get on the scoreboard, Paolo Garbisi slotting a penalty home, this time in spite of the ball once again falling off the tee, after Andy Christie was penalised at the breakdown after 30 seconds.
But Scotland responded brightly to that early setback and did so in the perfect manner just four minutes later, as Zander Fagerson crashed over from short range.
It was the product of relentless carrying from the visitors after an initial scrappy spell of possession, with George Horne impressively pulling the strings to edge his side closer to the whitewash before Fagerson burrowed over from a metre out.
Scotland were well on top by this point and took advantage of their bright start with a second try just five minutes later, Kyle Steyn the one to cross after the Italian defence found themselves stretched.
Duhan van der Merwe nearly found a gap out wide on the left but the ball was well recycled through the phases and eventually into the hands of Steyn, who had the simplest of finishes to extend his side’s lead to 14-3 with 13 minutes on the clock.
The game continued its frenetic opening and Italy hit back just two minutes later, Ignacio Brex racing onto a clever kick over the top by Martin Page-Relo to score by the posts and reduce the deficit to four.
After a busy start, the game eventually found its rhythm with a quarter of the game played and Scotland continued to see much of the ball with Horne and Finn Russell dictating the tempo.
A long penalty into touch proved the platform for another three points, with Russell slotting home a penalty after a tackle off the ball to nudge Scotland’s lead to 17-10.
Russell had found his groove by this stage, with the talismanic playmaker producing a stunning 50:22 to march his side up the field, and they took full advantage with a third try.
The resulting lineout was converted into a rolling maul that proved too much for the Italian defence and Pierre Schoeman darted over from close range.
Russell was unable to add the extras this time to leave the score at 22-10 at the half hour mark.
Italy fight back
That appeared to spark Italy back into life, with the hosts dialling up the pressure and eventually forcing a penalty that was expertly slotted home by Garbisi from 35 metres out.
The home side continued their momentum but almost found themselves conceding a fourth try as Christie intercepted a pass and raced up the pitch before his progress was halted by Brex.
Yet it would be the home side with the final score of a breathless first half, with Page-Relo this time the man behind another long-range penalty to further reduce the gap to six.
Italy almost stole a second try just before the whistle as Louis Lynagh hacked and chased a loose ball and Ange Capuozzo picked up the scraps but he was unceremoniously held up by Van der Merwe before he had a chance to dot down.
It appeared as if the scoring frenzy would continue immediately after the restart, as Horne dotted over after Huw Jones sliced through the Italian defence but the try was ruled out for an infringement from Schoeman in the build-up, the prop adjudged to be playing his man off the ball.
That was perhaps just the reprieve Italy required, as they had their second try just a minute later through debutant Louis Lynagh.
An excellent grubber kick from Garbisi set Lynagh in behind and the wing made no mistake to crash over, but Garbisi was unable to put Italy in front as his conversion cannoned off the upright to leave the hosts one behind, 22-21.
Hosts edge dramatic finish
The pendulum continued to swing back and forth as both sides searched for the vital next score, and it was Italy who found their way through with a Stephen Varney try.
It all stemmed from a break from Ross Vintcent, who carried into the 22 before quick ball from the hosts eventually led to Varney finding a gap from close range. Garbisi refound his range to fire home the conversion to give Italy a 28-22 led with 20 minutes to play.
Scotland were understandably reeling, with the Stadio Olimpico sensing a first win in Rome since 2013, but the visitors did well to stem the Italian tide and regain a foothold in the match.
A first real attack of the second half saw Van der Merwe set free but the wing was stopped in his tracks as the home defence continued to stand strong.
But an Italian victory was nudged closer by Garbisi, as he kicked another penalty from distance to extend the lead to 31-22 with just seven minutes remaining.
Scotland were not yet down and out, with quick hands in the corner almost setting Steyn free for his second of the afternoon before play was brought back for a penalty after a deliberate knock on.
That penalty was quickly converted into try line pressure, and Skinner was the one to crash over to set up a grandstand finish in Rome.
Scotland ran through phase after phase, with Russell attempting to dictate every pass but the Italian defence held out for a famous victory.
