There was as much relief as joy for Marc Wilmots' side at Belo Horizonte's Estadio Mineirao, although they ultimately merited a win that had looked unlikely until the coach began shuffling his pack.
Belgium had began the match as overwhelming favourites - considered even as dark horses for the Trophy - but it was Algeria who began by far the brighter of the two teams. After forcing a few early set-pieces and pinning the Europeans back in their own half, Les Fennecscarved out their first big chance on 18 minutes when an off-balance Riyad Mahrez sliced wide from the angle of the box after bursting through in the left channel.
They continued to look the livelier and more determined of the two teams, though, and on 25 minutes took the lead and ended their long wait for a FIFA World Cup goal.
Algeria hadn't scored at the tournament since 1986 - a run of 506 consecutive minutes of football - but with Bolivia's unwanted World Cup record (517 minutes) beckoning, they finally found a way through.
The goal came from the penalty spot, with Sofiane Feghouli - having been tugged back by Jan Vetonghen to win the kick - dispatching it himself with a coolly placed side-foot shot low to the left of the wrong-footed Thibaut Courtois.
Belgium attempted to respond, but were lacking in both penetration and ideas. Indeed, it was holding midfielder Alex Witsel - frustrated by the lack of success being enjoyed by those in front of him - who had Les Diables Rouges' best two efforts of the half, crashing in 30-yard shots that on each occasion forced Rais Mbolhi to parry hastily clear.
That same duo were reunited early in the second half, with the Algeria keeper flapping at a corner but applying just enough of a touch to prevent Witsel heading home. But the north Africans remained a threat at the other end and very nearly doubled their lead on 57 minutes when Carl Medjani's glancing header flashed just wide of the far post.
In their attempts to find a way through a packed Algerian defence, Belgium introduced Mertens, Fellaini and Divock Origi from the bench, and initially it was the latter who came closest to saving the day. But while the 19-year-old's pace took him through on goal and his low shot was powerfully struck, Mbolhi stuck out a leg to make a brilliant instinctive save.
Ultimately, it was Wilmots' final replacement who was finally able to restore parity, with Fellaini looping a magnificent header in off the underside of the bar from Kevin de Bruyne's inswinging cross. There were 20 minutes still to play at that stage, and with Belgian tails up, it was no surprise when they went on to snatch a winner with ten still remaining.
And, once again, it was a substitute who did the damage, with Mertens racing through on to a measured through-ball from Eden Hazard and smashing an unstoppable right-foot shot into the roof of the Algerian net.
Report courtesy: FIFA.com; Image: Wikimedia Commons