Danijel Subasic, battling through an apparent leg injury suffered late in regular time, stopped Russia’s first attempt from the spot and Mario Fernandes sent his shot wide, ending Russia’s unlikely run through the knockout stage.
The lowest-ranked side entering the competition, the hosts had taken a first-half lead though Denis Cheryshev’s blast from distance, only for Croatia to equalise through Andrej Kramaric soon after.
Domagoj Vida directed a header through traffic in the 101st minute to put Croatia ahead, but though Fernandes delivered an answer five minutes from time to force the shootout, his miss from the spot ended Russia’s hopes.
An exhausted and relieved Croatia side — only the second team to win consecutive shootouts in a single World Cup after Argentina in 1990 — will now advance to face England in the semifinals on Wednesday.
The home fans were behind Russia, screaming every time their team went forward in an open start, though Croatia had the better of the early chances.
Ante Rebic used a fine turn in the box to take a shot from a tight angle that was pushed aside for a corner, from which Rebic headed over the bar.
Ivan Rakitic had a good opportunity from a free kick just outside the box after a quarter-hour but could not keep his shot near the target.
Croatia then began to settle into possession, only to concede against the run of play in the 31st minute.
Following a long ball forward, Cheryshev gathered and played a one-two with Artem Dzyuba to get around Luka Modric, then stepped up to launch a shot from 25 yards into the top corner, leaving Croatia keeper Subasic stranded.
The crowd erupted, but Croatia were undeterred and equalised just eight minutes later.
Working the ball quickly upfield near the sideline, Croatia got the ball into the box on the feet of Mario Mandzukic, who fed in a cross for Kramaric to head home from eight yards.
The second half was more cagey as Croatia tried to probe through Russia’s back line, and Kramaric attempted a bicycle kick that was easily gathered by Russia goalkeeper Igor Akinfeev.
Russia made the first change of the match as winger Aleksandr Samedov limped off to be replaced by Aleksandr Yerokhin.
Croatia came within an inch of taking the lead on the hour mark as Akinfeev tried and failed to reach a Mandzukic cross, and the ball fell to Ivan Perisic, who tried to aim a shot inside the post only to see it come off the woodwork and roll through the six-yard box untouched.
It was Perisic’s last action as he was replaced by his Inter Milan teammate Marcelo Brozovic, while Russia brought on Fedor Smolov for Cheryshev.
Croatia continued to dominate possession in Russia’s half, though the hosts had a rare chance in the 72nd minute as Aleksandr Yerokhin found space in the box but sent his header over the bar.
Modric then had an opportunity in the centre of the Russian box but couldn’t get his feet right and his shot was blocked.
Yuri Gazinskiy became Russia’s third substitutes, coming in for Dzyuba, but Croatia nearly ran into trouble after making two more subs, Josip Pivaric and Mateo Kovacic on for Ivan Strinic and Kramaric.
But almost immediately after making their third sub in the 88th minute, Subasic fell to the pitch grabbing his hamstring, and he was forced to continue after receiving treatment and made a save at his near post.
Croatia pressed for a late winner, earning a pair of corners, but the game went to extra-time where after four minutes, right-back Sime Vrsaljko picked up an injury and Vedran Corluka became Croatia’s fourth sub.
Russia had an opportunity when Dejan Lovren gave the ball away in his own third, but was able to recover to muscle Fedor Smolov off the ball.
And almost immediately afterward, Croatia had their winner.
After earning a corner himself, centre-back Vida got on the end of Modric’s cross to head it down and somehow through two Russian defenders and Corluka into the corner.
Russia made their final change in response, bringing on Alan Dzagoev for Aleksandr Golovin, and they nearly had a chance when Subasic leapt for and fumbled a cross, but the whistle came to stop play and the goalkeeper again grabbed this thigh in pain.
Subasic was then tested again in the second period of extra time when he was able to corral Daler Kuzyaev’s half-volley from the edge of the box, then stretched to cut out a cross.
But Pivaric was whistled for handling the ball at the corner of the box, and Dzagoev’s cross was flicked inside the back post by Fernandes to level the game in the 115th minute.
Subasic was forced to make one more save to deny Roman Zobnin, and the game went to penalties.
Smolov began the shootout by sending a soft shot that Subasic was able to paw down after diving to his right, and Brozovic put Croatia ahead.
Dzagoev scored in the second attempt and Russia were back level when Kovacic scuffed his shot, allowing Akinfeev to save.
But the hosts were again on the back foot when Fernandes sent his attempt wide of the post, and though Akinfeev pushed Modric’s shot onto the post, it fell just over the line to put Croatia back in front.
Centre-backs then were successful in the fourth round, Vida answering after Russia’s Sergei Ignashevich found the net, and though Daler Kuzyaev kept Russia alive, Rakitic — who also won the shootout against Denmark in the last round — put Croatia through.
Russia became the third World Cup hosts to have their tournament end on penalties, after Mexico in the 1986 quarterfinals and Italy in the 1990 semifinals.