In 2016, the March equinox occurred at 04:30 (UTC) on March 20th.
Start the clock. Start the calendar.
The earth keeps rotating and begins its orbital trek around the sun. 365 days after we've started our clock, we approached 04:30 UTC on March 20th, 2017. The earth had rotated 365 times in relation to the sun (366 times in relation to the stars) during that trek. But the earth hasn't reached the same spot in its orbit as it was when we started the clock. It still has a little ways to go.
So the earth keeps spinning and we don't hit the equinox point (plane) until 10:28 UTC on March 20th, 2017. It doesn't change anything as far as our clocks are concerned. It still feels like 10:28 (or whatever time zone you're in). But the time of day of reaching that orbital spot has gotten later by almost 6 hours. We note it, but the earth is still spinning on its axis and we keep the clock rolling and keep ticking off the calendar as we start a new trip around the sun.
Another year passes and we confront the same situation as before. It's now March 20th, 2018, but the time of equinox is later in the day, again. Now it's 16:15 UTC. Same reason as before. The earth hasn't completed a full orbit in exactly 365 calendar days and still needs to finish the last bit of that orbit. But the earth keeps rotating and a little under 6 hours, we reach the equinox.
That's two trips around the sun and we've seen the equinox slip 11 hrs and 30 minutes. It isn't a set time difference each time. It varies, but it's within a range of 5 1/2 to 6 hours. But still we roll with it, start the trip around the sun again. On March 20th, 2019, the equinox arrives 21:58 UTC. Same reason. Same approximate time slippage.
A fourth trip around the sun produces yet another slippage in the time of equinox. It's now occurring at 03:50 UTC. But what day? If we didn't input a "leap day" sometime prior, the equinox would fall on 03:50 21 March (UTC). But we do, adding a day to February 2020 and catch the calendar up all in one fell swoop. We basically add 24 "extra hours" to make up for the fact that our calendar had slipped 23 hrs and 40 mins over the previous 4 orbits around the sun, using the equinox as a marker.
It's not perfect because we haven't restored the equinox to the exact minute that it was back in 2016, but that variance takes much longer to matter to our calendar date keeping. But those ~6 "extra hours" per year are simply due to the fact that the solar day DOESN'T "fit" into the solar year as a whole integer. The earth's rotation isn't perfectly matched with the earth's orbit about the sun. We complete the 365 solar rotations (or 366 sidereal rotations) just a little short of 1 full orbit about the sun.
And yes, there are other factors that affect the calculus, but those are less impacting on our annual calendars as is the mismatch of solar day with solar year. Precession, wobble, drag...some of those effects take centuries, millenia or eons to matter, phenomenological. They do matter to astronomers who must be precise, but not to our daily calendar life. Those "extra hours" do matter to us regular folks because we'd notice that seasons would appear to gradually shift during the calendar if we didn't make those "leaping" adjustments.
It would be nice and tidy if we didn't have to worry about those adjustments and the time piece of the sun/earth was tuned for our convenience. But it's not, and it doesn't have to be. We get "extra hours" in a solar year because of that and they don't appear out of nowhere.