the point the sun "sets" is where the rays of light transition from being curved towards the viewer, to being curved up in to the sky yes?
No.
So when does sun set occur then?
If there are rays pointing in all directions being pulled upwards away from the earth, then the sun can be seen when those light rays hit the observer, and NOT seen when those light rays DON'T reach the observer. Just look at that diagram in OP: the light rays at sun set are the final rays that are bending so much that the sun appears directly in front of the observer at sun set. If the observer is FURTHER away, or the sun is further away, then it follows that none of the rays of light will reach the observer any more, because they either hit the earth infront of the observer, or they bend upwards above the observer. No light rays reach the observer, therefore the sun has "set".
No matter what direction the light rays are emitted from the sun, there is no way to avoid the upward pull, just like an archer: they have a limit to the power they can fire an arrow, and gravity or "the force which is accelerating the ground upwards at 9.8m/s/s" is constant: they can aim higher or lower to affect the trajectory and the distance of the arrow, but there's still a limit to how far they can fire the arrow. The sun can emit light rays in all directions, but if all rays are being pulled upwards, then there's a limit to how far those sun rays can reach.