If the light is bent such that it comes at me in an upward direction then I see it below me.
Yes. In Bobby's extreme outer space scenario, the Sun will appear either next to or behind the Earth, and largely downwards from you.
Consider the sun reflected in a puddle of water. The sun looks like it's below me, in the puddle.
This is precisely because the inexplicable straightening you just proposed does
not occur. If your proposed change were implemented, you'd see the Sun in some completely unpredictable location.
As Parsifal already pointed out, all reflected light would accelerate just the same.