Plumb bobs point to Earth's center of mass because gravity acts as a vector field radiating toward that center. So basically, plum bobs point directly down. Do remember, though, that when I say down, I mean down as in the localized manifestation of Earth's gravitational field.