Once it's soft enough, wiggle the cup out as gently as you can. Get the elastic too, and throw the old stuff away.

Photo album created with Web Album Generator