throw up the sponge
From a custom of the boxing ring, the person employed to sponge a pugilist between rounds throwing his sponge in the air in token of defeat.