From 1st October you no longer need to display your existing tax disc. This is because the law has changed. For existing tax discs the DVLA advises these can be removed from your car and destroyed.
Remove at will if you wish. However you may wish to keep your tax disc. Tax disc collectors or velologists pay money for these bits of paper. Over £800 has been paid in the past for an old tax disc.
You may have to keep it for some time though. This one was from 1921!