This should be a no brainer because Gap insurance only covers the difference in the price paid and what the insurance company value the vehicle at and pay out should it become a total loss, the depreciation.
With a first owner new straight out off the forecourt vehicle that difference will be significant because of the big hit in depreciation as soon as the vehicle is handed over and should it become a total loss it will be valued as a one previous owner used hand vehicle.
With a used vehicle that initial big hit on depreciation does not exist. Assuming you are not paying over the odds for the vehicle a like for like replacement will be valued close to the price paid since the only depreciation will be in the mileage you put on the clock and the fact it has another owner (one previous owner becomes two previous owners etc).
EDIT: The exception being if your buying with a loan, assuming and check that the Gap Insurance covers theft, depreciation and the outstanding loan repayments.