A bit of statistical pedantry, if I may: as we get the 2024 US Presidential election occuring only the once, even when we have the result will we really be able to tell whether or not James Johnson (40% chance of Kamala Harris win) more or less right than those giving her an over 50% chance?
No... I discussed the Grimmer et al paper a few posts back about the difficulty of adjudicating between prediction models given the paucity of Presidential elections. But put differently - I would be very cautious about stating this election 'leans' any direction on the data we have.
A comment on sampling. While your algebra is correct for samples taken from a Gaussian distribution, what the British Polling Council recommends its members use as a statement is: 2/3 of the samples will be within ± 2% of the actual result and 90% (rather than 95%) of samples will be within ± 4%. Pollsters typically don't use pure random sampling (which should give a Gaussian distribution) but weight their samples according to demographics. What I think this does is to increase the likelihood that a sample will be very close to the true result but, because you cannot get something for nothing, a larger proportion of samples will be further from the true result (1 in 10 beyond 2 sigma rather than 1 in 20). Because a number of pollsters poll in both GB and USA, I expect they use the same methods in both. So I think the probability of a blow-out for one side or the other is larger than you would expect by treating it as a Gaussian distribution.
A bit of statistical pedantry, if I may: as we get the 2024 US Presidential election occuring only the once, even when we have the result will we really be able to tell whether or not James Johnson (40% chance of Kamala Harris win) more or less right than those giving her an over 50% chance?
No... I discussed the Grimmer et al paper a few posts back about the difficulty of adjudicating between prediction models given the paucity of Presidential elections. But put differently - I would be very cautious about stating this election 'leans' any direction on the data we have.
A comment on sampling. While your algebra is correct for samples taken from a Gaussian distribution, what the British Polling Council recommends its members use as a statement is: 2/3 of the samples will be within ± 2% of the actual result and 90% (rather than 95%) of samples will be within ± 4%. Pollsters typically don't use pure random sampling (which should give a Gaussian distribution) but weight their samples according to demographics. What I think this does is to increase the likelihood that a sample will be very close to the true result but, because you cannot get something for nothing, a larger proportion of samples will be further from the true result (1 in 10 beyond 2 sigma rather than 1 in 20). Because a number of pollsters poll in both GB and USA, I expect they use the same methods in both. So I think the probability of a blow-out for one side or the other is larger than you would expect by treating it as a Gaussian distribution.
Thanks Laurence that’s a very useful clarification, which I suspect also helps to explain the 538 probabilities for example.
Good work. I suspect Musk flagging up Polymarket’s odds last week and Trump’s subsequent price crash aren’t mere coincidences.