Stuart Altmann on wed 9 jun 99
A few days ago on Clayart, Wendy Rosen described major changes over one
year in the ranking of cities as sites for buying arts and crafts. The
rankings were based on a poll of readers of American Style Magazine. In
particular, the results of the poll suggested a major flight of buyers'
allegiances back to big cities as their favorite sources.
The magnitude of these changes in rankings make me suspect that there may
be explanations other than changes in peoples allegiances or buying habits
to explain the results. Here are some alternatives.
1. A change in wording may have altered readers' responses to ostensibly
the "same" question. (See, for examples, publications, such as Norm
Bradburn's, on the design of survey questionaires.)
2. The composition of the magazine's readership--or at least, of those
readers who responded to the survey--may have changed. (An examination of
zip codes of respondents might shed some light on this.)
3. Perhaps most important is the possibility that the putative changes are
largely the result of what statisticians call small-sample artifacts. We
are all familiar with this effect in other contexts, such as that many
families of, say, six children will deviate from an equal number of boys
and girls, even though on each birth, the chance of getting either sex is
50:50. For the same reason, a small sample of readers favoring any given
city would fluctuate from sample to sample (year to year) even with no
change in people's allegiances. Various methods are available for
examining such effects, such as calculating confidence limits on the
proportion of people favoring each city.
Stuart Altmann
email: salt@princeton.edu office 'phone: 609/258-4520
| |
|
