In a sampling analysis, two types of costs are involved: cost of collecting data and cost of an incorrect inference resulting from the data. A researcher must take into account the two causes of incorrect inferences:
systematic bias and sampling error. Systematic bias arises due to errors in the sampling procedures; also, it cannot be reduced or eliminated by increasing the sample size. The causes of these errors are detectable and can be rectified. Generally, systematic bias results due to one or more of the following factors:
- Inappropriate sampling frame: In case the sampling frame is inappropriate (a biased representation of the universe), it results in a systematic bias.
- Defective measuring device: When the measuring device shows constant error, it results in systematic bias. In a survey, if the questionnaire or the interviewer is biased, it results in systematic bias. Similarly, if the physical measuring device is defective, it shows systematic bias in the data collected through such a measuring device.
- Non-respondents: If all the individuals included in the sample are not involved, it might cause systematic bias. This is because, in such a situation the possibility of establishing contact from an individual is often correlated with what is to be estimated.
- Indeterminacy principle: Individuals act differently when kept under observation compared to non-observed situations. For instance, if workers are aware that they are being watched during a work study (which will determine their average length of time to complete a task and quota for piece work), they generally tend to work quite slowly. Thus, the indeterminacy principle may also be the cause of systematic bias.
- Natural bias in data reporting: Natural bias of respondents often causes systematic bias in many inquiries. We can find a downward bias in the income data collected by government, whereas we find an upward bias in the income data collected by some social organization. People tend to understate their income if asked about it for tax purposes. But, they overstate the same when it is a question of their social status.
Sampling errors are random variations in the sample estimates. They occur randomly and can be in either direction, are compensatory in nature, and their expected value of such errors is equal to zero. Sampling errors decrease with an increase in the sample size and are of a smaller magnitude in case of a homogeneous population.
Measurement of sampling error is usually called precision of the sampling plan. An increase in sample size improves the precision. But, increasing the sample size has its own limitations: a large-sized sample increases the cost of data collection and also adds to systematic bias. The most effective way to increase precision is to select a sampling design, which has smaller sampling error for a given sample size at a given cost. However, people prefer less precise design because it is easier to adopt the same and also because of the fact that systematic bias can be controlled in a better way in such a design.