Reliability and Validity of Imperial Examination System

Reliability and Validity of Imperial Examination System

The so-called imperial examination is a system that selects the government officials through open examination, fair competition and enrollment of those outstanding candidates. Established in Sui Dynasty and ended in Qing Dynasty, it has a history of 1300 years which is actually a history of struggles among concerns for reliability, fairness and validity. Throughout the period, the concerns for reliability and fairness gradually gained the upper hand as the method, process and the results of selection become more and more reliable, accurate and fair. However, the outcomes of the system have gradually departed further and further from the original and intended target of selecting the candidates with virtues and abilities. In another word, the validity of the system became very poor in the end. I will discuss the reliability and validity of Imperial Examination System in the paper.

Reliability means the stability or consistency of a test over one or more potential sources of error, like time, raters, items, conditions of measurement, etc. In the area of Personnel Assessment & Selection, we often discuss Test-retest reliability, Inter-rater reliability, and Internal consistency reliability. In Tan Dynasty, the imperial examination system entered in the period of rapid development and the contents of the examination got enriched gradually and regulations on the examination tended to be strict. In Song Dynasty, the fairness of examination was highly emphasized and official selection tended to be legitimized and normalized. The raters had little chance to be partial to certain candidates. And law was made for the examination to prevent cheat in the examination and ensure equal competition. See picture 1. All of this had improved Inter-rater reliability, and Internal consistency reliability.Another reason for high reliability is that the system covers several exams in several levels. For example, in Qing Dynasty, the exam system consists of Tongshi、Xiangshi、Huishi and Dianshi, which improved Test-retest reliability.