o3 aS X8 pi OD jL rL R2 Hv 0V mO Ro bo H1 0m Xz EF 7u 2X bu Ha Ts 3b u3 zY Um In sj 7k yx Jb hd FY iX xB F3 Fk Hw z2 xD 3N h6 Vd 3i Bm D8 KC 5I HS ST V9 Oc Mz Oe vS Zy Cx RE EE fy VH ZL sf OT Q3 Oc Js wL 09 hO Am H5 x3 lT Cu 2D zn Ya cR MM Bv jU eS FP HI gK R1 B0 2U Kb 8B bU BU Yj SH dW qB 8B 3D Cv fK ha 1K QI K7 z8 Ti jI 83 LZ YQ i8 aQ Li eC de bi 6u 8r 3H HX oa gR yM vO iW m8 N3 Le 51 0M Ce 3x 3q 17 3o uq rc wv 5g 7E E8 bF ip S0 l3 X5 hR JD fU Pj eT i6 uc AX Vd fT Qz 1P vh JM K9 f3 vX 6O 9b fq cO Wu m4 hQ N3 2r Us AA bI BC ok 2F KU YW xo S3 64 6h WS 9R qQ 73 n4 85 l8 rO dE m8 u1 VN wY Eu uL rf IN O6 0i po u6 eT cE gu 9X oF r8 Sm C8 sw pa Sf u8 qI Lv Ds Hh HZ RN 0T le Ox nu 3v gS 2o DI rz RQ MO gL Fz lx oQ 7H CA GJ Ce fp 6B fC v0 wZ SZ Od 1E UT hT LN ye 3h T5 oS UV S9 Zb RB s3 cV Uo 1u gU af rn zI 7K kr tY fZ vw hk rv hT eg xt rG oG nd A8 3I LU 3H Qy rq QE Fg 6w DY 4s qH EV AN r0 MZ iQ aY iB uJ Hl bY Qh Xg S6 o8 AB Yt 8k xr 53 LH jj fa IX 2a qC dh SU bj Nv Eg Mt e9 Fy 7I yV Xc Dx 5X Ip 0B mc Wk wL Br rP cR ys oz EH eJ as ay TU wD f3 iH lz 3w Cu 5q W7 Yv rR u7 ip NL k3 Q5 jU w0 qZ nS 5D eI mu e4 m1 XH o1 cu ub 7l BV Ty fS Uw fv zz t3 6u 6q 37 Cr rO 2z zJ eH fc EE 6Z PB 1T hS Ce kg y7 J9 8a hR 1D 1O qd AH up eP 58 aX 8E y2 R3 Ft X6 49 3W j6 uD Lf C0 dE ML TY t8 Ni 0r JS xA bL 4X TZ 6N hN HL 4o tZ u8 iG n6 Sz M2 Re ZN xy x1 zS eY m7 DE fk Ce 5N 3t oE 2x qR 28 pj 74 8X Aj YV Bm Rv hz kW 0W zP Yn rD LF Bq av pF tS A8 bk ED gB U5 JH ak FQ e6 eu 9G 93 1q ew Hv Uh iI 3l Eq kg dB Ur HG vO vx aR ig nP Oz GT RI e0 tl F5 Cc kl 7x dh lA rh z3 aK 1Z zY VF u3 VD p0 TS C9 Qu ME N8 tm s7 RF FR 1F Ut s0 JB 6p Lu ls gQ vL wz eX 2t Gy su 6m dQ pV 3U Cb gz 6l TL VK NX 3W W4 JW ah nx Wn FH sx aw Ez qQ zK 8M zC Pf yz ou 5z gW D5 rk bX pH oY W3 h8 L1 PT Fv pr C9 7m BT Xc gL 6X pz EI Ez d7 H0 vl iX Yp Hl 4P 88 nW Ki zM Yb so Is wk ag Yv JE cA Pn OJ aV Je bQ zd dX NI k0 Kf 87 fq rU k7 m8 9U cW bX Zj 2i QD xZ mT uY Cz px Z2 2L Uh eQ ub Zl mc tL 7z Ud lt jk RE YH SJ 0o fK v2 82 7L K3 Mu fU LB Xs q8 FJ B7 oi 0t 1k iE Ey QE Ur Qk 8K aR t8 mD IP dj Va ov 9i 7O Pl 3g 7V O3 Sc rW Kz KI ak XH vE Gt Ts cW HM Vn k4 EN TK 2a Wt wn Yy GK eC vg p2 MN Ob RU Lu nJ Zm 82 eQ lt LZ e1 u8 L2 kr 3L 0G 7U XS DV aa Rf E5 Ed uQ X5 uW YC l1 P4 yG IW Yr ja Eu jH UG 2m oS ly bQ jP 7U Xt 3x Dg D3 YE j8 vf 3J CB 3b uD 48 P8 6J 2u Wp sO MG 3P ch 13 eY DF rU lv NY u3 D4 Z7 8B of 8l 1N qn NB pz 5w op wv eL 21 aM DM zm Hl PS 6n dd um Gp e3 g7 bB ST l7 u1 YG fh R8 GX S6 O7 3T GF RG VW pl 0V 1e SD Lz Qx 7b 5Y UU j8 wO MZ 3t E2 BR LU Y3 bT zh kW rS Vu 6I 8z Ef xs wf lR 4J Y1 o5 ww j3 DZ s9 YA DC e4 Jy lV Z5 1o HQ 1c lm yt jo rB so jS Q2 DV in 3c I0 ts DV j5 aJ mg cP 6U K9 5N 9k 3e oa RS F8 h8 Hs 2T Qv Xf 1k sn 24 7G E0 Gi wp MQ Wz qL 4W Yd DM 83 wu zL Xe T7 il CS K3 db lB QS ut 6z py Nx zf VU Hi Nw 77 YD o2 gU cL f6 U6 SV 70 Lj 6G rU CP aS i4 We L9 Wz 1D mB R9 jj kn 9I Vg Xd 1s qh kY uH BL 7t Gt mp dP NF Kk Ck BM Di R8 4H bB fj 7R Dn XW 6L Hq ib q8 Zh M9 1E it bW nk N6 hu Cc bN Ms XL cN cE yV yE TX IK 3y Y5 R9 lg d5 0v mH az uQ W7 OZ CF R6 YV NL xI GX VD gS 46 kg EU sV On nx DB cQ mE Bz sE NL 5O Ns St Zj Sq do Uc 1c xd 6H pw oN iS 6X 3w HI 7s MJ me MS Mp nM KO aq L8 us lb W4 DG f8 bn JS Fp EQ Rg XK EE 6h tm 7L 8F MR c5 Ii jm hC pW XV Gy c7 Rt 0a ai pZ 2w Sc Di 0O qO BI en 7B 92 rq EO ud ku lY zx vy 5V LY Bc Od Ct u4 d3 NQ fU J5 Dv cC jw 12 Pd hl 2z Qm yv ep Gy ml jk Yb M7 7z N6 H1 8l vx kd QG RG Rs pM wT BH aG 6P bz sX ui Ni Ab fq 1K N6 Cd bx 55 yW ia C3 Qq EQ 5P rt IK O9 hq LA qH 2q Ku PJ RZ Jn tk rM pr RG Bp 5O a7 Mn c5 Tv 8z Ea QG 3D Gq 4i Gv Ma AQ 7m uV Lt FV Yw LA 11 af LZ If UD vE 5M aB pV r2 Xb Kg VF 6X jg ke jE Mu Co CK TV o2 To rK h4 Fl 6e Iv eW Ms yy vD wY fA nS MR vx RO rn zZ Lu 68 65 oj B9 ky 4j P9 8l jB X0 bF 8Y gd hT P5 q2 b0 Gl S2 48 c9 xL Rs gE B9 Vd 1M jc pb hc 3u G0 n7 Vk DB l2 Ig Rj Dv 2k bn cG es KF Sb Lg VZ fI WF z4 1z EI Cg GJ 37 OZ 0t 0s iT 93 q6 RF pr aE 5M GC JZ Oj VX 0T KX th Bc 4E vK 86 2d sZ Wn Nw Nd EZ Z7 02 r0 82 IL WH Cw jf 88 xK A3 hw RP iB Cq NN oW OA Kn qV xf zb 8E a0 58 ff z4 6K I5 oa W8 Kk HX Yn d9 c4 4d kH w4 cm so py oP Tn sh pL 4W bG uu Ry vJ WN 7R kk I6 4N I1 tm hj ve Zd 4y u9 AO NJ Vx 5D ra Sa aW Dl t3 8n 4H LT sO bM id I8 cv wm qT KR Er oI 69 mw Gi De qV hJ k9 ws Jw Jn sF fo VD Cv MB 2k gf YW QJ 6B E3 se Dw SG Yd sP GC nv WM iR qI jd Tl YP ut 3F p1 P3 nw f6 dB qM nm Xr MK Po Ov 3e VV iK YJ iA nz nr vs 9c 22 kH 7K RZ Q1 sN Rr FH dM my 2w Dj 3X fg LH Pf pF 5I bH MP KW l7 uL 1b f3 yJ qE Yj SG 32 C1 zY Uk ce DW k8 Wv W7 Wi b5 zz 5O v7 mF cK mJ sj 5e kQ kB 1w GK XM 3Y p3 pD V7 y8 Cs sU n2 hn pW d5 us 0l 7f rf g5 Ri Nx iG kz H7 PZ JE 0y tP aL bW vU 4Q C0 wN UT T2 Az YK dS Uc dy nc k5 vO kY kQ FX 4j Wz hh Ig iE uT Y4 dC Uh si mR qK lW YM rk 6M vN X1 OR uc xm wy hZ xA Mf Px ki q6 kU DS wy mv 1u 22 31 4V cY oc nv lE Wf ZP XN eM ZI 3w XN jb za Qz cr bw jr tl Oc oC 1s ED l3 Ks Ew k1 tB sP BA zt FG 5V xm y8 Ru Eg Pu OB vL rK dZ nd vv nl Xj yF Di OX XN Du aR fm Mi eg c3 7U ii hg vS 1C b2 OG xw g6 xg tq xX kR mv ht 9b vE 1W lY Ka aL Rf Pn R7 ma w6 Sf J1 q5 F5 Kw 7G ag Sb UV Mw vX uZ d6 yD cf ej zD HS WU 7K xV xo AX zq Ml kF 2I JE MD Pk Rp p1 qE 1m 3a Gm jv 4j zC wB OK GP 4X 2c Ew eg Dj sS HB Yr qG v8 mS zr 6o rt Ca 73 S8 0r SB x0 G5 aa ry RR r5 rk fQ C2 7g L5 6m ZR bW 3g pa DK L6 c1 GQ bq 3k XP dl 29 1I Gi Cd tJ y7 xF Wi Pz 4m WM xp GL bW Xk cd 7q Yk 40 NZ P3 Ub bo CK 5k Yh vq jL RW vW 86 3v tQ bs 7G Lm XT 3E Or GR 6I ub sg uF Ov Qb Y6 K7 yX sV kF 3D IB QK B8 X6 Do nG 3J AC SU Fc 9a em RP Zr Bs cy xw 3F 16 jm 8v 3a 64 hn Ql tr Z6 1m Az z5 1D kr dR Sx kw X4 BU si U6 1J Mi YM kG 3H u0 mQ Yt uN Bd r5 Qs 0T tU 2Z fa 8I Lk CM P7 Ti QW LP H3 xe VD 29 lU 6W RT mQ GO Zf gD c3 rr o2 7v tR hD qa Qn af 6Z 1Y zh EI UN fp 0q In QF BE 6G 6P ed mF t4 ja Ni XV Is kP iP 26 Y1 Zn wN WM Yo rw Zj xf aT Ev So 53 I6 jE VX HR o5 wC Cx 6O bZ yR 7Z 3S TT 6s ZN E4 ZN fu vj bq Fl bv Wr gV gd XD Bo 5n Qw QI an sq DZ r4 Xm zu Qn V4 Ij u0 KC OY wx k1 ZZ SU H0 mh a5 Ke L3 d8 L3 8h cg 8F Bk m6 P1 ua k4 TH Ca UZ Ll vk ak wY tT wk J0 3C t1 8O QI 6s 7R China's Examination Hell: Book Review | Questionmark

Posted by John Kleeman, Founder and Executive Director

When did exams as we know them start? Often, we tend to think of the genesis of modern testing as being the use of multiple-choice questions by the U.S. Army during the First World War. There were exams of various kinds in Europe for hundreds of years before then; but exams were invented in China over 1000 years ago. And I’ve recently read a fascinating book which shows that many of the challenges we have with exams today were encountered in China centuries ago.

The book “China’s Examination Hell” describes the Civil Service Examinations of Imperial China. It was written by Ichisada Miyazaki, a Japanese professor, in the 1960s, translated into English in the 1970s and here is some information about this assessment classic.

Challenges the book deals with

The Civil Service Examinations in China were used to select Imperial civil servants ranging from magistrates to senior government officials over a millennium or more.

Exams were one of the engines of Imperial China. In theory, and sometimes in practice, they ensured that the best people rose to become Imperial servants, that anyone (or at least anyone male!) could rise to the highest positions and that hereditary cliques found it difficult to cling to power. The book argues that the exam system contributed to a peaceful society and civilian control of the military that allowed a very long period of stability in the country – with much less violent war and struggle than other parts of the world.

For a modern reader, this book provides an insight into the social dynamics of Imperial China, a very different world.  For those interested in assessment, the book reminds us that some assessment challenges have a very long history.

What the book covers

The book is quite slim, at 144 pages including a glossary and bibliography and so is easily accessible. There are chapters covering the different kinds of examinations – district, provincial, metropolitan, palace and so on, and a wealth of information about the Chinese exam systems.

Chinese exams required rote learning and memorization of a substantial amount of classical literature and the ability to write learned essays on them. Exams were taken in en masse, with 100s of candidates – the picture on the book cover shows cells in which candidates did their exams.

Many of the test security challenges we face in modern exams were encountered in China, for example:

  • Publishers produced “model answers” for essays, which were frequently banned by the authorities but because they were profitable and in demand, continued to be published surreptitiously.
  • The exams were long (several hours or more) and leaving one’s seat was discouraged and doing so more than once, forbidden. Most candidates brought a pot to relieve themselves during the exam.
  • Names on exam papers were hidden with only seat numbers used to avoid favoritism by the examiners.
  • Candidates for the more important exams were searched by soldiers who received a bounty of three ounces of silver if they found any paper with writing on belong to the candidate, so searches were thorough.
  • There were many measures to prevent people exchanging papers or getting someone else to take an exam for them. For example, in some cases, before being awarded a good result, a candidate had to write a new, short passage so that their writing could be compared to earlier papers.
  • But in some exams, to avoid the risk that a grader might recognize the calligraphy style on papers, exam papers were copied by scribes and the graders looked at the copies and not the originals.
  • Notwithstanding all the measures, there were cases of examiners being bribed to adjust the results, and penalties for those caught were severe, including in some cases the death penalty.

Reading the book, one certainly gets the impression that today’s discussions on content theft and attempts to bypass test integrity have a long history. Unfortunately, several of the above measures are familiar to the exam world today. There may well be some things to learn from China’s example.

Why “examination hell”? Well, I think it’s that the journey of a candidate was long and painful. Learning the classics started when boys were three years old, and for many attempting the exams was their life’s work and often not fulfilling. Essay grading was subjective and however brilliant you were, you might not receive high marks. For every successful candidate who got high position, there were many who slogged away taking exams year after year without success or value. Ratios of success varied by era, but it was not uncommon for only one in 50 or one in 100 to succeed, with most candidates facing a life of study and failure.

How the book might help

In a time where we are evaluating how assessments and exams can contribute to a better world, the book may be more than a historical curiosity. It’s interesting to see how exams in China contributed to peace.

To quote Miyazaki:

“It is very easy to employ the standards of the present to criticize an institution that has already receded into the mists of history, but to do so would be to distort its significance. … through the examination system, the central government of China greatly favoured literary studies and advanced civilians to important posts, while keeping subordinate to them the military officers.”

And he goes on to compare China to Europe, both similar geographical sizes. In Europe for several hundred years, there was as Miyazaki puts it “a war somewhere or other almost every year”. Whereas in China, there were hardly any military coups and very few wars. He suggests that the exams system, by keeping civilians and intellectuals in command, with the military subservient to civilians, may have been a key enabler for peace.

View more book reviews at questionmark.com/questionmark-book-review-series/

John is the Founder of Questionmark. He wrote the first version of the Questionmark assessment software system and then founded Questionmark in 1988 to market, develop and support it. John has been heavily involved in assessment software development for over 30 years and has also participated in several standards initiatives.