The Art of Examination

Is examination an art? Many American school-teachers will at once reply: ‘No; it is an evil.’ Perhaps, if wrongly used, it may be both. War is an evil; but no one will deny that there is a military art which can be studied, ascertained, and applied. The Black Art of the Middle Ages, or that part of it called alchemy, has developed into the chemistry of the present day. The value of an art depends upon the purposes for which it is used, and the intelligence and precision of its application. Formerly employed chiefly for educational objects, measurements of natural and acquired capacity have of late been greatly extended. Civil-service examinations are one example of this; the methods of determining the fitness of candidates for industrial employment are another. Vocational guidance, whereof we hear much nowadays, is based upon discovering the aptitudes and qualities of boys or youths. Psychological tests — to which the recruiting of our soldiers in the World War gave wide prominence, and to which many colleges are giving attention for the purpose of admission or experiment — are a form of examination to measure ability. The whole faculty of one college is said to have submitted to such a test, with the result that the president made the poorest showing, scarcely above the level of a moron — an experience that may account for the hesitation of heads of colleges to accept the tests as an infallible measure of intelligence. But, crude as these things may be, they show the growing desire for some means of determining relative capacities—that is, for a form of examination that shall be reliable for the purpose to which it is applied.

Academic or educational examinations differ from purely psychological tests in that their object is to ascertain not only native ability but also proficiency in the study of specialized subjects. It may seem strange to say that in this country the art of examination is still in its infancy. That is, nevertheless, true; because in the case of school studies it is applied only to immature youths; and because in institutions of higher learning it has been employed chiefly by the instructor to measure the progress of students in a particular course taught by himself. Certainly we have not considered with sufficient care the object, scope, and utility of examinations. May they not be said to have three distinct objects? (1) To measure the progress of pupils; (2) as a direct means of education; (3) to set a standard for achievement.

I

One of our difficulties has come from paying almost exclusive attention to the first of these objects. So far, indeed, has this been carried that there is a tendency in some American schools to regard examinations as indictments for crime — ordeals to be imposed only upon the delinquent; to consider that the good scholar, who is known to have made satisfactory progress, should not be subjected to them. Such a view assumes that their object is only to measure the minimum of attainment, not the degree of excellence; and thereby fails to appreciate their use in stimulating the superior pupils. Boys and youths, who are more freshly aware than their elders of the motives influencing their conduct, naturally organize their sports on a competitive basis, and admire those who prevail therein. If playgrounds had no tests save a minimum athletic requirement they would be as deserted as gymnasia for the physically defective.

Examinations for admission to college — now conducted almost wholly by the College Entrance Examination Board, composed both of school-teachers and college instructors — have been increasingly directed to the single object of measuring the proficiency of the candidates. When the colleges received their students mainly from a small number of distinctly preparatory schools the examiners felt that one of their functions was to set a standard for that preparatory teaching; but the situation has changed since a large proportion of college students have come from schools only a small fraction of whose graduates have any intention of going to college. To direct by entrance examinations the general course of instruction in schools of that kind is clearly impossible, and the attempt to do so has been necessarily and properly abandoned. The examinations are used by the colleges that retain them to discover whether the candidates are, or are not, qualified to pursue college studies; in fact to measure their attainment in subjects deemed to make a good foundation for college work.

In an article in the Atlantic for July 1925, Mr. Morgan Barnes complains of college entrance examinations on the ground that they divert the aim of the teacher from the instruction of his pupils to preparing them for passing the examination. When there is a difference of opinion between the school and the Board about the kind of knowledge desired in any subject, this is no doubt true. If, for example, the Board is of opinion that the most important thing in algebra is the solution or fractioning of highly complex equations, while the teacher believes that the expression in algebraic form of comparatively simple facts is more valuable; or if the Board conceives that, in the study of a foreign language, grammar, syntax, or Greek accents is above all needful, and the teacher cares for little but intelligent and fluent reading; or if in English composition one or the other lays great stress on punctuation — then the teacher must give to his pupils who are preparing for the examination a training which differs from that of the other boys and is, in his opinion, inferior. Such a divergence concerns the object of the study. It is a difference of opinion on an educational question that ought in most cases to be settled by discussion and agreement among the members of the teaching profession.

When no difference of opinion exists on the aim in studying a subject, there seems to be no reason why a teacher should change his methods because of a prospect of examination. If he is teaching his pupils to translate Latin he ought neither to object to their being examined in translating the language nor to change his teaching in consequence. If the examination does not seem to him a fair test of the proficiency of his pupils, either his teaching or the examination is defective. Perhaps neither of them is perfect, for both teacher and examiner are in danger of giving peculiar weight to those aspects of a subject that appear to them most interesting and important, and of magnifying errors that strike them as vital, instead of looking on the matter from the broader standpoint of a general comprehension of the subject.

The writer recalls the case of a teacher of a high-school class in the Constitution of the United States who was misled by a catchword; and on the other hand he was told many years ago of an absurd marking of an examination paper in French. There occurred the words vieille fills. One candidate translated this ‘old maid,’ which was of course right; another translated it ‘old girl,’ which showed a failure to understand the idiom but was also considered right; while a third, who rendered it ‘old maiden lady,’ was marked wrong because there was nothing to indicate that she was a lady. Such a reader was obviously unfit to be an examiner.

Examinations designed principally to measure proficiency are, no doubt, exposed to dangers. One hears of the same book marked quite differently by two readers, because their estimates vary, or because one of them has some crotchet to which he attaches exaggerated importance. In such a case the standard of one or both of the examiners is wrong; and where such a divergence is possible the examination cannot be regarded as an accurate measure. To command confidence the mark should be very closely the same whoever reads the book; and this means that the examiners should have a uniform standard uniformly applied. In other words they should know the art and use it with precision. A divergence in marking shows inexperience, and its existence proves the art in its infancy.

Another danger lies in the nature of the questions. If they are such that of two persons examined, equally proficient, one may happen to know the answer to a question and the other not, there is an element of chance which lessens the accuracy of the test. This is true where the questions deal with details not in themselves significant — if, for example, they call, in an elementary subject, for small facts, for the identification of a list of somewhat obscure historical characters or unfamiliar quotations. In reading the papers of the College Entrance Examination Board one is struck by the fact that this is not the case; that in general the questions demand a comprehension and comment on facts with which every diligent schoolboy offering the subject must be familiar. Curiously enough, experience has shown that lists of names give the crammer his greatest opportunity, for their number is not very large and by coaching his pupil on a score or more of them he is almost certain to strike several of those that will appear on the paper. This article, however, is not concerned with the shortcomings of teachers or examiners, but with the proper objects and use of examinations.

Until recent years the examinations in college have also been in the main tests of knowledge on the part of the students. It is hard to avoid this where they are given for single courses, each including not more than a quarter of the whole work of the student for a year or half-year. Such a course cannot cover a large subject thoroughly, and the instructor, who must select those parts of it which he deems most important for his purpose, must confine his examination to what he has covered. Questions set by an outsider more or less unfamiliar with the course would obviously be unfair; and if the practice of outside examiners in single courses were general the instructor would be too much of a coach for an examination set by someone else. In this respect college courses are quite different from those taught in school, for the latter are simple, elementary, and comparatively well defined in scope; whereas college courses are far more elaborate, go much more into detail, and, with some exceptions in the earlier years, deal as a rule with parts of large subjects, which are capable of being treated in various ways. Examination by an outsider presents, therefore, in the case of single courses in college, difficulties that arise much less in the elementary subjects of a school curriculum. No system of outside examiners in single courses has, however, been attempted; and the primary object of the examinations set by the instructors is to ascertain whether the work has been faithfully and intelligently done or not, and how great has been the proficiency attained. Rarely is it the only means of measuring these things, but, save in research courses, it is essential. Not infrequently a student thinks he is doing well until an examination reveals to him how much he is below what he had supposed; and an inexperienced instructor is sometimes shocked, on reading his examination books, to find how little his class has been learning. We hear much, no doubt, of the evils of studying for marks instead of knowledge, but every faculty is aware that if an instructor has the reputation of setting easy papers and marking them leniently, many students in his courses will not do serious work, however good and inspiring his teaching may be.

The question of studying for marks rather than for knowledge, and the kindred matter of cramming for examinations, are not uninteresting and are often misunderstood. The popular impression of studying for marks is that a student whose primary object is a high grade devotes himself assiduously to memorizing small, and comparatively unimportant, points in a course, and thereby makes a better showing than a classmate with greater natural ability and perhaps a larger real command of the subject. The criticism is especially leveled at the so-called ‘grind,’who is very diligent but not very intelligent. As the questions are often made out and marked this result may, and does, occur. But if all examinations were so conducted as to be an accurate and complete measure of the education the course is intended to give, if the questions were so framed that mere diligence without a high degree of capacity would not earn the highest grade, then there would be no reason why the student should not work for marks, and good reason why he should. To chide a tennis-player for training himself with a view to winning a match, instead of acquiring skill in the game, would be absurd, because the two things are the same. The match is the best test of his skill; and if it were not it would lose its interest. If, on the contrary, marks in examination do not measure accurately comprehension of the subject as taught in the course and the power to handle it, the instructor is at fault, for his examination docs not measure what it should. However well adapted as a test of minimum diligence, it fails to measure excellence. The word ‘fault’ is, perhaps, too strong; because the art of examination is not only still imperfect but is also exceedingly difficult. We had better say that, if marks are not an adequate measure of what the course is intended to impart, then the examination is defective. If examinations were perfect the results would command universal respect, and high grades would be a more general object of ambition.

Of cramming, less need be said. Since a thorough grasp of the subjectmatter of a course must be the result of serious and prolonged study, which cannot be done in a night or even in a few days, the better the examination the less will cramming be useful in passing it. Reviewing is, of course, essential, and is of great value in bringing out the relation between the things learned. The need of review is, in fact, one of the educational benefits of examination. But the questions should not be susceptible of answer by merely committing to memory facts and formulæ. Yet it must not be assumed that the capacity to cram is altogether without value. A lawyer crams his case before going into court; so does an orator the facts to be presented in his speech; and this quality, like others, can be improved by practice.

A more serious evil of examinations set by the instructor, as those in single courses must be, is the tendency to give back to him what he has given, to absorb his ideas and repeat them rather than to think about them. All good university professors like independent thought on the part of their students; but undergraduates rarely appreciate this, and the path of least resistance is to repeat rather than reflect. The way to counteract such a tendency is to set questions that require thought more than memory; and this brings us to the second object of examinations, their use as a means of education.

II

Although examinations in single courses, whether in college or in school, have as their primary object to measure the progress of pupils, this is far from being their only object. They can and should be used also as a distinct element in the educational process, and as such can be made highly effective. Even if the sole aim of education at any stage were the knowledge of facts, the formal effort to recall those facts to mind unexpectedly, to exert pressure upon the memory and bring to consciousness things half-hidden there, is of great value by serving to make that faculty readily responsive to a call.

When the writer was a professor teaching a large freshman course he often told the class, before the midyear or final paper, that the art of passing examinations was one most useful to acquire. This was a surprise, and provoked a laugh; but it was explained that a lawyer trying a case in court was often confronted by an unexpected question of evidence and must at once try to recall any decisions bearing on the point he had ever come across, and that this was passing an examination; that a physician suddenly called to a suffering patient was in a similar position — at the bedside he also passes an examination. Throughout our lives we are constantly forced to muster all we can of our previous knowledge, and the habit of doing so can be cultivated by practice. How often when the occasion has passed do we ask ourselves, as a student does after an examination, why we did not remember some essential fact. The art of recalling quickly, fully, and accurately is certainly a valuable part of mental training. It is a special art, not the same thing as a rich store of knowdedge. Some men have all the knowledge they possess ready for use on demand; some require a certain time for reflection before they can produce it; and some can make use of it only in the solitude of their studies. The late Francis A. Walker said that every man had his personal equation, and that his own was two minutes. More than to scholars and writers is the value to men of affairs of recalling rapidly the knowledge that they need.

But a knowdedge of facts is a small part of education. We hear much today of teaching by problems; and rightly, because bare facts are of little value unless one knows how to use them. The important thing is to understand their relation to one another; to be able to correlate them, as the current expression goes; not merely to grasp and retain the relations one has been taught, but to perceive new relations, for no teacher can cover more than a minute fraction of the combinations actually met in the pursuit of any subject. The pupil must learn to apply principles to new and unexpected conditions, and the extent to which he can do so will largely determine the degree of his future effectiveness. Teaching by problems is peculiarly needed today because of the change in school methods that came a generation or more ago. Formerly schoolboys were set tasks to work out by themselves; then they recited. But now the teacher, instead of merely hearing the recitation, correcting mistakes, and helping to explain difficulties, takes a more direct part in instruction, thereby saving the pupil much labor, but also some of the benefit of personal effort. In a recent address at the California Institute of Technology, Professor William B. Munro remarked that when he began to teach in a small New England college he tried the experiment of giving his students a list of books which they were expected to read and study for themselves. This was an unheard-of innovation, far from popular; and one of his students said to him very frankly: ‘We don’t think you are playing fair. As we understand it you are paid by the college to read these books and tell us what is in them. Instead of that you tell us to go read them for ourselves.’ That young man had a common but wholly false idea of the relation of teacher and pupil and of the business in which they are engaged. His conception has been called the perambulator method of education. All real education is, in the ultimate analysis, selfeducation. The teacher can impart facts and principles, can point out the way, can interest and stimulate, but only the pupil can train his own mind. Without effort on his part the instruction will be as stale, flat, and unprofitable as lectures on swimming to a class that never enters the water. Moreover, to reach a high grade of intellectual activity the student must be something of an explorer. He must not only be critical of the ideas presented to him, but he must seek variations of them, or applications of them, new to him. That is the meaning of teaching by problems. They force the pupil to make a personal effort at their solution; and, with manifold differences in form, they are applicable to all subjects above the lowest elementary stage.

Problems may be used in a variety of ways; they may be given out to be solved in class or to be taken away and solved out of class. The theses required in college courses are really of that nature; and all these things are admirable for their appropriate uses, but none of them, save the larger kind of thesis, fills quite the purpose of a final examination. Problems to be worked in class or outside must relate to the part of the subject under consideration at the time, but the final examination comes at the end of a period of study and the problems can therefore be more comprehensive, involving the use of all that has been learned. It can also be of greater length than the classroom hour, and hence the problems can be more complex and searching. In short, it is a higher type of intellectual operation, for it can compel the student to reflect upon all that the course has covered, and give him a better conception than he would otherwise have of its scope and meaning. Someone may object that, as the final examination comes at the end, little good will be done to the student by knowing what he might have obtained from the course and did not. The answer to this is twofold. First, that the students have a keen appreciation of the kind of examination that will be given. They are told by their predecessors, even if they have not had experience at earlier examinations, and they look up previous papers in the course. Secondly, if they have not known what the nature of the examination in that course would be, they have learned what to expect in subsequent school or college years.

Such is the character that an examination may have when used as a recognized factor in the process of education, but in practice there is a distinct difficulty in combining this function with that of measuring the work of the students, ascertaining how far they have paid close attention to the lectures and done the reading required. Every instructor is confronted by the difficulty of setting a paper that will measure the diligence of the lower end of the class and at the same time give scope to the independent thought of the better scholars. In large classes experiments have been tried of alternative questions for honor men, some of them with much success, and a judicious use of options is one of the methods by which papers can be improved.

Both of the foregoing objects of examination — to measure the progress of pupils and as a direct means of instruction — should be kept in mind, for both are important; and, in its complete effect, the second more than the first. If it were proposed to lengthen the period of examination many teachers in all grades of education would instinctively object that, for both teacher and students, it would take away time that should be devoted to instruction. But such a view fails wholly to appreciate the value of examinations as an educational agency. It lays too much stress on instruction as the imparting of knowledge, in contrast with the personal efforts of the student to express, and therefore to comprehend and make his own, what he has learned. Teaching and examination are complementary processes, and each should be given the attention and time that experience proves to be wise.

III

The third object of examinations is to set a standard for achievement, and it is the most important of all. This touches the gravamen of Mr. Barnes’s criticism. He complains that they divert the teacher from the educational path he is following to a preparation for passing the papers of the College Entrance Board. He complains, in fact, that these papers set a standard to which he must conform, but a bad standard, or one not so good as his own.

The question whether the standards of the Board, composed as it is of special committees on the different subjects — forty per cent of whose members are school-teachers — are better or worse than that of the best, the average, or the poorer schools may well be left to the experts. It is clear, however, that for teachers, a considerable part of whose students expect to take the examinations of the Board, they do, although not so intended, set a standard of some kind. Of course a standard may be good or bad. It may excite keen intellectual effort or promote a dulling type of cram, and hence may be beneficial or injurious. There have been examinations that required merely committing to memory masses of useless data, and made drudges of those preparing for them; but that does not mean that a potent force for good should be discarded because, if not intelligently used, it is capable of abuse. It would be like arguing that, because there have been bad laws, legislation should be abolished; that, since codes of ethics have sometimes taught bad principles, ethics is a subject to be shunned; or — as some people argue — that, physicians with the best training in medical science having often made mistakes, all educated practitioners should be avoided.

That a standard, known to the students, is as important in education as in every other form of training will hardly bo denied. One would not train an athlete for a mile run without having him speed over the track, giving him the time he has made and the records for that distance. To be proficient in anything a man must have a standard and occasionally measure himself thereby. This is, indeed, most important for the capable men. They differ more among themselves than do the average or lower tier, and hence have more need of a standard of excellence. Now in education such a standard can be set by examination, and, in fact, whether so designed or not, examinations regularly taken at the close of a period of study inevitably do set a standard for the character and extent of the work done, both for the students and for the instructors. Moreover, experience proves that college students, and, no doubt, schoolboys also, will soon rise to any reasonable standard required of them. If low, a large part of them will be satisfied by attaining it; if high, they will put forth more effort. Observe the qualification that they will soon rise to the standard required. Although announcement has been made as thoroughly as possible, and has reached everyone concerned, a change of standard is not at once appreciated, and a number of students fall by the wayside for the first year or two; the proportion of failures then becoming normal again. For that reason changes should be made gradually, and not without abundant notice. Nor should the difficulty of the questions or the severity of the marking vary from year to year, — a complaint made of the mathematical papers of the College Entrance Board,—for such inequalities render the standard incomprehensible. A very large number of candidates are not likely to vary much from one year to the next in quality or preparation; so that, unless there is some obvious reason to believe them substantially better or worse, it is well to assume a larger or smaller per cent of failures due to a variation in the difficulty of the cpiestions. A difficult paper should be marked leniently, and an easy one rigorously, with the result that the marks attained will not differ much from year to year.

Since an examination sets the standard for achievement one cannot insist too much on the need of its being a good standard—one that calls for strenuous effort in the exercise of the qualities most important to cultivate. In mature students, such as upper classmen in college and members of professional schools, these are the ability to analyze a complex body of facts, to disentangle the essential factors, to grasp their meaning and perceive their relations to one another; in short, to master a subject as a whole and deal with it intelligently. Now we have seen the difficulty of examinations for this purpose in single undergraduate courses, arising from the fact that they cover only part of a large subject, and must be used primarily to measure the work of the students in that particular course. A number of colleges have, therefore, begun to require a general or comprehensive examination on the subject to which the student has devoted his principal attention; in which he has majored or concentrated, to use the technical term. Such an examination is conducted, not by the instructors in the courses, but by special committees appointed for the purpose. In some places it is required of all the students in a number of subjects, in others only of those intending to graduate with distinction. In some colleges their work is directed by a corps of tutors, who supplement the teaching in the courses, but, far from being assistants in those courses, deal chiefly with the parts of the subject not covered thereby, and treat the student as a man striving by all the means within his reach to master that subject. In other colleges the supervision is given less formally by members of the departments. But in all cases the object is the same: to set as a standard the mastery of the subject as a whole, attained in large part by the students’ independent reading under guidance.

Mastery of a subject depends on interest, but interest grows with mastery and with the personal exertion to acquire it, while both are aroused by the demand of the standard. No doubt the general examination in college, and in a medical school where it has been tried with success, is, and must be, a measure of proficiency (otherwise it would not be taken seriously); it is also a direct means of education; but its greatest importance lies in setting a standard of attainment, and no institution that has adopted it long enough to bear fruit will question its value.

The conclusion to be drawn from the views here presented is that examinations properly used are a vital part of the educational process, but that the art of using them to produce the best results is highly complex and difficult. They should, therefore, be entrusted to the mature teachers who appreciate their value and have had experience in preparing them; there should be consultation among the examiners; and so far as possible the questions should be framed and the books read by the same men; or, if this cannot be wholly done, the readers should be under careful supervision, and their grading constantly scrutinized.

To make a good examination paper is far more difficult than is commonly supposed. To do so requires much time and thought; but upon no part of the educational process can time and thought be better spent.