Examsmanship and the Liberal Arts
Examsmanship and the Liberal Arts
A Study in Educational Epistemology

By William G. Perry, Jr.
Bureau of Study Counsel
Harvard University
"But sir, I don't think I really deserve it, it was mostly bull, really." This disclaimer from a student whose examination we have awarded a straight "A" is wondrously depressing. Alfred North Whitehead invented its only possible rejoinder: "Yes sir, what you wrote is nonsense, utter nonsense. But ah! Sir! It's the right kind of nonsense!"

Bull, in this university, is customarily a source of laughter, or a problem in ethics. I shall step a little out of fashion to use the subject as a take-off point for a study in comparative epistemology. The phenomenon of bull, in all the honor and opprobrium with which it. is regarded by students and faculty, says something, I think, about our theories of knowledge. So too, the grades which we assign to examinations communicate to students what these theories may be.

We do not have to be out-and-out logical-positivists to suppose that we have something to learn about "what we think knowledge is" by having a good look at "what we do when we go about measuring it." We know the straight "A" examination when we see it, of course, and we have reason to hope that the student will understand why his work receives our recognition. He doesn't always. And those who receive lesser honor? Perhaps an understanding of certain anomalies in our customs of grading good bull will explain the students' confusion.

I must beg patience, then, both of the reader's humor and of his morals. Not that I ask him to suspend his sense of humor but that I shall ask him to go beyond it. In a great university the picture of a bright student attempting to outwit his professor while his professor takes pride in not being outwitted is certainly ridiculous. I shall report just such a scene, for its implications bear upon my point. Its comedy need not present a serious obstacle to thought.

As for the ethics of bull, I must ask for a suspension of judgment. I wish that students could suspend theirs. Unlike humor, moral commitment is hard to think beyond. Too early a moral judgment is precisely what stands between many able students and a liberal education. The stunning realization that the Harvard Faculty will often accept, as evidence of knowledge, the cerebrations of a student who has little data at his disposal, confronts every student with an ethical dilemma. For some it forms an academic focus for what used to be thought of as "adolescent disillusion." It is irrelevant that rumor inflates the phenomenon to mythical proportions. The students know that beneath the myth there remains a solid and haunting reality. The moral "bind" consequent on this awareness appears most poignantly in serious students who are reluctant to concede the competitive advantage to the bullster and who yet feel a deep personal shame when, having succumbed to "temptation," they themselves receive a high grade for work they consider "dishonest."

I have spent many hours with students caught in this unwelcome bitterness. These hours lend an urgency to my theme. I have found that students have been able to come to terms with the ethical problem, to the extent that it is real, only after a refined study of the true nature of bull and its relation to "knowledge." I shall submit grounds for my suspicion that we can be found guilty of sharing the students' confusion of moral and epistemological issues.

I:

I present as my "premise," then, an amoral fabliau. Its hero-villain is the Abominable Mr. Metzger '47. Since I celebrate his virtuosity, I regret giving him a pseudonym, but the peculiar style of his bravado requires me to honor also his modesty. Bull in pure form is rare; there is usually some contamination by data. The community has reason to be grateful to Mr. Metzger for having created an instance of laboratory purity, free from any adulteration by matter.

The more credit is due him, I think, because his act was free from premeditation, deliberation, or hope of personal gain.

Mr. Metzger stood one rainy November day in the lobby of Memorial Hall. A junior, concentrating in Mathematics, he was fond of diverting himself by taking part in the drama, a penchant which may have had some influence on the events of the next hour. He was waiting to take part in a rehearsal in Sanders Theater, but, as sometimes happens, no other players appeared. Perhaps the rehearsal had been cancelled without his knowledge? He decided to wait another five minutes.

Students, meanwhile, were filing into the Great Hall opposite, and taking seats at the testing tables. Spying a friend crossing I the lobby toward the Great Hall's door, Metzger greeted him and extended appropriate condolences. He inquired, too, what course his friend was being tested in.

"Oh, Soc. Sci. something-or-other."

"What's it all about?" asked Metzger, and this, as Homer remarked of Patroclus, was the beginning of evil for him.

"It's about Modern Perspectives on Man and Society and All That," said his friend. "Pretty interesting, really."

"Always wanted to take a course like that," said Metzger. "Any good reading?"

"Yeah, great. There's this book" - his friend did not have time to finish.

"Take your seats please" said a stern voice beside them. The idle conversation had somehow taken the two friends to one of the tables in the Great Hall. Both students automatically obeyed; the proctor put blue-books before them; another proctor presented them with copies of the printed hour-test.

Mr. Metzger remembered afterwards a brief misgiving that was suddenly overwhelmed by a surge of curiosity and puckish glee. He wrote "George Smith" on the blue book, opened it, and addressed the first question.

I must pause to exonerate the Management. The Faculty has a rule that no student may attend an examination in a course in which he is not enrolled. To the wisdom of this rule the outcome of this deplorable story stands witness. The Registrar, charged with the enforcement of the rule, has developed an organization with procedures which are certainly the finest to be devised. In November, however, class rosters are still shaky, and on this particular day another student, named Smith, was absent. As for the culprit, we can reduce his guilt no further than to suppose that he was ignorant of the rule, or, in the face of the momentous challenge before him, forgetful.

We need not be distracted by Metzger's performance on the "objective" or "spot" questions on the test. His D on these sections can be explained by those versed in the theory of probability. Our interest focuses on the quality of his essay. It appears that when Metzger's friend picked up his own blue book a few days later, he found himself in company with a large proportion of his section in having received on the essay a C+. When he quietly picked up "George Smith's" bluebook to return it to Metzger, he observed that the grade for the essay was A-. In the margin was a note in the section man's hand. It read; "Excellent work. Could you have pinned these observations down a bit more closely? Compare. in . . . pp . . . 11

Such news could hardly be kept quiet. There was a leak, and the whole scandal broke on the front page of Tuesday's Crimson. With the press Metzger was modest, as becomes a hero. He said that there had been nothing to it at all, really. The essay question had offered a choice of two books, Margaret Mead's "And Keep Your Powder Dry" or Geoffrey Gorer's "The American People." Metzger reported that having read neither of them, he had chosen the second "because the title gave me some notion as to what the book might be about." On the test, two critical comments were offered on each book, one favorable, one unfavorable. The students were asked to "discuss." Metzger conceded that he had played safe in throwing his lot with the more laudatory of the two comments, "but I did not forget to be balanced."

I do not have Mr. Metzger's essay before me except in vivid memory. As I recall, he took his first cue from the name Geoffrey, and committed his strategy to the premise that Gorer was born into an "Anglo Saxon" culture, probably English, but certainly "English speaking" Having heard that Margaret Mead was a social anthropologist, he inferred that Gorer was the same. He then entered upon his essay, centering his inquiry upon what he supposed might be the problems inherent in an anthropologist's observation of a culture which was his own, or nearly his own. Drawing in part from memories of table-talk on cultural relativity* and in part from creative logic, he rang changes on the relation of observer to observed, and assessed the kind and degree of objectivity which might accrue to an observer through training as an anthropologist. He concluded that the book in question did in fact contribute a considerable range of "'objective', and even 'fresh'," insights into the nature of our culture. "At the same time," he warned, "these observations must be understood within the context of their generation by a person only partly freed from his embeddedness in the culture he is observing, and limited in his capacity to transcend those particular tendencies and biases which he has himself developed as a personality in his interaction with this culture since his birth. In this sense the book portrays as much the character of Geoffrey Gorer as it analyzes that of the American people." It is my regrettable duty to report that at this moment of triumph Mr. Metzger was carried away by the temptations of parody and added, "We are thus much the richer."

In any case, this was the essay for which Metzger received his honor grade and his public acclaim. He was now, of course, in serious trouble with the authorities.

I shall leave him for the moment to the mercy of the Administrative Board of Harvard College and turn the reader's attention to the section man who ascribed the grade. He was in much worse trouble. All the consternation in his immediate area of the Faculty and all the glee in other areas fell upon his unprotected head. I shall now undertake his defense.

I do so not simply because I was acquainted with him and feel a respect for his intelligence; I believe in the justice of his grade! Well, perhaps "Justice" is the wrong word in a situation so manifestly absurd. This is more a case in "equity." That is, the grade is equitable if we accept other aspects of the situation which are equally absurd. My proposition is this: if we accept as valid those C grades which were accorded students who, like Metzger's friend, demonstrated a thorough familiarity with the details of the book without relating their critique to the methodological problems of social anthropology, then "George Smith" deserved not only the same, but better.

The reader may protest that the Cs given to students who showed evidence only of diligence were indeed not valid and that both these students and "George Smith" should have received E's. To give the diligent E is of course not in accord with custom. I shall take up this matter later. For now, were I to allow the protest, I could only restate my thesis: that "George Smith's" E would, in a college of liberal artist be properly a "better" E.

At this point I need a short-hand. It is a curious fact that there is no academic slang for the presentation of evidence of diligence alone. "Parroting" won't do; it is possible to "par-rot" bull. I must beg the reader's pardon, and, for reasons almost too obvious to bear, suggest "cow."

Stated as nouns, the concepts look simple enough:
cow (pure): data, however relevant, without relevancies.
bull (pure): relevancies, however relevant, without data.
The reader can see all too clearly where this simplicity would lead. I can assure him that I would not have imposed on him this way were I aiming to say that knowledge in this university is definable as some neuter compromise between cow and bull, some infertile hermaphrodite. This is precisely what many diligent students seem to believe: that what they must learn to do is to "find the right mean" between "amounts" of detail and "amounts" of generalities. Of course this is not the point at all. The problem is not quantitative, nor does its solution lie on a continuum between the particular and the general. Cow and bull are not poles of a single dimension. A clear notion of what they really are is essential to my inquiry, and for heuristic purposes I wish to observe them further in the celibate state.

When the pure concepts are translated into verbs, their complexities become apparent in the assumptions and purposes of the students as they write:

To cow (v. intrans.) or the act of cowing:
To list data (or perform operations) without awareness of, or comment upon, the contexts, frames of reference, or points of observation which determine the origin, nature, and meaning of the data *(or procedures). To write on the assumption that "a fact is a fact." To present evidence of hard work as a substitute for understanding, without any intent to deceive.
To bull (v. intrans.) or the act of bulling:
To discourse upon the contexts, frames of reference and points of observation which would determine the origin, nature, and meaning of data if one had any. To present evidence of an understanding of form in the hope that the reader may be deceived into supposing a familiarity with content.
At the level of conscious intent, it is evident that cowing is more moral, or less immoral, than bulling. To speculate about unconscious intent would be either an injustice or a needless elaboration of my theme. It is enough that the impression left by cow is one of earnestness, diligence, and painful naivete. The grader may feel disappointment or even irritations but these feelings are usually balanced by pity, compassion, and a reluctance to hit a man when he's both down and moral. He may feel some challenge to his teaching, but none whatever to his one-ups-manship. He writes in the margin: "See me."

We are now in a position to understand the anomaly of custom: As instructors, we always assign bull an E, when we detect it; whereas we usually give cow a C, even though it is always obvious.

After all, we did not ask to be confronted with a choice between morals and understanding (or did we?), We evince a charming humanity, I think, in our decision to grade in favor of morals and pathos. "I simply can't give this student an E after he has worked so hard." At the same time we tacitly express our respect for the bullster's strength. We recognize a colleague. If he knows so well how to dish it out, we can be sure that he can also take it.

Of course it is Just possible that we carry with us, perhaps from our own school-days, an assumption that if a student is willing to work hard and collect "good hard facts" he can always be taught to understand their relevance, whereas a student who has caught on to the forms of relevance without working at all is a lost scholar.

But this is not in accord with our experience.

It is not in accord either, as far as I can see, with the stated values of a liberal education. If a liberal education should teach students "how to think," not only in their own fields but in fields outside their, own - that is, to understand "how the other fellow orders knowledge," then bulling, even in its purest form, expresses an important part of what a pluralist university holds dear, surely a more important part than the collecting of "facts that are facts" which schoolboys learn to do. Here then, good bull appears not as ignorance at all but as an aspect of knowledge. It is both relevant and "true." In a university setting good bull is therefore of more value than "facts," which, without a frame of reference, are not even "true" at all.

Perhaps this value accounts for the final anomaly: as instructors, we are inclined to reward bull highly, where we do not detect its intent, to the consternation of the bullster's acquaintances. And often we do not examine the matter too closely. After a long evening of reading blue books full of cow, the sudden meeting with a student who at least understands the problems of one's field provides a lift like a drought of refreshing wine, and a strong disposition toward trust.

This was, then, the sense of confidence that came to our unfortunate section man as he read "George Smith's" sympathetic considerations.

II:

In my own years of watching over students' shoulders as they work, I have come to believe that this feeling of trust has a firmer basis than the confidence generated by evidence of diligence alone. I believe that the theory of a liberal education holds. Students who have dared to understand man's real relation to his knowledge have shown themselves to be in a strong position to learn content rapidly and meaningfully, and to retain it. I have learned to be less concerned about the education of a student who has come to understand the nature of man's knowledge, even though he has not yet committed himself to hard work, than I am about the education of the student who, after one or two terms at Harvard is working desperately hard and still believes that collected "facts" constitute knowledge. The latter, when I try to explain to him, too often understands me to be saying that he "doesn't put in enough generalities." Surely he has "put in enough facts."

I have come to see such quantitative statements as expressions of an entire, coherent epistemology. In grammar school the student is taught that Columbus discovered America in 1492. The more such items he gets "right" on a given test, the more he is credited with "knowing." From years of this sort of thing it is not unnatural to develop the conviction that knowledge consists of the accretion of hard facts by hard work.

The student learns that the more facts and procedures he can get "right" in a given course, the better will be his grade. The more courses he takes, the more subjects he has "had," the more credits he accumulates, the more diplomas he will get, until, after graduate school, he will emerge with his doctorate, a member of the community of scholars.

The foundation of this entire life is the proposition that a fact is a fact. The necessary correlate of this proposition is that a fact is either right or wrong. This implies that the standard against which the rightness or wrongness of a fact may be judged exists-someplace - perhaps graven upon a tablet in a Platonic world outside and above this cave of tears. In grammar school it is evident that the tablets which enshrine the spelling of a word or the answer to an arithmetic problem are visible to my teacher who need only compare any offerings to it. In high school I observe that my English teachers disagree. This can only mean that the tablets in such matters as the goodness of a poem are distant and obscured by clouds. They surely exist. The pleasing of befuddled English teachers degenerates into assessing their prejudices, a game in which I have no protection against my competitors more glib of tongue. I respect only my science teachers, authorities who really know. Later I learn from them that "This is only what we think now." But eventually, surely . . . . . Into this epistemology of education, apparently shared by teachers in such terms as "credits," "semester hours" and "years of French" the student may invest his ideals, his drive, his competitiveness, his safety, his self-esteem, and even his love.

College raises other questions: by whose calendar is it proper to say that Columbus discovered America in 1492? How, when and by whom was the year 1 established in this calendar? What of other calendars? In view of the evidence for Leif Ericson's previous visit (and the American Indians), what historical ethnocentrism is suggested by the use of the word "discover" in this sentence? As for Leif Ericson, in accord with what assumptions do you order the evidence?

These questions and their answers are not "more" knowledge. They are devastation. I do not need to elaborate upon the epistemology, or rather epistemologies, they imply. A fact has become at last "an observation or an operation performed in a frame of reference." A liberal education is founded in an awareness of frame of reference even in the most immediate and empirical examination of data. Its acquirement involves relinquishing hope of absolutes and of the protection they afford against doubt and the glib-tongued competitor. It demands an ever widening sophistication about systems of thought and observation. It leads, not away from, but through the arts of gamesmanship to a new trust.

This trust is in the value and integrity of systems, their varied character, and the way their apparently incompatible metaphors enlighten, from complementary facets, the particulars of human experience. As one student said to me: "I used to be cynical about intellectual games. Now I want to know them thoroughly. You see I came to realize that it was only when I knew the rules of the game cold that I could tell whether what I was saying was tripe."

We too often think of the bullster as cynical. He can be, and not always in a light-hearted way. We have failed to observe that there can lie behind cow the potential of a deeper and more dangerous despair. The moralism of sheer work and obedience can be an ethic that, unwilling to face a despair of its ends, glorifies its means. The implicit refusal to consider the relativity oi both ends and means leaves the operator in an unconsidered proprietary absolutism, History bears witness that in the pinches this moral superiority has no recourse to negotiation, only to force.

A liberal education proposes that man's hope lies elsewhere: in the negotiability that can arise from an understanding of the integrity of systems and of their origins in man's address to his universe. The prerequisite is the courage to accept such a definition of knowledge. From then on, of course, there is nothing incompatible between such an epistemology and hard work. Rather the contrary.

I can now at last let bull and cow get together. The reader knows best how a productive wedding is arranged in his own field. This is the nuptial he celebrates with a straight A on examinations. The masculine context must embrace the feminine particular, though-:itself "born of Woman." Such a union is knowledge itself, and it alone can generate new contexts and new data which can unite in their turn to form new knowledge.

In this happy setting we can congratulate in particular the Natural Sciences, long thought to be barren ground to the bullster. I have indeed drawn my examples of bull from the Social Sciences, and by analogy from the Humanities, Essay-writing in these fields has long been thought to nurture the art of bull to its prime. I feel, however, that the Natural Sciences have no reason to feel slighted. It is perhaps no accident that Metzger was a mathematician. As part of my researches for this paper, furthermore, a student of considerable talent has recently honored me with an impressive analysis of the art of amassing "partial credits" on examinations in advanced physics. Though beyond me in some respects, his presentation confirmed my impression that instructors of Physics frequently honor on examinations operations structurally similar to those requisite in a good essay.

The very qualities that make the Natural Sciences fields of delight for the eager gamesman have been essential to their marvelous fertility.

III:

As priests of these mysteries, how can we make our rites more precisely expressive? The student who merely cows robs himself, without knowing it, of his education and his soul. The student who only bulls robs-himself, as he knows full well, of the joys of inductive discovery, - that is, of engagement. The introduction of frames of reference in the new curricula of Mathematics and Physics in the schools is a hopeful experiment. We do not know yet how much of these potent revelations the very young can stand, but I suspect they may rejoice in them more than we have supposed. I can't believe they have never wondered about Lief Ericson and that word "discovered, or even about 1492. They have simply been too wise to inquire.>
Increasingly in recent years better students in the better high schools and preparatory schools are being allowed to inquire. In fact they appear to be receiving both encouragement and training in their inquiry. I have the evidence before me.

ach year for the past five years all freshmen entering Harvard and Radcliffe have been asked in freshman week to "grade" two essays answering an examination question in History. They are then asked to give their reasons for their grades. One essay, filled with dates, is 99% cow. The other, with hardly a date in it, is a good essay, easily mistaken for bull. The "official" grades of these essays are, for the first (alas!) C+, "because he has worked so hard," and for the second (soundly, I think) B+. Each year a larger majority of freshmen evaluate these essays as would the majority of the faculty, and for the faculty's reasons, and each year a smaller minority give the higher honor to the essay offering data alone. Most interesting, a larger number of students each year, while not overrating the second essay, award the first the straight E appropriate to it in a college of liberal arts.

For us who must grade such students in a university, these developments imply a new urgency, did we not feel it already. 'Through our grades we describe for the students, in the showdown, what we believe about the nature of knowledge. The subtleties of bull are not peripheral to our academic concerns. That they penetrate to the center of our care is evident in our feelings when a student whose good work we have awarded a high grade reveals to us that he does not feel he deserves it. Whether he disqualifies himself because "there's too much bull in it," or worse because "I really don't think I've worked that hard," he presents a serious educational problem. Many students feel this sleaziness; only a few reveal it to us.

We can hardly allow a mistaken sense of fraudulence to undermine our students' achievements. We must lead students beyond their concept of bull so that they may honor relevancies that are really relevant. We can willingly acknowledge that, in lieu of the date 1492 a consideration of calendars and of the word "discovered," may well be offered with intent to deceive. We must insist that this does not make such considerations intrinsically immoral, and that, contrariwise, the date 1492 may be no substitute for them. most of all we must convey the impression that we grade understanding qua understanding. To be convincing, I suppose we must concede to ourselves in advance that a bright student's understanding is understanding even if he achieved it by osmosis rather than by hard work in our course.

These are delicate matters. As for cow, its complexities ire not what need concern us. Unlike good bull, it does not represent partial knowledge at all. It belongs to a different theory of knowledge entirely. In our theories of knowledge it represents total ignorance, or worse. yet, a knowledge downright inimical to understanding. I even go so far as to propose that we award no more Cs for cow. To do so is rarely, I feel, the act of mercy it seems. Mercy lies in clarity.

The reader may be afflicted by a lingering curiosity about the fate of Mr. Metzger. I hasten to reassure him. The Administrative Board of Harvard College, whatever its satanic reputation, is a benign body. Its members, to be sure, were on the spot. They delighted in Metzger's exploit, but they were responsible to the Faculty's rule. The hero stood in danger of probation. The debate was painful. Suddenly one member, of a refined legalistic sensibility, observed that the rule applied specifically to "examinations" and that the occasion had been simply an hour-test. Mr. Metzger was merely "admonished."

* "An important part of Harvard's education takes place during meals in the Houses." -- An official Publication.

This essay was written at the request of the Committee on Educational Policy of the Faculty of Arts and Sciences, Harvard University, as a contribution to a collection of papers on the subject of examinations, March 1963. It can be found as one of the essays in the volume entitled "Examining in Harvard College: a collection of essays by members of the Harvard faculty," Cambridge: Harvard University, 1963, and is held in the Widener, Gutman, and Sociology libraries at Harvard, in addition to the Harvard University Archives.