Like Peeling an Onion…

What goes into designing a study?  It seems simple enough:  frame a research question, collect data, interpret data, share findings.  But, as the work of Fleckenstein, et al suggests, knowledge is relational, existing in the ways a researcher’s conceptualizations of her subject arrange things like data, interpretation, and questioning to form a coherent whole.  Broad, for example, cautions that a research question that is in place before data collection can “inappropriately guide” the collection of that data (Broad 24). 

Perhaps it would be more helpful to begin where Fleckenstein and company suggest:  with the selection and interrogation of a metaphor that will drive the way I conceive of my study, suggesting particular relationships between what I want to know, how I want to know it, and where I might find it.  It occurs to me that much theory focuses on the ways in which particular metaphors arrange the pieces within a specific field (cf. Stephen North’s The Making of Knowledge in Composition).  Brian Huot’s assertions that assessment be conceptualized as research rather than technology, in fact, start by interrogating the metaphors undergirding assumptions about what assessment is and does.  Rather than see assessment as a problem-solving tool (Huot 138), Huot suggests that teachers and theorists see assessment as research, as a way of asking and answering questions—in short, as an epistemology (Huot 163).

So how do I see the world?  More specifically, how do I see the classroom, the world in which I would like to conduct my study?  As a social constructivist, I see it as a site of epistemological inquiry, a place where knowledge is made.  And as a playwright, I see that epistemological inquiry as occurring in a specific place—on a “set,” which can be manipulated to resemble the world and bears a relation to it, but is not “the world” in its entirety—involving particular kinds of people—“role-players,” people who change “hats” or “masks” to fit the exigency of the moment, and whose “hat” or “mask” might inadvertently be changed for them by the constellation of role-players with which they interact—engaged in a particular kind of task—synthesis, the bringing together of disparate parts to create a whole that makes some kind of sense.  Call it a performance.  (It is.)  But don’t call it a sham.

More importantly, a play is an event during which factors act upon one another.  Drama, after all, comes from a Greek word meaning “action.”  Consequently, my metaphorical universe implies one question that undergirds all the others:  What does X do to Y?  That’s why, when Brian Huot introduces me to Lee Cronbach’s notion of validity as inherently rhetorical (Huot 93 – 94), I start scribbling in my notebook.  That’s why, when Fleckenstein and friends beseech researchers to adopt ecological metaphors that “privilege flexibility” and thus, account for change (Fleckenstein et al 399), I lean forward and make eye contact.

But, as Fleckenstein asserts, simply to have a metaphor is not enough.  To interrogate my own metaphor, I have to dip into the theory and practice of my “home” within English Studies:  creative writing.  Wendy Bishop has written both of writers’ pervasive use of metaphor to describe the act of writing, and of the dangers of that practice (Bishop 24, 32 – 33).  Writers, she intimates, tend to conceptualize writing as one of four activities:  cooking, mining, gardening, and fishing or hunting (24). I’ve written and spoken elsewhere of my own self-conception as a “maker” of plays, which, especially given my description of a plays task as synthesizing disparate parts into a whole, clearly puts me in the “cooking” camp.  What I need to determine now is whether my metaphor is ecological—whether, in Fleckenstein’s words, it “directs the researcher’s gaze to relationships, including the researcher’s own active involvement in and contribution to a research ecosystem” (395, emphasis mine).

There’s real danger here.  If I posit myself as the “writer,” I run the risk of seeing my participation in the study as above it all.  To use a metaphor from children’s theatre, I see myself as the puppeteer, without realizing my own strings.  But I know myself well enough to know I won’t do that.  How?  Because my MFA thesis was a play about a writer whose characters gained agency over him, who was held responsible for the effects of his own thoughts on his creations, and who, in the end, was revealed to be someone else’s creation.  (Notice the move I just made there—rather than leave the question “How?” implied, I actually asked it.  I did that purposefully, to show you that I am aware of at least some of the strings that tug on me.)

Speaking of strings that tug on me, let’s talk about the position statement on writing assessment.  Beginning from Brian Huot’s premise that assessment is and always has been a form of research, let’s look at the purposes espoused for assessment by the statement:

Writing assessment can be used for a variety of appropriate purposes, both inside the classroom and outside: providing assistance to students, awarding a grade, placing students in appropriate courses, allowing them to exit a course or sequence of courses, and certifying proficiency; and evaluating programs—to name some of the more obvious. Given the high stakes nature of many of these assessment purposes, it is crucial that assessment practices be guided by sound principles to insure that they are valid, fair, and appropriate to the context and purposes for which they designed. This position statement aims to provide that guidance (CCCC, 2006).

Clearly, research is meant to do things like “provide assistance to students,” and it takes the information collected by some form of research to award grades, place students in classes, and evaluate programs.  But nowhere in that introduction do I see the assertion that assessment itself is the act of collecting information.  From that point on, the statement becomes a collection of ironies for me (which perhaps displays my training as a New Critic):  The first guiding principle states that the purpose of an assessment should guide its use, but only after asserting what the “primary uses” of assessment should be:  “a means of improving teaching and learning.”  One might say that gathering information is an implicit part of improving at teaching and learning.  But one might also say that gathering information should be a more explicit part of teaching and learning, too.  So why isn’t it?

The second principle asserts that “writing is social,” a position my writing center training has long ingrained within me.  What I find curious is why assessment itself, purportedly the subject of the statement, is not conceptualized as social—as occurring within a context, as a form of communication, and above all, as a (communal) way of collecting, creating, and sharing knowledge.  Guiding principle number four appears to exist to slap the third principle in the face.  You thought writing was a social act?  Well, writing ability is individual—varied, fluctuating, context-specific, yes, but individual, nonetheless.  So there.

Of course, the phrase “best assessment practices” prefaces every bullet underneath each of the five guiding principles.  But take a careful look at the construction of the phrases, and you’ll see that it’s assessment that acts on writing.  Not the other way around.  Whether or not writing is typed or handwritten doesn’t influence assessment.  Whether or not writing “answers the assignment” (a form of response to the explicit aspects of assessment) doesn’t influence assessment.  Whether the writing includes pictures or graphics, what font it’s in, how long the sentences are, whether (as I do with “asserts” and Huot does with “contends”) writers favor particular expressions, or group their ideas in particular ways, none of that influences assessment.

(Do you get yet that I’m being facetious?  I mean, I know I’m not supposed to be explicit about irony—that would sap its power as a method of critique.  But I’m not supposed to be explicit about assessment, either, at least not the way Fleckenstein and Huot would ask me to be.  So I figured I’d be explicit about both.  Plus, I can underline how provisional I think things like irony, writing, assessment, weather reports, are—because this essay doesn’t just end with a period.  It also ends parenthetically.  “Don’t look at my conclusions,” it says.  Gloss them, but come back and read those later.  After the important stuff.  Except what does one do about the absence of text after the closing parentheses?)

 

Works Cited

Bishop, Wendy.  Released into Language:  Options for Teaching Creative Writing.  2^nd Ed.  Portland, ME:  Calendar Islands Publishers, 1998.  Print.

Broad, Bob.  What We Really Value:  Beyond Rubrics in Teaching and Assessing Writing.  Logan, UT:  Utah State UP, 2003.  Print.

Fleckenstein, et al.  “The Importance of Harmony:  An Ecological Metaphor for Writing Research.”  In College Composition and Communication.  December 2008:  60.2.  388 – 420.  Print.

Huot, Brian.  (Re) Articulating Writing Assessment for Teaching and Learning.  Logan, UT:  Utah State UP, 2002.  Print.

Conference on College Composition and Communication.  Writing Assessment:  A Position Statement.  6 December 2007.  www.ncte.org.  Web.

 

Safe Houses

Re-reading the title of Huot’s book, I’m reminded that in the first chapter, he indicates that assessment is not simply making a value judgment, but articulating it, which I take to mean elaborating on it, explaining it, relating it to its context—in a sense, acknowledging and fulfilling the inherently rhetorical aspects of an assessment.  That makes me think of Peter Elbow and Nancy Sommers.  Elbow essentially asserts, in “Ranking, Evaluating, and Liking” that generating the kind of descriptive, narrative feedback that might go with a holistic score often causes the evaluator to reconsider that score.  And Huot refers to Sommers’ article “Responding to Student Writing,” in which she contends that English teachers, having been trained to analyze literary texts, nonetheless carry expectations about student texts—expectations that lead them to find errors.

It occurs to me that, in the same way I could assert (or as Huot would say, contend) that T.R. Johnson’s anthology centers around authority, I could argue Huot’s book is entirely about expectations.  Responding to a text for another class, I noted that knowing what to expect can be a powerful indicator that an environment is “safe.”  And as I read Huot, or Elbow, (or for that matter, the quasi self-help book we’re reading in Creative Writing Pedagogy), it occurs to me that it’s safety that I’m struggling to define.  I’m uncomfortable with the way authority has been defined in writing classes—as having the most knowledge, as stemming from nuturance, as coming from the creation of a “safe” environment.  (What does “safe” mean?  That there’s no risk involved?  That it’s OK to look stupid?  That violence won’t be done to you?  For that matter, what is “violence”?)

Each of Huot’s chapters focuses on some aspect of assessment that can control what we expect of it, how the ensuing discussion will be bounded and in what direction it will move.  (So in a sense, each of Huot’s chapters asks the reader to consider the question “Do I feel safe or not?”)  His first chapter, before engaging on the typical “this is a summary of my book” move, focuses on the act of naming, perhaps the quintessential evaluative act.  What strikes me about this act is how invested Huot was in the (Re) of his book’s title, and in the absence of an explicit admission of this “obsession” of his, I find myself wondering whether or not Huot realized how entrenched he was about the title of his book.

Huot’s second chapter focuses on identifying two viewpoints from which most people interested in assessment organize their conceptualization of assessment.  A major part of this task is unpacking the assumptions that “situate” the “educational measurement community,” and those that situate the rhetoric and composition community.  To do this, Huot focuses on historical accounts of assessment from each “school of thought.”  I find myself interested in the way each field uses time to place boundaries around the topic of assessment, determining when it started and at what intervals it changed along with exactly what changed and why.  In a sense, White, Moss, and Yancey have created 15^th-century illuminated style maps of assessment, and at some point at either end along both axes, the map dissolves to the warning “Here There Be Monsters.”

Of course, along with the fifth chapter, the third chapter was the one to which I paid the most attention.  I’ve been both congratulated and chastised for being a “risky” teacher, mostly because I teach often without a script.  Often, I get the sense that my students don’t know what to expect next (which I’d equate with at least some sense of feeling “unsafe”).  I think that’s what Huot comes up against when he tries to get his students to evaluate each others’ work descriptively, without using quantitative, rank-implying language—students know what to do with a statement that implies competence or failure.  The next step after “This reads a lot like a letter to the editor of the school paper” or “I can close my eyes and see what this writing describes” isn’t always clear.  Why?  There’s no endpoint implied in a descriptive comment.  Clearly, a student could actually send a letter to the editor of the school paper, or film a video representation of a visually-descriptive piece of writing.  But they do that, in a sense, not knowing when they’ll fall off the face of the earth into the gaping maw of the F-monster.

In the fourth chapter, Huot unpacks the characteristics of what he would call particularly effective assessments.  But here, Huot focuses on tightening the boundaries around an assessment—focusing, for instance, on an assessment instrument’s validity for a particular occasion, or in a certain context, while implicitly rejecting the idea that an assessment should be universally applicable the way the Stanford-Binet, ACT, SAT, and other such tests strive to be.  It’s here that Huot monkeys most with the ideas of validity and reliability, thus putting those who rely on a definition of “assessment” that also relies on specific definitions of “reliable” and “valid” in an “unsafe,” risky position.  But he does this carefully—any time Huot writes about reliability and validity, he seems to select his words more carefully.  (It’s at these points in his book during which I think Huot feels the least “safe.”).

Chapter five focuses on the act of reading.  I remember reading an article in Kairos called the “Lo-Fi Manifesto,” in which the writer noted that “literacy” is the ability to both consume and create in a particular discourse, with a particular tool, etc.  That writer contended that the technical sophistication of many computer composing tools prevented many individuals from becoming fully literate by making it much harder to create electronic documents than to consume them.  In a sense, that’s what Huot is doing here—arguing that teachers have neglected half of their own literacy.  And what strikes me most about what he describes is the way his students create universes within which the writing they assess exist.  In a way, his students are creating a “safety net,” a constellation of stars by which to navigate the process of providing feedback.

To me, Chapter six turns the whole power dynamic on its head.  Rather than using assessment to confirm what teachers already know, Huot suggests that assessment is a tool by which people can investigate what they don’t know.  But more interesting to me is the way he engages with the term “technology.”  In asserting that assessment has become too driven by the technologies within which it has been developed, Huot asserts that assessment is itself a technology, one that drives teaching (a point that echoes Elbow’s in “Ranking, Evaluating, and Liking”).  What frustrates me most about the term “technology” is the way that term has become synonymous with “computer”—how many students realize that the Choose Your Own Adventure books they read are, in fact, hypertexts?  Or that they are inherently multimodal beings?  It’s refreshing to read someone who realizes that the ballpoint pen is as much a piece of technology as the Pentium chip.

I want to return to that theme of safety.  I think that Huot’s ideas are unsafe, in the sense that they ask those who adopt them to be willing not to know what to expect.  And I think that, whether it comes from, nature, nurture, both, or neither, humans want to know what’s coming next.  In a sense, people are driven by a narrative impulse—a tendency to see events as connected by cause and effect, even when they aren’t.  I think that teachers are particularly vulnerable to this way of thinking—after all, what is a lesson plan if it isn’t an effort to anticipate (and articulate) what the effects of a particular lesson might be?  And that brings up a question for me:  How could we (and why should we) train students not to know what to expect?

 

                                                                   Works Cited

Elbow, Peter.  386 – 406.  “Ranking, Evaluating, and Liking:  Sorting Out Three Forms of Judgment.”  Reprinted in Teaching Composition:  Background Readings.  3^rd ed.  Ed. T.R. Johnson.  Boston:  Bedford/St. Martin’s, 2008.

Sommers, Nancy.  “Responding to Student Writing.”  Reprinted in Teaching Composition:  Background Readings.  3^rd ed.  Ed. T.R. Johnson.  Boston:  Bedford/St. Martin’s, 2008. 377 – 386.

Assessment as (re)Search

This past weekend, [i.e., the weekend of February 13 - 15, 2009], I attended the annual conference of AWP—the Association of Writers and Writing Programs—which is the national organization for writers and teachers of creative writing.  Lo and behold, one of the sessions actually had to do with assessment:  a panel discussion the brochure description for which echoed Edward M. White’s call to assess ourselves before someone else does it.  As described in the brochure, and as introduced by its lead panelist (a former Dean of the College of Arts and Sciences at Vanderbilt who now directs the creative writing program there), the panel appeared to mistrust and fear assessment.  I doubt that most of the panelists would welcome an “assessment expert” if he or she were to introduce him or herself as such.

But at least three of the five panelists impressed me.  One, who appeared to be implicitly responding to another panelist’s call to mislead and distract those who would pass judgment on creative writing programs and practices, framed the situation rhetorically as an act of translation:  “Let’s figure out what they want, figure out what we call it, and then give it the name they will understand.  We’re good at that, after all.”  Another panelist first called the panel’s attention to the differences between summative and formative evaluation and between individual and program assessment, and then remarked, rather obviously, that everyone in the room already practiced assessment—in one-on-one conferences, by collecting portfolios and preparing students to defend book-length theses, even by line editing.

A third panelist, who coordinated a program in writing with a creative writing track, actually outlined a four-point assessment/teaching philosophy that blew me away, and then disavowed it, asserting that no one should utter out loud to the powers that be the kinds of things he was saying.  (Perhaps the fact that he was saying such things out loud meant that comment was made in jest.)  He talked about wanting students to be open to risk, vulnerable, ironically aware of themselves—the kinds of things that seem to me to be the point of taking a writing class—or any class at all.  Maybe he and I just think alike.  Maybe if Joan Mullin knew the kinds of things I was doing in my classroom, she’d demand that I be fired.  Maybe the fact that I also value reflectivity, risk-taking, and openness to change just means I’ve been inculcated into the same Fraternity of Insufferable Idiots that creative writing programs can sometimes be.  But I don’t think so.

It occurs to me that those same values—reflectivity, risk-taking, vulnerability—are the kinds of values that summative assessment can shield teachers from.  While a rubric can give students the ability to engage in reflection, risk-taking, and change by allowing them to name those heretofore indecipherable feelings and intuitions with which their public selves coexist and negotiate, it can also give the teacher the ability to resist such activities.  To both teachers and students, a rubric or set of “standards” can serve as a tool—but teachers can use that rubric as a way of justifying their impressions rather than interrogating and transforming them.

I think of the annotated bibliography assignment my students completed a while back.  In my experience, the most difficult aspect of research is resisting the impulse to come into a project with a result already decided.  One of my students, for example, wanted to use the materials she gathered to prepare her COM 110 speech asserting that abortion should be legal as the foundation of her annotated bibliography.  But in formulating the answer to that, as Gabe Gudding would say, “fraught” and political issue, she had overlooked the significance of a question, one that, if she investigated it, would help her resist the good/bad, black/white dichotomy in which so many political issues trap students:  For what reason do women seek out abortions?  There are answers to this question that might support the “do or do not” spirit of abortion legality.  But as a whole, the constellation of answers implied by the question also suggests that abortion’s place in society is more complex than whether or not it is legal or accepted.

What I’m getting at is Huot’s point in Chapter 6:  assessment is research.  It links judgment to curiosity rather than to gatekeeping.  Why do I like formative assessment?  Because it is understood as research—as a question seeking an answer rather than an answer seeking a justification.  Of course, summative assessment can do this, too.  But, like a research paper that tries to support an already-developed position, summative assessment that must judge student work in relation to an already-existing standard, or place a student in one of a set of already-existing classes, summative assessment (as it is understood now) can only do violence to students, by relegating those parts of themselves that don’t fit in the established order to marginalia, to a footnote.  Perhaps by even denying they exist, because the terms available to the assessor don’t quite fit the assessor’s intuiton.

Organically Grown

Just a few days ago, I had an interesting experience.  To begin a new class project involving the group creation of forums, I had brought several different examples of forums to class to help initiate a discussion of the concept of forum.  Among these were two different newspapers, a bound graphic novel, the Mercury Reader, and a university literary journal.  In groups of two to three students, I asked the class to answer two questions:  What would I name this forum?  And what characteristics or aspects of the example in front of me suggest that name fits this example?

As the class discussed the examples they had been given, they came up with some interesting definitions:  a graphic novel was “a comic that isn’t funny.”  A newspaper is “facts pulled together.”  The ways in which my students tried to both make sense of the texts they were encountering and to articulate that sense to their classmates was fun to listen to, even educational.  But one group’s analysis in particular disturbed me:  the decision that the Southern Review, a literary journal of poems, stories, book reviews, and the like, was a “scholarly journal.”

Ali, Chadman, and Jamie made a good case.  The publication came from a university-based press.  The journal contained no advertising.  The kinds of texts the group encountered in the journal were the same kinds of texts each member of the group was accustomed to seeing primarily in academic settings.  And the biographical information provided about each text’s author more often than not announced that author’s affiliation with an academic institution, either as holder of an advanced degree, or as a member of a particular institution’s faculty.  I noted the cogent, careful, and persuasive nature of the group’s case.  Still, something tugged at the back of my mind, and I announced that as a class, we would begin the next session by returning to Ali, Chadman, and Jamie, and their assessment of the Southern Review.

The next day, I did just that.  I decided to start the day by introducing my students to the concept of “situatedness,” the phenomenon whereby any claim or set of claims a person makes is informed (and limited) by the affiliations that person knits together into a sense of self.  I disclosed that I belonged to several different communities that shaped what I thought of as my “identity”—I’m a son, a brother, an uncle, a student, a teacher, and a playwright, among other things.  And then I revealed that my status as a playwright—and with it, some of the assumptions made by playwrights, and by “creative writers,” one of the bigger groups to which playwrights belong—had led me to place the Southern Review into an entirely different category of forum than the one into which Ali, Chadman, and Jamie had “situated” it.

How does that relate to Brian Huot’s ideas about assessment?  To answer that question, I have to reveal what happened later on during that second class session.  When we finally departed our discussion about individual forums, the class began a discussion about style.  To better situate the class within one of the contexts I knew they would be aware of, and perhaps anxious about, I asked them to read the row of the Project Grading Standards pertaining to Structure and Style.  In groups, I asked them to identify particular words, phrases, or sentences within that row that they either (1) found helpful, or (2) found unhelpful.  For each instance of helpful or unhelpful language, I asked each group to explain (a) how they would react to the helpful language, or (b) what information they saw as “missing” from the unhelpful language that would make it more helpful.

Brian, a former student of mine who elected to take my class again, piped up that the example of how not to conclude an essay (i.e., by repeating or summarizing oneself) was helpful because it indicated to him that he should avoid a strategy with which he was familiar (and thus might easily adopt without realizing how it was valued).  When I asked for an example of “unhelpful” language, Anna pointed out that “organic” didn’t help her because she didn’t understand what it meant—a reasonable explanation given the many ways in which “organic” might be taken, and the limited context within which she had encountered it.

What happened next is the key.  Though we only had another minute left in class, and I was prepared to let the students grapple privately with the meaning of “organic” and come back to it on Thursday, Chadman’s hand shot up, and I couldn’t ignore it.  “This definition comes from art,” he mused, “but it helps me understand what ‘organic’ might mean for a piece of writing.”  He related that to artists, an “organic” piece springs from the artist’s inspiration, rather than from a set format.  Though I felt the need to briefly re-phrase what he said in writing terms (i.e., that an organic piece of writing takes its structure from the writer’s purpose), I saw nodding heads throughout the room that indicated to me that Chadman’s understanding of the word “organic”—situated as it was in intersection with several different contexts, only one of which was writing—had answered their own, unvoiced questions about the concept.

 

Right There in Black and Whoops

[NOTE:  The events in this post actually occurred nearly three months before the posting date.]

 

I’ve always imagined myself to be open-minded.  Part of that means that establishing authority is an experiment in egalitarianism.  I like to ask my students to circle up their desks.  I frequently eschew the instructor “high chair” in the Stevenson computer labs for a student chair (nearly every day, at least one is empty).  I ask questions.  I encourage students to call me out, to question and to challenge my assumptions.  (Last Thursday, I spent the first few minutes of the period admitting that the class went right where I went wrong.)  Last Monday, during a teaching assistant’s first day leading the class, I bit my tongue, not wanting to impeach his authority.  (The next day, I bit down harder as Rob and Becky took charge of the class all by themselves.)

That’s one reason I mistrust summative (often holistic) assessment.  Not only does it turn a high-stakes decision into a one-shot deal, it does so while making the student the bearer of the burden of failure.  No, I don’t believe that a student should be able to argue that, because he or she paid twice for a class, he or she deserves a passing grade.  But I don’t congratulate myself when I give a student the “D” or “F” that might occasion such an argument.  More to the point, it doesn’t cost me much—not directly, anyway—to fail a student.  The student has to pay to take the course again, but I continue to register for classes for free, to collect a monthly paycheck.  I’m even awarded the mentorship of two future English teachers.  And on what grounds?  That I was a tough S.O.B.?

Turn that situation on its head, and you’ll discover what I value about formative assessment.  Formative assessment implies a negotiative, provisional, reciprocative relationship—one in which the teacher and the student share different, but (ideally) equal responsibility in the result of the assessment.  Learning isn’t just about what the student does, or what the teacher does:  it’s about what each does in dialogue with the other.  To put it in other terms, a successful act of formative assessment teaches me what I need to do just as much as it teaches the student what he or she needs to do.

Summative assessment—the kind of judgment that either gives credit for a class or doesn’t, that places a student in a college-credit course or doesn’t, isn’t a dialogue between student and teacher.  Before you interject “But the student is present in the form of his or her text!” let me illustrate what I mean by “dialogue.”  If a student paraphrases a teacher’s directions and the teacher says “No, that’s not what I meant,” that moment becomes one in which the student realizes his or her need to adjust how he or she listens, and the teacher realizes his or her need to adjust how he or she communicates the desired message.  Their words and actions not only have an effect on one another, but both teacher and student are welcome, even encouraged, to respond to that effect, to appeal directly to the most appropriate audience.  But if a student writes what he or she sees as a “passing” essay, or collects documents in what he or she sees as a “competent” portfolio of work, and a teacher says “No, that’s not what I meant” (for a portfolio or essay submitted as an instrument of summative assessment so often is an attempt to tell the teacher what he or she means), the ensuing moment is anything but a shared learning experience—although it might be possible for the student and teacher to learn from that moment, they do not do so together.  To put it another way, the effects of the student’s and the teacher’s words and actions either come from “out of the blue,” or they’re clearly visible, but inevitable in the way that the death of the cute girl is in a slasher movie.

And because they do not learn together, what they learn, so often, does not work together.  Summative assessment, it seems, is not designed to foster collaborative work—at least not between teacher and student.  Take the kind of assessment that “sums up” an academic term (such as a semester or a quarter).  Typically, teachers—acting alone—give single, holistic measurements of student performance, and students know exactly which teacher gave what measurement.  By contrast, students are usually encouraged to assess their teachers by answering a battery of questions covering everything from the teacher’s preparedness for class and availability for one-on-one assistance to their professional decorum and the appropriateness of the difficulty level of their assignments.  (Students are often asked to engage in self-assessment, too, in the [problematically holistic] form of “What grade do you think you’ll get in this class?” something teachers are seldom, if ever, asked to do in return.)  And students assess their teacher anonymously.

At this point, you’re probably arguing:  “But students need anonymity to be able to safely evaluate their teachers, whose power might otherwise silence them, or modify what they say!”  And:  “Teachers have already provided a mountain of narrative feedback by the end of a 15-week semester or 10-week quarter!”  But look past the typical ethical and practical objections, and what remains is this:  summative assessment exists to divide—not just to separate the strong from the weak, the competent from the incompetent, but to stratify former members of a community at the moment when they know the most about one another; the moment at which, perhaps, each could do the other the most good.

I’m describing, of course, complementarity.  That’s typical for someone of my training:  any good theatre geek knows that a play is a communal thing, and that each part of that play—every spotlight and blackout, every hat, dress, and cufflink, every gesture, step, thrust, and parry, every word, shout, shriek, sigh, and keening cry, every stage hand, props man, actor, sound tech, even every audience member’s stunned gasp or wet-eyed stare—cannot be understood without even one of the others.  That’s why, when I read someone like Sarah Warshauer Freedman, who asserts in describing a study she conducted that “readers judged consistently, their traits not affecting the variance in the scores” (289), and that she values Diederich’s thoughts about “how to train readers to lose their biases” (290), along with her attempts to ensure that “differences in evaluations caused by mode of discourse would not have to be examined” (291), I cringe.

I don’t question her motives.  (I had a Facebook friend who called “those with Ed.D. degrees” a special kind of “varmint.”  He’s not my friend anymore, simply because I asked him what he meant, and he didn’t feel obligated to reply.  In my view, one does not even implicitly question the motives of an educator without care and tact.)  But I do wonder what she’s missing.

I remember my first day teaching in a college classroom.  After asking my students to put their chairs in a circle (“How 1960s” I can hear Gabe Gudding quip from over my metaphorical shoulder, even as he helps me move a desk), I played a special kind of game I learned as a student teacher for a theatre class, which I like to call “The Dictionary Game.”  Starting with the classmate seated to my left, the first person makes up and spells an imaginary word:

“Farandola.  F-A-R-A-N-D-O-L-A.”

Their neighbor gives that word a part of speech and a definition:

“Farandola.  Noun.  A special kind of dance move that occurs just before one falls, usually unintentionally, on one’s behind.”

The next person then uses that term in a sentence:

“He tried to do the Running Man, but the resulting farandola sent Mom into the freezer for her ice pack.”

On that particular day, Rachel stalled, staring nervously at the ceiling.  Her neighbor, Chris, reminded her of the word and prompted her to use it in a sentence.  After a few moments of anxious silence, Rachel looked squarely at me and used all five of the words the class had created so far in a single sentence.

My first response was the same as Freedman’s:  I had lost control of the environment, and I wanted to stop, back up, even wipe the slate clean and start over.  Rachel had misunderstood my instructions, and in so doing, had introduced an unintended variable into my experiment.  But I resisted that initial impulse, and almost immediately, began to see the benefits behind what had happened.  Rachel had, in fact, done exactly what I had asked—illustrated an appropriate example of a term’s usage.  But she had also synthesized—pulling together all five words (correctly) into a single unit.  By doing that, Rachel taught me that the task I had set before my students asked for—and consequently, measured—skills for which I had not designed it.  I felt, I imagine, similar to the scientist who discovered aspartame (an artificial sweetener) while intending to create a new form of adhesive.  Or Rutherford, who, upon finishing an experiment designed to test his theory of the structure of the atom, remarked that he felt as though he had shot a cannonball at a sheet of tissue paper, and it had bounced off.

What complementarity suggests (what, I might add, the whole of human intellectual history suggests) is that nothing can be understood without being taken on its own terms—those terms being the variables that experimental procedure tries so hard to erase.  Assessing writing on its own terms, rather than controlling for variables, searches for them.  Interrogates them.  Even celebrates them.  That’s clearly a situated opinion—my training as a creative writer has taught me that the fulcrum of most effective writing is the element of surprise—but it’s not a lonely one.  Practitioners and theorists of writing assessment note that teachers tend to disagree.  Champions of communal writing assessment even assert that this disagreement can be mined for information.  And I can’t help but wonder:  What would happen if we introduced a student into the carefully-crafted environment of the “norming” session?  Not a “model” student, or a graduate assistant.  Somebody who, by our definition, doesn’t know anything.  What do we think would happen?  And more importantly, what might happen in spite of ourselves?

 

Work Cited

Freedman, Sarah Warshauer.  “Influences on Evaluators of Expository Essays:  Beyond the Text.”  Reprinted in Assessing Writing:  A Critical Sourcebook.  Ed. Huot, Brian and Peggy O’Neill.  Boston:  Bedford/St. Martin’s, 2009.  Print.  289 – 300.

Messick to the Rescue

Of course, as much as I try to deny them, there my actual interests are staring me in the face.  I’m interested in the assessment of creative writing.  In The Elephants Teach:  Creative Writing Since 1880, D.G. Myers writes that Ralph Waldo Emerson coined the term “creative writing,” not to describe the writing of imaginative stories, poems, plays, and other genres that would typically be considered “creative” in today’s formulation of the term, but to describe a kind of writing that encouraged students to develop problem-solving skills, as opposed to the typical writing of lines and declamation of already-written texts that dominated the pre-1880 writing classroom (cf. Myers 31). 

As I reflect on my first semester teaching at ISU, I realize that my students use the term in the same way—when they grouse that I’m teaching them “creative writing,” they do mean in part that some of the activities and assignments I set before them seem like writing stories, or poems, or plays.  But they also mean that I’m giving them tasks that thwart their attempts to find an easy answer or a rote response.

I have a list of courses that I’d love to teach:  The Rhetoric of Modern Comedy.  A creative writing seminar that, rather than organize itself around a particular genre, organizes itself around a theme, and makes genre selection part of the task of writing.  The Literature of Parody, Spoof, and Lampoon.  Using Verse in Playwriting.  I don’t know that I have all (or even any) of the knowledge and skills necessary to teach such courses.  But one such “imaginary” course, which I’ll call “Creative Genres as Metadiscursive Tools,” suggests a kind of study I might design.  (Again, a kind of study I might not yet know how to design—which suggests that this semester is about learning how to design it, and getting started designing it, whereas actually conducting the study might be a task for the future.)

As represented in the articles reproduced in the “Foundations” section of Assessing Writing, Samuel Messick’s ideas about using one form of inquiry to interrogate another appears to be the direction in which I’m headed.  (In fact, I’m already using something like this in the way that I teach.)  Specifically, I like to use writing “creatively” as a way to expose the tacit values and assumptions inherent in “academic” genres like the essay or the lecture-discussion class.  Examples of the kinds of tasks I might undertake or assign:

·         Write a play in which the typical parts of an academic essay (e.g. Thesis, Topic Sentence, Block Quote, Counterexample, etc.) talk to each other.  Give them a problem to solve, and let them find their way to a solution.  (Note that this problem should be academic in nature; e.g. “Block quote has amnesia, and can’t remember where he came from!”  “MLA and APA are arguing over which one should be included in Essay’s posse!”)

·         Imagine a conversation between three of your past teachers about one of the projects you’ve written for this class.  What would they like about what you wrote?  What would they dislike?  About which things would they agree or disagree?  What grade would they give you, and why?  (What if the essay itself were a character in the play?  What would it say to your teachers?  What would its self-image be?  How would it feel about you, its writer?)

·         Using the aspects of writing typical to poetry (e.g., condensed language, rhythm and meter, sound agreement, line breaks, etc.), write a “sentient” poem that expresses what its place is in the English Composition classroom.  Then, reply with a “sentient” essay that enters into dialogue with the poem using the elements typical of an essay (e.g., thesis, evidence, citation, etc.).  (What might the essay’s place be in the Creative Writing classroom?)

·         Using the examples from The Meaning of Liff and Revisionary Rhetoric, Feminist Pedagogy, and Multigenre Texts, create dictionary entries for at least five Types of College Student.  (Rather than name your students descriptively, i.e., “The Slacker,” you can choose to follow Douglas Adams and pick a naming principle, i.e., “Surnames of U.S. Presidents” or “Flavors of Ben and Jerry’s Ice Cream” as a mode of inspiration.  But you don’t have to.  Likewise, you can appropriate Jung’s sentence-fragment style to help describe your college student, if you wish.)

The problem now is designing a study that DOES something like what Messick describes, but that uses what I know, and answers a research question.  (The research question I want to start with is something like “What do Rhet Comp and Creative Writing value?”  And maybe “How might those values cross sub-disciplinary boundaries to interact productively?”

 

Works Cited

Myers, D.G.  The Elephants Teach:  Creative Writing Since 1880.  Chicago:  Chicago UP, 1996.

Williamson, Michael.  “The Worship of Efficiency:  Untangling Theoretical and Practical Considerations in Writing Assessment.”  In Assessing Writing:  A Critical Sourcebook.  Huot, Brian and Peggy O’Neill, eds.  Boston:  Bedford/St. Martin’s, 2009.

 

 

 

Walking the Talk

 

My students and I need to meet each other halfway, as they learn the kind of talk that I speak and write and I learn theirs, for the combination of these registers is more powerful than either alone.

Gerald Graff, from Clueless in Academe:

How Schooling Obscures the Life of the Mind (13)

 

Graff’s point in Clueless in Academe is, in part, the same one that David Bartholomae made in “Inventing the University” a decade and a half earlier:  students are entering into a new discourse, one that they can only attempt to approximate if they do not understand the value of such discourse.  In contrast to Bartholomae, however, who implies that it is the student who fails through the sheer impossibility that they might have access to (all of) what he or she needs to function within academic discourse, Graff asserts that it is academics who hide their labor behind an alchemic “cloud of smoke” (9).  As I re-read the writing of these two men, I’m reminded of Jim Mitsui, who taught creative writing at Oliver M. Hazen Senior High:  each week, Mr. Mitsui would ask us to ink one of our creations with one of an assortment of stamps.  “I don’t believe in grading creative writing,” he would intone from the front of the class, tracing elliptical orbits around his oratory with the strangely “z”-shaped pinky finger on his right hand.  “But I’ve got to grade something.  So you’ve got to pick something.”

I remember my favorite stamp—the Dragon, the scaly, sinuous, mysterious custodian of emperors under whose astrological sign I was born more than three decades ago.  Dragons—mercurial and ambitious, but also influential and well-liked—were special people, and I wanted to be special.  But as I picture the stamp now, what stands out for me is its arbitrary nature.  I chose it, but from a collection of choices determined for me, not selected by me.  If I could have created my own avatar, who would I have made myself out to be?  That memory haunts the decisions confronting me now.  I can open up a world of choice for my students, but can I ever be free of the specter of the past?  Can I offer my students a choice that no one has been offered before?  The path I might blaze for the initiates in my charge might seem new to them, but it is far from free—forever Peter Pan’s shadow, striving hopelessly to cast itself in the absence of its master.

The way I remember that Dragon stamp, I imagine, to the way students feel about being graded.  It’s not writing that scares them—they write e-mails and text messages, chatter on incessantly to each other between classes, during classes, after classes over chimichangas, cervezas, and cigarettes.  They know how to use language.  What they don’t know is what we—teachers, bosses, anyone in authority—want them to do with language.  To paraphrase Peter Elbow, students are caught standing in front of an audience whose mere presence has silenced them, forced them to search for someone else’s words to pour into the void that so many textbooks assert stretches between their ears (cf. Elbow, 174).

But students are not silent, one part of me asserts.  Website rating sites like http://www.ratemyprofessor.com show that students have plenty to say about the classes they take.  What I hear from teachers on this subject usually involves the ideological blinders placed on students by the categories available to them—teachers are either easy or hard, helpful or not, attractive or not.  Part of me is inclined to agree with the consensus of professorial condemnation.  Students so often focus on the quantitative nature of their classroom experience:  how much work do I have to do?  How easy are the assignments?  How likely is the professor to approve requests for extensions or to accept late work?  The part of me that has dealt day-to-day with this phenomenon, in which students seem forever willing to delay, to rationalize, to misplace blame, agrees with one director of FYC under which I have worked who theorized that perhaps students simply need to experience the consequences of their own decisions (i.e., to fail) and learn to adjust.

But if Bob Broad is right, and we have not yet given serious consideration to our own complicity in students’ focus on the grade (307), then perhaps the nagging feeling on the edge of my awareness is also right:  perhaps students have something to say to teachers about evaluation that teachers, so accustomed to dismissing student thought as shallow, uninformed, undeveloped, instrumentally-focused, have not yet heard.  Perhaps, to paraphrase Julie Jung, what students have to say to us is something we need to hear more than we want to hear it.  And if that’s true, then maybe we should go one step beyond Bartholomae:  instead of involving students “in scholarly projects, projects that allow students to act as though they were colleagues in an academic enterprise” (12, my emphasis), perhaps we should welcome them in earnest as the “members of the academic community” they are so often labeled as in university policy documents.  What that means is giving up the idea that the teacher’s job is to introduce students to the modes of thinking and inquiry that teachers already value, and opening ourselves to learning what’s valuable about ways of thinking teachers have traditionally dismissed.  To paraphrase Gerald Graff, it means assuming that, in signing up for our classes, students have already met us halfway, and it’s time for us to do our part.

 

 

Works Cited

Bartholomae, David.  “Inventing the University.”  Reprinted in Teaching Composition:  Background Readings.  Ed. T.R. Johnson.  Boston:  Bedford/St. Martin’s, 2008.  2 – 31.

Broad, Robert L.  “ ‘Portfolio Scoring’:  A Contradiction in Terms.”  Reprinted in Assessing Writing:  A Critical Sourcebook.  Ed. Brian Huot and Peggy O’Neill.  Boston:  Bedford/St. Martin’s, 2009.  301 – 314.

Elbow, Peter.  “Closing My Eyes as I Speak:  An Argument for Ignoring Audience.”  Reprinted in Teaching Composition:  Background Readings.  Ed. T.R. Johnson.  Boston:  Bedford/St. Martin’s, 2008.  172 – 194.

Graff, Gerald.  “Introduction:  In the Dark, All Eggheads are Gray.”  In Clueless in Academe:  How Schooling Obscures the Life of the Mind.  New Haven, CT:  Yale UP, 2003.  1 – 13.

Jung, Julie.  Revisionary Rhetoric, Feminist Pedagogy, and Multigenre Texts.  Carbondale, IL:  Southern Illinois UP, 2005.