I am posting this from my phone because we lost power at my house in this snow storm, so it’s been kind of chaotic around here. So let me just start this post about Cheryl Ball’s two articles, which more or less go together.
I’ve also invited Cheryl to stop by and join the conversation. I’ll hsve more to say later, but like I said, start without me.
I thought these were both pretty great reads. I like the idea of having students write to an actual publication like Kairos. I think this is a great way to get students thinking about tailoring their writing; they learn to read and understand the journal’s publication guidelines, think about who their audience is, and potentially get real experience with the process of submitting something for publication in a peer-reviewed journal.
I also liked that Ball gave her students the ability to actually help in the creation of the grading criteria for the assignment, though I wonder how time-consuming such a process can be in the classroom and how feasible it is for other assignments/classes?
What I liked *most*, though, was this:
“It was not feasible to judge students based on any finished product (or the process they used to complete the work) given that many first-time scholarly multimedia authors need a reasonable amount of feedback on their webtexts before those pieces are
considered ready to resubmit.” (75)
I think this is a great point and goes back to the idea that a text is never really finished. This raises challenges when it comes to assessment and grading because grades seem so finite and concrete, whereas the creation is not.
http://www.daniellebreann.wordpress.com
Danielle,I had similar thoughts on these articles, too. I had a teacher who let her students create the grading criteria of assignments, and it did not take as long as someone might think. We did this in creative writing and teaching writing courses and in both cases it took less than a class period to agree upon the criteria for assignments. Although the assignments were not webtexts, it was still an interesting process.
Ball’s idea of having students write to a publication was interesting, too, because an assignment like that would make students grow into better writers and consider their audience in a way that is much broader than the audience for an assignment.
I would agree, too, that a text is never really finished. Grades are concrete, but the text is always available for any change the writer wants to make. If an assignment is designed for a writer to grow, then grading or judging the assignment too early may show the writer what needs to be done to improve.
http://timstorm.wordpress.com
Danielle, you might like the short article I wrote that further discusses how I develop and mentor authors (which discusses my grading scale on 100% class participation, as well). It’s a three-parter, but I’m linking to the second part here: http://hybridpedagogy.com/Journal/files/Developing_Authors.html
http://ceball.com
I think this would speak to the argument about authentic audience we ran into last semester in Professor Baker’s class, as well as the aspect of students helping to build the assignment rubric. I think it is also interesting in the way the assignment is likely to empower student-writers; the assignment seems to follow the idea of “treat them like writers, and they will feel like writers,” which may seem silly but there is some truth/power to that line of thinking.
Though I had some mixed feelings while reading (I get it – you are the editor of Kairos), overall I thought Ball had some really great practices for the classroom. I’m not totally sold that writing for a publication necessarily makes students work that much harder or try on a different level, however. Having been in a class last semester that was tailored to submitting a paper to a conference (same concept), I can speak from experience that this didn’t really impact the level of thought/work I did on my writing. For the students that want to be published, it’s great. For those that don’t care or have no desire to continue in academia, eh, maybe not so much.
What I really loved, though, was her talk of the importance of peer review. It seems to me that peer review is something that graduate classes, and certain undergrad classes, forgot about. I haven’t had one class in this program that has asked me to do a peer review. And I love them – I think they are all Ball claims them to be. I definitely know that reviewing other students’ work helped me to reflect upon my own work. I also think the knowledge that someone other than the teacher would read my work helped push me too. Peer review is a powerful, hands-on approach that I think can really help solidify concepts/reflection for students.
I also agree that including students on the value criteria is a good way to go. I will say that I really like all the agency (for lack of a better word) Ball ascribes to her students. To say that student can create work she expects of first-time Kairos submitters is big. I don’t think students always get the credit they deserve and her classes seem very big on letting students step up and find out their worth. While at the same time promoting her journal, of course
http://msyapin.wordpress.com
Melissa,
This made me LOL. Like startle-the-cats-LOL.
It’s good to know that my informality doesn’t come across as cute to everyone. My friend Kristin is constantly reminding me of this. And I’m about — literally am supposed to be working on it instead of posting here — write another article about Kairos’ peer-review process and how the dev-editing works with authors (instead of students/classrooms). I will make sure it’s not too smarmy 
And if it is, you can bet that the British editors will womp me for it. lol.
Re the webtext assignment in classes, I should note that students had a choice of six journals to submit to: C&C Online, TheJUMP, Harlot of the Arts, Kairos, Enculturation, and X/changes. NONE chose Kairos, mostly because they couldn’t see themselves in that journal (for the same reason they didn’t choose C&C Online). Most chose Enculturation, because they liked it’s focus on pop culture and cultural studies. Some chose Harlot because they liked its public rhetorics perspective. None chose X/changes or TheJUMP because, although they liked that both featured ugrad writing, they couldn’t ever see a point of a teacher reading those journals. (And they’d decided that their strength, in most cases, would be to speak to teachers of digital writing about their experiences as students who use technology.) C&C Online they found too pedagogical; no way for them, as students (not teachers) to enter into the convo.
http://ceball.com
While I see your point about students not wanting to continue on in academia, I could make the same argument about peer reviews in a sense. I have had quite a few peer reviews that were more or less a waste of time, because all I got from some of those sessions were comments like, “Nice job,” “Heading in the right direction,” or a comma was added here and there. While some peer reviews were exactly as you described them (useful, insightful, another view point on my writing), I think again it comes down to what the peer values and sees as worth doing.
And I totally agree about the agency given to the students; like I said on Danielle’s post: “The assignment seems to follow the idea of ‘treat them like writers, and they will feel like writers,’ which may seem silly but there is some truth/power to that line of thinking.”
I got to talk to Cheryl Ball after our session at WIDE-EMU (Becky Morrison, Jana Rosinski, and I presented with her) this past fall, about how she structures her class using webtexts and the journal submission structure…it was an exciting and energizing conversation, and she shared with us many of the points brought up in both articles. I love how she uses the word “enacts,” reminding us that genre – and place/space/tools – carries out the content as a living thing, a happening.
For what it’s worth, EM-Journal has a poster published in the Kairos PraxisWiki that Cheryl Ball edits: http://kairos.technorhetoric.net/ (it’s at the bottom of the page).
I really love how she breaks assessment down into the following: “rhetorical, technological, ideological, institutional, professional, social, and other issues” to be discussed. Having elements to identify helps make this feel do-able. I also love that she solicited feedback from her students about what they expected in a webtext. I love this collaboration, and this is something I value in my own classroom.
I also appreciate something she points out in the “Manifesto” section: “Our goal was [to] create review criteria that reflected the Call For Manifestos while also allowing approaches that we really couldn’t have imagined until we received submissions.” When valuing exploration time, as we discussed earlier this week, I think that openness is really important in terms of ends, albeit maybe messy and uncertain.
I’m interested in the form/content she mentions in the assessment article, not just for multimedia texts, but for all texts…especially as it sounds like it would foster a great sense of autonomy and ownership over the work. This article is definitely something I’ll be revisiting.
The peer-review article emphasizes genre understanding as a necessity for successful peer-review, which I think is huge, and as a result is probably a challenge in a writing class that uses “mutt genres,” as she calls them. It is difficult to address audience and purpose concerns when they are manufactured, rather than observed in real-time/real-space. This links into publishing student writing as a means of opening that space, but again…how/when/where/etc. This was more along the lines of the presentation we did at WIDE, whereas Cheryl talked about a book series, I believe… anyway, I can see how positioning peer reviewers as an editorial board would work, likely much better than a teacher-facilitated “editing” (called revision) session…I like how peer-review here is actually revision, done through a letter which puts the reviewers in a position of power and collaboration and imitates the actual process.
And now I’ve written a novel-length comment, so I’ll stop.
http://parablematernal.wordpress.com
Hey Chelsea,
I want to take up this point for a sec:
EM-Journal is a good place, right? I may have mentioned when we talked that at ISU we have a similar journal (except it’s print), so it’s one outlet for publications that students can easily get into. I guess the reason I’ve cited your comment above is because of the phrase “student writing” –> for me, that diminishes students as only being able to produce writing that has the adjective “student” in front of it. And who publishes “student writing”? Well, really, only venues that want to praise students for writing well.
But, like I noted in my comment above, about X/changes and TheJUMP, who reads those venues? Other students don’t even read them. Faculty might read them, but why? It’s not like reading a scholarly journal, the point of which is to gain some new perspective on the discipline, which is not usually a perspective that undergraduates (on their own) can often offer.
So, then, it’s a matter of finding venues that will publish writing that is writeable and revisable within the constraints of a 15-week semester (or such). Venues where the word “student” doesn’t necessarily have to preface “writing”. And that can be lots of places, right?
http://ceball.com
Yeah. Actually, do you mind if we chat (maybe via email, if you have time) about this idea of “student writing”? I have a project for another class that this might be really useful for.
But for this space here, it sort of reminds me of how we internalize writing to the point that we forget writing is a technology itself. We also internalize “student writing,” not realizing that the adjective itself isn’t mandatory, isn’t an inseparable piece.
And I think that carries over into peer review, maybe? Are we approaching peer review with this attitude that it’s not going to go well? That it’s about student papers in a student space? And I think I remember you mentioning these other journals at WIDE when we talked afterwards; what you say now about students not feeling as if they belonged in those spaces is really important, especially in terms of entering the conversation. Which makes me wonder, do students actually feel more comfortable entering the conversation with multimedia texts, or is it equal to print-based work?
And all of that said, what role does publication – or even the course structure you lay out in these articles – play in influencing how we see “student” writing (ie: the “students cant write” crap)?
http://parablematernal.wordpress.com
Chelsea, sorry I got wrapped up last week and couldn’t get back to the blog. But, sure, email me if you want to talk more: s2ceball@gmail.com. I can’t guarantee a ton of time, since I’m on sabbatical and frantically trying to finish up some major projects before it ends, but I’m always happy to chat.
http://ceball.com
It’s a good point: we want to give students a publishing venue, but if no one reads that venue, well, what’s the point?
Of course, that’s also the case with a ton of academic genres, too. I am sure there are pieces in Kairos (and any other academic journal) that barely get read even when they are in the current issue. Some of the things that I’ve published before I know get read and taught, but I’ll bet you that fewer than 100 people have read the short stories I published back in the early 1990s. So what’s the point in that case?
For what it’s worth, I actually do assign some EM-Journal texts in one of my classes here. Of course, that has a lot to do with the fact that the writings in that issue of EM-Journal align with an assignment I give…
I enjoyed learning about webtexts. Before I read Ball’s article I didn’t think about what scholarly multimedia is and how it works. From what I understand webtexts are a form of scholarly multimedia that are like regular online articles only webtexts have a quality that helps us visualize the subject matter more. Ball said “Scholarly multimedia are article or book length, digital pieces of scholarship designed using multimodal elements to enact author’s arguments.” I admit I’m still not quite sure what this looks like exactly. Ball’s example seems to indicate that webtexts are sort of like scholarly slide shows you can click through.
I’m fairly new to the concept of “scholarly multimedia” (or whatever name you give to multimodal digital writing), and it’s kind of amazing to consider that at the time of her article, people were publishing online (at least in Kairos) for 16 years already…looking back I think that’s the “pre-print” for this coming fall. Still, that makes scholarly multimedia a simultaneously new and old genre.
Considering the timeline for this genre, it follows fairly closely with computer technology, and there is food for thought about the change in writing technologies changing the way we compose scholarly writing. I think the biggest thing I took away from these pieces is the idea that the technological expertise is both cultural and collaborative. She also makes the argument that form affects content (agreed) and I feel like I’ve read more about that somewhere…hmmm.
As others have pointed out above me, the concept of writing being an ongoing process is especially present in the scholarly multimedia format to a degree that is greater than other forms of scholarship in writing.
In all, the range of possibilities is striking for what can be “said” with this kind of writing…I kind of want to re-do my MOOC artefact.
http://visiblycynical.wordpress.com
This mode of scholarly writing reminds me somewhat of EMU’s LINGUIST List Project: http://www.easternecho.com/article/2012/06/linguists_join_forces_with_google_to_create_a_website_to_share_research
Which is kind of like a “hub” that brings people in the field together so they can pool resources and data and collaborate on pieces of academic writing.
Sarah K.,
You’re right: It’s really hard to get a grasp of what scholarly multimedia, or webtexts, are without seeing them in action. A print article can’t do it justice. I’d recommend going to Kairos and checking out a few of the webtexts there.
http://ceball.com
I enjoyed the Scholarly Multimedia article more than the Peer Review article, mostly because of my own mixed opinions of peer review. I really liked Ball’s discussion of aligning form and content from the get-go of the creation of webtexts, which I’ve found to be important whenever I’ve created… well, anything, really. I think form and content are also considered when writing traditional print texts, but I feel that the form of print texts, especially in academic settings, is so taught and ingrained that it seems almost instinctive. Students do not need to think so specifically on form because the form is familiar to them. Because the forms of webtexts are unfamiliar, then, students must afford more conscious thought to them, focusing deliberately on the interplay of form and content more than they might have to when writing a more common print text.
I also liked the stance Ball took on assessment–that it should be flexible and should vary depending on the situation. That’s not to say that the criteria for assessment shouldn’t be made as clear as possible, but is saying, I think, that the criteria that work in one situation, for one kind of webtext, do not always translate to all forms of webtext.
As for peer review… eh. I’ll be frank, I’ve had few good experiences with it, and I know many others who feel the same. But then, I also know some people who have really found value in it, and I’ve not yet been able to pinpoint the difference between creating valuable peer review and not-valuable peer review. It may simply be that the individual reviewer or class setting plays a huge part in the success of peer review, and I’ve been just been unlucky.
I will say that I felt Ball did a good job in acknowledging some of the scholarship surrounding peer review and when it was and wasn’t useful, and I commend her for getting her students to a place where they could both get and give reviews effectively. I wonder if part of the success might be due to the students that she has worked with in this particular class–juniors and seniors who, most likely, are interested and invested in the class, as opposed to students who are just taking a course to fulfill a requirement. Regardless, I also liked reading through the criteria of the peer review letter. I felt it gave very specific instruction as to what was and wasn’t appropriate, but still gave enough freedom and leeway for students to be able to write their own letters.
I think there are basically three ways that peer review can go wrong. The first (and in my experience the most common) is a lack of structure. Ball quotes someone as describing peer review sessions in a lot of classes is all about “chit-chat” opportunities, and I think one of the really useful things that Cheryl is doing in both of these pieces is sketching out the structure that could guide the peer review process. Have students come up with the criteria (or at least part of it), sure, but then hold them accountable to that criteria in their critiques.
Two slight tangents on this point. First, I always have students do something in writing in their peer reviews, which cuts down on the useless chit-chat factor, IMO. Second, a lot of what Cheryl is talking about here would be useful for non-multimedia projects too….
Second, peer review takes a lot of practice, and that’s another problem I see a lot in teachers– especially folks who are new to teaching, but also some that have been doing this for a while. The first peer review session for a class usually goes poorly, but as students get to know each other and as they have more practice and more familiarity with the language/value of peer review, it gets better.
And third, peer review takes students who are interested and willing to do it. That can sometimes be tough, both for students who frankly “don’t care” about their writing or their school work at all and also for students who just don’t trust the opinions of their peers. See the previous two points as ways to address that, but there is no doubt that it will never be perfect and there is also no doubt that students who are more invested and able with their writing will do better than those who aren’t– that is, peer review tends to work better in graduate level classes than it does for fycomp.
Yeah, keep in mind that I have a whole ‘nother article about how I taught multiple versions of this class for eight years at four different institutions and utterly failed at it. (And, heh, I co-wrote that article with two undergraduates
GOOD teaching is difficult and requires a LOT of reflection, tweaking, changing, re-using, re-mixing, etc. Grad students rarely have the opportunity to teach the same class more than once, most teachers (including faculty) like to radically change their syllabi if something didn’t work, instead of slightly shifting one thing, troubleshooting it for the weaknesses. Also, grad students and many faculty don’t read enough pedagogical scholarship and apply it to their teaching practices. This takes time and reflection and wisdom — three things that grad students are often still working towards.
This is all to say that peer review will work IF a teacher can make it work, but that takes a hella work across years, often, to achieve. I got it to work in my writing classes, but only after a decade of refining. Use this knowledge to make it happen more quickly for you.
And in whatever genres you want to focus on, whether those be heavily mediated and radically multimodal (like webtexts) or less mediated and transparently multimodal (like essays).
http://ceball.com
I think another issue in peer reviews that can come into play is the various “skill” levels of the writers. Just as “competent” student writers (sorry Ms. Ball, I know you don’t care for the “student writers” terminology much) can sometimes stall in progressing further in their writing from receiving only/mainly positive instructor feedback because they aren’t really challenged to improve their writing, so it could be with peer reviews.
But I think Cheryl is spot-on when she describes new writers to Kairos– people who are usually graduate students are college professors– as “developmental” because whenever a writer enters into a new discourse community, they have to learn the ropes again. Or to use your terminology here Bryan, their “skills” are not as “competent” and they are still learning like “student writers.”
I know what you’re getting at; you’re talking about the kind of student in a first year composition class who is incredibly ill-prepared for college and just doesn’t have the desire to do what it takes to succeed at school. I see that all the time, too. But a lot of it is also just not knowing the discursive practices. That’s what Cheryl is talking about and when it comes to graduate students who struggle with the kind of writing and demands of a graduate course, that’s what I’m talking about.
Of course, a lot student writers in any of these situations– first year comp, grad school, professor submitting to a journal, etc.– don’t really have the desire or the will to do what it takes and they give up. If students aren’t interested in the peer review process for example, there’s not a lot that a teacher can do. But if a teacher works at making peer review work, then it can help a lot, believe me.
Exactly, Steve. The biggest eye opener for the grad students (MA and PhDs) I’ve taught was having them submit their work to external readers for peer review. It was the first time in their lives they’d been critiqued and told their writing didn’t cut it on a first draft. I think if more teachers underwent the process of peer-review, they’d have a better understanding of how to teach writing and peer-review to make those processes effective for writers.
http://ceball.com
Jackie,
You’re right in that peer-review can go horribly wrong (as I see Steve has addressed in response). But to clarify one thing about the students in my class, from this article: Grad students aside, for the moment (they’re usually pre-invested in a topic by virtue of being in grad school…not always, but yes in this case). The undergrad class, however, is an elective in our curriculum. It’s open to any major and most students take it because it fills a slot in their schedule, not because they have any investment in multimodal composition. None have any idea what any of this stuff is about before they enroll.
If you check out the schedule on my syllabus, you’ll see how much prep work I provide to get them up to speed about the topic of mm comp, including the scholarly readings we do. It’s never easy to convince students to tackle a new topic, and it’s our jobs to convince them. (Some can never be convinced. Alas, some students don’t want to learn and there’s not much you can do about that beyond try to figure out why they’re not interested and to help them find a way in.)
http://ceball.com
Jackie, I’ve had some really great experiences with peer review and some awful ones. I think you’re right in saying that the level of investment/engagement in the class is a big factor. For a required class like ENGL120/121, I wonder how difficult it is to have “effective” peer review. To me, it seems like it’d require the instructors to really know their students, their personality types, their level of engagement in the class, etc. in order to put them into groups that would be productive. For example, you wouldn’t put a group of the four “best” (I’m using lots of quotation marks here because these are all evaluative terms that aren’t easily defined) students in their own peer review group, right? You’d mix them in with other groups?
Anyway, I would agree with Dr. Krause in that peer review works best in a graduate level class (at least in my experience) where the students really care about their work and want to be there.
http://www.daniellebreann.wordpress.com
At least I’m not alone in my bashing of peer reviews. However, even from those peer-review sessions that I thought were awful, I did walk away with some useful information. For example, seeing vague comments from peers like, “Nice job,” set me to thinking that comments needed to be specific enough so the writer knew what I was trying to say (i.e. rooted in their paper), but not too specific as to appropriate ownership of the author’s paper.
You are definitely not alone. I haven’t done a ton of peer reviewing, but my main complaints usually revolve around the experience being unguided. For instance, in ENG 300W, we did a lot of them, and used a worksheet, but we weren’t really “trained” in it. My other issue was that I didn’t feel like I had time to do a proper review of anyone’s work. I could do 4 of them about halfway, or 2 fairly completely. There simply wasn’t time to do justice to every item of the worksheets we were given. I wonder if there are good materials out there for this?
http://visiblycynical.wordpress.com
One other general comment I thought I’d add to the discussion here about both of these articles: I think that the way Cheryl sketches out scholarly multimedia in terms of both an authorship and a review process is good, but it also points to some of the dilemas of this kind of writing and how it plays out in both classrooms and in the scholarship.
For one thing, to do it well takes a tremendous amount of time and energy, and I’ll be blunt: I think some of the multimedia scholarship that appears in online journals like Kairos (and I am a big fan of Kairos, was even on the board when it started) and most of the multimedia work my students do is not very good. Now, the way Cheryl describes this in her teaching– that is, spending half a semester figuring out just what multimedia scholarship is and another half of the semester making a piece of multimedia scholarship– sounds great. But that’s not what happens in classes like fycomp when a group of students does some kind if multimedia text simply because there isn’t time. And I don’t think it is something a lot of the scholars in the field have done either, again because it takes a lot of time.
For another, the editorial stakes are high. To do this well requires a level of planning– scripting, storyboarding, etc.– that typically is not something most academics want to deal with. The revision stakes are even higher. As most academics and editors like Cheryl know, the vast majority of articles sent to academic journals are sent back to authors with “revise and resubmit,” meaning that reviewer of the article doesn’t think the piece is quite ready for publication but thinks that there is promise with some changes. Well, when you’re working with words in a row, that’s not that big of a deal. But when you’re working with video and audio and computer code– especially when you’re working with stuff like video interviews you captured that cannot really be “revised”– that is often a deal-breaker.
All of which is to say that even though places like Kairos have been publishing interactive/multimedia/not words in a row pieces for a long time now, we’re still not as sure what we’re doing with these publications as we are with more conventional words in a row pieces.
That’s very true, Steve. That’s why I refer to Kairos authors as developmental writers (of webtexts). Despite being in its 18th year now, potential authors of webtexts are still learning to recognize and understand the genre(s) of these texts, nevermind compose them.
That’s one of the main reasons Kairos has done some shifting in two ways: we better mentor authors up front and we better train our section editors to mentor authors (an invisible process to everyone but the staff) so that our quality can be more consistent across sections. As I tried to say very strongly in the TCQ article, I know these genres are particular to ME (lol, I go back to Melissa’s comment above
but when the journal has 7 section editors who are each responsible for doing developmental editing with their section’s authors, part of my job is to make sure that they’re mentoring authors in a way that I believe upholds the mission and quality of the journal. It has been pretty inconsistent up until recently (and even still now. But I’d argue we’re still putting out better webtexts than some of our sister journals 
Some of that is editor’s taste though, I admit.)
Because revision is more difficult in webtexts than words-in-a-row, we encourage authors to contact us at the idea-formation stage rather than at the complete-webtext stage. And, this is exactly what we do in our writing classes, as teachers, yes? (I’ve written a little more about how this works in my classrooms in a three-part Hybrid Pedagogy articles. They’re short.) By asking authors to contact us early, we can forestall any problem areas in content or form before authors spend 100s of hours doing it wrong.
That being said, Steve, Ima push you on this point:
The reason the above doesn’t work is because teachers treat multimedia like it’s still bells and whistles. Supplements. Add-ons that can be done at the last minute. And that’s just not the way it works, to be done well.
There are plenty of FYC classes around the U.S. that are doing multimedia as an integral part of their semesters (Michigan Tech, Ohio State, Kent State, Michigan State, Stanford, etc.) Some are doing it better than others, but the thing is: All texts are multimodal, even “mutt genres” and plain-ol essays. It’s not like you need to school students in the entire history of what multimodality is before you have them work on this stuff. Sure, the genre of a webtext takes a long time to explain, mostly because it’s not a solidified genre AND because the genre is wrapped up into stuff like academic peer-review, tenure, etc., all of which needs to be explained to students so that they understand the venues and disciplinary values of the publications they’re considering.
But, again, that’s MY example. Let’s take the example of a PSA (public service announcement), that’s the first assignment in a four-unit semester of a FYC class. (This, btw, is what my grad student, Jason Dockter, is doing as he’s studying his online FYC classes at a local community college.) PSAs come in many media (print/paper, videos, audio) and are distributed in any number of venues (bulletin board, local radio station, television, FB ads, etc.) So, these are (usually) short, multimodal texts. A teacher might spend one week having students read about and analysis examples of PSAs (out of which they create assessment criteria based on genre conventions of PSAs), another week designing them (including writing scripts or storyboards due on the first day and workshopped in class), a third week reviewing them, a fourth week revising them. (That last week, of course, might be out-of-class, if it’s a portfolio-based course.) Bam. Three weeks to practice in micro what I did in 15.
This absolutely can be done in FYC. (We’re doing it at ISU.)
http://ceball.com