Workshop

A workshop provides a place for the students in a class to see an example project, upload their individual projects, and see and assess each other’s projects. When a teacher requires each student to assess the work of several other students, the workshop becomes a powerful collaborative grading tool.

Workshop Strategies
Workshops can be ungraded, peer graded, instructor graded, or a combination of peer and instructor graded. Workshops enable you to create very specific assessment criteria for the graders to use. Also, workshops let you set the due dates for submitting work, and for grading work. You can use these and other features to build a strategy for making best use of workshops in your courses.

Peer Assessment of Assignments
One strategy for workshops is to have students assess each other’s work, before submitting that same work as a graded assignment. For example, you could create a workshop where students assess each other’s subject matter, outlines, and hypothesis for their term papers. Or they could assess each other’s photos for specific technical and artistic criteria before submitting them to the instructor for grading.

Timing of Submissions and Assessments
Workshops enable you to set different due dates for submitting work, and for assessing other student’s work. If you set the same due dates for both submission and assessment, many students might submit their work just before the submission deadline and they cannot all be assessed before the assessment deadline. Consider setting the submission deadline well before the assessment deadline. Then, before opening up the assessment ability to the students, examine the work submitted by them to ensure that it’s close to what you expected or were trying to elicit from the students. You might even want to use the time between submission and assessment to refine your assessment criteria, in response to the work submitted.

Creating a Workshop
The fields in the workshop window give you many choices. No matter what you enter into each field, your many decisions can be summed up as:

• What will you have each student do? Create a file offline and upload it to the workshop? Write a journal entry? Participate in an online chat? Perform some offline activity and report on it via email or wiki? While the workshop window enables the student to upload a file, you can also require any other activity from the student.
• Who will assess the assignments? Will the teacher assess all assignments? Will students be required to assess other students’ assignments? Will each student self-assess his or her work?
• How will the assignments be assessed? You can determine the number of criteria upon which each assignment is assessed, the grading scale, and the type of grading.
• When will students be allowed to submit their assignments? The assignment becomes available as soon as you show it. However, you can require students to assess an example before being allowed to submit their own work, and you also set a deadline for submission.

All the fields that we’ll discuss henceforth are variations of these questions. The online help does a good job of explaining how to use each field. Instead of repeating how to use each field here, we will focus on how your choices affect the student and teacher experience.

Workshop Fields
The workshop activity is the most complex tool currently available in Moodle. Workshops are designed so that a student’s work can be submitted and offered for peer review within a structured framework. Workshops provide a process for both instructor and peer feedback on open-ended assignments, such as essays and research papers. There are easy-to-use interfaces for uploading assignments, performing self-assessments, and peer reviews of other students’ papers. The key to the workshop is the scoring guide, which is a set of specific criteria for making judgments about the quality of a given work. There are several fields under workshop. They will be explained in the following sections. They provide a place for the students in the class as well as the teachers to make the best use of Moodle.

Title and Description
Your students will see and click on the Title. The Description should give instructions for completing the workshop. If you want to make printer-friendly instructions, you can upload a .pdf file to the course files area, and put a link to the document in the workshop description.

Grade for Assessments and Grade for Submission
These two fields added together determine the maximum points a student can earn for a workshop. Grade for assessments is the grade that the student receives for grading other submissions. This grade is based on how close the assessment a student completes is to the average of all assessments for that same submission. For example, say Student A submits his or her work. Students B, C, and D assess the work and give scores of 10, 9, and 5 respectively. The average assessment is 8, so students B and C would receive higher marks for their assessments as compared to student D. In essence, the Grade for assessments is the ‘grade for grading’.

Grade for submission is the maximum number of points a student can be given by the grader. If you choose Not Graded for the Grading Strategy, these grades
are irrelevant.
The submission grade can come from the teacher or other students. If the field Number of Assessments of Student Submissions is set to something greater than zero, then students assess each other’s work. If it’s set to zero, then only the teacher is assessing the work.
Making the maximum grade a multiple of the number of assessment elements enables the students to interpret their grades more easily. For example, suppose a workshop is assessed on five elements; for each element, the assessor will choose from the following four statements:

1. The workshop does not meet this requirement in any way (0 points).
2. The workshop meets this requirement partially (1 point).
3. The workshop meets this requirement (2 points).
4. The workshop exceeds this requirement (3 points).

You could assign a point value of zero for each A, one point for each B, two points for each C, and three points for each D. Then, each element would be worth a maximum of three points. With five elements, the workshop would have a maximum grade of 15. This would make it easier for the student to interpret his or her grade.

Grading Strategy
A workshop assignment is quite flexible in the type of grading scheme used. It can be one of the following:

Not Graded
When this is selected, students can comment upon each assessment element but do not select a grade. The teacher can grade the students’ comments. In that case, the workshop is transformed from one where students grade each other to where the teacher grades each student’s comments.
This may be especially useful when you want to have a structured discussion about the material that you present to the students. As the course creator, you can present the students with material uploaded to the workshop, or use the workshop’s description to direct the students to the material they must assess. After the students view the material, they enter the workshop and leave comments according to the elements presented. As the workshop presents the students with evaluation elements, and it requires that they complete each element, your discussion would be more structured if you used a wiki or a forum.

Accumulative
In the Accumulative grading strategy, the grade for each element is added to arrive at the accumulated grade. This style of grading enables you to present the reviewer with a numeric scale. You can also present the reviewer with Yes or No questions, such as ‘Does this workshop meet the requirement?’ Or, you can present the reviewer with a grading scale, such as ‘Poor, Fair, Good, and Excellent’. If you do use a Yes or No or a grading scale, you will assign a point value to each response. Consider informing the reviewer of the value of each response. For example, instead of
just writing:

• Poor
• Fair
• Good
• Excellent
• Consider writing:
• Poor (1 point)
• Fair (2 points)
• Good (3 points)
• Excellent (4 points)

Error Banded
When you choose this option, students evaluate a workshop using a series of Yes or No questions. Usually, you create questions to evaluate whether the workshop met a requirement, such as ‘Does the student present a variety of opinions?’
When writing an error banded question, make sure that it can be answered using only Yes or No. A sign that you need to revise your question is the presence of the word ‘or’. For example, don’t write ‘Did the student describe the plant well enough to distinguish it from others, or, is there still a doubt as to which plant the student is describing?’ Such a question cannot be answered Yes or No.

Making Best Use of Error Banded Questions
The answer to an error banded question is sometimes very clear, and sometimes subjective. For example, the question ‘Did the student describe the plant well enough to distinguish it from others?’ is subjective. One reviewer might think the student did an adequate job of describing the plant, while another might think otherwise. Error banded questions can be a good way to perform subjective peer evaluations of the students’ work.

If the work requires a more objective evaluation, such as ‘Did the student include
all five identifying features covered in this lesson?’ you may not need a workshop. That kind of objective evaluation can be performed easily by the teacher using
an assignment.

Criterion and Rubric
For a criterion grading scale, write several statements that apply to the project. Each statement has a grade assigned to it. The reviewers choose the one statement that best describes the project. This single choice completes the review. The rubric grading scale is the same as the criterion, except that reviewers choose a statement for multiple criteria. The following is a screenshot of an assessment element:

For a rubric, you would create several of these elements, and the reviewers would select a statement for each of them. For a criterion scale, you would create only
one of these:

Number of Comments, Assessment Elements, Grade Bands, Criterion Statements, or Categories in Rubric
This field determines how many elements will be evaluated. No matter which number you select, the reviewers will always be presented with a general comments field into which they can type text. If you set this field to zero, reviewers will see only the general comments field.

Allow Resubmissions
The name of this field implies that a student can replace a previous submission with a new one. Actually, if you turn this option on, students can submit more than once, but all previous submissions are retained. Also, the latest submissions are not likely to be evaluated any more than the earlier submissions. Each submission is equally likely to be assigned to a reviewer.
This has implications for the course management. For example, suppose for the field, Number of Assessments of Student Submissions, you select 3. Half way through the course, you run a report showing that most students have completed their three assessments (they have evaluated three other students’ work). Then, students begin resubmitting their work. These resubmissions will be distributed at random among the reviewers who have assessments left. As course manager, you need to determine if there are enough assessments left to cover the resubmissions.
The system keeps the highest grade of all the assignments submitted by the student (the highest grade is the largest teacher-peer combined score).

Number of Assessments of Examples from Teacher
Setting this field to a number greater than zero forces the students to assess that many number of example projects from the teacher. The student must comment upon and grade the example. The student’s assessment can be graded by the teacher. The student cannot submit her or his work until she or he has gone through the example the teacher provided.

Comparison of Assessments
Work is often assessed by both the teacher and students. The work being assessed can be examples provided by the teacher, or work submitted by the students. In either case, it can be assessed by both the teacher and the student.
When a student assesses a piece of work, the assessment can be graded. For example, suppose the teacher is conducting an online digital photography class. The teacher supplies a photo and asks the students to rate the photo’s contrast, brightness, focus, and so on. The students can be graded on their assessments. (Did they notice the overexposed area on the subject’s cheek? Did they notice that the eyes were a little out of focus?) Moodle grades a student’s assessment by comparing it to the teacher’s assessment of the same work. The closer the student’s assessment agrees with the one given by the teacher, the more points the student earns.

How close to the teacher’s assessment must the student’s be to earn a good grade? That is determined by this setting. When Fair is selected, random guessing will usually give a score of zero, or close to zero. The other settings range from Very Lax to Very Strict. You can change this setting on the fly, and evaluate its effect on student grades.

Number of Assessments of Student Submissions
This field determines how many other projects each student is asked to review. If there are more submissions than the allowed assessments, the reviewer will get only the number set in this field. Some projects will not be reviewed.

Weight for Teacher Assessments
This value can range from zero to ten. If set to zero, the teacher’s assessment for a piece of work carries no weight for the student’s grade. If set to 1, the teacher’s assessment carries the same weight as the student’s assessment of that piece of work. If set to 2, the teacher’s assessment counts as much as two student assessments, and so on. If students have consistently over or under-graded assignments in a workshop, this setting can be used by the teacher to raise or lower the overall grades.

Over Allocation
As students submit or upload their work to a workshop, Moodle allocates it to other students for assessment. The field Number of Assessments of Student Submissions determines how many submissions each student is required to assess. Ideally, everyone will submit their assignments on time, and the students will have plenty of time to evaluate each other’s work. For example, suppose there are ten students in the class, and the Number of Assessments of Student Submissions is set to 3. This means that each of the ten submissions is assessed three times. Moodle assigns the assessments as the work is submitted.
However, if a student submits work late, the students who are going to evaluate the late person’s work will need to wait before they can complete their assessments. Let’s suppose one student doesn’t submit his or her work by the deadline. This means that the class is three assessments short. As Moodle assigns the assessments evenly, three students will end the class one assessment short. Shall we penalize these students for not completing the required three assessments?

In our example, Over Allocation is set to zero, and each submission is evaluated three and only three times. If we set Over Allocation to one, and the deadline arrives, Moodle will over allocate some work to the students who still need to complete assessments. In this example, Moodle will randomly choose three pieces of work that have already been assessed three times, and assign them to the three students who are missing an assessment. These pieces of work will then be over allocated by one assessment each. Moodle allows a maximum over allocation of two.

Self Assessment
If this is set to Yes, each student is asked to evaluate his or her own work. This is in addition to the number of student submissions that the student is asked to evaluate.

Assessments Must Be Agreed
If this is set to Yes, then an assessment made by one student can be viewed by the other reviewers of the same work. If the other reviewers disagree, the evaluation process continues until they agree or until the assignment’s closing time is passed. This can be a useful tool for determining how clear your evaluation elements are. If there is a lot of disagreement among reviewers of the same work, revisit your evaluation elements and the instructions you give the reviewers.

Hide Grades Before Agreement
If this is set to Yes, the numeric parts of a project’s evaluation are hidden from other reviewers. The reviewers can see each other’s comments, but not the grades they’ve assigned. The grades will appear after the reviewers agree with each other.

League Table of Submitted Work
This creates a list of the best-rated assignments in this workshop. If it is set to zero, no list is created.

Hide Names from Students
When set to Yes, this hides the names of the students whose work is being evaluated. Note that the names of students are never hidden from the teacher. Also, if a teacher assesses a student’s work, the teacher cannot do so anonymously. This only hides the names of students who submitted work from the students who are evaluating
the work.

Use Password and Password
You can use these fields to password-protect the assignment.

Maximum Size
This field sets the size limit for project files uploaded to the workshop.

Start and End of Submissions/Assessments
These fields determine when the workshop opens and closes. On the closing date,
if any hidden grades appear, students can no longer upload files and evaluate
others’ work.

Release Teacher Grades
You can use this field to withhold the teacher’s assessments until a given date.

Group Mode
Just as in other activities, this determines if access is segregated by group.

Visible
This field shows the workshop or hides it from students.

Summary
Moodle offers several options for student-to-student and student-to-teacher interaction. When deciding which social activities to use, consider the level of structure and amount of student-to-student, student-to-teacher interaction you want. For example, chats and wikis offer a relatively unstructured environment, with lots of opportunity for student-to-student interaction. They are good ways of relinquishing some control of the class to the students. A forum offers more structure because entries are classified by topic. It can be moderated by the teacher, making it even more structured. A workshop offers the most structure, by virtue of the set assessment criteria that students must use when evaluating each other’s work. Note that as the activities become more structured, the opportunity for students to get to know each other decreases.