Staff Guide: Module Evaluation

Introduction

Welcome to Module Evaluation Questionnaires (MEQs), a valuable tool for gathering feedback from students on their module experiences and teaching staff. Managed by Academic Monitoring, MEQs offer insights to enhance teaching and learning practices.

MEQs typically occur towards the end of a taught module, such as in December for Semester 1 modules. They involve online anonymized questionnaires. Before MEQs commence, school staff can designate modules for evaluation, select additional questions, and nominate staff for evaluation.

During the evaluation period, students access the questionnaires via links in MMS and MySaint, leading them to the Evasys platform. Automated reminders prompt students to participate. Once the evaluation period concludes, staff can access a breakdown of responses.

Teaching staff can review module-wide feedback and responses related to their own performance, while coordinating staff can access all responses. Additionally, MMS offers a school-level overview known as the Tartan Rug, exclusive to staff with roles such as Head of School or Director of Teaching.

For more detailed information, visit: Module Evaluation Questionnaires.

  • Module evaluation plays a crucial role in gathering feedback to enhance teaching and learning experiences. Here’s how to set up a module for evaluation:

    1. Access Module Overview: Navigate to the Module Overview page, by selecting the module name anywhere in MMS or in the MyModule portlet in MySaint.
    2. Edit Module Details: In the Edit Module section, locate the Evaluation row (see Figure 1). Here, you can specify the evaluation method:
      • Do not evaluate: This option ensures the module won’t be evaluated, and no questionnaire will be generated.
      • Evaluate online: Ideal for standard undergraduate modules requiring evaluation.
      • Evaluate online, Dissertation/Project module: Tailored for dissertation or project-based modules.
      • Evaluate online, Group Dissertation/Project module: Suitable for group dissertation or project modules.
    3. Select Report Recipient: Choose the primary module coordinator from the Reports to dropdown list. Note that this selection is necessary for Evasys but won’t impact visibility of responses in MMS.
    4. Optional Questions: If additional questions are needed, contact [email protected] for the required code. Enter the code if applicable.
    5. Save Changes: Scroll down and select the green Save Changes button to confirm your settings.
    A screenshot of the Module Oveview page with the MEQ options indicated with blue arrows.
    Figure 1: Module Evaluation options on the Module Overview page.

  • Staff members with an academic role on a module can be designated for evaluation. Follow these steps to set up staff for evaluation:

    1. Access Module Overview: Navigate to the Module Overview page, by selecting the module name anywhere in MMS or in the MyModule portlet in MySaint.
    2. Select the Staff Tab: Within the Module Overview, locate and click on the Staff tab.
    3. Locate Staff Members: Use the table provided to locate the staff members to be evaluated. You can utilize the Filter table functionality by typing in a user or role to narrow down your search. Note: If a user does not appear in the staff table, they have not been added as staff for the module. Refer to this guide to add a user as member of staff on the module.
    4. Check the Evaluation Box: In the Evaluate column next to the respective staff member, tick the checkbox. A small dialogue box will appear on the screen indicating the successful assignment.

    Staff members must be set to be evaluated before the deadline given by the Module Evaluation team, or they will not be included in the questionnaire. If a member of staff needs to be evaluated, but has been added late, please contact the Module Evaluation team: [email protected]

  • Students can access Module Evaluation Questionnaires (MEQs) only during the evaluation period. Here are the ways students can access MEQs:

    1. MySaint: Students can find the Module Evaluation Questionnaire link under the respective module on the My Courses page in MySaint.
    2. MMS: Similarly, students can access MEQs by clicking the Module Evaluation Questionnaire link under the module on the Modules page in MMS.
    3. Reminder Emails: All students will receive automatic reminder emails during the evaluation period. These emails contain direct links to the MEQs. Since MEQs are anonymous, MMS sends reminders to all students regardless of whether they have completed the questionnaire.

    During the evaluation period, the MEQ tool also provides staff with a QR code. This QR code can be downloaded and added to PowerPoint presentations and handouts. Students can scan the QR code to access the questionnaire directly.

  • After the evaluation period concludes, staff can access student responses to Module Evaluation Questionnaires (MEQs). It’s important to note that not all staff members will see the same response options, as visibility is contingent upon individual permissions and roles within the system.

    Here’s how to view and interpret MEQ responses:

    1. Accessing MEQ Responses:
      • Click the Module Evaluation Questionnaire link for the respective module in MySaint or on the Modules page in MMS (refer to Figure 4).
    2. Adjust View Options:
      • Customize your view by selecting the required radio buttons (refer to Figure 5). These options determine the format and presentation of the responses.
    3. Interpreting Response Options:
      • Questions Stats: Displays statistics associated with each question, such as the number of responses, average response, and standard deviation.
        • Options:
          • None: Numerical stats are not displayed.
          • Present: Numerical stats are displayed under the graph.
      • Staff: Allows responses to be broken down by staff where applicable. Only available for coordinating staff.
        • Options:
          • Separate entries: Responses are broken down by staff member.
          • Combine by type: Responses relating to staff are grouped by question.
      • Free Text Questions: Displays responses to questions where students provide text responses.
        • Options:
          • Hide: Responses to free text questions are hidden.
          • Show count: Number of responses is shown, but individual responses are hidden.
          • Show comments: Typed responses are displayed, grouped by question.
      • Questions to Include: Determines which questions’ responses are displayed.
        • Options:
          • Centrally defined only (exclude school questions): Displays responses to default questions only.
          • All: Displays responses to all questions, including additional school-specific questions.
      • History: Allows viewing responses from previous instances of the module.
        • Options:
          • None: Only responses for the current instance are displayed.
          • Three years: Displays responses for the current academic year and the previous two.
          • Five years: Displays responses for the current academic year and the previous four (may take longer to load).

Tartan Rug

The Tartan Rug is a school/unit level report providing a high-level overview of Module Evaluation Questionnaire (MEQ) responses. It offers valuable insights into the overall feedback received from students. Access to the Tartan Rug is limited to staff members with specific roles at the school level such as:

  • Director of Teaching
  • Director of Postgraduate (Taught)
  • Head of School
  • Module Evaluation Officer

The Tartan Rug gets its name as the default view is a colourful grid. The color indicates how the score compares to a standard norm, which is the university-wide average across all modules evaluated in the same semester and academic year. If a module matches the norm exactly, the background is green. If it’s below the norm (usually indicating improvement), it’s blue, and if it’s above the norm, it’s red. The intensity of the color reflects how far the score is from the average, with pure blue or red indicating a deviation of 2 standard deviations. However, this assumes a normal distribution, which is not always the case.

For response rates, the color trend is reversed, as a higher response rate is considered favorable. The norm here is the average response rate across the university. Keep in mind that this is a rough guide, as modules with fewer students may have different response rate distributions than those with more students.

To access the Tartan Rug:

  1. Log into MMS: Use your credentials to log into the MMS system.
  2. Select School: At the top of the Modules page, select the relevant school from the drop down list.
  3. View School: Select the View School button.
  4. Access Tartan Rug: From the Assessment section, locate and select the MEQ Overview (aka ‘The Tartan Rug’) link.
  5. Select Academic Year and Semester: Choose the academic year and semester for which you want to view the Tartan Rug. You can select multiple options by holding down the shift button while clicking.
  6. Module Restrictions: The Tartan Rug can be restricted to certain modules. To view all modules in the unit for the selected year and semester, leave the Restrict Modules to: field blank. For information on how to restrict modules, refer to the Module Restrictions section.
  7. Generate Tartan Rug: Once you’ve made your selections, click the Change Display Settings button to generate the Tartan Rug.

  • The ‘Restrict Module’ box allows you to filter modules. You can use ‘.’ as a single character wildcard and square brackets to match any characters. For example, ‘..[34]’ matches any honors (level 3000 or 4000) modules, and ‘AB’ matches any modules starting with ‘AB’. While this box supports a subset of regular expressions, it does not support grouping operations.

  • There are two ways to view the tartan rug over multiple years. The easiest way is to select multiple years when from the options at the top of the table. This can be done on a PC by holding shift while you select each year.

    However it can also be done by:

    1. Access Tartan Rug: From the Assessment section, locate and select the MEQ Overview (aka ‘The Tartan Rug’) link.
    2. Select Academic Year and Semester: Select a single academic year and semester
    3. Module Restrictions: The Tartan Rug can be restricted to certain modules. To view all modules in the unit for the selected year and semester, leave the Restrict Modules to: field blank. For information on how to restrict modules, refer to the Module Restrictions section.
    4. Generate Tartan Rug: Once you’ve made your selections, click the Change Display Settings button to generate the Tartan Rug.
    5. Select the Show column isolation options checkbox.
    6. Select the Isolate Column heading in the table.

Recent Blog Posts

Cookie preferences