In the event that a researcher wishes to evaluate students’ behavior in school, there may be differences in their behavior in the classroom, in the canteen, in the sports hall, etc., and therefore observations should be made in all these places. The researcher can choose to be present in all these places and observe students’ behavior and take notes about all the processes and indicators of students’ behavior. It can be participatory observation if the researcher is a teacher or school worker who engages in all activities, or it can be nonparticipatory observation where the researcher follows the students throughout the school premises.
If it is participatory research and the students are familiar with the researcher, it is easier to follow students and to take notes, but the challenge can be in the fact that a researcher may already have existing opinions about the students. Sometimes this can cause a situation when minor data are not noticed as the researcher focuses his/her attention on students who can cause behavioural problems without noticing what is going on around the whole room. In the event that there are several teachers who are involved in data collection, there should be a common understanding on how to observe behavior, what kind of data should be the focus of attention, etc. In that case, structured observation could be a better solution to ensure that data from all the people who are taking notes are comparable.
If it is nonparticipatory research and students are not familiar with the researcher, this can change the dynamic of their behavior in the presence of an unfamiliar person, and the researcher should be patient to wait until students accept his/her presence and start to behave naturally. In both cases, notes of observational results should be taken, and the researcher should be aware of what can be observed.
Some examples of observational protocols are given below. In both cases, information should be included about the date, time, and place of observation, as well as how many students are present. In the case of structured observational protocol, researchers should agree on how the data will be collected as there is a possibility to insert information about particular students; alternatively, information can be collected about a group, in which case the structure is different.
An evaluation rubric can be used to ensure the objectivity of the data analyzed, but there still can be some subjectivity as the researcher’s perspective can be used to evaluate particular criteria. For example, during the evaluation of the experiences, the educational perspective can be the most dominant one, and during the evaluation, the students’ perspective can be kept in mind by using the researcher’s personal experience of working with students of different age groups. The selected applications can be evaluated, and the obtained results quantitatively and qualitatively evaluated, according to the comments made during the evaluation of each virtual museum application. The evaluation rubric can be used to evaluate the virtual museums and in the example provided below contains 25 criteria divided into three groups: (i) Technical performance (11 criteria); (ii) information architecture (5 criteria); and (iii) educational value (8 criteria), as well as one criterion for specifying the age groups for which the assessed virtual museum may be eligible. The boundaries of these groups are not strictly separated (see figure 1), because all factors interact and influence information perception processes, thus affecting the ability to store information in long-term memory, analyze it and synthesize new knowledge concepts, which is the primary goal of the educational process. Each of the criteria in the rubric can be evaluated on three levels, each of which has its own description (see table below). After evaluating each of the criteria, the possibility to add comments describing the specific criterion and the chosen rating is provided. During the quantitative analyses of the results, the levels are indicated by numbers where 1 indicated the lowest level, 2 indicated the medium level, and 3 indicated the highest level of the criterion.
Figure 1 Structure for the evaluation of virtual museums.
Criterions for the evaluation of VR/AR experiences in museums from an educational perspective. All the criterions should be evaluated by ticking the correct answer according to the evaluator’s opinion. Only in the last row, where the evaluator’s opinion on the age group should be given, can more than one answer be chosen.