Reflections

Reflection 1

In the seminar on designing courses and assessments, everyone shared one of the examples of the planned unit design or assessment brief. In the break-out group discussions, I shared one of my online unit designs for the postgraduate level. I shared information about how I reviewed my unit learning outcomes compared with the toolkit for crafting learning outcomes from Moodle.

In the break-out group discussions, the team members gave me some suggestions and they showed a very clear roadmap in the unit handbook, from aims to learning outcomes, and displaying the five criteria in enquiry, knowledge, process, communication, and realisation relevant to its learning outcomes. The unit aims, learning outcomes, and assessment could be created based on intrinsic motivation or external values. A culture of metrics and rankings is created to answer the question about what we are fostering or enabling for the students.

It can be clearly seen that the learning outcome with ‘successfully applying knowledge into practical’ cannot be purely examined by one method of assessment (O’Brien and Brancaleone 2011), for example, calculations or multiple choices. To review the learning outcomes with the assessment method, it should contain more than one format of assessments for the unit. How to develop and apply the knowledge beyond the lecture contents is difficult to test.

It is back to the original idea in the class, purpose, and perception of purpose are all in the loop (Addison, 2014). Once we write down the learning outcomes and assessment brief, it is helpful to review them again based on the intrinsic motivation or external values. It could be helpful to engage teaching team members to review the learning outcomes and assessment methods, rethink whether they meets the internal and industrial requirements, and also with a sufficient unbiased test method.

References:

Addison N., 2014, Doubting learning outcomes in higher education contexts: from performativity towards emergence and negotiation, The International Journal of Art & Design Education, 33(3) pp. 313-325.

O’Brien, S. and Brancaleone, D., 2011, Evaluating learning outcomes: in search of lost knowledge, Irish Educational Studies, 30(1), pp. 5–21.

Reflection 2

Data that is available to me is precious as I am a quantitative person. Thus, when I have been asked to write down my feelings about the data and to write down a poem for data. All I can think about is if I have the data that I need, just like the Simpsons lying on the doughnuts in heaven. But also, understand the data limitations and data bias. I shared one quote with my break-out group members, which is quite famous in quantitative analysis “All the models are wrong, but some are useful” – coined by the statistician, George Box (Berro, 2018).

Data provide many positive outcomes, i.e., to tell the stories, to identify the problems, etc… More importantly, we need to understand the limitations and how to interpret the data depending on the availability (Boudreau et al., 2001).

Considering the data that we collected for Unit/Course Evaluations, the data collection may have problems with the biased collection procedure or only collecting limited samples, to interpret the population. This is why when we reviewed the data from the institution, i.e., showing the percentages of home-based students and international students from Asia or the EU is important, because it tells some stories with demographic considerations. The value of these data is partly depending on how we interpret the data, and how to reflect it on our institutions for future improvements.

We need to critically review the existing university data. Even though we could interpret the data in a very positive way, for example, the students are more hard-working, and more happy to learn because of the innovative teaching methods. But also it should leave us to think about whether we are using the inappropriate assessment method or criteria for the students. Both positive and potential limitations could improve the institution’s performance as a whole.

References:

Berro J., 2018, “Essentially, all models are wrong, but some are useful” a cross-disciplinary agenda for building useful models in cell biology and biophysics. Biophys Rev.

Boudreau M. C., Gefen D., Straub D.W., 2001, Validation in information systems research: A state-of-the-art assessment, MIS Quarterly, pp. 1-16.

Reflection 3

This reflection focused on the exploring of institutional research, as the data collected from institutions could be rich in insight, compared with other general populations (Lepori et al., 2022). We also can benefit by doing the research as insiders because it is much easier to get permission from other students or staff. Also, existing internal statistics data is more possible to obtain from the institution, and we have prior experience and knowledge about our institution.

Not only focus on the positive side, but a few different perspectives also need to be aware before we decide to apply institutional research as the appropriate research method, i.e., the potential difficulties about how to deal with the bias or unexpected if the students are more vulnerable. By doing institutional research, more controlling methods that aim to release the possible tension between the roles of researcher and participants could be considered, because this tension during data collecting might affect how you communicate the research questions (McCormack, 2004).

Other perspectives argue that institutional research lacks the degree of reflexivity and the obvious conflict of interest that may occur (Grimpe, 2020). However, if institutional research has been stopped because of this reason, then there would be a significant lack of evidence to guide and direct future improvements for our institutions. Although conducting research within institution could receive criticises from colleagues, it is a way to review and change the institution.

After critically reviewing the pros and cons of institutional research, I will continually use my professional research skills to keep a neutral, relaxed, and unbiased relationship with participants when doing institutional research, aiming to minise the potential bias.

References:

Grimpe B. et al., 2020, From collaborative to institutional reflexivity: Calibrating responsibility in the funding process, Science and Public Policy, Volume 47, Issue 5, Pages 720–732.

Lepori, B., Borden, V. M. H. and Coates, H., 2022, Opportunities and challenges for international institutional data comparisons, European Journal of Higher Education, 12(sup1), pp. 373–390. 

McCormack C., 2004, Tensions between student and institutional conceptions of postgraduate research, Studies in Higher Education, 29(3), pp. 319–334. 

Reflection 4

This reflection will focus on the Ethical considerations when we supervise Final Major Projects at the underground level and Dissertations at the postgraduate level. Ethical considerations in research are critical in protecting the safety of study volunteers and safeguarding the legitimacy of results (Nii Laryeafio and Ogbewe, 2023).

Working in the supervisor’s position, I have introduced students to the importance of ethical considerations as a part of their primary research and highlighted the requirements of an approved ethical form from their supervisors before they start the primary data collection. Students are asked to draft their answers to the ethically relevant questions in the form in advance of the supervision meeting, and further clarifications also have been provided by students during the meeting. Then I signed the ethical form to approve the research plan to meet all the criteria in accordance with the Code of Practice on Research Ethics of the University of the Arts London.

Students are asked to provide clarification on a set of questions, i.e., sampling method, data collection method, data storage, confidential process, etc., which are the key concepts for ethical research (Drolet et al.,2022). The methods and actions that students are required to consider for their primary research include the requirements of participants internally within the university or externally outside the university, potential risks for both participants and researchers during the data collection, and how they are considered to minimise the effects.

I will continue to provide guidance on how to do appropriate research as a supervisor in my future work.

References:

Drolet, M., Rose-Derouin, E., Leblanc, J., Ruest, M. and Williams-Jones, B., 2022, Ethical issues in research: perceptions of researchers, research ethics board members and research ethics experts, Journal of Academic Ethics, Vol. 21 No. 2, pp. 269-292.

Nii Laryeafio, M. and Ogbewe, O.C., 2023, Ethical consideration dilemma: systematic review of ethics in qualitative data collection through interviews, Journal of Ethics in Entrepreneurship and Technology, Vol. 3 No. 2, pp. 94-110.