Upcoming panel presentation at #ICOM-11

The programme for the online part of the ICOM-11 conference 27 September, 2023 has been published. I will be exploring ethical aspects of education and assessment where AI is already visible on the horizon togehter with David Ruttenber and Laily Harti.

Programme

Florén, H. (2023, September 27). Multimodal assessment in higher education meets generative AI. In D. Ruttenberg (Chair) Multimodality and Future Landscapes: Meaning Making, AI, Education, Assessment, and Ethics [Online symposium panel]. ICOM-11 International Conference on Multimodality, Online & London, UK. https://internationalconferencemultimodality11.wordpress.com/programme/

Animating with AI

I have just explored AI for video animations. My first attempt is based on one of my recent blog posts on the same topic

The tool I used is called steve.ai.

Assessments, Texts and AI

When Artificial Intelligence [AI] can be prompted to produce text that can pass for being written by a human, and can generate images, or write music, and generate voice and image that ‘impersonate’ a specific person – then we need to reconsider how we teach and assess.

There are many opinions about assessment in education and several positions in research. I am in this post not going to go into any of these, but I do recognise that there are differences. However, regardless of how you see or approach assessments, if you are worried I suggest that you use more than one way to find out about your students’ learning.

It the students’ are writing – add something else as well. This could be a filmed presentation or a discussion in class, or some other form where the student has to apply knowledge in some way that can be traced and evidenced as their learning. There are many ways assessments can be constructed where AI would not be a problem. Why not use ChatGPT as part of an assignment – maybe analyse the resulting text? There is room for creativity here – map the student’s knowledge and learning in multiple ways.

We also need to start thinking about what AI can, as well as cannot, do or be.

Exploring Points of Reference and Norms Guiding Recognition of Quality in Multimodal Texts

I will be speaking about what may be guiding teachers’ assessements of multimodal texts at the Bremen-Groningen Online Workshops on Mulitmodality #BreGroMM Friday 1 April, 2022, 13-17.30 CET.

Data and analyses are part of my doctoral research at UCL Institute of Education. This is focused on assessement in higher education, and student produced multimodal texts, where students have represented their learning and knowledge in shapes and forms beyond conventional written texts.

Abstract

This presentation gives an account from a pilot study that is part of my doctoral research into  multimodal text practices and assessment in higher education. Using a multimodal social semiotic theoretical frame I explored norms guiding teachers’ assessments by examining three teachers’ points of reference for recognizing quality in five students’ multimodal texts. Data was collected from three different courses at a Swedish university and analysed using different forms of multimodal transcription. I identified four preliminary points of reference for the recognition of quality in multimodal text: shape, logic, register and selection of content. Results indicate that the teachers recognize quality in multimodal texts after a) relevant content, b)  coherent use of writing, image, colour and layout, c) a pleasing and effective design, and d) appropriate register. I tentatively suggest that these preliminary criteria indicate areas that could inform a future framework guiding multimodal assessment and supporting the recognition of knowledge and learning.

What drives my research interest?

There is always a reason, an underlying motivation, for any individual interest. This includes research interest. At a basic level I am interested in how people learn and in the conditions for learning. I think the roots of this goes far back into my own childhood. Somewhere, somewhen, there is that something that is the core of our individual existence. I remember looking in the mirror as a young child, and being aware of myself as separate from others. We are separate and unique but in other ways we all share basic feelings and experiences. One things we all have in common is the experience of being children, of being young and small.

My core belief is that most important of all, at any stage of education, is compassion and kindness. I hope that any future results of my research will improve and make better education for students and teachers alike. Teaching is a difficult profession where you often have to think on your feet, and I think teachers are often undervalued. That said there is room for improvement both in practice and in systems. Formal education takes place within systems and in institutions of learning. These are regulated by rules and structures and histories of practice. A student can easily become insignificant in these system. However, effects of a system will become part of the student’s lived experience for good or bad. Therefore I think it is important to reflect on the power educators have over students and how this power translates into teaching and assessment practices.

We measure and judge students and their knowledge and learning. We turn these measurements and judgements into marks and grades. These assessments can determine access to further education, jobs and affect feelings of self-esteem (Bezemer and Kress, 2016) making assessment practices a core influence on, not just students’ learning, but also their lives. Therefore it is important that assessments recognise signs of learning when expressed in different ways, using different media and in different shapes and forms. My ongoing research on digital text and assessment practices place focus on recognition of learning in higher education, but should be relevant for other levels of education.

Reference:

Bezemer, J., & Kress, G. (2016). Multimodality, Learning and Communication: A social semiotic frame. Routledge. https://doi.org/10.4324/9781315687537

Educational culture and assessment in Sweden

What is it about the Swedish educational culture that makes it so hard to decipher and explain?

I am researching text practices and assessment in higher education and I’m collecting my data in Sweden. This means I have to explain the context for an audience who are not familiar with the Swedish education systems and culture. My own experience of being a pupil and student spreads across different cultures; Swedish, French, British and US education systems. I know from experience that Swedish education is different. What I had not previously identified is how different this Swedish educational culture seems to be. However, to be able to explain the context I first have to understand it myself.

There is the focus on collaboration and group work, and the focus on children who need help and support, and no focus on gifted students. The ideal is inclusion and collaboration. Another Swedish cultural ideal is the concept of ‘lagom’ (not too much, not too little of anything), and for not standing out. Children in Sweden start school comparatively late, the year they turn seven, although most children attend F-class when they are six. I would say Sweden has a child centred educational culture and that there is a cultural norm for letting children be children and leaving a lot of space for children to play.

When it comes to assessment I find it especially tricky to decipher and explain the context. Contemporary Swedish education, including higher education, is influenced by sociocultural perspectives on education (Englund, Olofsson & Price, 2018) and has since the 1970s had a “grading-hesitant educational culture” (Dahl, Lien & Lindberg-Sand, 2009, p.61). Younger children are not graded at all. Grades are not introduced until year 6 when the children are 12, and grades are criterion based which theoretically means that everyone in a class can achieve the highest grade.

Higher education in Sweden is highly decentralised and each institution can decide on what grading scale to use, and this can be delegated down to the level of the subject level. In one and the same institution for higher education there can be several grading scales used. It is the teachers who teach the courses that also grade the students. Each and every course is graded and the most common grading scale is pass/fail which makes the topic of grade distribution “void of valuable information” (Dahl et al., 2009, p.72), although other scales are used such as the A-F  scale, or pass/pass with distinction/fail. There is no system of final exams and students get no grade for degrees, for example a BA, or professional (licensed) degrees such as a for teachers.

All of this seems to cause confusion for people from other educational cultures, especially where there is a firm paradigm of measurement and accountability. Swedish teachers at all levels of education work within a high trust and low accountability culture. I am finding it increasingly interesting even if it just the backdrop for my research and not the focus. This needs further thinking and writing about.

References

Dahl, B. L., & Lindberg-Sand, Å. (2009). Conformity or confusion? Changing higher education grading scales as a part of the Bologna Process: the cases of Denmark, Norway and Sweden. Learning and Teaching, 2 (1), 39–79. High Educ (2009), 58(53), 1-549. 

Englund, C., Olofsson, A. D., & Price, L. (2018). The influence of sociocultural and structural contexts in academic change and development in higher education. Higher Education, 76(6), 1051-1069. 

Assumptions I make

It is hard to talk about knowledge, learning and assessment

As I am researching multimodal text making and the recognition, evaluation and assessment of such texts in higher education it is becoming increasingly apparent how difficult it is to talk about these areas even with other researchers.

There is a need for some basic assumptions to be articulated, because we do not always make the same assumptions about basic (but complex) concepts such as learning and knowledge. I recognise that learning can be approached from different perspectives. However, in my research I make the assumption that learning and knowledge are interdependent. This is a basic assumption that comes with the theoretical framework of multimodal social semiotics (Jewitt, 2014; Kress, 2010) that I am using. Other assumptions are that all education is socially situated and that learning is context dependent.

I also assume that the activities of evaluation and assessment requires recognition of learning and knowledge. Students represents their learning and knowledge in some material way. In these material representations, where students can be using speech, writing, image, gestures, sounds etc., the teacher has to notice and recognise the knowledge and learning that the students are expressing or giving shape and form to (Kress, 2014). My point is that teachers cannot assess what has not been noticed. First we have to ‘see’ it. Then what the teacher sees has to be recognised as valid.

References

Jewitt, C. (Ed.). (2014). The Routledge Handbook of Multimodal Analysis (2nd ed.). London, England: Routledge.

Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London: Routledge.

Kress, G. (2014). The rhetorical work of shaping the semiotic world. In A. Archer & D. Newfield (Eds.)  Multimodal approaches to research and pedagogy: Recognition, resources, and access, (pp. 131-152). New York: Routledge 

Digital Storytelling to Document Learning in Higher Education #BbTLCEUR 2020

A version of my presentation at TLC Europe Online 2020 can be found here below, but one slide with extracts from my data has been excluded.

Digital Storytelling to Document Learning in Higher Education_Examples from a Swedish context_henrikafloren_BbTLCEUR from Henrika Florén

Abstract

This presentation will address the question, how can digital storytelling be used to document learning in higher education? Examples of assignments, from a Blackboard Learn environment in a Swedish context, from two different subjects and academic fields, will be highlighted in order to illustrate how students can represent their learning and knowledge, by creating multimodal texts in the shape of digital presentations with still and moving images and sound. In these examples the students are presenting, reflecting, illustrating and commenting as part of their academic learning, using presentation programs and screencasts. In higher education students increasingly work with various digital formats, creating multimodal texts that combine modes such as image, moving image, sound and writing. Such multimodal texts are complex representations of knowledge and learning. A multimodal social semiotic framework (Kress, 2010) here provides tools for studying and understanding the students’ multimodal texts (written, spoken, visual, gestural). The changing possibilities for students to represent learning and knowledge in multimodal texts also entail changes in how teachers design their teaching and in how they recognize and understand learning and knowledge. As a reflection of pedagogy, digital storytelling in the form of multimodal texts is therefore closely linked to teaching quality.

OECD and literacy

I’m looking at how reading is defined, understood and assessed in the Organisation for Economic Cooperation and Development (OECD).  A logical place to start is the OECD iLibrary, which is an online library that has books, papers and statistics available. Among other texts the PISA reports can be found found here.

PISA is the OECD Programme for International Student Assessment, which “assesses the extent to which 15 year old students near the end of their compulsory education have acquired the knowledge and skills that are essential for full participation in modern societies” (OECD, 2019, foreword). However,  the definition of reading literacy has not changed in any significant way and can for PISA 2018 “be summarised as understanding, using, evaluating, reflecting on and engaging with texts in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society”(Mo, 2019, p.2).

Today reading also include (multimodal) digital formats. PISA 2018 used a new framework for reading assessment, which “placed greater emphasis on the ability to find, compare, contrast and integrate information across multiple sources” (p.2) and included texts such as “an online forum with multiple posts, and a blog that links to a newspaper article” (Mo, 2019, p.2). Although PISA 2018 has taken digital reading environments into account the definition of text is still one that places writing as the dominant mode and image or other modes as secondary (OECD, 2016).

---

LINKS

OECD Education – http://www.oecd.org/education/

OECD iLibrary – https://www.oecd-ilibrary.org/

---

References

Mo, J. (2019), “How does PISA define and measure reading literacy?”, PISA in Focus, No. 101, OECD Publishing, Paris, https://doi.org/10.1787/efc4d0fe-en.

OECD. (May 2016). PISA 2018 DRAFT ANALYTICAL FRAMEWORKS MAY 2016. Retrieved from https://www.oecd.org/pisa/data/PISA-2018-draft-frameworks.pdf

OECD (2019), PISA 2018 Assessment and Analytical Framework, PISA, OECD Publishing, Paris, https://doi.org/10.1787/b25efab8-en.