Assessments, Texts and AI

When Artificial Intelligence [AI] can be prompted to produce text that can pass for being written by a human, and can generate images, or write music, and generate voice and image that ‘impersonate’ a specific person – then we need to reconsider how we teach and assess.

There are many opinions about assessment in education and several positions in research. I am in this post not going to go into any of these, but I do recognise that there are differences. However, regardless of how you see or approach assessments, if you are worried I suggest that you use more than one way to find out about your students’ learning.

It the students’ are writing – add something else as well. This could be a filmed presentation or a discussion in class, or some other form where the student has to apply knowledge in some way that can be traced and evidenced as their learning. There are many ways assessments can be constructed where AI would not be a problem. Why not use ChatGPT as part of an assignment – maybe analyse the resulting text? There is room for creativity here – map the student’s knowledge and learning in multiple ways.

We also need to start thinking about what AI can, as well as cannot, do or be.

Keeping structure in the PhD process

If you are at the beginning of your PhD process or if you have become lost in the process and the work load don’t lose heart. When talking to other doctoral student who are just starting I have been asked multiple times how I manage to do this and work at the same time. And the answer is structure. I hope this post can help.

I have been able to keep on track with my PhD process while I also work. Now how to I do this without breaking? Because I’m not. The answer is structure, not just in the writing or the research but a personal structure for the whole process. There are some key elements to this.

Keep track of time

First of all I calculated how much time each week (on average) I need to put into my PhD work.  Now I’m doing my PhD in the UK and part time which means I have five years, and not three, to complete my doctoral training and research.

Keep a log

Then I keep a combined log-planner for my work. I plan ahead for each week, what tasks need to be completed and approximately how many hours I should work each day. Some days I do more work and other days less. I also plan for time off. For my log-planner I use Google Docs, but you could use any application that you feel comfortable with and which allows you to plan and keep track of what you are doing.

The good thing about keeping a log-planner in Google Docs is that I can access this from multiple appliances and I’m careful to add the 5,10,15 minutes on the train or bus when I check my email, calendars, social media (research related posts). For each day I update my Log-Planner for what I’ve actually done and move things not complete ahead to the next day. So, the planner also functions as a ‘to-do-list’. In this way I save time because I keep everything in the same place. For meetings and such I use the University Outlook Calendar, but I add a note about meetings in my log. I also add short notes and reflections as I go along, taking care to also note what went well today. I keep one log for each term and the summer and save these in pdf format at the end of each term. This way I have a complete log of what I have been doing, I can pace my work so that my work load becomes reasonable and I don’t have to feel bad or stressed when I take time off. I then have an Excel document that calculates the total time for me, but you can also do that manually. I keep track or hours for each day, each week (+ or – ) and then for the whole year.

There are a number of ways you could do this, but they key take away should be

• Plan and keep track of your time
• Work when you have planned to (not later)
• And keep a record

Example Day – General structure for my PhD Log/Planner Document

Structure your reading

The third thing I recommend any doctoral (or post-graduate) student to do is to create a record of your reading. This you can do in multiple ways. The most simple option is to have a document template for your academic reading and notes, and create one document for each title you read.

I use a database solution instead, with one entry (page) for each title. This allows me to search across my readings using keywords. Creating a structure using some digital tool that will allow you to search your notes will save time and effort when you need to reference to previous research or other literature.

Enjoying the PhD process

I may be fortunate in that I can still enjoy the process two thirds into my PhD journey. As a part time PhD candidate the expectation seems to be that I would be over worked. In general, and from other people’s comments, I understand that the journey towards a PhD is often tough and stressful. Somehow I have managed to avoid this.

I should at this point own up to often being tired and sometimes frustrated with parts of the process, but life in general can be tiring and frustrating. I have nothing to complain about when it comes to research training, or about support from my supervisors and other people at UCL Institute of Education along the way of my PhD journey. One explanation is probably found in the structures and recourses available to me at UCL, but no matter how good training and supporting structures are there is also me as a person.

I think the two key things that keep me sane and sound, while both working full time (in academia as an educational developer) and conducting my doctoral research in parallel, are daily organised routines (habits) and the happiness I feel when working with my research. I work before and after my hours of ‘regular’ work, but I exercise regularly and I take care to get enough sleep.

I manage my research time, which means I work on my research most days, but never all day. I log everything I do which gives a very good overview of what I have done and how many hours I have worked. This means I can allow myself to stop even if I didn’t finish something I had planned to do. My planning is usually over-optimistic about the time required to do something. I know this about myself, and as long as I do the hours I have decided to work this particular week, that will be fine. This happens regularly and then I adjust my planning (to-do-lists) for the up-coming week. It’s about being systematic in work habits and not just the research project. So, I think key is to form habits of consistent time management. That includes time to sleep, eat, exercise, family and friends.

The other key ‘thing’ I think is my love for the research process. I may be fortunate in this, but I have found that I’m happy working with my data and writing. The closest I can come is the feeling I had as a child when playing and being completely immersed in that experience. Often I have to stop what I’m doing because my body says ‘stop’, but my mind wants to go on. I don’t know if this makes sense to you, but this feeling is very powerful. However, I do not wait for the feeling to surface before starting to work. It is the other way around – I start working and then after a while the feeling of flow, of play, of happiness, surfaces.

I recognise that we are all different, and that what works for me might not be a good solution for someone else, but if you are about to start working towards a PhD, then organising your time will be helpful. Stay fit, eat well, sleep, exercise (walks will do just fine) and have fun!

Exploring Points of Reference and Norms Guiding Recognition of Quality in Multimodal Texts

I will be speaking about what may be guiding teachers’ assessements of multimodal texts at the Bremen-Groningen Online Workshops on Mulitmodality #BreGroMM Friday 1 April, 2022, 13-17.30 CET.

Data and analyses are part of my doctoral research at UCL Institute of Education. This is focused on assessement in higher education, and student produced multimodal texts, where students have represented their learning and knowledge in shapes and forms beyond conventional written texts.

Abstract

This presentation gives an account from a pilot study that is part of my doctoral research into  multimodal text practices and assessment in higher education. Using a multimodal social semiotic theoretical frame I explored norms guiding teachers’ assessments by examining three teachers’ points of reference for recognizing quality in five students’ multimodal texts. Data was collected from three different courses at a Swedish university and analysed using different forms of multimodal transcription. I identified four preliminary points of reference for the recognition of quality in multimodal text: shape, logic, register and selection of content. Results indicate that the teachers recognize quality in multimodal texts after a) relevant content, b)  coherent use of writing, image, colour and layout, c) a pleasing and effective design, and d) appropriate register. I tentatively suggest that these preliminary criteria indicate areas that could inform a future framework guiding multimodal assessment and supporting the recognition of knowledge and learning.

The Digital in Education

The digital in education is interesting as it seems to cause tensions among teaching staff, scholars, educational developers and other categories of professions involved in education in different ways. A key issue is whether ‘the digital’ is to be seen as something separate from other educational activities and learning – or not. I would say that the digital in education today is interwoven in all areas of communication and as such always present in some way in educational situations.

If I now limit this reflection to higher education, I assume that students have smartphones and access to computers and internet connection (at least while in a university setting). I would argue that in contemporary (higher) education ‘the digital’ is always present. Whether we like it or not digital tools and spaces of different kinds are present. We use email, video conferencing, collaborate in writing and share documents, we store documents and media files in digital shape. Teachers and students use and make media texts – PowerPoint (or alternatives) and video (film) are common and used regulary. I find it hard to imagine contemporary (higher) education without all these digital tools and spaces.

To return to where I started – what do you think – is ‘the digital’ a special case?

What drives my research interest?

There is always a reason, an underlying motivation, for any individual interest. This includes research interest. At a basic level I am interested in how people learn and in the conditions for learning. I think the roots of this goes far back into my own childhood. Somewhere, somewhen, there is that something that is the core of our individual existence. I remember looking in the mirror as a young child, and being aware of myself as separate from others. We are separate and unique but in other ways we all share basic feelings and experiences. One things we all have in common is the experience of being children, of being young and small.

My core belief is that most important of all, at any stage of education, is compassion and kindness. I hope that any future results of my research will improve and make better education for students and teachers alike. Teaching is a difficult profession where you often have to think on your feet, and I think teachers are often undervalued. That said there is room for improvement both in practice and in systems. Formal education takes place within systems and in institutions of learning. These are regulated by rules and structures and histories of practice. A student can easily become insignificant in these system. However, effects of a system will become part of the student’s lived experience for good or bad. Therefore I think it is important to reflect on the power educators have over students and how this power translates into teaching and assessment practices.

We measure and judge students and their knowledge and learning. We turn these measurements and judgements into marks and grades. These assessments can determine access to further education, jobs and affect feelings of self-esteem (Bezemer and Kress, 2016) making assessment practices a core influence on, not just students’ learning, but also their lives. Therefore it is important that assessments recognise signs of learning when expressed in different ways, using different media and in different shapes and forms. My ongoing research on digital text and assessment practices place focus on recognition of learning in higher education, but should be relevant for other levels of education.

Reference:

Bezemer, J., & Kress, G. (2016). Multimodality, Learning and Communication: A social semiotic frame. Routledge. https://doi.org/10.4324/9781315687537

Multimodal Life and Research

Reflections

My general research interest lies in multimodality, in meaning making and in material expressions of communication and meaning. Transcription is here an important tool. I bring my background and experience in art, linguistics, IT and as a teacher with me as I approach my data. You could say that this is my library of professional experience. Add to that dance, music, parenting three children and having lived in different countries. All this together helps me approach my data with a wide lens, and different filters, that I can apply at will. A reflexive approach, certainly, and reflexive using different and multiple perspectives.

I have always loved colour, so I use colour in transcriptions. I find images fascinating and I include images in transcriptions. I remember playing and sorting things in different categories and patterns as a child. I do the same thing with data. I move and sort and categorize data, marking with colours, structuring and in this process I find patterns. The patterns reveal things that were hidden and then suddenly emerge in the data.

Creature (water colour by Henrika Florén, 2005)

Some of the best things aren’t planned. They just come, but they come as a result of working. Painting or working with data is not that different. No the productive surprises come as a result of hours of working, testing, trying out textures, patterns, combinations of ‘ingredients’ that may prove to be dead-ends but that also give surprising results. And now I should put this process into what will be an article for an academic journal with a focus on methodology. What I have described here does not fit in the format of that article, but it is no less important or relevant to the process.

I suppose I’m attempting to explain the feeling of the process here rather than describing the process so that it can be replicated. There are feelings involved in research. These can be feelings of frustration or joy over the work process. Sometimes (often) it is just a feeling of being so tired that I cannot think anymore. When that happens I know I have to let myself take a break, but I never feel I want to give up. No, the process of research is just too interesting, In the midst of all the hard work and long hours with only myself and my computer there is that moment of discovery that is exciting. It is that feeling of treasure, of revelation, of finding out, that to me is so satisfying that I would not want to do without it.

Educational culture and assessment in Sweden

What is it about the Swedish educational culture that makes it so hard to decipher and explain?

I am researching text practices and assessment in higher education and I’m collecting my data in Sweden. This means I have to explain the context for an audience who are not familiar with the Swedish education systems and culture. My own experience of being a pupil and student spreads across different cultures; Swedish, French, British and US education systems. I know from experience that Swedish education is different. What I had not previously identified is how different this Swedish educational culture seems to be. However, to be able to explain the context I first have to understand it myself.

There is the focus on collaboration and group work, and the focus on children who need help and support, and no focus on gifted students. The ideal is inclusion and collaboration. Another Swedish cultural ideal is the concept of ‘lagom’ (not too much, not too little of anything), and for not standing out. Children in Sweden start school comparatively late, the year they turn seven, although most children attend F-class when they are six. I would say Sweden has a child centred educational culture and that there is a cultural norm for letting children be children and leaving a lot of space for children to play.

When it comes to assessment I find it especially tricky to decipher and explain the context. Contemporary Swedish education, including higher education, is influenced by sociocultural perspectives on education (Englund, Olofsson & Price, 2018) and has since the 1970s had a “grading-hesitant educational culture” (Dahl, Lien & Lindberg-Sand, 2009, p.61). Younger children are not graded at all. Grades are not introduced until year 6 when the children are 12, and grades are criterion based which theoretically means that everyone in a class can achieve the highest grade.

Higher education in Sweden is highly decentralised and each institution can decide on what grading scale to use, and this can be delegated down to the level of the subject level. In one and the same institution for higher education there can be several grading scales used. It is the teachers who teach the courses that also grade the students. Each and every course is graded and the most common grading scale is pass/fail which makes the topic of grade distribution “void of valuable information” (Dahl et al., 2009, p.72), although other scales are used such as the A-F  scale, or pass/pass with distinction/fail. There is no system of final exams and students get no grade for degrees, for example a BA, or professional (licensed) degrees such as a for teachers.

All of this seems to cause confusion for people from other educational cultures, especially where there is a firm paradigm of measurement and accountability. Swedish teachers at all levels of education work within a high trust and low accountability culture. I am finding it increasingly interesting even if it just the backdrop for my research and not the focus. This needs further thinking and writing about.

References

Dahl, B. L., & Lindberg-Sand, Å. (2009). Conformity or confusion? Changing higher education grading scales as a part of the Bologna Process: the cases of Denmark, Norway and Sweden. Learning and Teaching, 2 (1), 39–79. High Educ (2009), 58(53), 1-549. 

Englund, C., Olofsson, A. D., & Price, L. (2018). The influence of sociocultural and structural contexts in academic change and development in higher education. Higher Education, 76(6), 1051-1069. 

Three preview readings of multimodal literature

When it’s hard to go the the library and browse the book shelves I find previews of books useful. I include three titles below.

The woes of upgrade writing

I don’t usually have problems writing. I like writing, but working on this upgrade text has been something else. Nothing has seemed to work. Oh, I have written. I’ve produced text, lots of text, but not the kind of text I need. It has been too much, too little, the wrong focus, and something underlying that has disrupted my whole process of writing. 

I have come to a point where I just cannot let it go. I work too much, too late, too long and then I go to bed. I have to sleep after all. I turn off the lights and my brain is still working with the text, and I have to get up again, because I realised something that I had to write down. Just as I don’t have problems writing (normally), I usually don’t have problems sleeping, but now I’ve woken up in the middle of the night dreaming about my data, my analysis, my text and I have to write. If I don’t write I can’t go back to sleep. I need to get that thought into text, but then in the morning I still have to get up the usual time after just a few short hours of sleep.

But today something finally shifted and I realised two things, two problems. One has to do with me, and my approach to writing the analysis, the other has to do with my data as such.

Last night I woke up again, got up and wrote down my whole analysis in one long flow of text. I know what is in the data. By now, I know my data very well. This morning I saw that I could use that text I wrote last night and go back add examples from the data. I have been trying to work from my data and build the text around that and now I reversed the direction, and this worked much better. The other thing I suddenly realised that has eluded me until now is a problem present in my empirical data. Once I identified this I could understand what I have been seeing for a while but not been able to conceptualise clearly until now.  Hopefully, I can now get down to the mechanics of writing and just get the text done. Write, re-write, polish, tie to literature. All those things that you do in academic writing.

It was there all the time, right in front of me!