The aim of this paper is to discuss and critique how movement can be articulated for the purposes of design, based on the texts by Loke and colleagues (2005, 2010). In addition to this, it will analyse how creating movement can contribute to a design project. This design project was part of a University course in which, the objective was to explore different movements and their experience while working with machine learning. Furthermore, this paper will discuss the findings that Loke, Larssen, and Robertson (2005) present about their investigations how Labanotation can be useful for the design practice.
We experience the world with our bodies. Walking to places, using our hands for different activities, working out and more, gives us the opportunity to interact with objects using our bodies. For a long time, movements and gestures has been used within technology to create better interactivity. For example, we can take the finger gestures used on our smartphones, swiping, typing and so forth. In their paper Loke & Robertson (2010) explain how the interactive technologies are becoming more embedded in our daily lives, are movement-based, which leads them to be more sought after. Consequently, designers try to design new movements for those technologies. To gain a better understanding of the movements, according to Loke and Robertson (2010, p. 12), designers can work with three different perspectives. “The mover” – which offers the first-person experience of the gestures, “The observer” – this perspective provides a view of the body and experience from outside standpoint; and last but not least “The machine”- this perspective offers to see how the machine itself recognises the movement. All three of these perspectives have their own unique value. The mover would allow the person to gather the full experience from working with a movement. From the physical experience to feeling the emotions gained from doing it. The observer, on the other hand, has a picture of how the movement looks like but it is not able to fully grasp the feelings of the movement as it is not the one performing it. Finally, the machine reacts to what it has been taught. That being said, in the following essay I will present why movements can be used for the purpose of creating better interactions.
Loke and Robertson (2010, p. 10) state that “the movements/choreographic ideas can be expressed or articulated through a combination of text, sketching, and images”. The documenting of the movement can usually be done in two ways. The first one would be through using written descriptions of the movement itself and the second, of visual representation of the main steps of the movement. According to Loke and Robertson (2010, p. 10), the written descriptions are a “written record of the choreographed movement, that details the specifics of how the body moves, the motivation for the movement and the kind of act in which the movement is contained.”. They also explain that the visual documentation can be used for informing the designers about which movement can be detected by the system. As part of the Interactivity course for Module III, the focus was to learn how to design movement and implement its data into a machine. Working with these methods of documenting was useful while exploring which movements would be interesting enough for us to work with. In order to portray the movement in the right way, the mover described how it felt doing the movement and what the movement looks like from their perspective. Another written description was taken from an observer’s point of view. These descriptions offered different angles for the movement to be looked at, since the mover undergoes through the full experience not just with the body but with feelings as well, while the observer can not grasp the whole experience since they do not go through the movement.
Furthermore, based on the paper from Loke and Robertson (2010), we can describe the changes of the movement with shape analysis. “The Shape analysis is a description of the changing forms and spatial qualities of the moving body.” (Loke & Robertson, 2010, p. 11). When analysing the movement, it can be split into different shapes, to gain a better understanding of what that movement is created from. In their paper, the authors analyse a movement where the transition of the body starts from what they call “ball-like” position to a splayed “wall – like” position on the floor and back to curled up position. The design practice in the course, afforded for us to be able to recognise and analyse movements according to the shape analysis. Breaking down a movement into a few different parts can help with gaining a better understanding of how the movement is done and why is it done that way. For example, the activity of climbing consists, of ‘problems’ that needs to be solved. Each of those problems can be solved in a few different steps. One of the more impressive moves while climbing is called ‘heel – hook’ when the climber uses his or her foot to create a hook to hold while going up. Diving the movement into shape would begin by standing in a ‘wall-like’ position, then the leg gets closer to the torso in a hook position, and the last part would be straightening up back to ‘wall – like’ position. After separating the movement into segments, it becomes much easier to record variations of that same movement by changing a few of its traits.
“A movement can be performed with kinetic variations of speed, scale, and direction to produce different patterns, dynamics and qualities of movement” (Loke & Robertson, 2010, p. 13). The differences in speed or scale can open a possibility to create new variations from the same movement. Since the machine perspective is not the same as the mover’s or observer’s perspective these variations can be important for the recognition in machine learning. Machines are not living beings. They work based on how they are programmed which indicate that they cannot think on their own, as a result they can only work and react to the data that is being inputted. Based on the design practice from the course, it was eye-opening that the small differences in the movement can affect the machine recognition a lot. For example, a simple gesture like moving the arms up and down would be recognised much easier than per say a more complicated embodied movement. The sensor placement also is a very important part of creating new movements. As much as it allows the designer to experiment it can also present a constraint as some parts of the body while moving offer better recordings of data. For example, recording the movement of baseball pitching in different ways resulted in different recordings of data. While the movement was recognisable by the observer, when done quickly the machine was not able to make difference between the details. When the sensor was placed in the hands rather than the thighs and done in much slower motion, the machine was able to recognise the movements.
Another way of analysing and describing movements is Labanotation. According to Hutchinson (1977), Labanotation is a system of analysing and recording movement, originally devised by Rudolf Laban in the 1920’s and further developed by Hutchinson and others at the Dance Notation Bureau, New York (as cited in Loke et al., 2005, p. 114). When creating interactions for input, Labanotation is a potential tool that can be used for analysing and notation of the movements. In their paper, Loke and the other authors discuss the advantages and disadvantages of using Labanotation in design. The main advantage they argue for is the possibility of easily linking the representation of a movement into the context of interaction. Furthermore, they argue that this can provide a good starting foundation when designing a movement-based interaction. “There are three essential forms of movement description in Labanotation – Motif, Effort-Shape and Structural”(Loke et al., 2005, p. 144). Following the definitions in the paper we can describe them as the following: The form of describing only the main parts of the movement or the motivation behind it is called Motif. When describing the aesthetic, emotional and expressive qualities we use Effort-Shape. Last but not the least when we want to describe a movement in a most specific description, we use the Structural form.
Labanotation was present when we were analysing and describing our movements in the final project. Most of our movements in it were described using the Motif. The reason for using that form in specific was that it was a project where the final outcome was not the goal rather than exploring and trying to design new movements. Since we were exploring baseball pitches, the aesthetic side of the movements was something that for me personally did not matter. The simple analysis provided us with descriptions fast enough to move forward with our project. The disadvantages of this form were probably the lack of details in the description of the movements. We never went to explore how the movements would feel if we change how the movements look like or what exactly they are expressing rather than where the hands go when throwing the ball. We also learned that the act of describing a movement based on its components and the motivation behind it can be challenging without a proper research of the movement itself.
All things taken into consideration, movements are included in the design of new interactive technology. With the progression of the advancements in technology and the new gesture-based technology, designers focus on creating new gestures and better interactive environments. As mentioned, describing the movement and analysing it through a few different systems and methods will be of assistance when combing the movements with the machine. The three perspectives, give the designer a different view on the movement, while the Labanotation offers three forms of analysis. Depending on how in-depth and detailed we want the analysis to be, each of these forms would probably fill the mold to our expectations. As stated by Loke, Larssen, and Robertson (2005, p. 120) “Labanotation and its underlying movement analysis system offer an understanding of the moving body and its movement potential that can act as a foundation for the design of movement-based interaction.”. With this base, the path for merging movements and interaction with technology is wide open.
Reference list
Loke, L., Larssen, A. T., & Robertson, T. (2005). Labanotation for Design of Movement-Based Interaction. Second Australasian Conference on Interactive Entertainment, 1996(November 2005), 113–120.
Loke, L., & Robertson, T. (2010). Studies of dancers: Moving from experience to interaction design. International Journal of Design, 4(2), 1–16.
Pink Prototyping 