Joyrex Labs

Outline

This Page

2. Gesture Columns on the Transcript

2.1 Form

The form column is used to specify the type of gesture. The different codes that can appear in this column are listed on the Gesture Coding Reference Page. Deictic gestures can be coded as points, palms, continuous points/palms, and holds. There are a lot of conventional gestures out there, and we occasionally add to our list. The most common conventionals are nods, shakes, flips, shrugs, and waves, but the possibilities are vast. The types of representational gestures include iconic and metaphoric gestures. Demo, beat, or sign can also be recorded in this column. Please use the shorthand listed on the coding sheet for marking each of the gesture forms—this makes computer searches a lot easier when we do analysis of different gesture forms.

2.2 Part of Body

The lrb column (meaning “left, right, both”) specifies the appendage(s) used to perform the gesture. Possible codes include R for right hand, L for left hand, H for head, B for both hands, W for whole body (rare) and RF and LF for right and left foot, respectively (extremely rare).

2.3 Object

The obj column is only used for deictic gestures, and we use it to denote what object is being referred to (held up, pointed at, etc.). Sometimes an iconic gesture may occur with an object as well, such as tracing the shape of an actual object or picture with the fingers. You will never have anything in this column for conventional gestures.

2.4 Gloss

In the gloss column, we write what the gesture means. For deictics, this column will almost always contain the exact same words that appear in the object column. There are only two exceptions to this rule. For palm gestures, sometimes the speaker is not only pointing at an object, but also requesting that the object be given to them (this typically involves extending an open palm facing upward next to a desired object). In this case, we would put the word give in the gloss column before re-writing whatever we had in the object column (e.g. we might put give cookie as a gloss, where just the word cookie appears in the object column). Similarly, sometimes a person will hold something up not just to refer to it, but to tell the other person to take it from them. In this case we put the word take before whatever was in the object column (e.g. take cookie).

For conventionals, the glosses are relatively strict. Once you have the form established, refer to the Gesture Reference Page and choose a gloss from those listed for the form. Even if none of the given glossees fit perfectly, you must choose an established gloss from the list. If none of the glosses seem to apply, it may be that a new gloss needs to be added for that form.

As for deictics, for representational gestures just put whatever the movement is supposed to mean. The meanings of representational gestures tend to fall into four categories: (1) action; e.g. flapping arms in the air to convey flying; (2) direction; e.g. moving index finger downward to indicate downward path — mark these as trace path in the orientation column; (3) attribute; e.g. holding cupped hands in the air to convey the shape and size of ball; and (4) shape; e.g. tracing the shape of a square with fingers on a drawing or in the air — mark these as trace shape in the orientation column.

2.5 Orientation

The orient column contains any extra information about how a gesture was performed. For points (and continuous points, etc.), we use this column to specify whether the object may have been tapped or touched. For palms, we specify tap or touch as well as whether the palm was palm up, palm down, palm side etc. For holds, we sometimes indicate that the object was shaken by writing shake. This is also the column where we write either trace shape or trace path for representational gestures that express information about the shape or path of an object. Except for the gestures listed above, it’s fine to leave the Orientation column blank if no codes are applicable.

2.6 Motion, Space, Description

The mspd column is used almost exclusively for representational gestures and demos. This is where you put a detailed description of what the gesture looked like. Be as descriptive as you possibly can here, because the second-level gesture coder relies on your depiction of the gesture. The only other thing besides a description that you would ever put in this column is sometimes, if somebody points at something with another object in the same hand, you would note that in the mspd column (e.g. using a pencil to point at objects in a book).