Specify the relationship between transcribed speech and gestures
entry ::= `code-set` [op `code-set`]* code-set ::= code [`set-op` code]* code ::= add | emphasize | reinforcing | "DA" | "MS" | "UC" | "X" add ::= "ADD" [".err" [".s"] | ".f" | ".met" | ".nr" | ".ns" | ".q" | ".s"] emphasize ::= "E" [".b"] reinforcing ::= "RF" [".a" | ".p"] set-op ::= "/" op ::= ";"
GS_Rel codes have the following interpretations:
| Code | Meaning | Description | Example |
|---|---|---|---|
| ADD | Adding information | Gesture adds info to speech such that G+S = one proposition | “Cookie” + [give gesture] |
| ADD.a | |||
| ADD.d | |||
| ADD.err | Matching error | Gesture and speech refer to same object but one incorrectly labels it (occurs mainly in infants) | “Cat” +[point to dog] |
| ADD.err.s | Matching error at sound level | Gesture and non-word sound refer to same object but one incorrectly labels it (occurs in infants) | “Meow” + [point to dog] |
| ADD.f | Adding info to a filler expression | Adding info to a filler expression | “OK” + [give gesture] |
| ADD.met | Adding metaphoric | Speech adds meaning to metaphoric gesture | |
| ADD.nr | Adding info to unrelated utterance | Gestures adds a new proposition or speech and gesture convey unrelated information such that G+S = two propositions | “Put the book on the table” + [hold up shoe] |
| ADD.ns | Adding info to unrelated sound | Speech adds unrelated meaningful sound | “Uh-oh” + [point to cat] |
| ADD.q | Adds info answering a spoken question | Gesture provides information (such as location or yes/no info) that answers the question posed in speech | “Where is the bird?” + [point to tree] “Is this the one?” + [head shake] |
| ADD.s | Adding info to a related sound | Speech adds related meaningful sound | “Meow” + [point to cat] |
| DA | Disambiguation | Gesture disambiguates speech | “Put it there” + [point to table] |
| E | Emphasizes w/o beat | Gestures that act like discourse markers by emphasizing what is said | “Hard to decide” + [head shake] |
| E.b | Emphasizes with beat | Hand, arm, or head movements where the gesture stroke syncronizes with emphasis in speech | “What I’m SAYING is” + [hand beat on “saying”] |
| ELAB.a * | |||
| ELAB.b * | |||
| FA | |||
| MS * | Adds to a meaningless sound | Gesture is accompanied by a meaningless sound (almost only occurs in infants) | “Da!” + [hold up toy truck] |
| RF | Reinforcing | Gesture reinforces the info provided in speech | “Cat” + [point to cat] |
| RF.a | Reinforcing attribute | Gesture reinfoces an adjective in speech describing perceptual features | “Here is a blue one” + [point to blue block] |
| RF.d | |||
| RF.met | |||
| RF.p | Reinforcing predicate | Gesture reinforces the spoken predicate describing an action the referenced object performs | “Crying” + [point to crying baby in book] |
| UC | Unclear relationship | The gloss for the gesture does not provide enough info or the speech is indecipherable | “I want ###” + [point to toy] |
| X | No speech | Gesture is produced on its own | [point to desired object] |
Warning
Codes marked with * are considered deprecated in Erica’s Second Tier Gesture Annotation
Ops have the following interpretations:
| Op | Meaning |
|---|---|
| ; | Separates all codes |
| / | Used to combine representational codes into single units |
None
GS_Rel is dependent on the G_Type Column, the Form Column, and the Enum Column.
| GS_Rel | G_Type, Form, and Enum Conditions |
|---|---|
| ADD | C, DP, DP.nl, DS, DSDP, G, or S in G_Type |
| ADD.a | R.a, R.a.e, or R.a.pp in G_Type |
| ADD.d | R.d, R.d.e, or R.d.pp in G_Type |
| ADD.err | DP, DP.nl, DS, DSDP, or G in G_Type |
| ADD.err.s | DP, DP.nl, DS, or DSDP in G_Type |
| ADD.f | C, DP, DP.nl, DS, DSDP, or G in G_Type |
| ADD.met | R.met in G_Type |
| ADD.nr | C, DP, DP.nl, DS, DSDP, or G in G_Type |
| ADD.ns | DP, DP.nl, DS, or DSDP in G_Type |
| ADD.q | R.m, R.m.e, or R.m.pp in G_Type |
| ADD.s | DP, DP.nl, DS, or DSDP in G_Type |
| DA | DP, DP.nl, DS, or DSDP in G_Type |
| E | C in G_Type and shake or nod in Form |
| E.b | E in G_Type |
| ELAB.a | R.a, R.a.e, or R.a.pp in G_Type |
| ELAB.b | R.m, R.m.e, or R.m.pp in G_Type |
| FA | FA in G_Type |
| MS | DP, DP.nl, DS, DSDP, G, FA, or S in G_Type |
| RF | C, DP, DP.nl, DS, DSDP, G, or S in G_Type |
| RF.a | DP, DP.nl, DS, DSDP, R.a, R.a.e, or R.a.pp in G_Type |
| RF.d | R.d, R.d.e, or R.d.pp in G_Type |
| RF.met | R.met in G_Type |
| RF.p | DP, DP.nl, DS, DSDP, R.m, R.m.e, or R.m.pp in G_Type |
| UC | Can be used with any Form code if appropriate |
| X | Can be used with any Form code if there is no utterance enumeration in Enum |