Joyrex Labs

Outline

This Page

Example Manual

G-S Relationship examples: Deictics (& some Iconics)

RF

Gesture replicates speech (includes match both at the basic and superordinate level)

Speech Gesture
Cookie point at cookie
Put it on the table point at table
What letter is this? point at letters in book
What color are his tires? hold up green tires
Put the animals hold up sheep and pig, one in each hand
A mommy point to picture of lady
mommy, daddy, baby point to big, medium, small penguin
Clean the house point to a wall in the house
Open the door point to door knob
Get her the boat point to drawer where the boat is
Me want hotdog point to refrigerator where hotdog is
It’s my hat point to hat
garbage point to kitchen where garbage is
pew! iconic: wave hand: it stinks
round iconic: make a circle with hand
Turn it over iconic hand turn
Quiet metaphoric: palm down, moves down
This side point to puzzle
This part. point to picture.
That piece. point to puzzle piece.
That end. point to other end of couch.

For speech that involves counting: If the pointing gesture is highlighting the number aspect, then call it an RF.

Speech Gesture
There is one point: picture in book
one, two, three point: bread piece1, bread piece 2, bread piece 3
You got one two three hold up a triangle, a heart, a circle

Gestures accompanying imaginative play:

Speech Gesture
There’s a stoplight. point to block representing stoplight (after first time item is introduced as a stoplight).
Want some tea? Hold up empty teacup.

RF.a

Gesture replicates the adjective in speech (only for physically apparent ones)

Speech Gesture
Happy point at a happy baby in book
There is the purple one point at purple bead
Purple point at purple bead
It is yellow. hold up yellow peg
How about the blue one? hold up blue peg
Hammer the green one hold up green peg
A long one and a short one point at long playdough then point at short playdough

RF.p

Gesture replicates the predicate in speech

Speech Gesture
Sleeping point at sleeping cat in movie
Crying point at crying baby in book
Crying point toward room where baby is crying

DA

Gesture clarifies a pronominal reference (he, they, it, this, that, etc.), a deictic reference (here, there), or a particle (up, down) in speech

Speech Gesture
What are these? point to noodles
What color is this? hold pink marker
These are mine. point to self
It’s my hat. point to self
Who is this? point to Sarah in picture
Who are those guys? point to red bird + point to blue bird
What is his name? point to boy in picture
That is not a bee. point to dragonfly
Is that an elephant? point to mammoth
It looks like an elephant point to mammoth
That is Julie’s stuff. point to Julie’s backpack
Where is their car? hold up Ken and Barbie
Put it over there? point to coffee table
Give it to me. point to cookie
You want to see this one? hold up a book
Up point to an upper shelf
Down point to floor
Put it through the back point to back of toy
Put the man on top point to top of toy
This one is raspberry point to raspberry juice box
This way point any direction

ADD

Gesture adds a new argument to what is conveyed in speech

Speech Gesture
Here hold up a toy block
See hold up a toy block
Look hold up a toy block
What color? hold pink marker
What shape? hold up triangle block
How about? point to empty space in puzzle
Zoe point at orange circle
Put it back in hold up plastic bag
kiss Santa and put it back point to a location
put it back on point to a location (a plant)
You need to hold on point to spin wheel
Put in point to hole in toy fridge
Put it back point to couch
You have some on your tray. point to food on tray
Little bit more. point to juice in cup
Want some? hold up a banana
You want the other half point to noodle
Want to feed? point to spoon
Not Elmo point to other toy on floor
They are camping point to tent
Water point to toy sink (where there is no water)
You are supposed to leave it point to other room
Want to say hi to him? hold up toy phone
In the cheese point to mouse in cheese toy
You want to scan the eggs point to scanner
Look what I have got! hold up ball + hold up barney
Look what I have got! hold up a toy block
Eat what’s on your tray. point to food on tray
You know where it goes point to puzzle board
All gone hold up snack wrapper
All done hold up the back cover of book
All gone hold up empty hands
Daddy point to street where daddy was previously
What do you want? point to toys in basket
What’s Mommy got in her hand? hold object

With adjectives

Speech Gesture
Broken hold up toy
Hot point to radiator
A big hold up truck
Yucky hold up green beans

ADD.nr

Gesture adds a new predicate to what is conveyed in speech

Speech Gesture
Let me see it open palm: give toy
Come here palm: give bottle
Who loves Mommy? come

ADD.s

For characterizing sounds (typically animal sounds), code ADD.s for speech that produce animal sounds, or some other sound characteristic of an entity (e.g., crying sound for a sad person, engine sound for a car)

Speech Gesture
Meow! point to cat in book
Woof! point to cat dog
Wa wa wa! (crying sound) hold up a sad bear face
Choo choo! hold up a toy train

ADD.ns

For evaluative sounds, code ADD.ns for speech that act as an evaluative marking on a gesture (surprise sound Uh-oh!, question sound: Hmm?, excitement sound: Wow!, etc.)

Speech Gesture
Oh! hold up: Winnie the Pooh
Hmm? point at cat

ADD.f

Speech Gesture
Yeah point to bottle
Thank You hold cup of milk
OK point to TV

Also: ByeBye, Hello/hey, alright, sure, please, oh well, it’s all right, you’re welcome, oh my, excuse me, amen, there you go, here you go, it’s ok, I mean, hold on, oh my gosh, that’s right, I’m sorry, bless you, right.

ADD.q

Gesture answers Wh-type question in speech

Speech Gesture
Where are the apples?. point to apple tree in book
Where’s baby Mikey? point to baby in picture
Which fish did he catch? point to fish caught by character in a book
Where is the cow? point to board where cow is located

ADD.err

Gesture adds an incorrect match to speech

Speech Gesture
cat point at dog
frog palm up: give cat

ADD.err.s

Gesture adds an incorrect match to speech that includes an animal or artifact sound

Speech Gesture
Meow point at dog
Ruff point at dinosaur

G-S Relationship examples: Conventionals (& some Functional Acts)

GIVE

  • Any indication of object transfer in speech gets “RF”
  • Other speech with verbs typically gets “ADD.nr”
  • Fillers (e.g., OK, yeah, thank you) gets “ADD.f”
  • Ignore pronouns in give gestures; so no “DA” coding

RF

Speech Gesture
Give me the pacifier open palm: give pacifier
Can I have the ball? open palm: give ball
Let me have it. open palm: give ball
You want me to take it? open palm: give ball
Throw it to mom! open palm: give ball
Roll it to mom! open palm: give pacifier
Send it back open palm: give ball
Hand it (to me) open palm: give ball
Bring your bike here palm up: give bike

ADD

Speech Gesture
My pacifier open palm: give pacifier
Ball open palm: give ball
More open palm: give cracker
Yeah over here open palm: give ball
Ball open palm: give ball
Daddy open palm: give ball
Na-na open palm: give pacifier (nana=pacifier)
throw it open palm: give ball

ADD.nr

Speech Gesture
Want me to fix it? open palm: give toy
Can I see your book? open palm: give book
Want me to put them together? open palm: give lego
Let me see open palm: give toy
That was your Winnie the Pooh snack? open palm: give snack wrapper
Hot. give food (the food is hot)

ADD.err

Speech Gesture
Ball give red ribbon

ADD.f

Speech Gesture
Yeah open palm: give ball
Thank you open palm: give ball
OK open palm: give ball

Also: ByeBye, Hello/hey, alright, sure, please, oh well, it’s all right, you’re welcome, oh my, excuse me, amen, there you go, here you go, it’s ok, I mean, hold on, oh my gosh, that’s right, I’m sorry, bless you, right.

FLIPS

RF

Code “RF” when flips come with a question of any sort. These include sentences that are declarative, but are produced with a question intonation.

Speech Gesture
What will Molly tape? flip her hands: “don’t know”
Does that hurt? flip her hands: “don’t know”
That is your blue book? flip her hands: “don’t know”
Blue? flip her hands: “don’t know”

Code “RF” when flips match the discursive intonation of speech.

Speech Gesture
I can’t do it anymore flip w/shrug: give up
I didn’t do it flip: don’t know
uh-oh! flip: where is it?
Look how nice it is! flip: exclamation
Big boy! flip: exclamation

Code “RF” when flips come with speech that marks absence of objects or people.

Speech Gesture
I don’t have any more grapes flip: all gone
No kitty cats flip: all gone

ADD

Code “ADD” when flip adds a question to what is said; this does not happen very often, typically children use this, but sometimes the moms may use it as well.

Speech Gesture
Mommy flip: where is she?

NODS & SHAKES

Nods produced with affirmative words or words of positive emotional valence are considered reinforcing (RF). Shakes produced with words of negation or words of negative emotional valence are considered reinforcing (RF).

Nods (affirmation) * Yes * Yep * Yeah * Right * Uh-huh * Sure * Ok * Absolutely * Right on * Mm-hmm

Shakes (negation) * No * Nope * Nah * Not * Don’t * Didn’t * Won’t * Wouldn’t * Can’t * Uh-uh

Nods (positive emotion) * Great * Awesome * Good * Sweet * Fabulous

Shakes (negative emotion) * Bad * Horrible * Yuck * Bleagh * Disgusting * Etc.

RF

Speech: Gesture
Yeah nod
yeah? nod
No? shake
Don’t shake
Yucky shake

ADD

Gesture adds a negation to an affirmative sentence that is not a question

Speech Gesture
Got to find a new place for that shake
the CD is out shake

E

Gesture that comes with questions (or declarative sentences in speech), but does not answer the question. It just emphasizes it. I typically change the gloss of the gesture from “yes” or “no” to “emphasis”

Speech Gesture
you want to go bye bye, don’t you? nod
Hard to decide shake
Where are you going? nod

OTHER CONVENTIONALS

RF

Speech Gesture
No, no, no, no naughties
Leave me alone naughties
Take a breath Palm out (wait gesture): slow down
That is enough Palm down (wait gesture): slow down
come/come here/come on, etc. pick up
Up pick me up

ADD.nr

Speech Gesture
Hug beckon
Let’s go beckon

MS

Code MS for unconventional sounds. This typically happens with infants, rarely with adults

Speech Gesture
Ba! holds up a bottle
Ba! holds up a toy lid
Da! points at food on table

FUNCTIONAL ACTS (FA)

For all functional acts (hold-outs and demonstrators) code gesture-speech relation as “FA”

Speech Gesture
Want me to help you up? open palm: take hand
Get up? open palm: take hand
Want more juice? hold out: take juice bottle
This is how you turn the handle demo: hand turns the handle

Semantic Role examples

For further info see Levin (1955); Lyons (1977), Croft (1991)

Confusing example sentences

  • “You need to match this one to this one” +POINT (to puzzle piece) +POINT (to other puzzle piece) = TH & IE
  • “This one matches this one” +POINT (to shirt) +POINT (to pants) = ED & ED
  • “Pink” +POINT (to shoe) = ED
  • “This one is pink”” +POINT (to shoe) = ED
  • “Fast” +POINT (to toy car) = ED
  • “I think” +POINT (to brownies) = X (unable to determine role)

Transitive Agent [TA]

  • The boy ate the pretzel.
  • Susan read the book.
  • William put the frog in a bowl.
  • Emily bought a new bike.
  • The robber turned himself in.
  • Matt sang an old song.

Intransitive Agent [IA]

  • The boy ate.
  • The cat slept.
  • Charles sang.
  • Jonah jumped into the lake.
  • The bird flew up to the branch.

Possessor [PO]

  • Brian has two turtles.
  • I have more cookies than Paul.
  • This is Greg‘s purple hat.
  • My name is Franklin. +POINT (to self)
  • These are mine. +PALM (to self)
  • The video is his. +POINT (to other)

Experiencer [EX]

Emotion Perception Sensory Perception
  • I am morose.
  • Hugging my dog makes me happy.
  • James is in love with Dorian.
  • Kelsey likes playing jacks
  • Spiders frighten Wanda.
  • I want a cookie!
  • Sam knows Lois.
  • Walter understood the theorem.
  • Richard saw the bear cub first.
  • The flowers smelled nice to Amanda.
  • Everyone heard the explosion

Patient [PA]

  • Thomas ate all the cheese
  • The fire destroyed the building
  • Goats groom each other with their teeth
  • Can you open the jar?
  • Cut it!
  • I read the book.
  • The rain stopped.

Theme [TH]

  • Damien put the jar on the shelf.
  • He gave the flowers to Jennifer.
  • I drove Sarah to the mall.
  • She took her wallet so she could shop.
  • Andrew told Kayla a secret.
  • Humphrey fell off the couch.
  • Daisy made Zora late for her appointment.

Entity-Locative (static) [EL]

  • The cup is on the table.
  • Dad is in his office.
  • Frogs live in the river.
  • The possum is on the patio.

Entity-Owned (static) [EO]

Only animate things (or characters) can own entities. Otherwise possessive descriptions are Described Entities.

  • This is mine. +POINT (to object)
  • Mine. +**POINT(to object)**
  • Brian has four dump trucks.
  • The shoes belong to Jane.
  • Mom’s slippers.
  • My freckles.

Entity-Perceived [EP]

  • I saw two dogs playing.
  • Look at that! +POINT
  • He could smell dinner all the way upstairs.
  • Touch the rabbit on his back.
  • This dress looks weird in this light.
  • I want cookies!
  • Want to look? +POINT
  • See mommy? +POINT

Entity-Described [ED]

  • The rabbit is soft.
  • Candles are hot.
  • This purple dress.
  • Frogs are nice.
  • Max is my friend.
  • There are four. +POINT
  • It‘s for me.
  • Is that one fast?
  • Lots of sugar.
  • Pink box
  • Squishy +POINT
  • How about the last corner? + POINT to puzzle piece

Entity-Named [EN]

  • This is a turtle.
  • Larry.
  • Sugar.
  • Pink.
  • What’s this?
  • This one +POINT
  • POINT
  • Is that it?

Animate Endpoint [AE]

  • I gave two quarters to Hector.
  • The ball rolled to the dog.
  • Fred’s bike crashed into me.
  • Hand Lilly the corkscrew.
  • The horse walked up to Kaye.
  • Bring back my bonnie to me.

Inanimate Endpoint [IE]

  • I put two quarters into the machine.
  • The meatball rolled under the sofa.
  • Put your feet in your shoes.
  • The bunny hopped into the river.
  • She dropped two tablets into the glass of water.

Animate Source [AS]

  • The fleas jumped from the poodle to Sam.
  • These presents are from Larry.
  • You gave me a great idea.
  • The rabbit jumped out of my hands.

Inanimate Source [IS]

  • The train left the station.
  • Ralf escaped from his terrarium.
  • This wine is from Paris.
  • Then he went from here to there.
  • I was hiding and then I jumped out of the box.

Path [PT]

  • The leaf fell down.
  • Then you make it go up and around.
  • Now you go under.
  • The car drove through it.
  • I hopped around the table.
  • I’m far away.

Static Comparative [SC]

  • Jude is taller than Jack.
  • My flower is prettier than yours.
  • Americans are fatter than Lithuanians.
  • Superman is stronger than Batman.

Active Comparative [AC]

  • I can eat more cheese than you.
  • Frogs can jump higher than mice.
  • Superman can run faster than Batman.
  • Batman chases more ladies than Superman.

Instrument [IN]

  • She cut the pie with the knife.
  • I opened the box with scissors.
  • The knife cut his heart out.
  • Doug used poison to get revenge on his ex-wife.

Accompanying Actor [AA]

  • He ate dinner with his wife
  • I want to go play with Susie.
  • Charles likes to watch television with his grandpa.

Beneficiary [BE]

  • I put the dishes away for my mom.
  • She took night classes to improve her Spanish for her boyfriend.
  • Graham painted a picture for Rebecca.
  • Dad made me deviled eggs.

Time [TI]

  • Yesterday, all our troubles seemed so far away.
  • Now, it looks as though they’re here to stay.
  • Jasmine got one last week.
  • My dad comes home at 5 PM.
  • We get to do a puzzle everyday.

Location [LO]

  • I threw a penny into the well at the park.
  • The top is spinning on the table.
  • At school, our teacher gave us stickers.
  • Iona picked up the most trash on the beach.