Body Movement Tracking Robot

Body Sense can find the shoulders, elbows, wrists and knees of a person in the webcam picture, all on your computer with no internet. In this project MIEO copies you. Raise your right hand above your shoulder and MIEO lifts its servo arm and smiles; put your hand down and MIEO lowers its arm. The trick is comparing two heights: if your wrist is higher than your shoulder, your arm must be up.

• How a computer finds the joints of a body in a picture
• Comparing two positions to decide what a person is doing
• Copying a movement with a servo
• Only sending MIEO a command when something changes

• Mieo board
• Battery
• 3D-printed mountings
• Screws and screwdriver
• A servo motor, from the MIEO kit
• Card, scissors and tape to make the arm
• A webcam; most laptops have one built in
• A USB cable, or Bluetooth on your computer
• The Robolab app (free)
• A webcam (most laptops have one built in)

1. Build the waving arm as in Servo Waving Hand: the servo plugged into S1 and taped to the side of MIEO, with a card arm on its horn, pointing down at 20 degrees.
2. Switch MIEO on and connect it to your computer, then stand it beside the screen.
3. Stand back from the webcam until the camera picture shows you from your head down to your waist.

Want MIEO Bot to guide you?

Robolab's built-in tutor has a lesson on the same idea, “Hands Up to Drive”. Open MIEO Bot in Robolab, pick Maker, then Mieo, and it shows you the blocks one at a time and checks your work as you go.

About MIEO Bot's lessons

1. Open Robolab, choose Mieo, connect and set the switch at the top right to Arena. Add the Body Sense extension and make a variable called up.
2. Start with “when green flag clicked”, “switch camera on with picture 0 % faded”, “show body markers”, “set servo on Servo 1 to 20 angle”, “show neutral” and “set up to 0”.
3. Add a “forever” loop with “find body poses in camera” inside, then “if … then” with “number of people > 0”.
4. Inside it, add “if … else” with “y of right wrist on person 1 > y of right shoulder on person 1”. On the stage, y gets bigger the higher something is.
5. Arm up: in the top part, add “if up = 0 then” with “set servo on Servo 1 to 160 angle”, “show happy” and “set up to 1”.
6. Arm down: in the else part, add “if up = 1 then” with “set servo on Servo 1 to 20 angle”, “show neutral” and “set up to 0”.
7. Click the green flag, allow the camera, and raise your right hand.

Arena mode: the program runs on your computer and talks to MIEO live over USB or Bluetooth, so keep MIEO switched on and connected.

when green flag clicked
switch camera [on v] with picture (0) % faded
[show v] body markers
set servo on Servo [1 v] to (20) angle
show [neutral v]
set [up v] to (0)
forever
  find body poses in [camera v]
  if <(number of people) > (0)> then
    if <(y of [right wrist v] on person [1 v]) > (y of [right shoulder v] on person [1 v])> then
      if <(up) = (0)> then
        set servo on Servo [1 v] to (160) angle
        show [happy v]
        set [up v] to (1)
      end
    else
      if <(up) = (1)> then
        set servo on Servo [1 v] to (20) angle
        show [neutral v]
        set [up v] to (0)
      end
    end
  end
end

Click the green flag: the camera picture shows you with dots on your joints, and MIEO's arm points down. Raise your right hand above your shoulder and MIEO swings its arm up and smiles. Lower your hand and MIEO lowers its arm. Wave slowly up and down and MIEO waves with you.

If it doesn't work

• MIEO copies your other arm? The camera picture is a mirror. Change right to left in both blocks, or raise the other arm.
• Nothing happens? Stand far enough back for the camera to see your shoulders and wrists, in good light.
• MIEO's arm flickers up and down? Lift your hand clearly above your shoulder, not level with it.

1. Easy: play a sound each time MIEO's arm goes up.
2. Medium: add a second servo in S2 for the other arm, copying your other wrist.
3. Hard: when both your hands are above your head, make MIEO play its win sound and show a light pattern.