SE367 / HW4 / Group B

Motor Expertise

Debidatta Dwibedi

Q1. Which two instructions in the "programming language" of the 2011 HW would be the most difficult for robots to follow?
A. We thought the following steps in the suggested algorithm were the toughest:

• Hold pencil at p = 2cm , with M and T affectors
  Holding onto the pencil is a tough task because the force on the pen needs to be balanced and there are no sensors on the tips of the robot's fingers. If the robot had 2 points of contact, it becomes even more difficult to balance the torque about the pen's center of mass, which gets displaced as sson as there is either an imbalance in force applied or torque applied. The Kalakrishnan robot has three points of contact which makes balancing of torque easier. Humans have compressible fingers which allows torque to be balanced without any extra effort.
  
  
• Rotate point of contact between I-affector and object, about axis defined by line passing through points of contact of M and T affectors (with object), by an angle such that P-axis lies at perpendicular distance of r from source point.
  The robot needs to successfully decrease the pressure applied to hold on to the pen so as to loosen the grip, so that it can rotate the pen, but not so much that the pen slips and falls down.

Q2. The robot following the learning paradigm as in Kalakrishnan is clearly gaining some expertise. Which aspects of the execution may be called implicit or automatic, and which aspects may be more explicit? What could be the "chunks" in this structure?
(Here we assume that implicit refers to the instructions that the robot performs on its own and explicit refers to the part where it requires a human to specify certain parameters for him. The words explicit and implicit don't refer to implicit/tacit or explicit knowledge respectively.)
A. From [1]

• Explicit aspects:Initial policy is set by kinematic demonstration and forces/torques are also initialized to zero.
• Implicit aspects:Policy Improvement with Path Integrals is done automatically. The robot explores new possible paths,executes one and evaluates its success and based on that updates its policy automatically.
• Chunking:The algorithm samples for new trajectories that are close to the current policy, measures their cost by executing them on the robot, and subsequently updates the policy as a weighted average of the samples. So this can be thought of chunking as it is remembering the last policy and several polcies close to it, which it uses to generate the new path.

Q3. Comment on whether human learning may also be following similar "reward" based processes? Consider the learning process for the fire-fighting expert who knows how to fight complex fires.
A. A lot of learning in humans is based on rewards and punishments, which usually are feedback of our actions in the environment. In fact a lot of the times people behave in a manner that increases rewards and hence become better at tasks that provide them with rewards. These rewards need not be provided by the environment always and may be a resultant of intrinsic motivation.

In the fire-fighting video, the novice wanted to perform the task of fighting the fire by opening the water in regular but small intervals and looking for its effects on the fire. He definitely wanted to use the abating fire to reward himself and keep following that line of action. However, because of his lack of expertise he was unable to contain the fire. But fire-fighters are given training in which they are given instructions on what to do. If they successfully follow an instruction and the fire abates they are able to reward themselves intrinsically and learn that instruction and tend to do that more. His fellow fire-fighters and seniors might also provide him with rewards in form of words of praise to ensure that they learn the correct method of fire-fighting. In this case, a learning fire-fighter can reward himself intrinsically because he knows he must somehow keep the fire in check and any action that helps him in doing this needs to be rewarded. Because of a lot of training and the rewards the fire fighter gives himself, he is able to perform the instructions as actions and anticipates its effects, intuitively.

References

[1] Kalakrishnan, Mrinal, et al. "Learning force control policies for compliant manipulation." Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on. IEEE, 2011.