Click Here to go to image.
| Playing soccer | Brushing your teeth | Playing a tennis match | Practicing tennis against a wall |
Deciding what item to take at hostel lunch |
|
| Deterministic vs Stochastic | Stochastic | Deterministic | Stochastic | Deterministic | Deterministic |
| Episodic vs Sequential | Sequential | Episodic | Sequential | Sequential | Sequential |
| Static vs Dynamic | Dynamic | Static | Dynamic | Dynamic | Static |
| Discrete vs Continuous | Continuous | Continuous | Continuous | Continuous | Discrete |
| Fully observable vs Partially observable |
Partially observable | Fully observable | Partially observable | Fully observable | Fully observable |
| Single agent vs Multiagent | Multiagent | Single agent | Multiagent | Single agent | Single agent |
Playing Soccer
Stochastic :: In a soccer match next state is not completely determined by the action executed by the agent. The environment is complex. One can never predict the behaviour of other players exactly and thus have to worry about uncertainty in passing/shooting/defending. Therefore, it is stochastic in nature.
Sequential :: In a soccer match, a success i.e. a goal can be result of someone passing from midfield to striker few minutes ago. Thus the choice current action i.e. passing another mid fielder/ striker can affect my future decision. Short term consequences ( eg pass is correct or not) have long term consequences making them sequential.
Dynamic :: In a soccer match, other players keep moving while a algorithm computes what to do next. As the environment can change while an agent is deliberating; the environment is dynamic for the agent.
Continuous :: Soccer match is a continuous state and continuous time problem ; as the speed and position of other players keeps on changing through continuous range of values, smoothly over time.
Partially observable :: In a soccer match, an agent can not determine what other agents are thinking and thus is partially observable.
Multi-agent :: In a soccer match, a striker needs to account position, speed and direction of defenders of opposite team. Each player needs to account for the action of the other player making it multiagent.
Reason for few of the labels
Practicing tennis against a wall - Deterministic :: The next state is completely determined by the current state and action executed by agent i.e force applied, direction in which it is applied etc.
Practicing tennis against a wall - Dynamic :: As the player keeps on moving according to the ball while computing what to do next.
NOTE In deciding what item to take at hostel lunch there is assumption that there is enough food for every student.
Deciding what item to take at hostel lunch - Sequential :: If an agent takes dry vegetable then it prefer chapatti else if I take vegetable with curry it prefers rice with them. Thus choice of current action can effect th future action making it sequential.
