Expected Capacity or Reward
Consider that each state of the system has some (throughput) capacity or reward. For example, consider the two-state system shown in Figure 8-6. Assume a gain of 100 units (dollars, for example) can be obtained from the system when it is in state 1 per time unit. Similarly, assume a loss of 25 units (dollars) when the system is in state 2. Therefore, the expected gain from state I per unit time (at a specified time point t) is , cIPI(t) where ciI is the gain (or capacity or reward) from state I per unit time. Therefore, the total gain of the system can be obtained by summing the gains of all states of the system. Integrating this over a specified time interval gives the total gain within that interval.