Consider that each state of the system has some (throughput) capacity or reward. For example, consider the two-state system shown in
Figure 8-6. Assume a gain of 100 units (dollars, for example) can be obtained from the system when it is in state 1 per time unit. Similarly, assume a loss of 25 units (dollars) when the system is in state 2. Therefore, the expected gain from state
I per unit time (at a specified time point
t) is ,
cIPI(t) where
ciI is the gain (or capacity or reward) from state
I per unit time. Therefore, the total gain of the system can be obtained by summing the gains of all states of the system. Integrating this over a specified time interval gives the total gain within that interval.