From: A deep learning method for counting white blood cells in bone marrow images
Pseudocode of the embedded Soft Attention Mechanism |
---|
Initialize |
Global variables, model parameters \(\theta ,\theta_{v}\) and counter T = 0 |
Parameters of RL agents: \(\theta^{{\prime }} ,\theta_{v}^{{\prime }}\), t ← 1 |
Repeat |
Initialize the gradients \(d\theta \leftarrow 0,d\theta_{v} \leftarrow 0\) |
Synchronize the RL agents \(\theta^{{\prime }} = \theta ,\theta_{v}^{{\prime }} = \theta_{v}\) |
Initialize the initial states of LSTM \(c_{0} ,h_{0}\) |
\(t_{start} = t\) |
Take the entire image \(x_{t}\) |
Repeat |
FPN extracting the feature vectors \(v_{t}\) of \(x_{t}\) |
Using \(v_{t}\), the previous state of LSTM \(h_{t - 1}\) |
to obtain an attentive state \(Z_{t}\) |
Input \(Z_{t} ,c_{t - 1} ,h_{t - 1}\) to LSTM |
LSTM output \(h_{t}\) |
\(s_{t} \leftarrow h_{t}\) |
Take regression action \(a_{t}\) by \(\pi (a_{t} |s_{t} ;\theta^{{\prime }} )\) |
Obtain reward \(R_{t}\) and a new image \(x_{t + 1}\) |
\(t \leftarrow t + 1\) |
\(T \leftarrow T + 1\) |
Until reaching the terminal state \(s_{t}\) or \(t - t_{start} = t_{max}\) |
\(G = \left\{ {\begin{array}{*{20}l} 0 \hfill & {terminal\,states_{t} } \hfill \\ {V\left( {s_{t} ;\theta_{v}^{{\prime }} } \right),} \hfill & {non - terminal\,states_{t} } \hfill \\ \end{array} } \right.\) |
\({\mathbf{for}}\,\, i \in \left\{ {t - 1, \ldots ,t_{start} } \right\}{\mathbf{do}}\) |
\(G \leftarrow R_{t} + \gamma G\) |
Calculate the gradients \(\theta_{v}^{{\prime }}\): \(d\theta_{v} \leftarrow d\theta_{v} + \frac{{\partial \left( {G_{t} - v\left( {s_{t} ;\theta_{v}^{{\prime }} } \right)} \right)^{2} }}{{\partial \theta_{v}^{{\prime }} }}\) |
\(\theta^{{\prime }}\): \(d\theta \leftarrow d\theta + \nabla_{{\theta^{\prime}}} \log \pi (a_{t} |s_{t} ;\theta^{{\prime }} )\left( {G_{t} {-} V\left( {s_{t} ;\theta_{v}^{{\prime }} } \right)} \right) + \beta \nabla_{{\theta^{{\prime }} }} H\left( {\pi \left( {s_{t} ;\theta^{{\prime }} } \right)} \right)\) |
End for Until \(T > T_{max}\) |