Skip to main content

Table 2 Pseudocode of the embedded Soft Attention Mechanism; tmax the maximum number of the training episodes of each regression, Tmax is the number of training termination

From: A deep learning method for counting white blood cells in bone marrow images

Pseudocode of the embedded Soft Attention Mechanism

Initialize

Global variables, model parameters \(\theta ,\theta_{v}\) and counter T = 0

Parameters of RL agents: \(\theta^{{\prime }} ,\theta_{v}^{{\prime }}\), t ← 1

Repeat

Initialize the gradients \(d\theta \leftarrow 0,d\theta_{v} \leftarrow 0\)

Synchronize the RL agents \(\theta^{{\prime }} = \theta ,\theta_{v}^{{\prime }} = \theta_{v}\)

Initialize the initial states of LSTM \(c_{0} ,h_{0}\)

\(t_{start} = t\)

Take the entire image \(x_{t}\)

Repeat

FPN extracting the feature vectors \(v_{t}\) of \(x_{t}\)

Using \(v_{t}\), the previous state of LSTM \(h_{t - 1}\)

to obtain an attentive state \(Z_{t}\)

Input \(Z_{t} ,c_{t - 1} ,h_{t - 1}\) to LSTM

LSTM output \(h_{t}\)

\(s_{t} \leftarrow h_{t}\)

Take regression action \(a_{t}\) by \(\pi (a_{t} |s_{t} ;\theta^{{\prime }} )\)

Obtain reward \(R_{t}\) and a new image \(x_{t + 1}\)

\(t \leftarrow t + 1\)

\(T \leftarrow T + 1\)

Until reaching the terminal state \(s_{t}\) or \(t - t_{start} = t_{max}\)

\(G = \left\{ {\begin{array}{*{20}l} 0 \hfill & {terminal\,states_{t} } \hfill \\ {V\left( {s_{t} ;\theta_{v}^{{\prime }} } \right),} \hfill & {non - terminal\,states_{t} } \hfill \\ \end{array} } \right.\)

\({\mathbf{for}}\,\, i \in \left\{ {t - 1, \ldots ,t_{start} } \right\}{\mathbf{do}}\)

\(G \leftarrow R_{t} + \gamma G\)

Calculate the gradients \(\theta_{v}^{{\prime }}\): \(d\theta_{v} \leftarrow d\theta_{v} + \frac{{\partial \left( {G_{t} - v\left( {s_{t} ;\theta_{v}^{{\prime }} } \right)} \right)^{2} }}{{\partial \theta_{v}^{{\prime }} }}\)

\(\theta^{{\prime }}\): \(d\theta \leftarrow d\theta + \nabla_{{\theta^{\prime}}} \log \pi (a_{t} |s_{t} ;\theta^{{\prime }} )\left( {G_{t} {-} V\left( {s_{t} ;\theta_{v}^{{\prime }} } \right)} \right) + \beta \nabla_{{\theta^{{\prime }} }} H\left( {\pi \left( {s_{t} ;\theta^{{\prime }} } \right)} \right)\)

End for Until \(T > T_{max}\)