- Research article
- Open Access
Using Kinect to classify Parkinson’s disease stages related to severity of gait impairment
BMC Bioinformatics volume 19, Article number: 471 (2018)
Parkinson’s Disease (PD) is a chronic neurodegenerative disease associated with motor problems such as gait impairment. Different systems based on 3D cameras, accelerometers or gyroscopes have been used in related works in order to study gait disturbances in PD. Kinect Ⓡ has also been used to build these kinds of systems, but contradictory results have been reported: some works conclude that Kinect does not provide an accurate method of measuring gait kinematics variables, but others, on the contrary, report good accuracy results.
In this work, we have built a Kinect-based system that can distinguish between different PD stages, and have performed a clinical study with 30 patients suffering from PD belonging to three groups: early PD patients without axial impairment, more evolved PD patients with higher gait impairment but without Freezing of Gait (FoG), and patients with advanced PD and FoG. Those patients were recorded by two Kinect devices when they were walking in a hospital corridor. The datasets obtained from the Kinect were preprocessed, 115 features identified, some methods were applied to select the relevant features (correlation based feature selection, information gain, and consistency subset evaluation), and different classification methods (decision trees, Bayesian networks, neural networks and K-nearest neighbours classifiers) were evaluated with the goal of finding the most accurate method for PD stage classification.
The classifier that provided the best results is a particular case of a Bayesian Network classifier (similar to a Naïve Bayesian classifier) built from a set of 7 relevant features selected by the correlation-based on feature selection method. The accuracy obtained for that classifier using 10-fold cross validation is 93.40%. The relevant features are related to left shin angles, left humerus angles, frontal and lateral bents, left forearm angles and the number of steps during spin.
In this paper, it is shown that using Kinect is adequate to build a inexpensive and comfortable system that classifies PD into three different stages related to FoG. Compared to the results of previous works, the obtained accuracy (93.40%) can be considered high. The relevant features for the classifier are: a) movement and position of the left arm, b) trunk position for slightly displaced walking sequences, and c) left shin angle, for straight walking sequences. However, we have obtained a better accuracy (96.23%) for a classifier that only uses features extracted from slightly displaced walking steps and spin walking steps. Finally, the obtained set of relevant features may lead to new rehabilitation therapies for PD patients with gait problems.
Parkinson’s Disease (PD) is a chronic neurodegenerative disease in which there is a dopaminergic deficit, due to the loss of nigroestriatal cells, causing motor impairment with tremor, rigidity and akinesia/bradykinesia as cardinal sign . Gait impairment is among the motor problems associated with PD causing as significant disability that in advanced cases is not improved with the dopaminergic treatments .
Kinect Ⓡ is an innovative motion capture device developed by Microsoft  that is low-cost and constitutes a non-intrusive tracking device. With Kinect, it is possible to build systems without markers that allow users to control and interact with applications by using an interface that recognizes gestures, voice commands and objects without physical contact. In the literature, there exist some works that use Kinect whose goal is to provide solutions for people that have to perform some kind of rehabilitation, in general, and for people with Parkinson Disease (PD), in particular. Those systems are of three different types: a) systems that help in performing rehabilitation therapies, b) systems that help in the process of monitoring or evaluating users, and c) systems that try to help in the process of diagnosing PD.
There are several systems whose goal is to help in the rehabilitation processes, some of them not related to PD [4, 5] but others related to PD [6–9]. In  the authors present a prototype of a game-based telerehabilitation system that tries to prove the adequacy of using Kinect for telerehabilitation therapies. Kinerehab  is an occupational therapy system where patients can perform three different exercises: lift arms front, lift arms sides and lift arms up. In  they propose a game aimed at training dynamic postural control system for people with Parkinson Disease. In , they assess the effects on postural control of patients with Parkinson Disease when playing with Kinect. In  the authors present a Kinect-based virtual reality system that provides patients having Parkinson Disease (PD) with a motivating way to improve their rehabilitation. In  they present a Kinect-based system that can assess gait by identifying different gait features such as step length and states of the gait cycle by using a Finite-State Machine.
With respect to systems whose goal is to monitor or evaluate users, some works can be found in the rehabilitation area. In , D. González-Ortega et al. present a system for cognitive rehabilitation that tracks human body joints (head and hands) and the face and facial features (eyes, nose, and ears), achieving a successful monitoring percentage of 96.28%, using the depth information provided by the Kinect device. In  Antón et al. present a method that achieves a 91.9% accuracy in posture classification and 93.75% accuracy in trajectory recognition of rehabilitation exercises performed in front of Kinect. In  Clark et al. claim that Kinect can validly assess kinematic strategies of some postural control tests (forward reach, lateral reach and single-leg eyes-closed standing balance), and affirm that the results are comparable to those obtained with a 3D camera-based motion analysis system. The goal of the previous works is to accurately monitor postures of people that are standing in front of Kinect. However, the monitoring of the gait of people that suffer from Parkinson Disease (PD) is different because they are not standing in front of Kinect; they are walking towards Kinect or moving away from it. In that case, it is required that the Kinect sensor provides accurate depth data. In  Galna et al. compared the accuracy of the Kinect sensor with a 3D motion analysis system (Vicon) with 9 people with PD and 10 controls and concluded that Kinect could accurately measure some temporal and spatial features of clinically relevant movements but not for smaller movements such as hand clasping or toe tapping. In another similar work  where Kinect sensor accuracy was also compared with accuracy obtained with Vicon, but only with healthy people, Kharazi et al. concluded that Kinect could accurately track the knee and hip during the gait cycle, but not the ankle. In , Gabel et al. claim that Kinect sensor offers accurate and robust measurements of gait features including gait kinematics. However, more recently, in , Springer and Yogev present a review report of 12 similar studies that concluded that Kinect offers good validity for only some spatiotemporal gait parameters but poor validity for gait kinematics variables.
There are some systems that try to diagnose PD by using Kinect. In  Rocha et al. present a system to distinguish between PD patients and non PD people (4 PD patients and 5 non PD patients or controls which took part in the experiment) by using some measures extracted from the Kinect v2 such as velocity and acceleration of joints, distances and angles between joints, and other calculated measurements such as duration, length, average velocity and cadence of gait cycles. They identified the angle at the elbow as a significant feature to distinguish between PD and non-PD persons. In  Tupa et al. present a system to distinguish between PD patients, young healthy people and aged-matched people (18 PD patients, 18 aged-matched people and 15 students). Their classification algorithm obtained an accuracy of 97.2% by using some extracted gait features: leg length, normalized average stride length and gait velocity, all of them estimated with the data collected from Kinect. In  Eltoukhy et al. concluded that Kinect was sensitive enough to detect between group differences among PD and non-PD persons (11 non-PD and 8 PD) for some spatiotemporal and kinematics features such as stride length, stance and swing duration, gait and swing velocity. Moreover, they found these results were similar to those obtained with a 3D movement analysis system (BTS).
Moreover, there are also many systems whose goal is to diagnose PD by using other devices different from Kinect. Godinho et al.  present a review of 168 proposals that make use of 73 different devices (22 wearable that include sensors such as accelerometers, gyroscopes or magnetometers, 38 non-wearable that make use of force plates, infrared cameras, video recording cameras, ultrasounds, radio waves, etc., and 13 hybrid devices that make use of a combination of them). Among the gait parameters analyzed by those systems we can find: stride length and velocity, cadence gait cycle, swing phase, stance phase and double support ratio, trunk movements, number of steps, peak velocity, step time, symmetry of limbs, tremor, etc. These gait features can also be calculated with the data captured by Kinect, is a low-cost and less intrusive non-wearable device. It has to be said that 2 of the 9 recommended systems by Godinho et al. are also non-wearable devices. The first one , based on Nintendo Wii Balance Board, is used as an assessment tool for postural instability and not for gait, and it is considered a cheap and widespread device. The second one is the GAITRite system , an electronic pathway that contains pressure sensitive sensors and that provides information about gait features: walking speed, cadence and step length.
The goal of this work is to try to characterize the gait of PD patients by using the raw information collected from the Kinect device, and some features that can be calculated (or more correctly, estimated) from those raw data, knowing that this information is less precise than the information collected from more precise and sophisticated devices. With respect to this goal, it is known that in the early-moderate stage, the predominant form is short-stepped, with slow velocity and decreased amplitude of the segmental movement, which is related to hypokinesia . Clinical characteristics of the locomotor pattern include reduced angular excursion of joints such as shoulder and knee  in one hemibody, which extend to the other hemibody as the disease progresses. Progressively, this gait becomes more abnormal and features such as flexion of the trunk, stopped posture, short steps, and shuffling, the latter associated with reduced ground clearance and festination, become more prominent . Indeed, Freezing of Gait (FoG) is one of the most debilitating features of PD as it causes falls  and reduces mobility and quality of life . It is described as “a brief, episodic absence or marked reduction of forward progression of the feet despite the intention to walk”. Furthermore, it is interesting to note that it seems to be associated with postural instability  but not with cardinal features of PD .
Moreover, we want to identify different stages of PD in order to detect the locomotion changes that progressively occur as the disease progresses. To achieve this, we will undertake a cross-sectional clinical study with three groups of patients: 1) early PD patients without axial impairment; 2) more evolved PD patients with higher gait impairment but without FoG; and 3) patients with advanced PD and FoG. To the best of our knowledge, this is the first work where such a clinical study has been done by using Kinect.
In this section we present relevant information of the clinical study and the methods applied for the gait analysis of PD patients. Thus, we present the test scenario and the participants, the datasets about patients and Kinect recordings, the data preprocessing of the Kinect recordings and the feature selection and stage classification.
Test scenario and participants
The test consisted of recording with Kinect the gait of the group of patients that took part in the clinical study, whose goal was to distinguish among the PD stage of patients. This test was performed in a corridor of the Hospital Donostia in San Sebastian during two different days (in November 2015 and January 2017).
The participants in the test were 30 patientsFootnote 1, of whom 8 were in the first stage of the disease (early PD patients without axial impairment), 11 in the second stage (more evolved PD patients with higher gait impairment but without FoG), and the remaining 11 in the third stage (patients with advanced PD and FoG). The stage of each patient was diagnosed by the neurologists of the Hospital Donostia.
The corridor was long enough for the patients to walk about 4.5-5.5 meters in both directions. They had to walk along the corridor forwards and backwards four times. Therefore, in total, each patient performed a walk of approximately 40 meters, including their corresponding 7 turnings or spins required to change the direction.
With the aim of covering the entire walk, including the spins that are very significant to identify PD patients, and also to capture the whole body of the patients (without cuts in the upper or below body), two Kinect devices were needed, each recording approximately 2.5 meters of the total route. The reason is that, with one Kinect device, only 2-3 meters can be covered. The two cameras were placed next to the wallFootnote 2 (distributed as shown in Fig. 1), and tried to record simultaneously the complete walk. Each camera was connected to an independent computer that ran a data capture program that saved the data generated by the Kinect cameras during each patient walk. It is important to remark that there is a zone of overlap between the two cameras so that the walk can be recorded entirely, which also means that there is an area that is recorded twice (once for each camera).
The data available for the subsequent analysis is of two types: one is about clinical data of patients and the other one is the data captured with Kinect.
Dataset about patients
All patients were diagnosed according to UKPDS Brain Bank Criteria . Each participant was also assessed on the Hoehn and Yahr staging and the Freezing of Gait Questionnaire . None of the patients were deemed as having dementia according to Movement Disorder Society PD Dementia criteria . All patients were assessed in the practically defined “off” state having stopped dopaminergic drugs 12 h before. This study was approved by the local ethics committee and written informed consent was obtained from all patients prior to study participation.
With respect to the clinical data of patients, we had the following information for each one: gender, age, weight, height, years of evolution of the PD, stage of the disease assigned by the neurologists (1, 2 or 3) based on the gait impairment as explained previously, and disease stage according to the Hoehn and Yahr (HY) scale , also assigned by the neurologists. As said before, 8 patients were assigned to the first stage of the disease, 11 to the second stage, and 11 to the third one, based also on the gait impairment assessment. According to the HY classification, there were 4 patients in stage 1.5, 7 in stage 2, 8 in stage 2.5, 10 in stage 3 and 1 in stage 4, with a strong Pearson correlation between the HY scale and the one used in the study: 0.793411. The first affected body side was also assessed for each patient. Table 1 shows the summary of patient information.
Dataset of Kinect recordings
With respect to the data captured by Kinect from the patients, it can be said that each Kinect was capable of recording a maximum of 30 frames per second. The information that was saved for further processing was the following: depth images and skeletons. Regarding the size of the depth images, their dimension was 800 * 600 pixels. As to the skeleton structure, each frame is composed of 25 joints (distributed as shown in Fig. 2) from which the overall position of the skeleton of the frame is obtained and whether it is inferred by the Kinect or not. Nevertheless, only information of the joints (and not from the depth images) has been used for this work.
All joints are composed of 4 values, 3 of which indicate the joint position, with the fourth value indicating if the position value has been inferred by Kinect or not. A position value for a joint is inferred at a particular moment, if that joint is hidden for the infrared sensor of the Kinect camera, and the value is then calculated by the Kinect software by using previous or subsequent position values. If a joint position has been inferred, then the confidence of that data is low. The position values consist of the joint coordinates in X, Y and Z axes:
The point X corresponds to the lateral (horizontal) position with reference to Kinect.
The point Y corresponds to the vertical position with reference to Kinect.
The point Z corresponds to the depth position, that is, the distance in depth from Kinect to the person.
Apart from the mentioned Kinect feature of inferred joint positions, the Z value of the joints is not so accurate because Kinect has been mainly designed for people to stand in front of the camera (the depth position does not need to be so accurate) and not for people that walk in front of the camera. Related to this, it has to be said that when people are not looking at the camera or when they are moving away from the camera, Kinect tries to capture their skeletons as if they were in a position looking at the camera. The result is that the obtained data is of lower quality when patients walk away the camera; in Fig. 3, an impossible skeleton can be seen with the shoulders and the feet on the back side. It is interesting to know these facts about the quality of Kinect data so that an appropriate data cleaning can be performed during data processing.
Data preprocessing: data cleaning and feature extraction
Before finding an accurate classifier, it is required to preprocess the raw data captured from Kinect with the goal of cleaning the data and extracting candidate features for the classifier. The preprocessing was performed according to the following steps: initial data cleaning, identification of gait steps, data cleaning of those steps, frames labeling, final data cleaning of steps, and feature extraction.
Initial data cleaning
In this first step, frames with many inferred points were removed because, as explained in a previous subsection, those values provide low reliability. The threshold to remove a frame was fixed on four inferred skeleton points.
Identification of gait steps
After the initial data cleaning, a gait step-based approach was followed to further reduce noise and to extract relevant features for the PD stage classification. A gait step is defined as a sequence of frames from the moment when one foot strikes the floor to the other foot striking the floor. A strategy based on local maximums of the distance between the feet time series was used in order to detect the gait steps in a sequence of frames. The frames registered between two consecutive local maximums were considered a gait step.
The spine base joint on XZ plane (floor projection) of each two consecutive local maximums belonging to a valid step were used to calculate the displacement direction. Figure 4 shows the spine base joint on XZ plane time series for two steps. The right step in the figure represents a straight walking step (a step whose displacement direction angle is less than 0.1 radians from the straight line), with all its frames fully aligned with the displacement direction. The left step the Fig. 4 represents a slightly displaced step from the straight line (a step whose displacement direction angle is more than 0.1 radians from the straight line and less than 0.2 radians) and it may correspond to the beginning of a spin. Black dots in the figure correspond to frames aligned with the displacement direction (spine base less than 0.1 radians variation from the computed displacement direction). Red dots in the figure correspond to frames not aligned with the displacement direction (spine base more than 0.1 radians variation from the computed displacement direction).
Since the walking direction of a person is not always straight to the location of the camera (an example is provided in Fig. 4, left), an angular rotation over the y-axis (vertical axis) was applied in order to align the direction of walking of each person with the position of the Kinect sensor.
Data cleaning of steps
After identifying gait steps, another data cleaning process was carried out: each detected step was considered valid if only one foot was detected as moved. The frames corresponding to invalid steps were not considered for the next analysis step (frames labeling). At this stage, frames with a large difference in length between limbs of the same type were also removed for further analysis.
After the data cleaning process of steps, it was attempted to categorize every frame as belonging to one of these types:
straight walking step, frame aligned with the displacement direction, displacement towards the Kinect sensor
slightly displaced walking step, frame aligned with the displacement direction, displacement towards the Kinect sensor
slightly displaced walking step, frame equally aligned or not aligned with the displacement direction, displacement towards the Kinect sensor direction
At least 5 frames of the same type (a, b, c) had to be detected in the same step, in order to label such frames and to consider them for further analysis. This threshold was used in order to avoid noisy or outliers frames and it was empirically established.
Final data cleaning of steps
At this point, frames whose displacement direction was not towards the Kinect sensor were removed, unless they were identified as part of a spin. As it has been said in a previous subsection, data provided by Kinect when people are walking away the camera are not very accurate. However, when performing a spin it is important to maintain the data that may provide information about when the user is going towards or away from the camera.
For each frame labeled as a, b or c, a set of angles was computed as follows:
Limbs angles: angles of the projections on the XY and YZ plane (frontal and lateral plane, respectively) for each of the following bones: humerus, forearm, thigh, and shin, both left and right side. These measurements correspond to 16 angles. (4 limb types * 2 sides * 2 plane projection types = 16 angles). As an example of the computing of these angles, the angle α in Fig. 5 and α′ in Fig. 6 show the right forearm angle projection on xy and yz respectively. These kinds of measurements were successfully used previously for gait based recognition with a Kinect sensor  and we intended to test them here for the Parkinson stage classification.
Bent angles related to Parkinson disease. Using the principles outlined in  we have calculated three body angles related to Parkinson disease. Namely, we calculated neck-flexion (NF), fore-bent (FB) and lateral-bent (LB) angles. Angle β in Fig. 5 and β′ in Fig. 6 show the lateral-bent and fore-bent, respectively. The LB and FB angles were computed using the spine-shoulder and spine-mid joints provided by Kinect. The NF angle was computed in a similar way to FB angle, using the neck and spine-shoulder joints provided by Kinect.
A local regression (lowess) was applied to each angle series and then the mean and standard deviation were computed for each angle series. In this way, 19 mean angle-values were computed for each type of frames (a, b, c). These 57 values characterize postural states during the walking test. An additional 57 standard deviation values have been computed in a similar way. These values characterize the amount of variation of the corresponding angles during the walking test.
On the other hand, for spin sequences, the Kinect sensor was not intended to obtain sideways data. Still, we have extracted the number of steps in spin for those steps whose displacement direction was more than 0.2 radians from the straight line. As it has been said before, the displacement direction could be towards the Kinect sensor or moving away.
Feature selection and stage classification
The features mentioned above (115 features, 38 for each a, b, c type of frame and 1 for spin steps) were computed for all the subjects (30 patients) recorded with Kinect. For classification purposes, the instances were balanced in the training set, which means that the instances were reweighted in the data so that each stage class had the same total weight.
There are different methods in the machine learning area that could be used to discriminate between PD stages. Several classification algorithms have been evaluated for each feature subset obtained previously. We evaluated different methods, for example, decision trees (the J48 algorithm provided by WEKA), Bayesian networks (BayesNet algorithm in WEKA), neural networks (multilayer perceptron algorithm in WEKA) and K-nearest neighbors classifier with (IBk algorithm in WEKA with K=1) among others. All the classification algorithms evaluated were run 100 times, using different random seed and a 10-fold cross validation schema. The performance of each classification method was measured in terms of accuracy (percentage of correctly classified instances). Additionally, the standard deviation over the different folds was computed. As the data was balanced, we did not consider another performance measure. All the evaluated methods were run by using the default WEKA parameters settings .
Once a classification method was selected, several experiments were carried out in order to improve the results of the classification method by selecting a better subset of features. Not all the computed features were relevant for the classification task and some of them could be redundant. In order to select the most relevant ones, we tested several filter methods: correlation-based feature selection (CFS), information gain and consistency subset evaluation. The filter methods have the advantage that they select variables regardless of the classification model and usually are robust to overfitting. All these methods were tested using the implementation provided by WEKA tool. Each filter method obtained a feature subset of a training set and the classification accuracy for that feature subset was computed over a test set. Once again 10-fold cross-validation was used an evaluation schema.
The list of experiments run in order to choose the best classification and feature selection method are summarized in Table 2. The first two columns in Table 2 identify the classification method. The third columns (No feature selection) shows the accuracy (average and standard deviation) of each classification method tested when no external feature selection algorithm is used. The last three columns show the accuracy of each classification method after applying the corresponding feature selection algorithm. Note that some classification methods (for example J48, Bayes Net) have their own built-in feature selection.
One of the best feature selection methods turned out to be the Correlation-based Feature Selection (CFS), as can be seen in Table 2, fourth column. The CFS method considers both the individual predictive ability of each attribute and the degree of redundancy between them .
Table 3 shows the list of the features selected by the CFS algorithm, using data from all the subjects. Notice the small number of features (7) obtained by this method, compared to 115 features initially considered. It also highlights the fact that most of the selected features (5 features) are related to the spin execution, namely F3, F4 and F5 (related to type b frames), F6 (related to type c frames) and F7 (number of steps in spin).
In terms of classification methods, Bayesian networks are the method that provided the better by far accuracy. Specifically, the accuracy obtained, using 10-fold cross-validation and CFS feature selection method, was ≈93.40% (see Table 2).
Bayesian network model
Bayesian network method  is a Directed Acyclic Graph (DAG), which is a graph with no cyclic paths. Each node in the DAG represents a random variable, while the edges between the nodes represent statistical dependencies between the corresponding random variables . Figure 7 shows the structure of the Bayesian Network induced model for PD stage classification. The nodes correspond to the stage to be classified (1, 2 or 3) and the selected features (F1, F2, …, Fn). Specifically, the CFS algorithm selects seven features related to PD stage to be classified. The arrows between stage node and Fi nodes indicate that a value taken by the variable stage depends on the values taken by variables Fi (with i∈[ 1..7] in the final Bayesian model obtained using data from all the subjects). These relationships are encoded by conditional probability distributions (CPDs) of the form P(Fi|stage) (the probability of Fi given stage, as stage is parent for Fi nodes as may be observed in Fig. 7).
The CPDs of the final model (P(Fi|stage)), obtained using the data of all the subjects, are shown and discussed below.
Table 4 shows the probability distribution for the F1 feature (standard deviation for left shin projection angle on YZ plane). The classification algorithm discretized the range of the F1 feature in two intervals: low for values in [0,0.462431) and high for values in [0.462431, +∞]. As may be observed, stages 2 and 3 are related to low values for this variable (that means P(F1=low|stage=3)=0.872) and P(F1=low|stage=2)=0.624). Stage 1 is strongly related to high values for this feature (P(F1=high|stage=1)=0.955). This feature characterizes the intensity of movement in depth of the left shin during straight walking test and we associate low values for this for this feature to small step length reported in Parkinson’s disease (Pearson correlation coefficient of this feature with step length is 0.60).
Table 5 shows the probability distribution for the F2 feature (standard deviation for left humerus angle on XY plane for frames of type a, straight walking). As may be observed, stage 3 is related to high values for this feature (P(F2=high|stage=3)=0.955), that means a higher intensity of the lateral movement of the left humerus. The high value was fixed in this case to the interval [0.048687, +∞]. This feature allows distinguishing the cases labeled as stage 3 from those labeled as stage 1 and 2.
Table 6 shows the probability distribution for the F3 feature (mean frontal bent for type b frames). As may be observed, stage 1 is associated with low values for this feature (P(F3=low|stage=1)=0.587), that means low frontal bent, the subject walks upright, without leaning forward, stage 2 is associated with medium values and stage 3 to high values, respectively.
The F4 variable of the Bayesian network model is the standard deviation for lateral bent (LB) angle measured for frames of type b. In this case the CPF (see Table 7) associated stage 1 with low values. Stage 2 and 3 are associated with high values of F4. This feature allows distinguishing the cases in stage 1 from those in stage 2 and 3.
The F5 feature is a postural one, namely the mean for the left forearm angle projected on YZ plane and measured for type b frames. In this case, the CPD related stage 3 to low values for F5 (0.789 probability, for more details, that means that the position of the left forearm is mainly vertical for type b frames, see Table 8). (slightly displaced walking steps, frame aligned with the displacement direction).
The F6 feature is the standard deviation for left humerus angle on XY plane for type c frames. This feature is similar to F2 feature. Stage 1 is mainly related to low values for this variable, stage 2 to medium values and stage 3 to high values, respectively (for more details, see Table 9).
The F7 feature is the number of steps for spin walking. In this case the CPF related the stage 3 mainly to high values (0.872 probability, that means that subjects of stage 3 are more likely to take many steps in spin, see Table 10). This feature allows distinguishing the cases in stage 3 from those in stage 1 and 2.
Bayesian network classifier versus other classification methods
The Bayesian network induced in the previous subsection is very similar to a Naïve Bayesian classifier. In particular, a Naïve Bayes classifier may be seen as a specialized form of a Bayesian network with a structure similar to the one in Fig. 7, when depicted as a Bayesian network . Therefore, differences in the classification results of Bayesian network and Naïve Bayes methods reported in Table 2 need further analysis.
When analyzing the two methods, we could conclude that the differences in the results are due to the fact that Bayesian network method discretizes the continuous values of the features used for classification. Discretization is the process of transforming a continuous-valued feature into a discrete one by creating a set of contiguous intervals that spans the range of the feature’s values, as has been done in the previous subsection with the selected features (F1, F2, …, F7). Indeed, when Naïve Bayes was used in conjunction with CFS method and feature discretization, the PD stage classification results obtained were similar to the ones obtained when using Bayesian network with CFS. Results over the impact of discretization over Naive-Bayesian classifier have been already observed in previous studies, such as  and the improvement of classification performance with discretization on biomedical datasets has been reported in literature in related works, such as .
In view of the improvements that the discretization can bring, several tests have been carried out in order to verify the performance for PD stages classification of other methods, when used in conjunction with feature discretization method. Among the classification methods initially tested in Table 2, Multilayer Perceptron, Ibk, Kstar and SVM improved their classification accuracy towards values similar to Bayesian network method when used in conjunction with feature discretization method.
As in this step we identified several methods with similar accuracy, we compared them using other criteria such as training and testing time. Table 11 shows the average training and testing time (calculated in millisecondsFootnote 3) for each classification method with similar accuracy.
From the point of view of costs (see Table 11), Bayesian networks continue to be a very good option as they obtain low times for both training and testing. Nevertheless, Naïve Bayes with feature discretization overtakes Bayes Net method, especially in training times, since for Naïve Bayes structural learning is not required. Moreover, our obtained Bayesian network model is similar to a Naïve Bayes classifier with feature discretization because features are independent and not related among them. In any case, these two methods provide models which are easier to understand by a human than the other candidate methods, as have been seen in the previous subsection, through the network representation and the corresponding CPDs.
In this work, we have studied gait impairment in PD patients by using Kinect. As it has been said in “Background” section, systems that try to diagnose PD have used different gait features such as stride length and velocity, cadence gait cycle, swing phase, stance phase and double support ratio, trunk movements, number of steps, peak velocity, step time, symmetry of limbs or tremor. Moreover, by using Kinect, we have been able to experiment with other features derived from the data about joints captured with the Kinect device, and we have found that some of those features are relevant in the obtained Bayesian network model. In particular, the movement and the position of the left arm proved to be very relevant when classifying the PD stage. As may be observed in Table 3, the CFS algorithm selected three features related to the left arm position as relevant, that means almost half of the selected features. One of those features was a postural feature, related to positions held during slightly displaced walking steps (F5). The other two measures were the intensity of the lateral movement of the left humerus (F2 and F6). Moreover, the straight walking sequences provided useful information related to the intensity of movement of lower limbs (F1), meanwhile the slightly displaced walking sequences provided useful information related to trunk position (F3 and F4). We can also add that the spin and slightly displaced steps sequences of the test provided more useful information for classifying PD than the straight walking sequences. It is worth saying that although not all of the patients were initially affected in the same hemobody (left, right or both) (see Table 1), it seems that the left part of the body is somehow more affected, which may constitute an interesting and novel finding. However, as the patients were mainly right-handed, it cannot be concluded that similar results would be found with left-handed patients.
With respect to the biological/clinical significance of the selected features of the classifier, we can say that locomotion in FoG patients is characterised by (1) a profound and incremental decrease in stride length; (2) highly reduced joint ranges in the hip, knee, and ankle; (3) disordered temporal control of gait cycles and (4) high-frequency alternate trembling-like leg movements . These gait deficits are unrelated to disease severity, and, although FoG has traditionally been viewed as a motor symptom of advanced PD, it does not correlate with the cardinal features of parkinsonism: rigidity, bradykinesia, and tremor . However, it correlates with some characteristic cognitive domains decline such as executive dysfunction, set-shifting and conflict resolution. Although functional imaging studies have found primarily bilateral impairments in FoG in PD [44, 45], others have shown asymmetrical cortical functioning (i.e., hipoactivity in the right fronto-parietal cortices)  (Bartels et al 2008) and structural abnormalities in the right cortical hemisphere and brainstem  in FoG in PD, irrespectively of the side of basal ganglia (BG) degeneration. Because the right hemisphere plays an important role in monitoring sensorimotor information , it seems reasonable to expect that right cortical dysfunction in addition to right BG damage in FoG PD patients could exacerbate right hemisphere dysfunction. Thus, it might be expected that interruption of right hemisphere sensorimotor processes could lead to more severe gait impairments. Supporting this, recent research has found evidence that right BG damaged FoG PD patients also experience more FoG episodes in specific situations that demand increased sensory processing compared to left BG damaged FoG PD . In these sense, our results show that the most relevant features in these patients in the classifier are morphological motor features related to previously reported right hemisphere impairment such as left shin, humerus and forearm angles. In addition, cholinergic neurons of the pedunculopontine nucleus (PPN) are particularly important for gait, as shown by posture and gait abnormalities induced in monkeys with cholinergic lesions in the PPN . This is related to the involvement of frontal and lateral bents in our classifier, probably related to posture and balance impairment in these patients. Finally, in FoG PD patients it is known that a decrease in step length with shuffling becomes particularly apparent during gait initiation or turning , which will be represented by the number of steps during spin variable in the classifier.
Finally, it is difficult to compare our obtained accuracy (93.40%) with those obtained in other related works because, to the best of our knowledge, there are no similar Kinect-based systems that try to classify between different stages of PD. However, there are previous studies showing the reliability of Kinect distinguishing between healthy and PD patients [18, 20, 51, 52]. In our case, we have performed additional experiments with our system in order to classify between PD people and controls. We have tested our system with 11 healthy people, aged 65-70, and found that the accuracy of the classifier to distinguish between the 11 controls and the 30 PD people was 93.51%. This PD vs non-PD classifier has been created by using the Bayes Net method, after selecting features with CFS method. The training data have been balanced, and the experiments have been run 100 times. The set of relevant features for this classifier is different than for the classifier that distinguishes among the 3 PD stages, but that is normal because it is a different problem.
In this paper, we have presented a Kinect-based system able of classifying between three different stages of Parkinson Disease related to severity of gait impairment. As Kinect is a low-cost and non-intrusive device, the system is less expensive and more comfortable than systems based on accelerometers, gyroscopes or exoskeletons. Moreover, the obtained accuracy of the classifier has shown that using Kinect is feasible to build such a system, although previous involving about Kinect presented some contradictory results.
The classifier was built with data captured by the system from a clinical study performed with 30 patients whose PD stage has been reported by neurologists. In the process of selecting the best classifier, different classification methods and techniques to select relevant features among the 115 features identified were tried. The best one, with an accuracy of 93.40% using 10-fold cross validation, has been a Bayesian Network classifier combined with a correlation-based feature selection method. The method obtains a model which is easily interpreted by humans. Feature discretization has been shown to improve the classification results of other methods that could also be suitable for the problem treated. In relation to the relevant features, it was found that almost half of them were related to the movement and the position of the left arm, most of the relevant features were related with spin performing and gait sequences with slightly displaced steps, and for straight walking sequences, some features were related to intensity of movement of lower limbs. Other classic features for the characterization of PD were also tested (the average length of the step, average speed, average cadence), but using them did not improve the classification results (although they have been found to have a certain degree of relevance for stage classification).
Moreover, this set of relevant features that correspond to specific body joints may lead to new rehabilitation therapies for PD patients with gait problems. Many studies have shown the efficacy of rehabilitation therapy at improving specific impairments and functional limitations in individuals with PD and FoG. Those rehabilitation therapies should try to improve the values for the obtained relevant features in PD patients with the goal of delaying as much as possible the progression from the initial stages of PD to the stage where FoG appears. Our Kinect-based system could be used in order to monitor the evolution of those PD patients.
We plan to perform prospective studies to analyse if the previously mentionated rehabilitation treatement delays the progression towards more severe forms of gait dysfunction.
All the patients were carefully recruited by experienced neurologists according to UKPDS Brain Bank Criteria. In  they worked with 48 PD patients; in the other 17 works that we have checked they worked with less than 30.
In the test performed in November 2015, the angle of the Kinect camera was perpendicular to what should be a straight displacement of the patient. In the test of January 2017, the angle has been 60 degrees to what should be a straight displacement of the patient. The method presented in this paper is valid for different orientations of the Kinect cameras.
Each training and testing time is obtained as a mean over 100 runs. An iMac with 3,06 GHz Intel Core i3 processor has been used for the tests.
Correlation-based feature selection
Conditional propability distributions
Directed Acyclic graph
Freezing of gait
Hoehn And Yahr scale
SVM with sequential minimal optimization
Support vector machine
- UKPDS brain bank:
United Kingdom Parkinson’s disease society brain bank
Hughes A, Daniel S, Kilford L, Lees A. Accuracy of clinical diagnosis of idiopathic parkinson’s disease: a clinico-pathological study of 100 cases. J Neurol Neurosurg Psychiatry. 1992; 55(3):181–4.
Galna B, Lord S, Burn D, Rochester L. Progression of gait dysfunction in incident parkinson’s disease: impact of medication and phenotype. Mov Disord. 2015; 30(3):359–67. https://doi.org/10.1002/mds.26110.
Lütkebohle I. Kinect for windows. 2014. http://www.microsoft.com/en-us/kinectforwindows/discover/features.aspx. Accessed 21 Nov 2017.
Lange B, Chang C-Y, Suma E, Newman B, Rizzo AS, Bolas M. Development and evaluation of low cost game-based balance rehabilitation tool using the microsoft kinect sensor. In: Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE: 2011. p. 1831–4.
Chang Y, Chen S, Huang J. A kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Res Dev Disabil. 2011; 32(6):2566–70. https://doi.org/10.1016/j.ridd.2011.07.002.
Galna B, Jackson D, Schofield G, McNaney R, Webster M, Barry G, et al. Retraining function in people with parkinson’s disease using the microsoft kinect: game design and pilot testing. J Neuroeng Rehabil. 2014;11(60). https://doi.org/10.1186/1743-0003-11-60.
Pompeu JE, Torriani-Pasin C, Doná F, Ganança FF, da Silva KG, Ferraz HB. Effect of kinect games on postural control of patients with parkinson’s disease. In: Proceedings of the 3rd 2015 Workshop on ICTs for Improving Patients Rehabilitation Research Techniques. REHAB ’15. New York: ACM: 2015. p. 54–7. https://doi.org/10.1145/2838944.2838958, http://doi.acm.org/10.1145/2838944.2838958.
Palacios-Navarro G, García-Magariño I, Ramos-Lorente P. A kinect-based system for lower limb rehabilitation in parkinson’s disease patients: A pilot study. J Med Syst. 2015; 39(9):1–10. https://doi.org/10.1007/s10916-015-0289-0.
Cancela J, Arredondo M, Hurtado O. Proposal of a kinect(tm)-based system for gait assessment and rehabilitation in parkinson’s disease. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE: 2014. p. 4519–22. https://doi.org/10.1109/EMBC.2014.6944628.
González-Ortega D, Díaz-Pernas F, Martínez-Zarzuela M, Antón-Rodríguez M. A kinect-based system for cognitive rehabilitation exercises monitoring. Comput Methods Programs Biomed. 2014; 113(2):620–31.
Anton D, Goñi A, Illarramendi A. Exercise recognition for kinect-based telerehabilitation. Methods Inf Med. 2015; 54(2):145–55.
Clark RA, Pua Y, Bryant AL, Hunt MA. Validity of the microsoft kinect for providing lateral trunk lean feedback during gait retraining. Gait Posture. 2013; 38(4):1064–6. https://doi.org/10.1016/j.gaitpost.2013.
Galna B, Barry G, Jackson D, Mhiripiri D, Olivier P, Rochester L. Accuracy of the microsoft kinect sensor for measuring movement in people with parkinson’s disease. Gait Posture. 2014; 39(4):1062–8. https://doi.org/10.1016/j.gaitpost.2014.01.008.
Kharazi MR, Memari AH, Shahrokhi A, Nabavi H, Khorami S, Rasooli AH, Barnamei HR, Jamshidian AR, Mirbagheri MM. Validity of microsoft kinecttm for measuring gait parameters. In: 2015 22nd Iranian Conference on Biomedical Engineering (ICBME). IEEE: 2015. p. 375–9. https://doi.org/10.1109/ICBME.2015.7404173.
Gabel M, Gilad-Bachrach R, Renshaw E, Schuster A. Full body gait analysis with kinect. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE: 2012. p. 1964–7. https://doi.org/10.1109/EMBC.2012.6346340.
Springer S, Yogev SG. Validity of the kinect for gait assessment: A focused review. Sensors. 2016; 16(2):194. https://doi.org/10.3390/s16020194.
Rocha AP, Choupina H, Fernandes JM, Rosas MJ, Vaz R, Cunha JPS. Kinect v2 based system for parkinson’s disease assessment. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE: 2015. p. 1279–82. https://doi.org/10.1109/EMBC.2015.7318601.
Ťupa O, Procházka A, Vyšata O, Schätz M, Mares J, Valis M, Mařík V. Motion tracking and gait feature estimation for recognising parkinson’s disease using ms kinect. Biomed Eng Online. 2015; 14:97.
Eltoukhy M, Kuenze C, Andersen M, Oh J, J S. Prediction of ground reaction forces for parkinson’s disease patients using a kinect-driven musculoskeletal gait analysis model. Med Eng Phys. 2017; 50:75–82. https://doi.org/10.1016/j.medengphy.2017.10.004.
Godinho C, Domingos J, Cunha G, Santos A, Fernandes R, Abreu D, Gonçalves N, Matthews H, Isaacs T, Duffen J, Al-Jawad A, Larsen F, Serrano A, Weber P, Thoms A, Sollinger S, Graessner H, Maetzler W, Ferreira J. A systematic review of the characteristics and validity of monitoring technologies to assess parkinson’s disease. J Neuroeng Rehabil. 2016;13(24). https://doi.org/10.1186/s12984-016-0136-7.
Clark R, Bryant A, Pua Y, McCrory P, Bennell K, Hunt M. Validity and reliability of the nintendo wii balance board for assessment of standing balance. Gait Posture. 2010; 31(3):307–10. https://doi.org/10.1016/j.gaitpost.2009.11.012.
Menz H, Latt M, Tiedemann A, Mun San Kwan M, Lord S. Reliability of the gaitrite walkway system for the quantification of temporo-spatial parameters of gait in young and older people. Gait Posture. 2004; 20(1):20–5. https://doi.org/10.1016/S0966-6362(03)00068-7.
Giladi N, McMahon D, Przedborski S, Flaster E, Guillory S, Kostic V, Fahn S. Motor blocks in parkinson’s disease. Neurology. 1992; 42(2):333–9.
Giladi N, McMahon D, Przedborski S, Flaster E, Guillory S, Kostic V, Fahn S. Automatic motion analysis of gait in patients with parkinson disease: effects of levodopa and visual stimulations. Frenc Rev Neurol. 1996; 152:128–34.
Kemoun G, Defebvre L. Clinical description, analysis of posture, initiation of stabilized gait. Frenc Presse Med. 2001; 30:452–9.
Kerr G, Worringham C, Cole M, Lacherez P, Wood J, Silburn P. Predictors of future falls in parkinson disease. Neurology. 2010; 75:116–24. https://doi.org/10.1212/WNL.0b013e3181e7b688.
Nutt J, Bloem B, Giladi N, Hallett M, Horak F, Nieuwboer. Freezing of gait: moving forward on a mysterious clinical phenomenon. Lancet Neurol. 2011; 10:734–44. https://doi.org/10.1016/S1474-4422(11)70143-0.
Giladi N, Mcdermott MP, Fahn S, Przedborski S, Jankovic J, Stern M, et al.Freezing of gait in pd: prospective assessment in the datatop cohort. Neurology. 2001; 56:1712–21.
Giladi N. Gait disturbances in advanced stages of parkinson’s disease. Adv Neurol. 2001; 86:273–8.
Mancini M, Priest K, Nutt J, Horak F. Quantifying freezing of gait in parkinson’s disease during the instrumented timed up and go test. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE: 2012. p. 1198–201. https://doi.org/10.1109/EMBC.2012.6346151.
Gibb W, Lees A. The relevance of the lewy body to the pathogenesis of idiopathic parkinson’s disease. J Neurol Neurosurg Psychiatry. 1988; 51:745–52.
Giladi N, Shabtai H, Simon E, Biran S, Tal J, Korczyn A. Construction of freezing of gait questionnaire for patients with parkinsonism. Parkinsonism Relat Disord. 2000; 6(3):165–70.
Emre M, Aarsland D, Brown R, Burn D, Duyckaers C, Y M, et al.Clinical diagnostic criteria for dementia associated with parkinson’s disease. Mov Disord. 2004; 19(9):1043–9.
Hoehn M, Yahr M. Parkinsonism: onset, progression and mortality. Neurology. 1967; 17(5):427–42. https://doi.org/10.1212/wnl.17.5.427.
Bobillo F, Dranca L, Bernad J. A fuzzy ontology-based system for gait recognition using kinect sensor. In: Proceedings of the 11th International Conference on Scalable Uncertainty Management (SUM 2017). Granada: Springer: 2017.
Oeda T, Umemura A, Tomita S, Hayashi R, Kohsaka M, Sawada H. Clinical factors associated with abnormal postures in parkinson’s disease. PloS ONE. 2013; 8(9):73547.
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH. The weka data mining software: an update. ACM SIGKDD Explor Newsl. 2009; 11(1):10–8.
Hall MA. Correlation-based feature selection for machine learning, PhD Thesis. New Zealand: Department of Computer Science, Waikato University; 1999. Available on-line here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37.4643%26rep=rep1%26type=pdf.
Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers; 2014.
Bouckaert RR. Bayesian network classifiers in weka for version 3-5-7. Artif Intell Tools. 2008; 11(3):369–87.
John GH, Langley P. Estimating continuous distributions in bayesian classifiers. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers: 1995. p. 338–45.
Dougherty J, Kohavi R, Sahami M. Supervised and unsupervised discretization of continuous features In: Prieditis A, Russell S, editors. Machine Learning Proceedings 1995. Morgan Kaufmann: 1995. p. 194–202. https://doi.org/10.1016/B978-1-55860-377-6.50032-3.
Lustgarten JL, Gopalakrishnan V, Grover H, Visweswaran S. Improving classification performance with discretization on biomedical datasets. In: AMIA Annual Symposium Proceedings, vol. 2008. American Medical Informatics Association: 2008. p. 445. https://www.ncbi.nlm.nih.gov/pubmed/18999186.
Shine JM, Matar E, Ward PB, Bolitho SJ, Gilat M, Pearson M, Lewis SJ. Exploring the cortical and subcortical functional magnetic resonance imaging changes associated with freezing in parkinson’s disease. Brain. 2013; 136:1204–15.
Shine JM, Matar E, Ward PB, Bolitho SJ, Pearson M, Naismith SL, Lewis SJ. Differential neural activation patterns in patients with parkinson’s disease and freezing of gait in response to concurrent cognitive and motor load. PLoS ONE. 2013; 8(1):1204–15.
Bartels A, Leenders K. Brain imaging in patients with freezing of gait. Mov Disord. 2008; 23(Suppl 2):461–7.
Fling B, Cohen R, Mancini M, Nutt J, Fair D, Horak F. Asymmetric pedunculopontine network connectivity in parkinsonian patients with freezing of gait. Brain. 2013; 136:2405–18.
Serrien D, Ivry R, Swinnen S. Dynamics of hemispheric specialization and integration in the context of motor control. Nat Rev Neurosci. 2006; 7:160–6.
Ehgoetz Martens K, Pieruccini-Faria F, Silveira C, Almeida Q. The contribution of optic flow to freezing of gait in left- and right-pd: Different mechanisms for a common phenomenon?.Parkinsonism Relat Disord. 2013; 19:1046–8.
Karachi C, Grabli D, Bernard F, et al.Cholinergic mesencephalic neurons are involved in gait and postural disorders in parkinson disease. J Clin Invest. 2010; 120:2745–54.
Eltoukhy M, Kuenze C, Oh J, Jacopetti M, Wooten S, J S. Microsoft kinect can distinguish differences in over-ground gait between older persons with and without parkinson’s disease. Med Eng Phys. 2017; 44:1–7. https://doi.org/10.1016/j.medengphy.2017.03.007.
Arango Paredes J, Muñoz B, Agredo W, Ariza-Araújo Y, Orozco J, Navarro A. A reliability assessment software using kinect to complement the clinical evaluation of parkinson’s disease. In: Conf Proc IEEE Eng Med Biol Soc. IEEE: 2015. p. 6860–3. https://doi.org/10.1109/EMBC.2015.7319969.
This research work was funded by the Spanish Ministry of Economy and Competitiveness (grant FEDER/TIN2016-78011-C4-2-R). The funding bodies had no role in the design or conclusions of this study.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The project “Analysis of the freezing of gait in Parkinson’s Disease” was approved by Clinical Research Ethics Committee of the Health Area of Gipuzkoa (Protocol Code: FOG-2015). All participants were informed about the study and gave their informed consent. The consent of the patients was obtained in written form, according to the protocol and the Patient Information Sheet and Informed Consent. All patients were of legal age and none were unconscious patients.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Dranca, L., de Abetxuko Ruiz de Mendarozketa, L., Goñi, A. et al. Using Kinect to classify Parkinson’s disease stages related to severity of gait impairment. BMC Bioinformatics 19, 471 (2018). https://doi.org/10.1186/s12859-018-2488-4
- Classification methods
- Bayesian networks
- Parkinson disease