Fig. 3
From: Visualizing complex feature interactions and feature sharing in genomic deep neural networks
Input-dependent visualizations produce unstable results on XOR logic and fail to capture the XOR interaction. Three types of input-dependent visualizations on example positive and negative sequence from synthetic data set I. The visualization using positive example (left) only highlight two of the 3 predefined patterns because a positive sample can only contain one of GCTCAT,CGCTTG, while the third pattern will always be missing. When using negative example which contains all three patterns as the input, all of the methods assign either all positive or all negative scores to the three patterns (right), failing to capture the XOR interaction between GCTCAT and CGCTTG. The saliency methods predict negative score for CAGGTC, a pattern that should always exists in positive examples, suggesting that these methods are not stable enough when dealing with complex logic