Data summary

Data analysis is a huge topic but any data analysis should begin with simple data summary. It is so easy to overlook this step especially when you are dealing with the data is presented in an overly complicated way.

Recently, I was handed longitudinal data collected at a two-time point with 2 years interval between them. The data had about 20 columns including the socio-economic factors, parasite burden and growth indicators which were normalized to z-scores. 

Now, since it was longitudinal data, the attention quickly diverted on how to model the impact of parasites on the growth parameters. Various models including fixed level, random effects, and GEE models were investigated on whether or not they can be used for this kind of data. This took so many hours of research on the web regarding similar data analysis. Eventually, we found that GEE was the appropriate analysis for this kind of analysis (https://stats.stackexchange.com/questions/16390/when-to-use-generalized-estimating-equations-vs-mixed-effects-models). The key word here is that we were interested in "marginal" effect and not conditional effect. We went ahead with the GEE and found the significance for few of the parasites on the growth parameters as the outcome. This looked all good and obvious. 

However, since the parasite burden had highly skewed distribution, we decided to transform it. It was not absolutely necessary for us to go forward with transformation with this analysis.  We went for the transformation anyway. Using rcompanion package and transformtukey function, we found that some values were not transforming at all. Remaining values seems to be "normally" transformed. It turns out that most values were "zeroes". These zero values were not missing or an anomaly in the measurement. These were all genuine zeroes since most of the subjects did not have any parasites.  In fact, about 90-95% of cases had no parasites and here we are trying to find their impact on the growth of these subjects.

This was missed in the beginning because we were overly focused on the longitudinal part.  If we had gone along doing simple data summary, we would have caught it a lot earlier. Lesson learned. 

Now we know the problem, how to deal with this? Do we model them separately which would leave us with only 5% of the cases for the parasite group or do we lump them with the same group? This definitely impacted the statistics of parasite impact.  This lead to another marathon of research on whether on or not to conduct the analysis using GEE with all the data or analyze them separately. No answers have been found yet.  Possible solution include mixture models, mixed level modeling but we have not implemented any of those yet.  I may update this post when we find out conclusively. 

Character Encoding

Google released Colaboratory as a data science tool for the purpose of providing computational and hosting Jupyter notebooks to experiment. It also provides a good demonstration of Tensorflow machine learning library. It saves files as python notebook (.ipynb extension) within a designated folder inside your Google drive account.

I got all excited with this and not to say the least that it provides users with 13 GB of ram, Intel dual-core Xeon processors as shown below. It is rationed from some VM but I am not aware of the internal details.

At the writing of this post, it just provides Python 2 and 3 kernels. R and other languages are supposed to be added in future. It seems they are serious since they have added Jake Vanderplas (http://vanderplas.com/)  as a visiting researcher beginning of this year.
I took a plunge in the Colaboratory by trying to see if I can analyze my local data. There is a Jupyter notebook provided as a documentation to load the data from local machine, Google drive, Google sheets, Google cloud as expected: https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/io.ipynb however it does tell us how can we "use" those files. It seems to be outside the scope of documentation so it was of no help to me. I kept trying to read the file after loading it into an object

##Fail

import pandas as pd

from google.colab import files
uploaded = files.upload()

for fn in uploaded.keys():
   print('User uploaded file "{name}" with length {length} bytes'.format(
   name=fn, length=len(uploaded[fn])))

#To see if the file is in the current folder but I could not find it

!ls

#I loaded the file anyway to see if it was present

pd.read_csv('YCOM-Web2016_2017-11-29.csv')

But there was no file. It kept saying file could not be found. 

The problem was pretty trivial, it turns out that uploaded files are "never" uploaded to the hard drive but are stored in RAM as Python objects and we need to work with those objects as I found the solution  using this stack overflow link: 

https://stackoverflow.com/questions/48340341/read-csv-to-dataframe-in-google-colab

##Parital solution

import pandas as pd
import io

from google.colab import files


uploaded = files.upload()


for fn in uploaded.keys():
   print('User uploaded file "{name}" with length {length} bytes'.format(
   name=fn, length=len(uploaded[fn])))

##Output

YCOM-Web2016_2017-11-29.csv(text/csv) - 1243703 bytes, last modified: 1/19/2018 - %100 done
User uploaded file "YCOM-Web2016_2017-11-29.csv" with length 1243703 bytes

##End of output

df = pd.read_csv(io.StringIO(uploaded['YCOM-Web2016_2017-11-29.csv'].decode('utf-8')))

df

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 505403: invalid start byte

What is this UnicodeDecodeError? It turns out that my file did not have utf-8 encoding which I supposed. I needed to convert the io object string into file format which Pandas library could swallow. Again, google come to the rescue, I got this page where we have the list of different encodings: https://docs.python.org/3/library/codecs.html#standard-encodings

for which Python 3 has the support. 

I changed my line in the code to 

df = pd.read_csv(io.StringIO(uploaded['YCOM-Web2016_2017-11-29.csv'].decode('ISO-8859-1')))
df

and it loaded with full glory!

Notice we have the argument to decode changed from utf-8 to ISO-8859-1 which allows characters not supported by utf-8. I am still mystified since, the file was supposed to be a plain old csv file but well for the future references, always check your file encoding using this command:

file -I file_name.csv

which on my mac shows up as "unknown-8bit".  Not very helpful but we do know that it is not plain old "us-ascii". 

TLDR: Check your encoding before you load the file and do not make quick assumptions especially when working with new systems!

Dealing with low sample size significance testing

Recently I had to analyze data with very few data points in the range of 3-15. The data consisted of 3 groups and multiple subgroups. The most obvious choice, in this case, was to use a non-parametric statistical test such as Wilcox test. The problem with the Wilcox test is that we have the problem of losing power/sensitivity. A t-test, on the other hand, may give us false-positive especially with the sample size of 3. How do we deal with this? This issue is exacerbated especially for p-value calculation. P-values seems to be necessary "evil" but below are the points to address this problem.

These are some of the links which helped me to understand this issue:
https://stats.stackexchange.com/questions/14434/appropriateness-of-wilcoxon-signed-rank-test
https://stats.stackexchange.com/questions/2492/is-normality-testing-essentially-useless/2498#2498
https://stats.stackexchange.com/questions/121852/how-to-choose-between-t-test-or-non-parametric-test-e-g-wilcoxon-in-small-sampl

I was intrigued further to carry informal "research" and found possible options.

• If we cannot determine normality, should we just use t-test anyways? 
• The idea is that this experiment is to detect potential vaccine candidate in the very preliminary stage so we need to be slightly "lenient" and err on the side of allowing few false positives.

• Just select Wilcox.test since it is appropriate for non-parametric and enough power to detect a difference
• https://stats.stackexchange.com/a/66235/124490

• Use bootstrapped values: 
• http://biostat.mc.vanderbilt.edu/wiki/pub/Main/JenniferThompson/ms_mtg_18oct07.pdf
• Requires more than 8 samples.
• https://stats.stackexchange.com/questions/33300/determining-sample-size-necessary-for-bootstrap-method-proposed-method
• However, some say we may require more than 20
• https://speakerdeck.com/jakevdp/statistics-for-hackers

• Using Permutation test:
• It works with a fewer sample size as compared to bootstrapping but it cannot generate confidence interval. 
• In fact, Wilcox test is a subset of a permutation test. 

• Plainly displaying the data points with a confidence interval.

• Using Effect size to illustrate the "significance". Site: https://garstats.wordpress.com/2016/05/02/robust-effect-sizes-for-2-independent-groups/
• Some of the recommendations include 

• Cohens.d (Not to use for non-normal/non-parametric data) 
• Cliff's delta (Non-parametric ordinal data)
• Mutual information (MI)
• Kolmogorov-Smirnov
• Wilcox & Muska’s Q

• Equivalence test. Site: https://support.minitab.com/en-us/minitab/18/help-and-how-to/statistics/equivalence-tests/supporting-topics/why-use-an-equivalence-test/ 
• This option requires knowing the "difference" which has some biological/Clinical significance.

Overall, this gives us many options but still, this is no panacea. The small sample size is a very difficult issue and these "solutions" can help to minimize the pain.