Thursday, 7 April 2016

Sequencing is not counting!

Posted by: Greg Gloor

The idea of using compositional data analysis approaches (CoDa) to examine microbiome samples appears to be gaining traction. There have been a recent spate of papers (and not just from us) that are going in the right direction by acknowledging that we have compositional data.

However, to my mind, most papers do go far enough, and still make strong assumptions about the data. I want to briefly outline why we must embrace the idea of CoDa in its entirety.

Fundamentally, sequencing is not counting. To see this, lets set up a very simple thought experiment. In ecology, it is common to count species within some area, and then to use these counts for the analysis.

In the counting example, we have 5 tigers and 20 ladybugs in some defined area which is a random sample of the environment. We can assume that the animals are free to move both within the box and between the box and the environment, and that the density of animals inside and outside the box is about the same. Therefore, if one more tiger should wander in from the outside, that not alter the number of ladybugs and vice-versa. Moreover, if a space alien happens to land in the counting box, it will likely not alter the count of tigers or ladybugs (unless it lands on and squishes one!). We can see that the abundance of tigers and ladybugs can be essentially treated as uncorrelated. In addition, we can normalize for different box sizes rather simply by scaling the size of the box. So if one student measures a box of size 1, and another a box of size 2, we can adjust for sampling effort fairly simply. This is the rationale behind normalizing for sequencing depth, and if we had counts would work marvelously for sequencing data.


But, this is not the case with sequencing. Every DNA sequencing instrument has a fixed upper limit on the number of reads that can be delivered, and this is illustrated by having exactly 25 cells inside the box. Visualized in this way, we can see the difference in how the same counts of 5 tigers and 20 ladybugs must behave in the face of a perturbation. Either a tiger or a ladybug must be displaced if another tiger wanders onto the sequencing grid. Another way of saying this is that the tiger and ladybug numbers observed are correlated, since an increase in one necessarily involves a decrease in the other. If the space alien now enters the sequencing grid, we see that again we must displace a tiger or a ladybug. This causes all manner of problems for traditional approaches because of spurious correlation, negative correlation bias and sub-compositional correlation effects.

This problem generalizes to any number of species, and is not alleviated by increasing the number of cells in the grid. Attempts to normalize read counts, which are the same as normalizing the number of squares in the grid are doomed to fail because of this property.

So what can we do? We have to embrace the idea that the actual numbers we observe from a sequencing run are irrelevant, and the only information available is relative information: that is, that the ratios between the ladybugs and tigers is the only measurable property. We have begun the process of adapting the full measure of CoDa tools to microbiome studies and our first paper is just out in Annals of Epidemiology: It's all relative: analyzing microbiome data as compositions, and I hope you give it a read. There is also a supplement that contains all the code used to generate the paper.

Although there are many issues to be resolved, mainly because sparse data can be problematic with  CoDa approach, I think that it is worth checking out for your datasets. In a later post, I will outline how the general approach we use to account for the sparsity problem.

Tuesday, 27 October 2015

Wading through the deluge of scientific literature

Posted by: Jean M Macklaim

The amount of science published every day is massive, making it extremely difficult to keep up with literature in your given field. I used to rely on keyword searches and RSS feeds or email updates on specific topics ("microbiome", "bacterial vaginosis", "probiotics", etc.) but this becomes a chore to keep up, and missing a day or two leave you with dozens of emails to catch up with. Not only that, but each weekly update on a topic will consist of 50-80 article titles that you have to read and sort and determine which are relevant enough to follow up on. I stumbled across this article in Nature (How to tame the flood of literature, Elizabeth Gibney) outlining some of these issues and suggested solutions.

So *is* there a better solution? Maybe not a catch-all tool, but here are some tools and software that have helped me out.

Automated recommendation tools


Google Scholar (online)
Google scholar is my primary tool for searching and finding publications. Once you make a scholar profile (linked to your Google account),  you'll get an automated list of your own publications and citations (see mine here), including some metrics on your citation rate. Once you have a profile, Google will automatically suggest publications that are related to your work (they appear under "My Updates"). I find these updates are the most relevant to my research and what I am interested in.



Additionally, you can save articles you find via Google Scholar to "My Library". This keeps a list with links to the articles, along with citation metrics and information. However, this does not replace a citation manager as there is currently no way to export your list of articles, or automatically insert citations into a manuscript.

PubChase (online)
This is a new tool I am using, but I already see the potential. Like Google Scholar, it uses information about what you've published, but with the addition of articles you've saved to your Library as a resource to recommend articles relevant to you. Your Library can be populated by uploading a BibTeX (.bib) file, which nearly any reference manager will export for you.

Pubchase has some other useful features I haven't yet explored. It automatically links your library entries to PDF files (where possible) so you can read papers directly from your account. They also provide tablet, mobile, and computer apps for OSX, iOS, and Android, allowing you to read and access your library from any device. You can also make your library public for other members to follow, or follow someone else's library.


Reference managers

A reference manager is essential for writing a manuscript and keeping track of citations. I don't have a favourite yet, but here are a few I've tried and liked:

Papers (Mac and iOS only; $)
An app you can install on your Mac and iPhone/iPad to access and view citations, and make notes and mark up PDF articles. It had a nice interface, and easily links to most word processing software (citations can be added to Word, plain text apps, Google Docs, etc.). The one major downside is this software has a cost, although there is a student discount rate.

Zotero (Mac, Windows, Linux; free)
Cross-platform, and free. Does the job, but the interface is not as clean as I'd like (seems a little 90s Windows-esque). Ability to insert citations and build reference lists for most document types.

Mendeley (Mac, Windows, Linux, iOS, Android; free)
Another nice looking application. It manages papers and citations. Has more functionality than I need though, and I find myself lost in menus sometimes. Seems to currently only support adding citations and building reference lists in Microsoft Word documents.


Other tools

Evernote (Mac, iOS, Windows, Linux, Android; free)
A multi-multi-purpose digital notebook. I use this for everything in my life, and it has become my digital lab book. You can add notes, PDFs, images, code, basically anything you want. You can compile notes into notebooks and tag topics. You can share with your workmates. It basically keeps my scientific life organized. Did I mention its FREE (I've found no need to pay for the premium subscriptions).


Blogs and social networking sites

Another source I use to keep up-to-date. A number of researchers maintain blogs with current information about their research or research in the field. I recommend using an RSS reader to receive the posts as they become live. Some I recommend:

CommaFeed (online)

Feedly (online, and available apps)

Also, the current Safari and Chrome browsers have RSS functionality built in.

Other sources include Facebook, Twitter, ResearchGate, and Google+. I find it's only useful to follow these if you are already an avid user. It's not feasible for me to check with every social networking site every day, and so I don't rely on these to keep up.


In case you are interested, here are some of the blogs I follow:



Simply Statistics (Jeff Leek, Roger Peng, and Rafa Irizarry)

Bits of DNA (Lior Pachter)




Please comment below on any other software tools that make your research life easier!

Monday, 24 August 2015

Science is hard

Posted by: Jean M Macklaim


An interesting article on Christie Aschwanden on FiveThirtyEight - worth a read!

Wednesday, 20 May 2015

From the blogosphere - batch effects in sequence data

Posted by: Jean M Macklaim

We've performed a number of sequencing runs lately in replicates (replicated in PCR amplification, library preparation, sequencing run, and sequencing platform) to explore the technical variation, including batch effects. We know technical variation can change your results in big ways.

A good example of batch effect has been brought to attention recently:

http://simplystatistics.org/2015/05/20/is-it-species-or-is-it-batch-they-are-confounded-so-we-cant-know/

http://www.the-scientist.com/?articles.view/articleNo/43003/title/Batch-Effect-Behind-Species-Specific-Results-/

In summary, two major papers (1 and 2) nearly a decade apart report the same conclusion: human tissues are more similar in gene expression to any other human tissue than to the equivalent tissue in mouse.

Whoa, what?

So human lung tissue is more similar to human eye tissue than to mouse lung?

As it turns out, this likely isn't true. The observation is due to the confounding batch effect. In other words, what the authors thought was explained by differences in species was actually confounded by the differences in platforms/analyses (in the first case e.g. a different microarray with a different set of probes were used for mouse vs human). The most recent 2014 study has been refuted for confounding batch effect too in at least one publication.

So how does one deal with confounding variables?

The most robust way is to sufficiently randomize your samples so technical parameters don't line up with the biological parameters you are testing, and ensure you maintain a consistent protocol throughout.

This, of course, does not guarantee you won't confound your data, but it should help prevent a systemic bias in sample preparation. If you process all your controls on Tuesdays and your affected samples on Fridays, you won't be able to determine if the effect is due to the condition or the day of the week. As another example, it's important to randomize sample loadings to ensure you don't line up all of condition1 in row 1 of your 96-well plate, and condition2 in row 2 etc. This is where is is essential to collect sufficient metadata about your samples in order to explicitly examine confounding variables.

And of course, ALWAYS look at your data

Tuesday, 12 May 2015

The count you get is just a guideline - part 2

Posted by: Greg Gloor

In previous posts I have shown the effect of sequencing on the underlying shape of the data, and that the data are better thought of as likelihoods of observing a particular count for a particular gene or OTU. Now I'm going to show you the effect of combining those two ideas into an approach to examine datasets of this type that minimizes false positives while maintaining power.

The two-step method is implemented in the ALDEx2 package on Bioconductor. Step one is to model the technical variance by sampling instances from the Dirichlet distribution. This distribution is essentially a multivariate Poisson distribution, and is a good estimator for the underlying technical variation that we have seen in previous blog posts. Step two is to use the centered log-ratio transformation to re-capitulate the underlying shape of the data. These two steps are implemented in the aldex.clr function.

We can now use these transformed instances for standard statistical tests. Below shows an example for a dataset where we are interested in knowing which variants differ in abundance between two conditions. What we want is to find out which variants will give us the same answer regardless of the underlying technical variation. The way we do this is to calculate P values, and Benjamin-Hochberg adjusted P values for each Dirichlet instance, and then to tabulate the mean values. The mean P value for random data should converge on 0.5. The mean P value (and BH adjusted values) for real differences should be relatively invariant to technical replication. That is, we should make the same conclusions if we sequence today or tomorrow.

The figure shows four plots of what we get from a different numbers of Dirichlet instances (number on the left), coupled with a histogram of the P values and the number of BH adjusted values that pass a threshold of <0.05 (number on the right). P values <0.05 are in orange, BH <0.05 are in red. As you can see we start off with 138 'significant' variants with a single instance, this is also about what we get if we just use our raw data. After only 5 Dirichlet instances we have reduced this to 80 'significant' variants, and by 100 or so, we stabilize on a value of 75 or so.


Below is an animated gif that walks through 2000 instances. It is pretty clear that inferred technical variation has a large effect on what is called 'significant', and that we can remove at least some of this stochastic noise quite simply. Also it is clear that the number of significant variants stabilizes and is not affected by further random sampling.

(Click the image to restart the gif animation)

Thursday, 23 April 2015

Weighted UniFrac in all its forms

Posted by: Ruth Grace Wong

It turns out, weighted UniFrac is slightly different depending on what pipeline you’re using. Here’s the lowdown:

Original weighted UniFrac

The formula used in the original UniFrac paper (Lozupone et al. 2005) is thus:
Given a rooted tree for which reads are associated with the leaves on the tree. For each edge of the tree, you can determine the proportional abundance of the reads associated with all its descendant leaves. The weighted UniFrac measure is calculated by multiplying each branch length of the tree with the difference in proportional abundance of its descendants between the two samples, and taking the sum.


The supplementary material includes a proof that weighted UniFrac is a distance metric, by showing that, given samples a, b, and c, the distance between a and b must always be less than or equal to the sum of the distance between a to c and c to b. This satisfies the triangle inequality requirement. The same tree is used for all three distances calculated in the proof, and the distances can range from zero to the total branch lengths * the height of the phylogenetic tree (which is greater than one). Also, as stated in the UniFrac Revealed blog post, this form of weighted UniFrac produces the same distance matrix no matter how the phylogenetic tree is rooted.

Normalized weighted UniFrac

A later publication by the same group (Lozupone et al. 2007) describes a normalized weighted UniFrac, which outputs values between 0 and 1:

The first step in applying weighted UniFrac is to calculate the raw weighted UniFrac value (u), according to the first equation:



If the phylogenetic tree is not ultrametric (i.e., if different sequences in the sample have evolved at different rates), clustering with weighted UniFrac will place more emphasis on communities that contain quickly evolving taxa. Since these taxa are assigned more branch length, a comparison of the communities that contain them will tend to produce higher values of u. In some situations, it may be desirable to normalize u so that it has a value of 0 for identical communities and 1 for nonoverlapping communities. This is accomplished by dividing u by a scaling factor (D), which is the average distance of each sequence from the root, as shown in the equation as follows: This is accomplished by dividing u by a scaling factor (D), which is the average distance of each sequence from the root, as shown in the equation as follows:


Some may prefer the normalization so that the range of possible distance values is the same between experiments. However, this disrupts the triangle inequality property. Take the following example data set with three samples and 12 taxa:

          0   1     2   5    4    8    7   13   10   11   18   15
sample1 1141 330 12382 477  483  504 1809 1373   96   78 4514   63
sample2  697 260  6340 544 1700 2869  469  424   28   62   86 3066
sample3 1696 332  1078 290 2795 6125  434  121 3056 2788   63 5772

The corresponding tree, in Newick format, is

(((7:0.48127,(2:0.01401,13:0.02789)0.840:0.07613)0.626:0.04214,(8:0.21777,5:0.03645)0.881:0.37634)0.747:0.16584,(((((0:0.1161,4:0.20244)0.876:0.07231,1:0.05902)0.726:0.25695,15:0.47267)0.336:0.04988,18:0.01402)0.871:0.23856,(10:0.05189,11:0.03639)0.925:0.20682)0.553:0.00433);

If UniFrac is calculated with the normalization between all three samples, the distance between sample 1 and 3 ( 0.5884264) is 0.0202271 larger than the sum of the distances between samples 1& 2 ( 0.3404242) and 2 & 3 ( 0.2277751). If UniFrac is calculated without the extra division, the distances are: 1 to 2 = 0.09914178, 2 to 3 = 0.0835136, and 1 to 3 = 0.1814773, and the triangle inequality is satisfied, as expected.

Implementation in QIIME (FastUniFrac)

True to form, the Qiime implementation is identical to the original formula

return (branch_lengths * abs((i/float(i_sum))-(j/float(j_sum)))).sum()

If you look at a distance matrix output by Qiime, you will probably find at least one value greater than one. Qiime functions can also be run with normalized weighted UniFrac by using the dist_weighted_normalized_unifrac parameter rather than dist_weighted_normalized_unifrac (see the Qiime documentation for details).

Implementation in Mothur

The mothur weighted unifrac code implements normalized weighted UniFrac, as noted in the Discussion section of the Mothur weighted UniFrac algorithm page.

Implementation in GUniFrac

The GUniFrac paper (Chen et al. 2012) erroneously cites the normalized weighted formula as follows:

The (normalized) weighted UniFrac distance (Lozupone et al., 2007) weights the branch length with abundance difference and is defined as


Instead of using the average distance of each sequence to the root in the denominator, they’ve used the branch length.

They say in their paper that normalized UniFrac overweights long branch lengths and isn’t very sensitive to differential proportional abundance on short branch lengths. To fix this, they propose their own normalized GUniFrac formula:

To attenuate the weight on branches with large proportions, we may instead use the relative difference |piA − piB|/(piA + piB) (∈ [0,1]) in the formulation. We denote this distance measure as


This formulation does not fit the triangle inequality either. When run with the three example microbial communities used above, it yields the same distances as the normalized weighted UniFrac measurement.

Closing  thoughts

To summarize, there are three different types of weighted UniFrac out there (original, normalized, and GUniFrac). The original and normalized version are the most common, and are usually analyzed with Euclidean statistical methods, such as PCoA plots, even though the normalized version is not a Euclidean metric.