Crowdsourcing science project for phylogenies?

Image via WikipediaThe idea behind crowdsourcing is that the answer to a question is often more likely to be correct if you average the answers from a large number of non-experts rather than a single expert in the field. The term "crowdsourcing" has also been used for projects that outsource repetitive or challenging work to a crowd via the internet.
I have been thinking of outsourcing the problem of conversion of embedded phylogenies in PDFs back to newick/nexus format and have been looking at various science projects that have used crowdsourcing.

The most impressive from my point of view is Galaxy Zoo which has already resulted in a number of publications and impressive discoveries. Astrophysicist use the crowd to categorise 1000s of galaxies and have expanded the crowd tasks to include matching images of galaxies with randomly simulated images.

Stardust@Home is another astrophysics project which asks that the crowd looks through images for dust particles brought back to earth by a spacecraft in 2006.

Another cool project is the Open Dinosaur Project which asks that the crowd aggregates published measurements of dinosaur limb bones for many different taxa from the literature and directly measured from specimens to study the evolutionary transitions from bipedality to quadrupedality.

Foldit is a computer game enabling the crowd to contribute to our understanding of how protein folds. Figuring out which of the many, many possible structures is the best one is regarded as one of the hardest problems in biology today and current methods take a lot of money and time, even for computers. The idea of using human's spare time to get further insight is genius!

Another game that might not be directly relevant to science is Google Image Labeler which I found rather addictive. Google gets users to label/tag images as a side-effect of playing a game and this is probably used to improve image searches on the web. I list it hear because I came across a few images of animals that in some cases were labeled down to the latin binomial.

UPDATE: An interesting new crowd sourcing project at http://www.oldweather.org/ to help gather information about past climates from hand written nautical records.

Hacking the JPEG/PDF tree format

Just back from giving a presentation at the Scottish Phylogenetic Discussion group in Edinburgh. Nice mix of talks. I think I got a lot of people thinking and talking about the way we are doing things and how we could improve the way we do things. The slides are available on Nature Precedings but it is a bit slow to load, so I have added it to slideshare as well.

Hacking the JPEG/PDF tree format

View more presentations from josephhughes.

The phyloscape changes quickly, we need to build a better way to keep track of it

Since Hennig's 1969 major publication on the phylogeny of hexapod orders (Insecta + Entognatha), I have found more than 60 publications of phylogenies on the ordinal relationships within the group. Some may be re-analyses of the same data but it is still quite a large number of studies. Thirty-seven of these have been published in the last decade and this rapid change in the phylogenetic landscape of this group (and this is probably the case for many other lineages) is increasingly becoming hard to keep track of. Sure, you could do a regular Pubmed or WoS search for phylogen* + insecta but you then need to extract the phylogeny and put it in the context of previously published studies. Sure there are databases like TreeBase and PhyLoTa that provide ready-made phylogenetic reconstructions but the former has limited content and the latter has limited resolution at many nodes of interest.

It is important to have an up-to-date and complete image of the phylogenetic landscape of the groups we work on, even if the overall picture is blurry. This would provide a better idea of areas that require further taxonomic sampling and/or a larger number of characters to resolve the relationships of interest, it would also provide a valuable resource for comparative studies. For this to work, information needs to be integrated between different databases like PhyLoTA, TreeBASE, GenBank, Treefam etc. in an automated fashion as well as defrosting phylogenetic reconstructions from previously published studies (see my previous post). Perhaps a simple repository of third-party phylogenetic reconstructions would help: submitter - publication reference - figure number - phylogeny (newick, nexus, phyloxml, nexml ....). Although would anybody submit data? Perhaps I need to think of a way to reward those that do/

Reference

Hennig, W. 1969. Die Stammesgeschichte der Insekten. Frankfurt am Main, Germany: Kramer.

TreeRipper: towards a fully automated optical tree recognition software

Unfortunately my TreeRipper program has been rejected from BMC bioinformatics for now because there are too many delegate programs that the reviewers didn't manage to install successfully. So until I manage to find time to make a makefile that can deal with the installation on multiple platforms, I have put the manuscript on Nature precedings and you can find the code at google code. I think I am the first to attempt to fully automate the conversion of a tree image into something more useful for researchers and I hope that what I have done can be built upon and improved. I have attached to the code a set of images and tree files that might be useful for training and/or benchmarking future programs.

Why I think a Phylogenetic Standard might not work

Currently we have many different formats for phylogenies (e.g., nexus, newick, extend newick, phylip, xml) and there is no doubt that putting in place a Phylogenetic Standard would be a good thing but just because a group of researchers get together and decide that a Phylogenetic Standard would be good doesn't make it happen. Unless the phylogenetic community is behind the decisions being made which is unlikely to happen any time soon as phylogeneticist already have a poor record of submitting their phylogenies to databases like TreeBase.
These are obviously points that you are aware of as you mention them in your statement. I only remark that unless publishers enforce a particular format and submission of the phylogenetic data into repositories (like GenBank/EMBL), then however noble the idea of a Phylogenetic Standard is, it is unlikely to be put into practice by phylogeneticists.

Installing tesseract command line OCR on MacOS X

Installing libpng from source:
http://kenno.wordpress.com/2006/04/20/compiling-libpng-for-mac-os-x/

fink install libjpeg, aspell, aspell-en

I will want to create my own aspell dictionary using taxonomic names:
http://www.mail-archive.com/code4lib@listserv.nd.edu/msg01545.html

Download and installing tesseract following install instructions:
http://code.google.com/p/tesseract-ocr/downloads/list

fink xpdf for pdfimages to extract images from a pdf:
>pdfimages -j LandPlants_paper.pdf LandPlantImg

To convert in imagemagick to tif for tesseract :
convert LandPlantImg.jpg -compress None test.tif

Using tesseract:
tesseract test.tif out.txt

I have now got a script to extract the names and check them against a dictionary of taxonomic names from spira.
I am thinking that using information from the article itself might provide even better results. When tesseract 2.0 comes out, there will also be a way of training the program to improve the character recognition. OCRupus also looks like an interesting program for layout detection but it doesn't work on MacOSx yet
The line extraction is proving to be much more difficult than first thought mainly because the lack of consistn format and the labelling at the nodes that get in the way of edge detection. I have tried a number of methods for cleaning up the image and bit by bit I will get there, I hope.

Branch thinning and tracking

I think I might have found a way to convert phylogenetic trees from image to nexus format through a process similar to that used in GIS map vectorization. This approach should also enable me to deal with trichotomies. The pattern matching approach that I had used previously turned out to be unsuccessful because of the many inconsistencies in tree drawing making it difficult to find a pattern that would always match a tip or a node. Line tracking offers hope!
All this would be so unnecessary if only researcher submitted their phylogenies to TREEBASE. Still, there are a number of phylogenies published prior to Treebase.