Re: [Tdwg-phylo] Publishing a tree in RDF
Yes, Triplify was the inspiration to the web2py plugin.
Also, I've been working on a phylogentic tree editor called tred. http://code.google.com/p/tred/
Its still a little rough, but usable/stable. Realizing that many Biologists do not have their data in a relational database one could upload their newick, nexus, etc to tred, edit the tree then export to CDAO. I'd be happy to add functionality to tred for this purpose if people will use it.
On Wed, Oct 20, 2010 at 5:00 AM, tdwg-phylo-request@lists.tdwg.org wrote:
Send tdwg-phylo mailing list submissions to tdwg-phylo@lists.tdwg.org
To subscribe or unsubscribe via the World Wide Web, visit http://lists.tdwg.org/mailman/listinfo/tdwg-phylo or, via email, send a message with subject or body 'help' to tdwg-phylo-request@lists.tdwg.org
You can reach the person managing the list at tdwg-phylo-owner@lists.tdwg.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of tdwg-phylo digest..."
Today's Topics:
- Re: Publishing a tree in RDF (Hilmar Lapp)
Message: 1 Date: Tue, 19 Oct 2010 17:27:21 -0400 From: Hilmar Lapp hlapp@nescent.org Subject: Re: [Tdwg-phylo] Publishing a tree in RDF To: Chris Baron topher.baron@gmail.com Cc: "tdwg-phylo@lists.tdwg.org" tdwg-phylo@lists.tdwg.org Message-ID: FE252B2A-D7C2-4873-B5B0-4695D382CABE@nescent.org Content-Type: text/plain; charset="us-ascii"
Hi Chris - you probably know the Triplify project? http://triplify.org/
-hilmarOn Oct 19, 2010, at 1:03 PM, Chris Baron wrote:
I've written a plugin to transform relational database data into RDF : http://www.web2py.com/semantic
If anyone keeps their data in a relational database, this would be useful.
On Tue, Oct 19, 2010 at 11:58 AM, Arlin Stoltzfus arlin@umd.edu wrote: Chris--
It sounds like you have the skills and interest to do this. Our first goal is very modest: just assess what's possible given currently available tools. I only know about one so far, which will translate a file into CDAO with the option to add it to a triple store. A web server interface is here:
http://www.cs.nmsu.edu/~cdaostore/phylows.phphttp://www.cs.nmsu.edu/%7Ecdaostore/phylows.php
If you know about other tools, that would be great.
The next step (weeks from now) would be to look ahead to the future and identify gaps. If you are interested, all you need to do is to sign up for TDWG twiki access (if you don't have it already) and start working.
Arlin
On Oct 19, 2010, at 12:33 PM, Chris Baron wrote:
Hi,
My name is Chris Baron - a software developer, computer science graduate student interested in linked data, working in BioSync at the Field Museum on tree visualization. I have a few ideas on how to publish trees in RDF (have even created a tool to do so) and would like to speak to Biologists about how their data exists at publication so I can form a proper solution.
On Tue, Oct 19, 2010 at 5:00 AM, <tdwg-phylo- request@lists.tdwg.org> wrote: Send tdwg-phylo mailing list submissions to tdwg-phylo@lists.tdwg.org
To subscribe or unsubscribe via the World Wide Web, visit http://lists.tdwg.org/mailman/listinfo/tdwg-phylo or, via email, send a message with subject or body 'help' to tdwg-phylo-request@lists.tdwg.org
You can reach the person managing the list at tdwg-phylo-owner@lists.tdwg.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of tdwg-phylo digest..."
Today's Topics:
- publishing trees in CDAO as linked open data? (Arlin Stoltzfus)
Message: 1 Date: Mon, 18 Oct 2010 12:32:39 -0400 From: Arlin Stoltzfus arlin@umd.edu Subject: [Tdwg-phylo] publishing trees in CDAO as linked open data? To: CDAO list cdao-discuss@lists.sourceforge.net, Phylogenetics Standards Interest Group tdwg-phylo@lists.tdwg.org Message-ID: BAAFA68C-2622-409A-B530-1CABADC36016@umd.edu Content-Type: text/plain; charset="US-ASCII"; format=flowed; delsp=yes
Dear all--
At the Biodiversity Information Standards (TDWG) meeting last month, a sub-group of the phylogenetic standards working group started a project to assess "current best practices for publishing trees electronically". At present, probably the best way to publish a tree is to use the TreeBASE submission process, which makes it possible to upload a NEXUS file and create associations of OTUs with taxon IDs, and of data rows with accessions, and so on. But there are also some other possibilities. The preliminary report is being assembled here:
http://wiki.tdwg.org/twiki/bin/view/Phylogenetics/LinkingTrees2010
Our plan is (over the next 6 weeks) to get feedback on this preliminary report, then analyze the results and prepare a more extensive report for publication.
One possibility that we haven't examined yet is to publish a tree electronically by rendering it in RDF in the language of CDAO (Comparative Data Analysis Ontology) and then contributing it to the Linked-Open-Data cloud in some way.
Does anyone out there want to explore this idea and contribute the results to the TDWG project? If so, please let me know.
Arlin
Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
End of tdwg-phylo Digest, Vol 2, Issue 3
-- Chris Baron <ATT00001.txt>
Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
-- Chris Baron _______________________________________________ tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
--
: Hilmar Lapp -:- Durham, NC -:- informatics.nescent.org :
On Oct 20, 2010, at 2:43 PM, Chris Baron wrote:
Also, I've been working on a phylogentic tree editor called tred. http://code.google.com/p/tred/
Interesting. How does your editor compare to the dozens of tree editors and visualizers already available. For example PhyloWidget, or jsPhyloSVG?
I may sound like a broken record here, and I don't want to discourage you at all from developing tools for phylogenetics. We certainly need more developers. I also do think though that as a community we make the most progress if we aren't sprouting new tools for the same purpose all the time.
-hilmar
The purposes of tree visualization are many and varied, so it seems natural to me that tool development reflects that diversity. One distinction of tred is that it exposes server-side and client-side functionality.
-Rick
On Wed, Oct 20, 2010 at 3:17 PM, Hilmar Lapp hlapp@nescent.org wrote:
On Oct 20, 2010, at 2:43 PM, Chris Baron wrote:
Also, I've been working on a phylogentic tree editor called tred. http://code.google.com/p/tred/
Interesting. How does your editor compare to the dozens of tree editors and visualizers already available. For example PhyloWidget, or jsPhyloSVG? I may sound like a broken record here, and I don't want to discourage you at all from developing tools for phylogenetics. We certainly need more developers. I also do think though that as a community we make the most progress if we aren't sprouting new tools for the same purpose all the time.
-hilmar
=========================================================== : Hilmar Lapp -:- Durham, NC -:- informatics.nescent.org : ===========================================================
I am yet to consider the visualizing RDF trees, however, I have am interest in visualizing and interacting with trees and networks in a web context.
I've been looking at the various web tree visualizers, as well as more generic visualization libraries, for the last week or so. Understanding the functionality, maturity, user community and inter-browser compatability is time consuming, and not always simple to determine.
Specifically, I am looking for a tool that supports, or can extended to support;
1. Both both trees and networks, 2. The selection of multiple nodes, e.g. subtrees 3. The browsing of large trees/networks - I get some weird effects with large trees in jsPhyloSVG. 4. (Readable) Labels at all nodes. 5. Alternate layouts (circular, hyperbolic, etc.) 6. Works in as many browsers as possible. 7. A variety of I/O formats - I store my trees in PhyloDB, which I am thinking of generalizing for networks.
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
I would be very interested in any comments regarding existing tools and the above issues (an others I have not thought about). I would be happy to compile all comments and perhaps some comparative demos somewhere.
- Dave
David M. Kidd
Research Associate Center for Population Biology Silwood Park Campus Imperial College London 0207 594 2470
-----Original Message----- From: tdwg-phylo-bounces@lists.tdwg.org [mailto:tdwg-phylo-bounces@lists.tdwg.org] On Behalf Of Richard Ree Sent: 20 October 2010 21:47 To: Hilmar Lapp Cc: tdwg-phylo@lists.tdwg.org Interest Group; Chris Baron Subject: Re: [Tdwg-phylo] Publishing a tree in RDF
The purposes of tree visualization are many and varied, so it seems natural to me that tool development reflects that diversity. One distinction of tred is that it exposes server-side and client-side functionality.
-Rick
On Wed, Oct 20, 2010 at 3:17 PM, Hilmar Lapp hlapp@nescent.org wrote:
On Oct 20, 2010, at 2:43 PM, Chris Baron wrote:
Also, I've been working on a phylogentic tree editor called tred. http://code.google.com/p/tred/
Interesting. How does your editor compare to the dozens of tree editors and visualizers already available. For example PhyloWidget, or jsPhyloSVG? I may sound like a broken record here, and I don't want to discourage you at all from developing tools for phylogenetics. We certainly need more developers. I also do think though that as a community we make the most progress if we aren't sprouting new tools for the same purpose all the time.
-hilmar
=========================================================== : Hilmar Lapp -:- Durham, NC -:- informatics.nescent.org : ===========================================================
_______________________________________________ tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
Hi Dave,
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
Rod Page has coded a web widget (I believe all javascript) that has a small preview window for the whole tree and a larger "zoomed in" view. "TreeJuxtaposer" is a java app(let?) that allows you to contract and expand selections of nodes/clades. I think these come closest to what you are talking about, though neither operates on networks.
Cheers,
Rutger
Hi, Dave and Rutger:
My own tree viewer "Archaeopteryx" provides such an overview when zoomed in, plus some other features described as "missing" in most current tools.
See: http://www.phylosoft.org/archaeopteryx/
Example: http://www.phylosoft.org/archaeopteryx/examples/mollusca.html
Archaeopteryx also provides other useful features (at least for comparative genomics use cases). For example, the ability to infer internal taxonomies (if all external nodes have _some_ taxonomic information associated with them; standalone version only; via uniprot taxonomy database).
Please let me know if you'd like to know more or have suggestions for improvement (although keep in mind that this Archaeopteryx is just a peculiar hobby of mine).
Christian
On 10/21/2010 3:39 AM, Rutger Vos wrote:
Hi Dave,
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
Rod Page has coded a web widget (I believe all javascript) that has a small preview window for the whole tree and a larger "zoomed in" view. "TreeJuxtaposer" is a java app(let?) that allows you to contract and expand selections of nodes/clades. I think these come closest to what you are talking about, though neither operates on networks.
Cheers,
Rutger
QED. The technology is out there. IMHO, the continual propagation of new tree-viewers by developers, and the sense of users that there is no tree viewer that satisfies their needs, is (in the medium-term and long-term) a cultural-organizational-educational problem and not at all a technical problem.
If this is true, then in order to solve the real problem, we need to think about things like changes in funding structure, standards development, and modes of user engagement, not new graphics libraries that do just the right thing with only a few commands.
Arlin
On Oct 21, 2010, at 12:57 PM, Christian M Zmasek wrote:
Hi, Dave and Rutger:
My own tree viewer "Archaeopteryx" provides such an overview when zoomed in, plus some other features described as "missing" in most current tools.
See: http://www.phylosoft.org/archaeopteryx/
Example: http://www.phylosoft.org/archaeopteryx/examples/mollusca.html
Archaeopteryx also provides other useful features (at least for comparative genomics use cases). For example, the ability to infer internal taxonomies (if all external nodes have _some_ taxonomic information associated with them; standalone version only; via uniprot taxonomy database).
Please let me know if you'd like to know more or have suggestions for improvement (although keep in mind that this Archaeopteryx is just a peculiar hobby of mine).
Christian
On 10/21/2010 3:39 AM, Rutger Vos wrote:
Hi Dave,
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
Rod Page has coded a web widget (I believe all javascript) that has a small preview window for the whole tree and a larger "zoomed in" view. "TreeJuxtaposer" is a java app(let?) that allows you to contract and expand selections of nodes/clades. I think these come closest to what you are talking about, though neither operates on networks.
Cheers,
Rutger
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
------- Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
Thank you Rutger, Christian, Cindy, Hilmar and Arlin,
Whatever the technological maturity of tools, there remains the hurdle of finding the viewers, assessing their functionality and assessing whether a task can be completed with the available tools given a time-scale and skill set. I have just started to put together a comparison site describing functionality and where the same trees can be viewed in the different viewers.
Sorry for interrupting the RDF flow - it has been very interesting to follow.
- Dave
David M. Kidd
Research Associate Center for Population Biology Silwood Park Campus Imperial College London 0207 594 2470
-----Original Message----- From: tdwg-phylo-bounces@lists.tdwg.org [mailto:tdwg-phylo-bounces@lists.tdwg.org] On Behalf Of Arlin Stoltzfus Sent: 22 October 2010 15:30 To: tdwg-phylo@lists.tdwg.org Interest Group Subject: Re: [Tdwg-phylo] Publishing a trees in RDF
QED. The technology is out there. IMHO, the continual propagation of new tree-viewers by developers, and the sense of users that there is no tree viewer that satisfies their needs, is (in the medium-term and long-term) a cultural-organizational-educational problem and not at all a technical problem.
If this is true, then in order to solve the real problem, we need to think about things like changes in funding structure, standards development, and modes of user engagement, not new graphics libraries that do just the right thing with only a few commands.
Arlin
On Oct 21, 2010, at 12:57 PM, Christian M Zmasek wrote:
Hi, Dave and Rutger:
My own tree viewer "Archaeopteryx" provides such an overview when zoomed in, plus some other features described as "missing" in most current tools.
See: http://www.phylosoft.org/archaeopteryx/
Example: http://www.phylosoft.org/archaeopteryx/examples/mollusca.html
Archaeopteryx also provides other useful features (at least for comparative genomics use cases). For example, the ability to infer internal taxonomies (if all external nodes have _some_ taxonomic information associated with them; standalone version only; via uniprot taxonomy database).
Please let me know if you'd like to know more or have suggestions for improvement (although keep in mind that this Archaeopteryx is just a peculiar hobby of mine).
Christian
On 10/21/2010 3:39 AM, Rutger Vos wrote:
Hi Dave,
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
Rod Page has coded a web widget (I believe all javascript) that has a small preview window for the whole tree and a larger "zoomed in" view. "TreeJuxtaposer" is a java app(let?) that allows you to contract and expand selections of nodes/clades. I think these come closest to what you are talking about, though neither operates on networks.
Cheers,
Rutger
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
------- Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
_______________________________________________ tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
There are other issues here as well. Technology advances rapidly, and what once seemed a good choice may rapidly become dated. Without wishing to start a flame war, I think Java applets are dead (sorry Phylowidget), Flash is fading away (thank you Steve), and increasingly we will see lightweight SVG and Canvas-based browsers, non-web based browsers that exploit specific hardware (iPad anyone), and Google- Earth based browsers.
I doubt one-size-fits all will work. Standards themselves won't address this issue, and I personally doubt RDF is where we want to be focussing efforts anyway. A decent JSON format for trees and associated metadata would be much more palatable for developers. It's worth remembering that the great success of Newick (and Nexus) was largely due to the ease of parsing.
Regards
Rod
On 22 Oct 2010, at 15:29, Arlin Stoltzfus wrote:
QED. The technology is out there. IMHO, the continual propagation of new tree-viewers by developers, and the sense of users that there is no tree viewer that satisfies their needs, is (in the medium-term and long-term) a cultural-organizational-educational problem and not at all a technical problem.
If this is true, then in order to solve the real problem, we need to think about things like changes in funding structure, standards development, and modes of user engagement, not new graphics libraries that do just the right thing with only a few commands.
Arlin
On Oct 21, 2010, at 12:57 PM, Christian M Zmasek wrote:
Hi, Dave and Rutger:
My own tree viewer "Archaeopteryx" provides such an overview when zoomed in, plus some other features described as "missing" in most current tools.
See: http://www.phylosoft.org/archaeopteryx/
Example: http://www.phylosoft.org/archaeopteryx/examples/ mollusca.html
Archaeopteryx also provides other useful features (at least for comparative genomics use cases). For example, the ability to infer internal taxonomies (if all external nodes have _some_ taxonomic information associated with them; standalone version only; via uniprot taxonomy database).
Please let me know if you'd like to know more or have suggestions for improvement (although keep in mind that this Archaeopteryx is just a peculiar hobby of mine).
Christian
On 10/21/2010 3:39 AM, Rutger Vos wrote:
Hi Dave,
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
Rod Page has coded a web widget (I believe all javascript) that has a small preview window for the whole tree and a larger "zoomed in" view. "TreeJuxtaposer" is a java app(let?) that allows you to contract and expand selections of nodes/clades. I think these come closest to what you are talking about, though neither operates on networks.
Cheers,
Rutger
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
--------------------------------------------------------- Roderic Page Professor of Taxonomy Institute of Biodiversity, Animal Health and Comparative Medicine College of Medical, Veterinary and Life Sciences Graham Kerr Building University of Glasgow Glasgow G12 8QQ, UK
Email: r.page@bio.gla.ac.uk Tel: +44 141 330 4778 Fax: +44 141 330 2792 AIM: rodpage1962@aim.com Facebook: http://www.facebook.com/profile.php?id=1112517192 Twitter: http://twitter.com/rdmpage Blog: http://iphylo.blogspot.com Home page: http://taxonomy.zoology.gla.ac.uk/rod/rod.html
On Oct 22, 2010, at 11:00 AM, Roderic Page wrote:
There are other issues here as well. Technology advances rapidly, and what once seemed a good choice may rapidly become dated
True.. but another aspect is just that in order to explore whether a particular graphical concept or tool will work, or will benefit people, you have to knock out prototypes and implementations. e.g., drag-and-drop branch rearrangement in MacClade was great for small trees but not for big ones, hence phylowidget was looking to implement something that worked for large trees (via concentric menus on nodes) and to experiment with auto-pruing, etc. People have to keep knocking out GUI software efforts to test the waters for new features. And the search is not over, for example:
- Do we really have a completely satisfying way of visualizing patterns of gene duplication within a species tree? - If I handed you a tree with 200k nodes, is there a visual / GUI way that would allow you to easily find interesting patterns in it, such as points of incongruence with a conventional taxonomy? - If I gave you 1,000 trees, is there a visual / GUI way that would allow you to see which branches in each tree probably crossed an ancient land bridge together?
So I think we still have lots more GUI / visual / ergonomic challenges that still need to be solved (even if lots of challenges have already been solved). And we should not expect each new idea to be implemented in a pristine killer app that does everything that everyone wants out of a tree visualizer -- that takes too long to build, and the right person for dreaming up a new idea is not necessarily the best person for creating clean, robust, off-the-shelf software.
We need to be okay with seeing a rich plethora of quick-and-dirty efforts, each focusing on articulating/investigating sets of novel ideas -- despite some obvious redundancy in some of the more general functions. And then periodically, someone's got to assemble the best of these ideas into a robust, jack-of-all-trades, off-the-shelf software package.
bp
On Oct 22, 2010, at 12:22 PM, William Piel wrote:
We need to be okay with seeing a rich plethora of quick-and-dirty efforts, each focusing on articulating/investigating sets of novel ideas -- despite some obvious redundancy in some of the more general functions. And then periodically, someone's got to assemble the best of these ideas into a robust, jack-of-all-trades, off-the-shelf software package.
That sounds nice indeed, but frankly I've never seen it happen. The quick-and-dirty efforts that I have seen become standard parts of reusable and sustainable software have almost all started from their early beginnings as parts of reusable software. For a while rather experimental and unstable parts, obviously, but the future course was charted not as an afterthought.
Truth of the matter is that we're all scientists. No scientist is interested in, or gets promotion, scientific recognition, or publications from assembling entirely incompatible, mostly redundant except for a few features, quick-and-dirty pieces of software into some grand jack-of-all-trades software package.
My $0.02, from about 13 years of software engineering.
-hilmar
This discussion doesn't really belong here, but I'm not sure where else it belongs, and I sure would hate to stop, because its interesting and valuable.
Hilmar, are you referring to various quick-and-dirty BioPerl modules, for instance, as cases in which software that ended up being reusable started out as part of a reusable software package? Are there other kinds of contexts where this would work? How could we get different developers working on the same architecture, building a shared foundation but creating different solutions from it?
One idea used in nexplorer3 (and in some other viewers) is a Model- View-Controller design pattern that separates the business logic from the user interface.
It seems to me that ATV has been pretty successful. I've seen half a dozen sites that use it (that's a success by my standards).
Arlin
On Oct 22, 2010, at 12:38 PM, Hilmar Lapp wrote:
On Oct 22, 2010, at 12:22 PM, William Piel wrote:
We need to be okay with seeing a rich plethora of quick-and-dirty efforts, each focusing on articulating/investigating sets of novel ideas -- despite some obvious redundancy in some of the more general functions. And then periodically, someone's got to assemble the best of these ideas into a robust, jack-of-all-trades, off-the-shelf software package.
That sounds nice indeed, but frankly I've never seen it happen. The quick-and-dirty efforts that I have seen become standard parts of reusable and sustainable software have almost all started from their early beginnings as parts of reusable software. For a while rather experimental and unstable parts, obviously, but the future course was charted not as an afterthought.
Truth of the matter is that we're all scientists. No scientist is interested in, or gets promotion, scientific recognition, or publications from assembling entirely incompatible, mostly redundant except for a few features, quick-and-dirty pieces of software into some grand jack-of-all-trades software package.
My $0.02, from about 13 years of software engineering.
-hilmar
=========================================================== : Hilmar Lapp -:- Durham, NC -:- informatics.nescent.org : ===========================================================
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
------- Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
On Oct 22, 2010, at 1:06 PM, Arlin Stoltzfus wrote:
Hilmar, are you referring to various quick-and-dirty BioPerl modules, for instance, as cases in which software that ended up being reusable started out as part of a reusable software package?
Yes.
Are there other kinds of contexts where this would work? How could we get different developers working on the same architecture, building a shared foundation but creating different solutions from it?
I think I gave some examples in my other email. The key is that contributing be openly welcome, have very little barriers that aren't directly related to the code itself, and have an advantage for the developer. The advantage, for example, to adding on to BioPerl is that as a developer there's an awful lot of code I can easily reuse for all the mundane stuff, my code will be seen by and be useful to a lot more people, and I can be part of a community that I can learn from or exchange ideas with.
-hilmar
Both technology and users' needs evolve rapidly - so the "gaps" will never be closed, at least in the time frame required by grants, etc. Also, in phylogenetics a lot of developers are themselves users, so quick and dirty solutions to scratch individual itches is inevitable. It's a good thing, IMO.
-Rick
On Fri, Oct 22, 2010 at 11:22 AM, William Piel william.piel@yale.edu wrote:
On Oct 22, 2010, at 11:00 AM, Roderic Page wrote:
There are other issues here as well. Technology advances rapidly, and what once seemed a good choice may rapidly become dated
True.. but another aspect is just that in order to explore whether a particular graphical concept or tool will work, or will benefit people, you have to knock out prototypes and implementations. e.g., drag-and-drop branch rearrangement in MacClade was great for small trees but not for big ones, hence phylowidget was looking to implement something that worked for large trees (via concentric menus on nodes) and to experiment with auto-pruing, etc. People have to keep knocking out GUI software efforts to test the waters for new features. And the search is not over, for example:
- Do we really have a completely satisfying way of visualizing patterns of gene duplication within a species tree?
- If I handed you a tree with 200k nodes, is there a visual / GUI way that would allow you to easily find interesting patterns in it, such as points of incongruence with a conventional taxonomy?
- If I gave you 1,000 trees, is there a visual / GUI way that would allow you to see which branches in each tree probably crossed an ancient land bridge together?
So I think we still have lots more GUI / visual / ergonomic challenges that still need to be solved (even if lots of challenges have already been solved). And we should not expect each new idea to be implemented in a pristine killer app that does everything that everyone wants out of a tree visualizer -- that takes too long to build, and the right person for dreaming up a new idea is not necessarily the best person for creating clean, robust, off-the-shelf software.
We need to be okay with seeing a rich plethora of quick-and-dirty efforts, each focusing on articulating/investigating sets of novel ideas -- despite some obvious redundancy in some of the more general functions. And then periodically, someone's got to assemble the best of these ideas into a robust, jack-of-all-trades, off-the-shelf software package.
bp
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
Inevitable. Good thing inded. But not enough to make significant progress and fill the gaps. I agree with Hilmar.
Nico
On Oct 22, 2010, at 12:39 PM, Richard Ree wrote:
Both technology and users' needs evolve rapidly - so the "gaps" will never be closed, at least in the time frame required by grants, etc. Also, in phylogenetics a lot of developers are themselves users, so quick and dirty solutions to scratch individual itches is inevitable. It's a good thing, IMO.
-Rick
On Fri, Oct 22, 2010 at 11:22 AM, William Piel william.piel@yale.edu wrote:
On Oct 22, 2010, at 11:00 AM, Roderic Page wrote:
There are other issues here as well. Technology advances rapidly, and what once seemed a good choice may rapidly become dated
True.. but another aspect is just that in order to explore whether a particular graphical concept or tool will work, or will benefit people, you have to knock out prototypes and implementations. e.g., drag-and-drop branch rearrangement in MacClade was great for small trees but not for big ones, hence phylowidget was looking to implement something that worked for large trees (via concentric menus on nodes) and to experiment with auto-pruing, etc. People have to keep knocking out GUI software efforts to test the waters for new features. And the search is not over, for example:
- Do we really have a completely satisfying way of visualizing patterns of gene duplication within a species tree?
- If I handed you a tree with 200k nodes, is there a visual / GUI way that would allow you to easily find interesting patterns in it, such as points of incongruence with a conventional taxonomy?
- If I gave you 1,000 trees, is there a visual / GUI way that would allow you to see which branches in each tree probably crossed an ancient land bridge together?
So I think we still have lots more GUI / visual / ergonomic challenges that still need to be solved (even if lots of challenges have already been solved). And we should not expect each new idea to be implemented in a pristine killer app that does everything that everyone wants out of a tree visualizer -- that takes too long to build, and the right person for dreaming up a new idea is not necessarily the best person for creating clean, robust, off-the-shelf software.
We need to be okay with seeing a rich plethora of quick-and-dirty efforts, each focusing on articulating/investigating sets of novel ideas -- despite some obvious redundancy in some of the more general functions. And then periodically, someone's got to assemble the best of these ideas into a robust, jack-of-all-trades, off-the-shelf software package.
bp
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
On Oct 22, 2010, at 12:39 PM, Richard Ree wrote:
Both technology and users' needs evolve rapidly - so the "gaps" will never be closed, at least in the time frame required by grants, etc. Also, in phylogenetics a lot of developers are themselves users, so quick and dirty solutions to scratch individual itches is inevitable. It's a good thing, IMO.
I agree entirely with the "solutions to scratch individual itches is inevitable and a Good Thing(tm)".
What I don't agree with is if this is portrayed as being mutually exclusive with reusable and more sustainable software. Projects like Bioconductor, the Bio* libraries, and other similar ones demonstrate that it clearly does not need to be mutually exclusive if we don't want it to be. Every piece of code in BioPerl is there because it first scratched someone's itch. The rate of innovation in Bioconductor is pretty rapid.
The barrier, if there is perceived to be one, is social, not technical or scientific. If we as a community of practice don't care about sustainable software, it won't happen, and we'll continue to lose software. If we want to think that the software we produce is an integral part of our science, we lose part of our science every time a piece of software is no longer maintained. And if we want to think that writing a new piece of software is part of our scientific progress, then every time someone writes code for something that code has already been written for, our rate of scientific progress is slowed down unnecessarily.
Again, just my own $0.02.
-hilmar
On Fri, Oct 22, 2010 at 11:56 AM, Hilmar Lapp hlapp@nescent.org wrote:
I agree entirely with the "solutions to scratch individual itches is inevitable and a Good Thing(tm)".
What I don't agree with is if this is portrayed as being mutually exclusive with reusable and more sustainable software.
I never suggested this, but it seemed there was a sentiment that scratching on the part of individual developers was a bad thing - counterproductive to some common goal - and should be discouraged. That struck me as odd. I do agree that modularity and reusability should indeed be encouraged, e.g. in educating user-developers in MVC design patterns and library-based development.
-Rick
On Oct 22, 2010, at 10:29 AM, Arlin Stoltzfus wrote:
QED. The technology is out there. IMHO, the continual propagation of new tree-viewers by developers, and the sense of users that there is no tree viewer that satisfies their needs, is (in the medium-term and long-term) a cultural-organizational-educational problem and not at all a technical problem.
To clarify this-- I didn't mean to imply that there are no gaps. In the *short-term*, there may be real gaps between what users want (or think they want) and what developers are providing and maintaining. I'm just saying that, by virtue of repeated failures, we should have learned by now that the way to close these gaps is NOT for the Nth independent developer to go out and developer the Mth independent tree viewer using the latest GUI fashions.
Arlin
If this is true, then in order to solve the real problem, we need to think about things like changes in funding structure, standards development, and modes of user engagement, not new graphics libraries that do just the right thing with only a few commands.
Arlin
On Oct 21, 2010, at 12:57 PM, Christian M Zmasek wrote:
Hi, Dave and Rutger:
My own tree viewer "Archaeopteryx" provides such an overview when zoomed in, plus some other features described as "missing" in most current tools.
See: http://www.phylosoft.org/archaeopteryx/
Example: http://www.phylosoft.org/archaeopteryx/examples/ mollusca.html
Archaeopteryx also provides other useful features (at least for comparative genomics use cases). For example, the ability to infer internal taxonomies (if all external nodes have _some_ taxonomic information associated with them; standalone version only; via uniprot taxonomy database).
Please let me know if you'd like to know more or have suggestions for improvement (although keep in mind that this Archaeopteryx is just a peculiar hobby of mine).
Christian
On 10/21/2010 3:39 AM, Rutger Vos wrote:
Hi Dave,
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
Rod Page has coded a web widget (I believe all javascript) that has a small preview window for the whole tree and a larger "zoomed in" view. "TreeJuxtaposer" is a java app(let?) that allows you to contract and expand selections of nodes/clades. I think these come closest to what you are talking about, though neither operates on networks.
Cheers,
Rutger
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
------- Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
David--
This raises two issues that I don't quite understand.
The problem of visualizing large trees, e.g., hundreds of thousands or millions of nodes, has been solved again and again by computer scientists. We've all seen some visualizations of the internet done with such tools. There is no fundamental technology barrier. The technology exists to visualize large trees and zoom around and navigate and so on.
The problem has to do with how to develop stable and adaptable software for a community that is poor, dispersed, fractious (unable to agree on standards), and constantly changing in its needs.
Likewise, computer scientists and hackers have repeatedly solved the problem of designing a framework for automating bioinformatics workflows. So why don't more bioinformaticians use automated workflows?
Probably the group that is getting closest to what you want is the TreeViz group of EoL at the Field Museum. Karen Cranston (cc) has some cool slides on this.
The second issue that I don't understand is: what is the proper netiquette when a list message is cc'ed to half a dozen people? If we all keep repeating the cc pattern, recipients may be getting lots of messages that they don't want. If we stick to the list only, they could miss out.
Arlin
On Oct 21, 2010, at 5:20 AM, Kidd, David M wrote:
I am yet to consider the visualizing RDF trees, however, I have am interest in visualizing and interacting with trees and networks in a web context.
I've been looking at the various web tree visualizers, as well as more generic visualization libraries, for the last week or so. Understanding the functionality, maturity, user community and inter- browser compatability is time consuming, and not always simple to determine.
Specifically, I am looking for a tool that supports, or can extended to support;
- Both both trees and networks,
- The selection of multiple nodes, e.g. subtrees
- The browsing of large trees/networks - I get some weird effects
with large trees in jsPhyloSVG. 4. (Readable) Labels at all nodes. 5. Alternate layouts (circular, hyperbolic, etc.) 6. Works in as many browsers as possible. 7. A variety of I/O formats - I store my trees in PhyloDB, which I am thinking of generalizing for networks.
The ability to browse large trees seems to be a particular limitation of existing tools (I'd love to be corrected if I am wrong). Having a tree larger than the widget, as in Phylowidget, is one approach, however, an overview window would be nice to orientate your view in relation to the entire tree. I have also been considering displaying only a subset of nodes and then having 'expand', 'contract' and 'pan' (by expanding and contracting) functions for navagation. The ability to display node subsets is probably more important for networks than trees as reticulation will often result in visual occlusion.
I would be very interested in any comments regarding existing tools and the above issues (an others I have not thought about). I would be happy to compile all comments and perhaps some comparative demos somewhere.
- Dave
David M. Kidd
Research Associate Center for Population Biology Silwood Park Campus Imperial College London 0207 594 2470
-----Original Message----- From: tdwg-phylo-bounces@lists.tdwg.org [mailto:tdwg-phylo-bounces@lists.tdwg.org ] On Behalf Of Richard Ree Sent: 20 October 2010 21:47 To: Hilmar Lapp Cc: tdwg-phylo@lists.tdwg.org Interest Group; Chris Baron Subject: Re: [Tdwg-phylo] Publishing a tree in RDF
The purposes of tree visualization are many and varied, so it seems natural to me that tool development reflects that diversity. One distinction of tred is that it exposes server-side and client-side functionality.
-Rick
On Wed, Oct 20, 2010 at 3:17 PM, Hilmar Lapp hlapp@nescent.org wrote:
On Oct 20, 2010, at 2:43 PM, Chris Baron wrote:
Also, I've been working on a phylogentic tree editor called tred. http://code.google.com/p/tred/
Interesting. How does your editor compare to the dozens of tree editors and visualizers already available. For example PhyloWidget, or jsPhyloSVG? I may sound like a broken record here, and I don't want to discourage you at all from developing tools for phylogenetics. We certainly need more developers. I also do think though that as a community we make the most progress if we aren't sprouting new tools for the same purpose all the time.
-hilmar
=========================================================== : Hilmar Lapp -:- Durham, NC -:- informatics.nescent.org : ===========================================================
tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo _______________________________________________ tdwg-phylo mailing list tdwg-phylo@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-phylo
------- Arlin Stoltzfus (arlin@umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD tel: 240 314 6208; web: www.molevol.org
On Oct 21, 2010, at 9:30 AM, Arlin Stoltzfus wrote:
Probably the group that is getting closest to what you want is the TreeViz group of EoL at the Field Museum. Karen Cranston (cc) has some cool slides on this.
Yes, except that Karen is now at NESCent :-) and she has been directly involved in an iPlant team working on a tree viewer that can efficiently read and display very large trees. She presented that back at the iEvoBio conference.
BTW Arlin - it didn't sound like such a solved problem according to the computer scientists involved in that project. But I'll leave it to Karen to comment further.
The second issue that I don't understand is: what is the proper netiquette when a list message is cc'ed to half a dozen people?
Keep everyone unless you have a good reason to drop someone (such as someone asking to be dropped :). BTW I think most if not all of those copied are on the list anyway (Rick - are you?).
-hilmar
The problem of visualizing large trees, e.g., hundreds of thousands or millions of nodes, has been solved again and again by computer scientists. We've all seen some visualizations of the internet done with such tools. There is no fundamental technology barrier. The technology exists to visualize large trees and zoom around and navigate and so on.
There is a distinction between a generic solution for visualizing very large graphs / networks and a solution tailed for phylogenetic trees / acyclic graphs. The visualizations of the internet (for example) are trying to convey a different type of information - clusters of connectivity, hubs - rather than the specific branching pattern of a phylogeny. We've invited smart comp sci & visual folks to workshops on treeviz, and they haven't told us the problem is already solved. That's not to say that isn't anything of value to learn from these generic projects, but the reality is that we don't have a user-friendly, widely accepted solution for visualizing large (>100K leaves) phylogenetic trees.
The iPlant project that I've been working with has an interactive solution that works for trees of ~500K leaves (it was 1M with the standalone version, but we lost some performance when we moved to a web-based solution). We've been focused on the topology up to this point, but now it is time to start developing an interface for attaching metadata and annotations. Wiki pages for the project: https://pods.iplantcollaborative.org/wiki/display/iptol/Tree+Visualization prototype implementation (no ability to upload trees here yet): http://flanders.iplantcollaborative.org/
The problem has to do with how to develop stable and adaptable software for a community that is poor, dispersed, fractious (unable to agree on standards), and constantly changing in its needs.
This is definitely part of the problem with treeviz. We're starting (finally) to talk about this in the iPlant project. Having a key resource like TreeBASE to point to is going to be helpful in this process.
The second issue that I don't understand is: what is the proper netiquette when a list message is cc'ed to half a dozen people? If we all keep repeating the cc pattern, recipients may be getting lots of messages that they don't want. If we stick to the list only, they could miss out.
Problem here seems to be determining who is on the list and who isn't. I am, so I don't need to be copied separately.
Karen
Dave:
On Oct 21, 2010, at 5:20 AM, Kidd, David M wrote:
I store my trees in PhyloDB, which I am thinking of generalizing for networks.
PhyloDB can already deal with networks, or at least was designed that way - what made you think that it doesn't? Is there something missing?
-hilmar
participants (11)
-
Arlin Stoltzfus -
Chris Baron -
Christian M Zmasek -
Hilmar Lapp -
Karen Cranston -
Kidd, David M -
Nico Cellinese -
Richard Ree -
Roderic Page -
Rutger Vos -
William Piel