That looks great to me. Thanks for your consideration of this request.<br><br><div><span class="gmail_quote">On 7/27/06, <b class="gmail_sendername">"Döring, Markus"</b> <<a href="mailto:m.doering@bgbm.org">m.doering@bgbm.org
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">I have implemented the log-only request now and would like to suggest the following:
<br><br>1) add "log-only" attribute to responeOperationGroup. Ive chosen "log-only" cause we already have there an attribute "apply-xslt"<br>2) add a mandatory boolean attribute "logRequestsDenied" to the operation element in a capabilities request
<br>3) use existing responses for the log-only request. So if you do a search with log-only active, you will get an empty search response back. Thats much easier to implement and doesnt require any change in the schema. The same works with inventories. Pong, Capa & Meta responses dont cost much anyway, so we could do a normal response (if anyway uses log-only with those requests at all)
<br><br><br>Does everyone agree to this?<br>The schema is already updated for this.<br><br><br>Markus<br><br><br><br>-----Original Message-----<br>From: <a href="mailto:pywrapper-devel-bounces@lists.sourceforge.net">pywrapper-devel-bounces@lists.sourceforge.net
</a> on behalf of John R. WIECZOREK<br>Sent: Tue 7/25/2006 5:29 PM<br>To: Döring, Markus<br>Cc: PyWrapper Developers mailing list; <a href="mailto:roger@tdwg.org">roger@tdwg.org</a>; <a href="mailto:tdwg-tapir@lists.tdwg.org">
tdwg-tapir@lists.tdwg.org</a><br>Subject: Re: [PyWrapper-devel] [tdwg-tapir] RE: WG: tapir: capabilities<br><br>Why make this so hard? We're talking about a portal sending out one message<br>to n providers and not having to wait for a response. Right now we would
<br>have to send out a message and wait for all of the responses (up to a<br>timeout). This is a net improvement.<br><br>Why do we need metadata from providers to know if they want logging<br>requests? We don't ask them if they want metadata requests, or how often.
<br>Just send the requests. If they want to log them, they will configure their<br>provider to do so. Otherwise the provider will ignore them.<br><br>In the meantime, always log on the portal side. Not only will that give
<br>providers with flakey connections a place to see usage statistics, but it<br>will also generate information will be interesting on its own - summary<br>information about how the portal is used that you wouldn't get from the
<br>providers.<br><br>To me, this usage business is important enough that I would even go so far<br>as to certify portals as being in compliance with meeting this social<br>contract. That way a provider could release access to certified portals and
<br>disallow access for those who don't abide by the contract. Remember, the<br>semblance of control is really important to a lot of our providers. If you<br>don't think so, have someone do a survey of existing providers to see if
<br>they would want it or not. It would be a sample biased against needing<br>logging (since they are already doing without). If that survey turned up<br>interest in logging anyway, then it's worth doing. My feeling is that it is
<br>so easy to implement (if you don't try to get unnecessarily fancy) that it<br>should just be done - it would be easier than conducting a survey about it.<br><br>On 7/25/06, "Döring, Markus" <<a href="mailto:m.doering@bgbm.org">
m.doering@bgbm.org</a>> wrote:<br>><br>> I agree that logging via GUID doesnt help in many cases where the provider<br>> wants to know what was searched for.<br>><br>> But searching on a portal-cache to find data from 20 different providers
<br>> in 1 search and then sending of 20 log requests could also be annoying. Plus<br>> the burden of the portal of checking the registry if a providers really<br>> wants logging.<br>><br>> The most efficient is probably a portal specific logging as Donald
<br>> suggests. But then providers would have a hard time agglomerating the<br>> logging data from several totaly different portals.<br>><br>> To get a comparable logging across different portals though it seems to me
<br>> that Renatos suggestions are worth a try. It would definitely need guidlines<br>> for portal developers to know when to use a log request and how to use it.<br>> How to treat paging and map data are good examples where there is no obvious
<br>> correct behaviour.<br>><br>><br>> -- Markus<br>><br>><br>> > -----Ursprüngliche Nachricht-----<br>> > Von: <a href="mailto:tdwg-tapir-bounces@lists.tdwg.org">tdwg-tapir-bounces@lists.tdwg.org
</a><br>> > [mailto:<a href="mailto:tdwg-tapir-bounces@lists.tdwg.org">tdwg-tapir-bounces@lists.tdwg.org</a>] Im Auftrag von<br>> > John R. WIECZOREK<br>> > Gesendet: Montag, 24. Juli 2006 18:28<br>> > An:
<a href="mailto:roger@tdwg.org">roger@tdwg.org</a><br>> > Cc: PyWrapper Developers mailing list; <a href="mailto:tdwg-tapir@lists.tdwg.org">tdwg-tapir@lists.tdwg.org</a><br>> > Betreff: Re: [PyWrapper-devel] [tdwg-tapir] RE: WG: tapir:
<br>> > capabilities<br>> ><br>> > Logging that a GUID was used isn't sufficient; it doesn't<br>> > tell how the data were used. What I'm after is to log the<br>> > actual query that would have had to go to the provider to
<br>> > produce the results used. The example in your second example<br>> > shouldn't happen, the query should specify a record limit per<br>> > provider.<br>> ><br>> ><br>> > On 7/24/06, Roger Hyam <
<a href="mailto:roger@tdwg.org">roger@tdwg.org</a>> wrote:<br>> ><br>> ><br>> > I thought this sounded like a good idea but since<br>> > reading Renato's message I am now confused.<br>> >
<br>> > If a user does a search on a portal and gets 100<br>> > results and looks at the first 10 does the provider of the<br>> > 11th record get notified? Their data has been used because it<br>> > has been given in the count. Another example would be if a
<br>> > portal gave a distribution map to 10km squares based in data<br>> > from multiple providers. Each data point is made from several<br>> > suppliers data and removal of any one supplier's data may not
<br>> > change the map. Do we notify them all?<br>> ><br>> > I could envisage a GUID based system just about. The<br>> > call to the log function would basically say "Some one has<br>> > accessed the data that I got from you that you tag with this
<br>> > GUID" but I can't see how this would work on a search based<br>> > system. The log call would mean "Some one searched for<br>> > something that made used of data I got from a search I did on
<br>> > you once".<br>> ><br>> > So really the only service we need is a GUID based one.<br>> > Perhaps extending the LSID resolution spec would be more appropriate?<br>> ><br>> > Roger
<br>> ><br>> ><br>> ><br>> > Renato De Giovanni wrote:<br>> ><br>> > Hi John,<br>> ><br>> > Implementing the log request was never a<br>> > problem. We discussed about
<br>> > that again during the Madrid meeting, and only<br>> > after that a feature<br>> > freeze was suggested. It's true that PyWrapper<br>> > is being adjusted now<br>
> ><br>> > to conform to the new specs, and considering<br>> > that DiGIR2 (or wasabi)<br>> > postponed implementation of TAPIR, I suppose it<br>> > should not be a big
<br>> > problem to make additional changes if necessary.<br>> ><br>> > The main problem I had with the log request was<br>> > that it would<br>> ><br>> > probably not solve the issue behind it, which
<br>> > is to track usage by<br>> > data aggregators. I still have the same<br>> > feeling, and I can easily<br>> > imagine situations when it would not be easy or<br>> > even possible to
<br>> > translate searches on top of cached databases<br>> > to TAPIR requests.<br>> ><br>> ><br>> > But maybe I'm wrong, and if you all think it's<br>> > a good feature then we
<br>> > can try to include it. However, I do think that<br>> > providers should be<br>> > able to advertise as the part of capabilities<br>> > if they want to receive<br>
> ><br>> > log requests or not.<br>> ><br>> > To me it also sounds like a new operation,<br>> > especially if it's only<br>> > related to search. It could make sense for
<br>> > view, inventory and<br>> > metadata operations. Maybe capabilities too.<br>> > But it doesn't make<br>> ><br>> > sense for ping. Well, maybe it could make sense
<br>> > for ping if the data<br>> > aggregator monitors provider status and accepts<br>> > similar requests on<br>> > top of its results...<br>> ><br>> > So, yes, it could be a new attribute "logOnly"
<br>> > as part of the<br>> ><br>> > operationRequestGroup with an answer<br>> > </received> (just after the<br>> > response header). And we could add an attribute
<br>> > "acceptLogRequests"<br>> > in the <operations> element in capabilities<br>> > responses. The other<br>> ><br>> > option would be to include a new operation, but
<br>> > maybe it's better to<br>> > just have it as an optional attribute for all<br>> > operations.<br>> ><br>> > Best Regards,<br>> > --<br>> > Renato
<br>> ><br>> > On 19 Jul 2006 at 10:32, John R. WIECZOREK wrote:<br>> ><br>> ><br>> ><br>> > I appreciate that you will consider<br>> > this request. I always thought
<br>> > it<br>> > would be trivial to implement. Your<br>> > simulation mode sounds very much<br>> > like what I had in mind. I hadn't
<br>> > thought it necessary to get a<br>> ><br>> > response from a log request, but if<br>> > there was a simple response, it<br>> > could be used as a ping, or it could be
<br>> > used to retry logging until<br>> > the provider did respond. So, something<br>> > like <log request received>.<br>> ><br>> > I think the addition oflogOnly
<br>> > attribute is a good one, and could<br>> > apply to every request type.<br>> ><br>> > Javi, I don't disagree that portals<br>> > SHOULD log the data usage,
<br>> > especially to cover the situations<br>> > where a provider doesn't respond.<br>> ><br>> > I also think that having the<br>> > information logged at the provider is a
<br>> > responsible course of action, since<br>> > they will have immediate access<br>> > to the usage statistics that way. It<br>> > will be much easier for a
<br>> > portal<br>> ><br>> > builder to send log requests than it<br>> > will be to build the<br>> > infrastructure and interfaces to logs,
<br>> > therefore it is more likely<br>> > to<br>> > actually get done.<br>> ><br>> ><br>> > On 7/18/06, Javier de la Torre
<br>> > <<a href="mailto:jatorre@mncn.csic.es">jatorre@mncn.csic.es</a>><br>> > <mailto:<a href="mailto:jatorre@mncn.csic.es">jatorre@mncn.csic.es</a>> wrote:<br>> > I am not sure about this,
<br>> ><br>> > I still think that portals should<br>> > be gathering this data and<br>> > making<br>> > it available for data providers...
<br>> ><br>> > But in any case if you like it then<br>> > I agree with MArkus that the<br>> ><br>> > best<br>> > is to include another parameter in
<br>> > the operationRequestGroup.<br>> ><br>> > I havent checked but what happens<br>> > if you do an extension there<br>> > with<br>> > an attribute that is implementation
<br>> > specific? A qualified<br>> ><br>> > attribute.<br>> > Will this still validate against<br>> > our schema? You were<br>> > discussing
<br>> > about qualification of attributes before no?<br>> ><br>> > Javi.<br>> ><br>> > On 18/07/2006, at 10:41, Döring,
<br>> > Markus wrote:<br>> ><br>> ><br>> > > John,<br>> > > all changes going on with TAPIR<br>> > right now are really only<br>
> > changes<br>> > > in terminology or removing<br>> > inconsistencies we did not detect<br>> > before<br>> > > we started the documentation and
<br>> > final implementation.<br>> ><br>> > ><br>> > ><br>> > > But nevertheless I would support<br>> > your request. Especially from
<br>> > the<br>> > > implementation side of view this<br>> > is a trivial change to the<br>> > code.<br>> > > So why dont add it? Just some
<br>> > additional thoughts:<br>> ><br>> > ><br>> > ><br>> > > - Ive added a "simulation" mode
<br>> > already to my code where no<br>> > SQL<br>> > > gets executed but just logged. So<br>> > you can test<br>> > configurations
<br>> > > without risking sending off<br>> > killer statements. Thats similar<br>> ><br>> > to<br>> > > logOnly I guess, returning
<br>> > nothing but diagnostics. What would<br>> > you<br>> > > suggest to be returned for a<br>> > logOnly request? just the empty<br>> > TAPIR
<br>> > > envelope? Nothing? <OK>?<br>> > ><br>> ><br>> > > - would this log-only request not<br>> > be needed for all requests?
<br>> > at<br>> > > least for inventories? So it<br>> > would be easiest to have a new<br>> > logOnly<br>> > > parameter in the header or
<br>> > "request element" just after the<br>> ><br>> > header?<br>> > > something like <search logOnly="true"><br>> > >
<br>> > ><br>> > ><br>> > > -- Markus<br>> > ><br>> > >
<br>> > >> -----Ursprüngliche Nachricht-----<br>> > >> Von:<br>> > <a href="mailto:pywrapper-devel-bounces@lists.sourceforge.net">
pywrapper-devel-bounces@lists.sourceforge.net</a><br>> ><br>> > >><br>> > [mailto:<a href="mailto:pywrapper-devel-bounces@lists.sourceforge.net">pywrapper-devel-bounces@lists.sourceforge.net
</a><br>> > ] Im<br>> > >> Auftrag von John R. WIECZOREK<br>> > >> Gesendet: Montag, 17. Juli 2006 23:30<br>> > >> An: Renato De Giovanni
<br>> > >> Cc:<br>> > <a href="mailto:pywrapper-devel@lists.sourceforge.net">pywrapper-devel@lists.sourceforge.net</a><br>> > <mailto:<a href="mailto:pywrapper-devel@lists.sourceforge.net">
pywrapper-devel@lists.sourceforge.net</a>> ; tdwg-<br>> > <a href="mailto:tapir@lists.tdwg.org">tapir@lists.tdwg.org</a><br>> ><br>> > >> Betreff: Re: [PyWrapper-devel]
<br>> > [tdwg-tapir] RE: WG: tapir:<br>> > >> capabilities<br>> > >><br>> > >> A little off topic, but it
<br>> > occurs to me that a great deal<br>> > of<br>> > >> work is still ongoing with<br>> > TAPIR, which suggests to me that<br>> ><br>> > >> it may be warranted to re-state
<br>> > my request for a simple<br>> > >> message type - a log request.<br>> > This request would be the<br>> > same<br>> > >> as a search request, except that
<br>> > the caller doesn't need a<br>> ><br>> > >> response. Providers would use<br>> > this type of request to log<br>> > >> data usage if the data were
<br>> > retrieved from a cache<br>> > elsewhere.<br>> > >> I remember talking about this in<br>> > Berlin, at which time<br>> ><br>> > there
<br>> > >> was supposed to be a feature<br>> > freeze. Clearly we've gone<br>> > >> beyond that, so I'm requesting it again.<br>> > >>
<br>> > >><br>> > >> On 7/17/06, Renato De Giovanni<br>> > <<a href="mailto:renato@cria.org.br">renato@cria.org.br
</a>><br>> > <mailto:<a href="mailto:renato@cria.org.br">renato@cria.org.br</a>> wrote:<br>> > >><br>> > >>Hi,<br>> > >>
<br>> > >>If I remember well, the "view"<br>> > operation was re-included in<br>> > the<br>> > >>protocol just to handle query
<br>> > templates, especifically<br>> ><br>> > >> for TapirLite<br>> > >>providers. So if someone wants to<br>> > query a provider using
<br>> > some<br>> > >>external output model that should<br>> > be dynamically<br>> > >> parsed, then the<br>> > >>"search" operation must be used
<br>> > instead (using either<br>> ><br>> > >> XML or simple<br>> > >>GET request). View operations are<br>> > really bound to query
<br>> > >> templates,<br>> > >>and they are not allowed to<br>> > specify "filter" or<br>> > >> "partial" parameters.
<br>> ><br>> > >>--<br>> > >>Renato<br>> > >><br>> > >>On 17 Jul 2006 at 21:26, "Döring,
<br>> > Markus" wrote:<br>> > >><br>> > >>> I was just about to edit the<br>> > schema and realizing<br>> ><br>> > >> that output models
<br>> > >>> are only specified for<br>> > searches. but what about<br>> > >> views? they use<br>> > >>> query templates, yes. but only
<br>> > the ones listed in<br>> > >> capabilities? we<br>> ><br>> > >>> should have dynamic ones here<br>> > as well I think. And
<br>> > >> they link back to<br>> > >>> static/dynamic models.<br>> > >>><br>> > >>> So should models maybe become a
<br>> > seperate section not tight<br>> ><br>> > to<br>> > >>> search/view operations? I am<br>> > going to modify the<br>> > >> schema nevertheless
<br>> > >>> already to accomodate the<br>> > changes below - ignoring<br>> > >> views for now.<br>> ><br>> > >>>
<br>> > >>> Markus<br>> ><br>> ><br>> > _______________________________________________<br>> > tdwg-tapir mailing list<br>> >
<br>> > <a href="mailto:tdwg-tapir@lists.tdwg.org">tdwg-tapir@lists.tdwg.org</a><br>> > <mailto:<a href="mailto:tdwg-tapir@lists.tdwg.org">tdwg-tapir@lists.tdwg.org</a>><br>> >
<a href="http://lists.tdwg.org/mailman/listinfo/tdwg-tapir">http://lists.tdwg.org/mailman/listinfo/tdwg-tapir</a><br>> ><br>> ><br>> ><br>> ><br>> ><br>> ><br>> > --<br>> >
<br>> > -------------------------------------<br>> > Roger Hyam<br>> > Technical Architect<br>> > Taxonomic Databases Working Group<br>> > -------------------------------------
<br>> ><br>> > <a href="http://www.tdwg.org">http://www.tdwg.org</a> <<a href="http://www.tdwg.org">http://www.tdwg.org</a>><br>> > <a href="mailto:roger@tdwg.org">roger@tdwg.org</a><br>
> > +44 1578 722782<br>> > -------------------------------------<br>> ><br>> > _______________________________________________<br>> > tdwg-tapir mailing list<br>> >
<a href="mailto:tdwg-tapir@lists.tdwg.org">tdwg-tapir@lists.tdwg.org</a><br>> > <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-tapir">http://lists.tdwg.org/mailman/listinfo/tdwg-tapir</a><br>> > <
<a href="http://lists.tdwg.org/mailman/listinfo/tdwg-tapir">http://lists.tdwg.org/mailman/listinfo/tdwg-tapir</a>><br>> ><br>> ><br>> ><br>> ><br>> ><br>> ><br>> _______________________________________________
<br>> tdwg-tapir mailing list<br>> <a href="mailto:tdwg-tapir@lists.tdwg.org">tdwg-tapir@lists.tdwg.org</a><br>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-tapir">http://lists.tdwg.org/mailman/listinfo/tdwg-tapir
</a><br>><br><br><br><br></blockquote></div><br>