Re: [thredds] TDS initialization

[Roland Schweitzer <Roland.Schweitzer@xxxxxxxx>]

Thanks John.  Among the groups we collaborate with there are some folks 
that are quite concerned about the scaling issue.  Personally, my direct 
experience at this point that indicates that the performance is just 
fine (at least so far) even with our largest catalogs.

What's the experience of the list?  Are folks seeing unacceptable TDS 
initialization because of time spend reading catalogs?  The thread from 
John Maurer about aggregation access issues notwithstanding.

Roland

On 01/03/2011 07:34 PM, John Caron wrote:
> On 1/3/2011 10:53 AM, Roland Schweitzer wrote:
> > Hi,
> >
> > We're starting to put together some "big" server-side configuration 
> > catalogs (both with "lots" of dataset elements and "lots" of 
> > catalogRef elements).  We are wondering about the process TDS goes 
> > through to read the catalog when is starts.  What gets cached?  Does 
> > it have a way to know a referenced catalog is unchanged?  When do 
> > referenced catalogs get scanned?  And so on.
> >
> > Is there some documentation or a flow chart on how TDS initializes 
> > itself?
> >
> > Thanks,
> > Roland
> >
> > _______________________________________________
> > thredds mailing list
> > thredds@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe,  visit: 
> > http://www.unidata.ucar.edu/mailing_lists/ 
>
> Hi Roland:
>
> The sad answer is theres not much documentation. Weve been on the 
> verge of redoing the initialization sequence for a few years now, so 
> weve been waiting so we can document the clean, cool refactor instead 
> of the crufty, lame current one.
>
> Anyway, the TDS reads in all the config catalogs at startup. It caches 
> all of them, and uses the "expires" attribute on the catalog to decide 
> if/when it needs to reread a catalog.  It needs to read all catalogs, 
> including catalogRef, because it has to know what the possible dataset 
> URLs are, and there is no contract that a client has to read a catalog 
> first.
>
> Obviously this doesnt scale forever. Ethan can probably fill in some 
> details.
>
> see:
> http://www.unidata.ucar.edu/projects/THREDDS/tech/catalog/v1.0.2/InvCatalogSpec.html#catalog 
>
>
> John
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/