Steve Emmerson wrote:
Dear LDM user,
I'm working on a replacement for the LDM and I'd like your help. I need
user-stories (i.e., high-level usage scenarios) so I don't forget
anything or leave out some useful feature. I've created a wiki to help
gather the stories (and for anything else for which it might be useful).
The wiki is at
<https://wiki.ucar.edu/display/unidata/LDM+Replacement>. It contains
the link "User Stories" in which I've put one story. Please read it and
if you think something is missing, then please add your own story, edit
any existing one, add comments, etc.
In order to add or edit, I'll have to create an account for you on the
UCAR wiki and add you to the "LDM-Replacement" group. Just reply to
this email with your full name and the email address you'll want to use
as your identifier. Naturally, I reserve the right to be human (i.e.,
picky :-).
Now's your chance!
Lacking access to the wiki, and also having a very strong opinion that
Wiki's are for documentation, and not active communication, I'm gonna
start here.
One of the listed Design Goals is an automatic adaptation to topology.
I strongly agree with this one. I think the static approach we're so
comfortable with could benefit here from some change.
Automatic adaptation to bandwidth is a little fuzzier. If the idea is
to adapt on the fly to varying bandwidth requirements, TCP does a fairly
good job of that. It also handles congestion management pretty well.
I'd not recommend hacking a UDP distribution mechanism to make UDP
artificially ack packets, but IIRC, there's been some changes to TCP ack
requirements over time for LDM to improve throughput. There might be
some places where we can work on this. Having said this, another data
distribution system I'm involved in moves a lot of data around its
high-level servers using UDP and sees precious little packet loss: With
strict accounting we see something approximating 0.2% but practical
appreciation says we see no adverse impact from that. Using binary
files instead of short files certainly helps this, though: For our use,
0.2% packet loss could be a major failure real fast.
I strongly support the ability to obtain/retrieve realtime _and_
archived data. I'm not so sure what static and dynamic directories
are/mean but I think I've got a good idea... Ditto finite vs infinite
files, and "local data holdings". I think we already do that with LDM.
I'm already using LDM and PQACT for event driven activities and they
just work nicely. Am I doning something wrong? If DDDAS had discovered
LDM 5 years ago, they'd be a bit further ahead.
Minimizing b/w and requiring a site to receive just wha tit needs are
laudable goals and appropriate for leaf nodes, but further upstream
unless we have a really adaptable system (not impossible but this is
starting to sound like a major CompSci project) but non-trivial if
you're also feeding downstream nodes. I can envision a couple of ways
to approach this, however, that might work.
Stand-alone app AND libraries is good. Very good.
Enhancing scalability is laudable but save with limitation some
feedtypes and some file sizes, I think we're OK here. Scalability WRT
the number of sites? I think there's likely an upper bound but I'm not
positive we've reached it yet.
I don't do Windows. I can't stand the downtime. Server installs are
(still) increasingly going to *nix. My particular preference is Linux
but it's not burned in stone. I can survive with BSD (with or without
Apple) or one of the commercial Unices, including Solaris, if I really
have to. But Windows? Really? Maybe, just maybe for an end-user leaf
node.
Cool names are always good.
And after a little more reading... I don't have a browser on my LDM
machines. They're servers. Nor would I (like Tyler) support java in a
high performance data handling environment. Java's got its place, but
in my experience (and unlike Tyler, I do write C/C++/java code) it's not
here.
So that's my initial impression.
--
Gerry Creager -- gerry.creager@xxxxxxxx
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843