Hi Everette,
When were these workflows submitted yesterday or today? If its today, 10
workflows were submitted, 7 finished fine, 1 wrf went crazy, 2 are
running fine now they should finish up some time soon.
But as I was saying the workflow successful message we are seeing might
be misleading. The workflow engine sends a success message if WRF tells
it is done, we are not checking if it has finished with good output or
bad output. But we need to get those bad outputs experiment information
from users and pass those data files to the met. group to refine the
configurations. Not sure what he notification problem could be, will
look forward to see their bug reports.
Thanks,
Suresh
Everette Joseph wrote:
LEAD - Master Mailing List
We submitted runs between 15z and 21z per discussion on Monday.
Failure rate seemed high and there were some weird experiences with
output and notification--- your email explained the latter -- I
think. Our testers should be sending reports to "contact us".
EJ
Suresh Marru wrote:
LEAD - Master Mailing List
Hi All,
Just want to update on how test runs has been so far this week.
The workflows are running good for most of the configurations, we
certainly came down quite a bit from 100% success rate but now sure
exactly how much, hopefully not below 90%. With good amount of
testing from all of you we have a found a problem which prevented you
from launching a workflow this morning. Restarts of couple of
services helped and everything is fine now but we still did not
resolve the problem and will continue to look why the unscheduled and
unwanted down time happened.
If you have noticed yesterday between morning and 3pm (EST) workflows
failed due to some change in a software. We reverted that change and
all failed workflows were resurrected late in the evening and
finished successfully but not sure if they did a forecast or a post
cast. The consequence of this will be duplicates of data in your
workspace for the experiments created yesterday.
We have seen some WRF runs going unstable in past 2 days, not sure if
they gave workflow success or failed status. I did not completely
understand the reasons Dan Weber and Brian Jewett have analyzed but
you guys might, which is "Looking at the plots Brian generated from
the earlier run, convection starts on the southern boundary, which
usually means doom for a forecast, especially if it is on an inflow
condition (which is the case for the run). No convection was visible
in the the ADAS analysis during the forecast area. It seems that the
main problem is the placement of the lateral boundary in a region
that is mountainous and where convection developed. This is the one
of the problems with limited domain forecasts, such as the 5km
201x201x51 grid" -- We still need to parse this to english and then
make the changes to data search or namelists.
All of this aside, please continue to use our resource reservations
at NCSA from 1pm to 6pm (EST) and test and report problems as usual.
Happy Testing,
Suresh
_______________________________________________
LEAD-All mailing list
LEAD-All@xxxxxxxxxxx
http://www.caps.ou.edu/mailman/listinfo/lead-all
==============================================================================
To unsubscribe leadusers, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================