Re: [netcdfgroup] NetCDF4 MPI-IO and compression

[Rob Ross <rross@xxxxxxxxxxx>]

In defense of the HDF team, this is difficult capability to enable in  
practice. This is because prior to compression you don't know how much  
space a given write will require, so parallelizing writes (especially  
if they aren't performed collectively) is not straightforward and  
requires a significant amount of coordination.

Rob

On Feb 13, 2008, at 12:07 PM, Ed Hartnett wrote:

> Thomas Green <thomas.green@xxxxxxxxxxxxxxxx> writes:
>
> > NetCDF users,
> >
> > Just a quick question.  Can NetCDF4 use MPI-IO if also using
> > compression?  I read on the HDFgroup website that MPI-IO and  
> > compression
> > are not compatible but I am not sure if this still applies and how it
> > affects NetCDF4.
> >
> > The HDFgroup FAQ related to MPI-IO and compression:
> > http://www.hdfgroup.com/hdf5-quest.html#p5comp
> >
> > Many thanks,
> >
> > Tom
>
> No, sorry, that is not allowed by HDF5.
>
> If you want parallel I/O you must use uncompressed data.
>
> I have asked the HDF5 team whether this might be changed in a future
> HDF5 release...
>
> Ed
>
> --
> Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/