Date: Sun, 11 Apr 2004 08:32:24 -0600 (MDT) From: "John R. Cary" <cary@colorado.edu> To: Ireneusz SZCZESNIAK <iszczesniak@iitis.gliwice.pl> Cc: Quincey Koziol <koziol@ncsa.uiuc.edu> Subject: Re: graphics/opendx and dropping science/hdf Message-ID: <Pine.LNX.4.44.0404101841390.24692-100000@geta.colorado.edu> In-Reply-To: <Pine.LNX.4.58.0404101924410.2607@irek.iitis.gliwice.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
A few more features possibly of interest: dxhdf5 also allows the user to downselect at time of import. One can choose, e.g., to load on average 1/5 of the particles, selected at random, but only those with 0 < x < 0.1. Thus, not all particles are brought into memory, only those relevant. Similarly, one can load into memory only, e.g., the x component of some field, sampled at every 10th point in each direction, and of only some sub-slab of the data. This is useful for large data sets, which could overwhelm the memory on a single processor. Finally, dxhdf5 has a very liberal, BSD-style license, so it can be used in other products, both commercial and non commercial. This is because our goal is to maximize use and, so, feedback, so that dxhdf5 can "field-hardened" for our own use. FYI, most of the work was done by Mr. Szczesniak, only a little by myself. He did this while working in my research group at the University of Colorado for the last two years. Unfortunately, he will not be able to return this summer, should extensive development be needed. JRC On Sat, 10 Apr 2004, Ireneusz SZCZESNIAK wrote: > Hi, > > I am Ireneusz Szczesniak. John Cary and I are the authors of dxhdf5. > As far as I know there are only two packages that import HDF5 data > into OpenDX: dxhdf5 and ImportHDF5. There are a few very good things > about dxhdf5. > > 1. dxhdf5 is user friendly: building and installing is standard and > robust (like a GNU package). dxhdf5 builds and installs with > Automake, Autoconf and Libtool. Now the package easily builds and > installs on Linux, Mac OS X, IRIX, AIX and possibly on others, but > we have not tried others. > > 2. It's reliable. Since the package was relased in summer 2002, we > have not heard complaints that the module crashes in OpenDX. Well, > the only exception was with AIX, but we found out there was a bug > in OpenDX, and not in dxhdf5. > > 3. dxhdf5 is cleanly implemented and can be easily extended. When I > work on dxhdf5, it's easy for me to change it, even though I > completely forgot the code. > > 4. Aside from importing fields, dxhdf5 imports sets of particles and lets > you choose the particles you want. See here: > http://www-beams.colorado.edu/dxhdf5/ImportHDF5Species.html > > 5. It imports scalar and vector fields. Plus, dxhdf5 automatically > finds out what kind of data you are importing: integer, float or > double. Some examples of fields may be found here: > http://www-beams.colorado.edu/dxhdf5/ImportHDF5Field.html > > We presented dxhdf5 at the 18th International Conference on Numerical > Simulation of Plasmas (ICNSP '03), http://web.mit.edu/ned/ICNSP/, and > people there were quite interested in dxhdf5. Some of them, > unfamiliar with dxhdf5, were saving their data in plain text files > even tough they knew about HDF5, because can import text data into > OpenDX and other software. Back then we got some feedback on what > people really need and we got some good ideas for further development. > > If you have some questions, please let us know. > > > Best, > Ireneusz Szczesniak > > ******************************************** > * Ireneusz SZCZESNIAK - research assistant * > * Tel: +48 (32) 231-73-19, extension 204 * > * http://www.iitis.gliwice.pl/~iszczesniak * > ******************************************** > > On Thu, 8 Apr 2004, Mike Folk wrote: > > > Date: Thu, 08 Apr 2004 11:06:53 -0500 > > From: Mike Folk <mfolk@ncsa.uiuc.edu> > > To: Quincey Koziol <koziol@ncsa.uiuc.edu>, > > Mikhail Teterin <mi+kde@aldan.algebra.com> > > Cc: ports@FreeBSD.org, hdfnews@ncsa.uiuc.edu, ijliao@FreeBSD.org, > > yuri@irfu.se, corecode@corecode.ath.cx, tilman@arved.de, > > flynn@energyhq.homeip.net, aa8vb@nc.rr.com, > > toyonaga@msd.ts.fujitsu.co.jp, tg@FreeBSD.org, kay_lehmann@web.de > > Subject: Re: graphics/opendx and dropping science/hdf > > > > I completely forgot: Ireneusz Szczesniak has done an open dx > > implementation called "dxhdf5", and has some nice documentation and other > > materials at http://www-beams.colorado.edu/dxhdf5/. We need to be working > > with her. > > > > Mike > > > > ------------------------------------ > > (Thanks for cc'ing me Quincey.) > > Just a couple other comments about open dx: > > > > The CACTUS project has been using open dx with hdf5 for several years. I > > strongly suspect there are others, but don't know specifics. As far as I > > know, the CACTUS folks haven't documented their work (I asked them a few > > times, but that was a couple of years ago), so I don't know exactly how > > they organize the data. > > > > Open dx and hdf5 are a natural combination, and it's a pity we haven't > > married them in some formal way, and we'd love to work with you if you move > > in that direction. > > > > Mike > > > > At 09:27 AM 4/8/2004, Quincey Koziol wrote: > > >Hi Mikhail, > > > > > > > I intend to drop the science/hdf port. It was obsoleted by hdf5 long ago > > > > (was it?), it breaks on some some 64bit platforms, it conflicts with > > > > math/netcdf. > > > > > > > > All the technical issues are, probably, fixable, but the obsoleteness > > > > makes them not worth the effort. Or so it seems. > > > > > > > > The only port relying on science/hdf is graphics/opendx and only if > > > > WITH_HDF is defined. Unfortunately, opendx does not (yet?) support hdf5. > > > > > > > > So... > > > > . Does hdf5 really obsolete hdf4? > > > No, they are different file formats & libraries, etc. > > > > > > > . Does hdf support add much value to opendx at all, > > > > or can we just remove it? > > > I don't know. > > > > > > > . If not, should I (or some other helping hand) try to patch > > > > opendx to use hdf5? > > > It would certainly be nice if opendx supported hdf5. > > > > > > > or > > > > . bring the old hdf4 into the 21st century (fixing the types, > > > > depending on math/netcdf instead of building its own)? > > > It shouldn't be _too_ hard to fix the type information in hdf4 (I can > > > help > > >you with that, if you'd like), but it would be _very_ hard to depend on > > >math/netcdf instead of its own. > > > > > > Quincey > > > > > > > > > > > Thanks for any ideas, > > > > > > > > -mi > > > > > > > > -- > > Mike Folk, Scientific Data Tech (HDF) http://hdf.ncsa.uiuc.edu > > NCSA/U of Illinois at Urbana-Champaign 217-244-0647 voice > > 605 E. Springfield Ave., Champaign IL 61820 217-244-1987 fax > > > > -- John R. Cary Professor, Dept. of Physics, University of Colorado, Boulder, CO 80309-0390 cary@colorado.edu ph. (303) 492-1489 fax (303) 492-0642 cell (720) 839-5997
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.LNX.4.44.0404101841390.24692-100000>