Corrupted memory when reading int32 connectivity into a int64 dataspace (investigated parallel reads), HDF5


Problem: Reading a CGNS file (which was created for a default cgsize_t = int) with a CGNS program which has default cgsize_t = long int)

The issue is CGNS is passing a different memory type to HDF5 than what the memory buffer is. I’ll try to explain it for the case of an int32 dataset.

(1) You pass CGNS a buffer that is int64 (cgsize_t); this is what the dataset gets read into memory as.

(2) However, CGNS gets the memory data type of the dataset from the CGNS file, which in this case is int32, i.e.,

(2a) CGNS sets the dataset memory space in H5Dread to be int32.
(2b) But, CGNS lied, in step (2a), to HDF5 by telling HDF5 that it was reading into an int32 space, but it’s actually reading it into an int64 memory space

(3) The result, you get junk in the memory from HDF5.

Now, if CGNS is honest with HDF5 and instead reports the memory space as int 64,

if (rw_mode == CG_PAR_READ) {
type_id = H5T_NATIVE_INT64;
herr = H5Dread(data_id, type_id, mem_shape_id,
data_shape_id, plist_id, data[0].u.rbuf);

then HDF will convert the int32 dataset into an int64 dataset in memory, and everything will be correct.

So, the issue is CGNS assumes that the user is reading into memory a datatype buffer that is the same as the dataset type in the file. What CGNS should be doing instead is setting the memory space to match the memory buffer and not the file space buffer type. Then HDF will automatically handle the conversion to correct memory space datatype.

CGNS needs to do in readwrite_data_parallel something like (untested):

switch (type) {
case CGNS_ENUMV(Character):
type_id = H5T_NATIVE_CHAR;
case CGNS_ENUMV(Integer):
if(sizeof(cgsize_t) == 8)
type_id = H5T_NATIVE_INT64;
type_id = H5T_NATIVE_INT32;




Scot Breitenfeld




Fix versions

Affects versions