Windows fails VS17, large files > 2GB

Description

I recently switched from VS10 to VS17, and I then built CGNS 3.2.1 from scratch (3.3.x still buggy on Windows). All was fine for the last few months until tonight when I had to work with a 3.2GB CGNS file (produced on Linux).

On cg_open() calls, I was getting “file not found”, but I traced the failure to the cgio_check_file() procedure (in cgns_io.c) shown below. The “stat” command (highlighted) is failing on Windows when the CGNS file is over 3GB (or maybe 2GB). Note that, I had configured my CGNS for “Large File Support”, and the HAVE_LSEEK64 macro was properly defined by cmake in the build files, but it was still failing.

I found that, the Windows “stat” function/struct is only for 32-bit file-sizes, but in prior versions of the Microsoft CRT, it appears that the “stat” function did not return an error when “stat”ing a large file. I’m guessing that, in the newer MS CRT versions (VS 15, VS 17), that this 32-bit “stat” function now (properly) fails when you try to 32-bit “stat” a file that is over 2-3Gb (because the stat struct has the 32-bit “st_size” field which would be incorrect for large files).

So, because the CGNS library does not appear to ever use “stat” to get the size of the file, this “defect” of using the “struct stat” for large files (on Windows) was never found. Now that the 32-bit “stat” fails for larger files, the CGNS usage of this CRT API has to be modified.

I added a define (stat à _stat64) to remap the function and struct to the proper 64-bit API (#define highlighted below), and this worked. There are several places in the MLL where “stat” is used and it will always fail on large CGNS files on Windows unless the remap is used. I only changed to _stat64 for this one function below so I could confirm the behavior/fix – it needs to be altered across the MLL, and there would be an interaction with the MLL “large file support” CMake/configure switch… Of course, I imagine there are some 32-bit OS die-hards out there (I’m not one of them), but I really with “large file support” for Windows was the default unless a 32-bit Windows OS was detected – it’s annoying to have to explicitly configure this CMake switch (On 64-bit Windows, but 64-bit Linux is fine).
=========

/ Had to add this on Windows (VS17) to get stat() to work.

  1. define stat _stat64
    int cgio_check_file (const char *filename, int *file_type)
    {
    int n;
    char buf[32];
    FILE *fp;
    static char *HDF5sig = "\211HDF\r\n\032\n";
    struct stat st;


if (ACCESS (filename, 0) || stat (filename, &st) ||
S_IFREG != (st.st_mode & S_IFREG)) {
last_err = CGIO_ERR_NOT_FOUND;
return last_err;
}

Environment

None

Status

Assignee

Unassigned

Reporter

Scot Breitenfeld

Components

Fix versions

Affects versions

Priority

Blocker
Configure