Sporadic crash on Windows when appending images to TIFF file

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Sporadic crash on Windows when appending images to TIFF file

Dinesh Iyer
Hi everyone,
I am using libTIFF 4.0.0 and I am receiving a sporadic error/crash when I am trying to write multiple images to a TIFF file only on Windows. 

I have not yet been able to generate simplified reproduction steps but the outline of the code that results in this error is below:

// Write the first image
TIFF* tif = TIFFOpenW(fileName, "w"); 

// Set Tags before writing images
// Write the first image
TIFFClose(tif);

// Call the code below repeatedly say 500 times. An error/crash happens sporadically
TIFF* tif = TIFFOpenW(fileName, "a"); 

// Set Tags before writing images
// Write the first image
TIFFClose(tif);


There are no calls to TIFFWriteDirectory or TIFFSetDirectory in my code.

In some instances, opening the file fails because there are no write permissions. Other times, I get a crash with the following trace:
Stack Trace (from fault):
[  0] 0x00000000084321a0 TIFFFindField at C:\temp\amake_2263_6088\win64\tiff\libtiff\tif_dirinfo.c:490 
[  1] 0x000000000842dd47 OkToChangeTag at c:\temp\amake_2263_6088\win64\tiff\libtiff\tif_dir.c:691
[  2] 0x000000000842f574 TIFFSetField at c:\temp\amake_2263_6088\win64\tiff\libtiff\tif_dir.c:726
[  3] 0x000000000842c3df writeTIFFTags (my function)
<SNIP>

I am always writing to a local disk and not any network location. Just for testing, I added a pause of 0.5 seconds after writing each image and it reduced the frequency of the error/crash. Any thoughts?

Also, is there a way for me to build libtiff in debug mode on Windows so that i can do some more investigation on my end?

Regards,
Dinesh

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Bob Friesenhahn
On Wed, 24 May 2017, Dinesh Iyer wrote:

> Hi everyone,
> I am using libTIFF 4.0.0 and I am receiving a sporadic error/crash when I
> am trying to write multiple images to a TIFF file only on Windows.

Your first step should be to update to libtiff 4.0.8, which was
released this past weekend.  It does not make sense to use an old
version which contains many bugs which are already fixed.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Dinesh Iyer
Hi Bob,
Does that mean that this issue has been fixed in 4.0.8? The process of upgrading a library version is non-trivial in our workplace and so if this bug has been indeed fixed, I would be happy to move to the latest version.

Dinesh

On Wed, May 24, 2017 at 12:21 PM, Bob Friesenhahn <[hidden email]> wrote:
On Wed, 24 May 2017, Dinesh Iyer wrote:

Hi everyone,
I am using libTIFF 4.0.0 and I am receiving a sporadic error/crash when I
am trying to write multiple images to a TIFF file only on Windows.

Your first step should be to update to libtiff 4.0.8, which was released this past weekend.  It does not make sense to use an old version which contains many bugs which are already fixed.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Bob Friesenhahn
On Wed, 24 May 2017, Dinesh Iyer wrote:

> Hi Bob,
> Does that mean that this issue has been fixed in 4.0.8? The process of
> upgrading a library version is non-trivial in our workplace and so if this
> bug has been indeed fixed, I would be happy to move to the latest version.

I am unable to tell you if the problem is fixed.  What I can tell you
is that active libtiff developers are not usually using Windows for
development (although Windows is supported) so it is not getting as
much developer testing under Windows as it should.  We rely on the
larger community to help assure that libtiff is working correctly
under Windows.

A large number of bugs and security issues have been fixed since
4.0.0.  The fixes we are aware of are mentioned in the change logs,
but some issues which were fixed could have unknown side effects.

To look at the change logs, first go to
http://www.simplesystems.org/libtiff/v4.0.8.html, and after reading
that page, click on v4.0.7, read that page, click on v4.0.6, etc.

Bob

>
> Dinesh
>
> On Wed, May 24, 2017 at 12:21 PM, Bob Friesenhahn <
> [hidden email]> wrote:
>
>> On Wed, 24 May 2017, Dinesh Iyer wrote:
>>
>> Hi everyone,
>>> I am using libTIFF 4.0.0 and I am receiving a sporadic error/crash when I
>>> am trying to write multiple images to a TIFF file only on Windows.
>>>
>>
>> Your first step should be to update to libtiff 4.0.8, which was released
>> this past weekend.  It does not make sense to use an old version which
>> contains many bugs which are already fixed.
>>
>> Bob
>> --
>> Bob Friesenhahn
>> [hidden email], http://www.simplesystems.org/users/bfriesen/
>> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
>>
>

--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Dinesh Iyer
Thanks Bob!

That makes sense ... I will upgrade to the latest version and let you know if this bug has been fixed as a side effect.

Dinesh

On Wed, May 24, 2017 at 3:19 PM, Bob Friesenhahn <[hidden email]> wrote:
On Wed, 24 May 2017, Dinesh Iyer wrote:

Hi Bob,
Does that mean that this issue has been fixed in 4.0.8? The process of
upgrading a library version is non-trivial in our workplace and so if this
bug has been indeed fixed, I would be happy to move to the latest version.

I am unable to tell you if the problem is fixed.  What I can tell you is that active libtiff developers are not usually using Windows for development (although Windows is supported) so it is not getting as much developer testing under Windows as it should.  We rely on the larger community to help assure that libtiff is working correctly under Windows.

A large number of bugs and security issues have been fixed since 4.0.0.  The fixes we are aware of are mentioned in the change logs, but some issues which were fixed could have unknown side effects.

To look at the change logs, first go to http://www.simplesystems.org/libtiff/v4.0.8.html, and after reading that page, click on v4.0.7, read that page, click on v4.0.6, etc.

Bob



Dinesh

On Wed, May 24, 2017 at 12:21 PM, Bob Friesenhahn <
[hidden email]> wrote:

On Wed, 24 May 2017, Dinesh Iyer wrote:

Hi everyone,
I am using libTIFF 4.0.0 and I am receiving a sporadic error/crash when I
am trying to write multiple images to a TIFF file only on Windows.


Your first step should be to update to libtiff 4.0.8, which was released
this past weekend.  It does not make sense to use an old version which
contains many bugs which are already fixed.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/



--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Vincent Torri
In reply to this post by Dinesh Iyer
On Wed, May 24, 2017 at 5:42 PM, Dinesh Iyer <[hidden email]> wrote:

> Hi everyone,
> I am using libTIFF 4.0.0 and I am receiving a sporadic error/crash when I am
> trying to write multiple images to a TIFF file only on Windows.
>
> I have not yet been able to generate simplified reproduction steps but the
> outline of the code that results in this error is below:
>
> // Write the first image
> TIFF* tif = TIFFOpenW(fileName, "w");
>
> // Set Tags before writing images
> // Write the first image
> TIFFClose(tif);
>
> // Call the code below repeatedly say 500 times. An error/crash happens
> sporadically
> TIFF* tif = TIFFOpenW(fileName, "a");
>
> // Set Tags before writing images
> // Write the first image
> TIFFClose(tif);
>
>
> There are no calls to TIFFWriteDirectory or TIFFSetDirectory in my code.
>
> In some instances, opening the file fails because there are no write
> permissions. Other times, I get a crash with the following trace:
> Stack Trace (from fault):
> [  0] 0x00000000084321a0 TIFFFindField at
> C:\temp\amake_2263_6088\win64\tiff\libtiff\tif_dirinfo.c:490
> [  1] 0x000000000842dd47 OkToChangeTag at
> c:\temp\amake_2263_6088\win64\tiff\libtiff\tif_dir.c:691
> [  2] 0x000000000842f574 TIFFSetField at
> c:\temp\amake_2263_6088\win64\tiff\libtiff\tif_dir.c:726
> [  3] 0x000000000842c3df writeTIFFTags (my function)
> <SNIP>
>
> I am always writing to a local disk and not any network location. Just for
> testing, I added a pause of 0.5 seconds after writing each image and it
> reduced the frequency of the error/crash. Any thoughts?
>
> Also, is there a way for me to build libtiff in debug mode on Windows so
> that i can do some more investigation on my end?

Are you using Visual Studio or MSYS/mingw ?

Drmemory is a good tool to find problems on Windows (it's a valgrind-like tool)

Vincent Torri
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Dinesh Iyer
In reply to this post by Dinesh Iyer
Hi Bob,
Just a quick follow up on this issue. 

Full disclosure: I have not tried this on 4.0.8.

I was able to reproduce this issue by just calling TIFFOpenW(fileName, "a") followed by TIFFClose() repeatedly without writing any image data. The TIFF library error generated is:

TIFFOpen: myfile.tif: Cannot open.

which is generated when the call to CreateFileW in tif_win32.c fails? Again this failure happens sporadically.

I added a guard in my code to retry opening the file if an error occurs with a pause 0.1 seconds between retries (max 5 retries) and the errors seemed to have gone away.

This might be a Windows file handling issue.

Dinesh

On Wed, May 24, 2017 at 3:33 PM, Dinesh Iyer <[hidden email]> wrote:
Thanks Bob!

That makes sense ... I will upgrade to the latest version and let you know if this bug has been fixed as a side effect.

Dinesh

On Wed, May 24, 2017 at 3:19 PM, Bob Friesenhahn <[hidden email]> wrote:
On Wed, 24 May 2017, Dinesh Iyer wrote:

Hi Bob,
Does that mean that this issue has been fixed in 4.0.8? The process of
upgrading a library version is non-trivial in our workplace and so if this
bug has been indeed fixed, I would be happy to move to the latest version.

I am unable to tell you if the problem is fixed.  What I can tell you is that active libtiff developers are not usually using Windows for development (although Windows is supported) so it is not getting as much developer testing under Windows as it should.  We rely on the larger community to help assure that libtiff is working correctly under Windows.

A large number of bugs and security issues have been fixed since 4.0.0.  The fixes we are aware of are mentioned in the change logs, but some issues which were fixed could have unknown side effects.

To look at the change logs, first go to http://www.simplesystems.org/libtiff/v4.0.8.html, and after reading that page, click on v4.0.7, read that page, click on v4.0.6, etc.

Bob



Dinesh

On Wed, May 24, 2017 at 12:21 PM, Bob Friesenhahn <
[hidden email]> wrote:

On Wed, 24 May 2017, Dinesh Iyer wrote:

Hi everyone,
I am using libTIFF 4.0.0 and I am receiving a sporadic error/crash when I
am trying to write multiple images to a TIFF file only on Windows.


Your first step should be to update to libtiff 4.0.8, which was released
this past weekend.  It does not make sense to use an old version which
contains many bugs which are already fixed.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/



--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/



_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Bob Friesenhahn
On Fri, 26 May 2017, Dinesh Iyer wrote:
>
> which is generated when the call to CreateFileW in tif_win32.c fails? Again
> this failure happens sporadically.
>
> I added a guard in my code to retry opening the file if an error occurs
> with a pause 0.1 seconds between retries (max 5 retries) and the errors
> seemed to have gone away.
>
> This might be a Windows file handling issue.

There are two possible I/O implementations available under Windows.
The implementations are tif_win32.c and tif_unix.c in the libtiff
subdirectory.  Either of these may be used and you can switch your
build to use tif_unix.c rather than tif_win32.c.

There is a known issue with tif_win32.c in that libtiff only preserves
the lower 32-bits (or maybe 31 bits) of the Windows file handle so if
the handle requires more bits, then there would be a failure.

In recent libtiff 4.0.X I would expect tif_unix.c to function as well
as tif_win32.c, and in some cases better.  I am not sure if unicode
filenames are supported with tif_unix.c though.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Sporadic crash on Windows when appending images to TIFF file

Dinesh Iyer
Thanks Bob. I will give this a shot as part of upgrading to 4.0.8

On Fri, May 26, 2017 at 11:42 AM, Bob Friesenhahn <[hidden email]> wrote:
On Fri, 26 May 2017, Dinesh Iyer wrote:

which is generated when the call to CreateFileW in tif_win32.c fails? Again
this failure happens sporadically.

I added a guard in my code to retry opening the file if an error occurs
with a pause 0.1 seconds between retries (max 5 retries) and the errors
seemed to have gone away.

This might be a Windows file handling issue.

There are two possible I/O implementations available under Windows. The implementations are tif_win32.c and tif_unix.c in the libtiff subdirectory.  Either of these may be used and you can switch your build to use tif_unix.c rather than tif_win32.c.

There is a known issue with tif_win32.c in that libtiff only preserves the lower 32-bits (or maybe 31 bits) of the Windows file handle so if the handle requires more bits, then there would be a failure.

In recent libtiff 4.0.X I would expect tif_unix.c to function as well as tif_win32.c, and in some cases better.  I am not sure if unicode filenames are supported with tif_unix.c though.


Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/