Libtiff 4.0.8 released

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Libtiff 4.0.8 released

Bob Friesenhahn
Libtiff 4.0.8 is now released thanks to the considerable efforts of
Even Rouault.

All of the changes are bug and security fixes.

Information on the release may be found at
http://www.simplesystems.org/libtiff/ or http://libtiff.maptools.org/ 
as soon as they update their content from the source repository.

The release files are avilable for download from
"ftp://download.osgeo.org/libtiff/".

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Libtiff 4.0.8 released

PSIRT
Is there a specific page that documents all the security fixes?

Thanks
Lisa



-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Bob Friesenhahn
Sent: Sunday, May 21, 2017 3:28 PM
To: Tiff List <[hidden email]>
Subject: [Tiff] Libtiff 4.0.8 released

Libtiff 4.0.8 is now released thanks to the considerable efforts of Even Rouault.

All of the changes are bug and security fixes.

Information on the release may be found at http://www.simplesystems.org/libtiff/ or http://libtiff.maptools.org/ as soon as they update their content from the source repository.

The release files are avilable for download from "ftp://download.osgeo.org/libtiff/".

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email] http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: Libtiff 4.0.8 released

Bob Friesenhahn
On Mon, 22 May 2017, PSIRT wrote:

> Is there a specific page that documents all the security fixes?

Yes. If you had visited one of the two referred web sites, you can
click on the "v4.0.8" link to see the details for the release.  Each
libtiff release has always been documented in this way.  For example

http://www.simplesystems.org/libtiff/v4.0.8.html

Bob

>
> Thanks
> Lisa
>
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Bob Friesenhahn
> Sent: Sunday, May 21, 2017 3:28 PM
> To: Tiff List <[hidden email]>
> Subject: [Tiff] Libtiff 4.0.8 released
>
> Libtiff 4.0.8 is now released thanks to the considerable efforts of Even Rouault.
>
> All of the changes are bug and security fixes.
>
> Information on the release may be found at http://www.simplesystems.org/libtiff/ or http://libtiff.maptools.org/ as soon as they update their content from the source repository.
>
> The release files are avilable for download from "ftp://download.osgeo.org/libtiff/".
>
> Bob
> --
> Bob Friesenhahn
> [hidden email], http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
> _______________________________________________
> Tiff mailing list: [hidden email] http://lists.maptools.org/mailman/listinfo/tiff
> http://www.remotesensing.org/libtiff/
> -----------------------------------------------------------------------------------
> This email message is for the sole use of the intended recipient(s) and may contain
> confidential information.  Any unauthorized review, use, disclosure or distribution
> is prohibited.  If you are not the intended recipient, please contact the sender by
> reply email and destroy all copies of the original message.
> -----------------------------------------------------------------------------------
>
>

--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

A bug in libtiff error/warning handling

Paavo Helde
In reply to this post by Bob Friesenhahn

Hi,

I would like to report what I think is a bug in libtiff error and
warning handling. There are two error handlers which can be installed
(via TIFFSetErrorHandler and TIFFSetErrorHandlerExt) and which are
called with a va_list. However, if both handlers are installed they will
both iterate through the same va_list without reinitialization which is
not allowed (seems to crash randomly with gcc on Linux, for example). I
believe it should be the task of libtiff to reinitialize va_list between
the calls. Ditto for warnings.

A patch file is attached, hopefully in a usable format.

Cheers
Paavo


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

TiffHandlerPatch (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Even Rouault-2

On mardi 4 juillet 2017 14:04:35 CEST Paavo Helde wrote:

> Hi,

>

> I would like to report what I think is a bug in libtiff error and

> warning handling. There are two error handlers which can be installed

> (via TIFFSetErrorHandler and TIFFSetErrorHandlerExt) and which are

> called with a va_list. However, if both handlers are installed they will

> both iterate through the same va_list without reinitialization which is

> not allowed (seems to crash randomly with gcc on Linux, for example). I

> believe it should be the task of libtiff to reinitialize va_list between

> the calls. Ditto for warnings.

>

> A patch file is attached, hopefully in a usable format.

 

That's a good point, but I wonder why you would have both error handlers installed ? That isn't really expected.

 

I'm wondering if we shouldn't use the HandlerExt on priority over the Handler to avoid the warning/error to be reported twice ? What do other devs think ?

 

But if we want to keep the current behaviour of reporting twice, your patch looks indeed right.

 

Even

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Roger Leigh
On 2017-07-04 12:30, Even Rouault wrote:
> p, li { white-space: pre-wrap; }
>
> On mardi 4 juillet 2017 14:04:35 CEST Paavo Helde wrote:
>
>> I would like to report what I think is a bug in libtiff error and
>
>> warning handling. There are two error handlers which can be
> installed
[...]
> That's a good point, but I wonder why you would have both error
> handlers installed ? That isn't really expected.

A larger application might use several libraries which might use
libtiff.  One library migth use one handler, and another library might
use the other.  Neither would be aware of the other's actions.

In the libraries I maintain, we use a thread-safe RAII wrapper around
libtiff to set and unset the handlers on the fly.  It's horrible, but it
does make this safer.  Though it still can't cope with another thread
directly setting the handlers behind its back.

> I'm wondering if we shouldn't use the HandlerExt on priority over the
> Handler to avoid the warning/error to be reported twice ? What do
> other devs think ?

Why can't Handler delegate to HandlerExt so that there's only ever a
single handler irrespective of which call you use?  Can't Handler
register its own private handler with HandlerExt which strips off the
thandle and calls the user-supplied handler?  Internally, only the Ext
variant need be invoked, with Handler wrapping it transparently.


Regards,
Roger

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Paavo Helde
In reply to this post by Even Rouault-2

On 4.07.2017 14:30, Even Rouault wrote:

On mardi 4 juillet 2017 14:04:35 CEST Paavo Helde wrote:

> Hi,

>

> I would like to report what I think is a bug in libtiff error and

> warning handling. There are two error handlers which can be installed

> (via TIFFSetErrorHandler and TIFFSetErrorHandlerExt) and which are

> called with a va_list. However, if both handlers are installed they will

> both iterate through the same va_list without reinitialization which is

> not allowed (seems to crash randomly with gcc on Linux, for example). I

> believe it should be the task of libtiff to reinitialize va_list between

> the calls. Ditto for warnings.

>

> A patch file is attached, hopefully in a usable format.

 

That's a good point, but I wonder why you would have both error handlers installed ? That isn't really expected.


The standard handler is initialized to unixErrorHandler by default and there is no indication I should set it to NULL when installing the extended error handler. Also, what I'm developing is a library which works in the same process with several other third-party libraries making use of the same libtiff library instance, some of which install their own handlers (not too many of them luckily, otherwise this would be a much more severe problem). When I reviewed common libraries which we share process with I discovered they are using TIFFSetErrorHandler(), that's the main reason why I used TIFFSetErrorHandlerExt() myself, to reduce conflicts with other libraries (however, I do not attempt to dereference the passed-in tif->tif_clientdata pointer because this may actually point to foreign data (e.g. libTIFF was called in another thread by another third-party library and has actually nothing to do with our library)).

A proper fix would be to deprecate the global handler variables in libtiff (which are not thread-safe anyway and will conflict between different libraries anyway) and store the handler function pointer in the TIFF structure instead, next to tif_clientdata. This would make the handler specific to a single client and then I would indeed agree there would be no need to call more than one handler at any time. I think it has been agreed earlier that this is something what is needed, but to get this fixed is obviously much more work.

Cheers
Paavo



_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Even Rouault-2
In reply to this post by Roger Leigh

 

> Why can't Handler delegate to HandlerExt so that there's only ever a

> single handler irrespective of which call you use? Can't Handler

> register its own private handler with HandlerExt which strips off the

> thandle and calls the user-supplied handler? Internally, only the Ext

> variant need be invoked, with Handler wrapping it transparently.

 

Hum, I'm not sure we can do that (or that would be tricky) since TIFFSetErrorHandler() must return the previous handler with a type of TIFFErrorHandler. Or actually that would mess a sequence fo calls of TIFFSetErrorHandler() and then TIFFSetErrorHandlerExt(), where the later would then return our internal ErrorHandlerExt wrapper instead of NULL currently.

 

I've just applied Paavo's patch for now. This should fix crashing situations.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Bob Friesenhahn
In reply to this post by Paavo Helde
On Tue, 4 Jul 2017, Paavo Helde wrote:
>
> A proper fix would be to deprecate the global handler variables in libtiff
> (which are not thread-safe anyway and will conflict between different
> libraries anyway) and store the handler function pointer in the TIFF
> structure instead, next to tif_clientdata. This would make the handler
> specific to a single client and then I would indeed agree there would be no
> need to call more than one handler at any time. I think it has been agreed
> earlier that this is something what is needed, but to get this fixed is
> obviously much more work.

This would not work for the common case of reporting an error when
there is no TIFF handle.  There would need to be additional parameters
added for the functions which open a a TIFF.

If OS thread APIs can be used, then there is the option to use thread
specific data without any API change, but a possible added library
dependency.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Paavo Helde

On 4.07.2017 21:38, Bob Friesenhahn wrote:
On Tue, 4 Jul 2017, Paavo Helde wrote:

A proper fix would be to deprecate the global handler variables in libtiff (which are not thread-safe anyway and will conflict between different libraries anyway) and store the handler function pointer in the TIFF structure instead, next to tif_clientdata. This would make the handler specific to a single client and then I would indeed agree there would be no need to call more than one handler at any time. I think it has been agreed earlier that this is something what is needed, but to get this fixed is obviously much more work.

This would not work for the common case of reporting an error when there is no TIFF handle.  There would need to be additional parameters added for the functions which open a a TIFF.

If OS thread APIs can be used, then there is the option to use thread specific data without any API change, but a possible added library dependency.

Thread-specific handlers would work fine for us. Our software is a large C++ app containing a ton of thread-specific data already. In C++ the thread local storage is even standardized since C++11 with the special thread_local storage class specifier, no third-party libraries needed, but in C the things might be different.

Paavo


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Edward Lam
In reply to this post by Bob Friesenhahn
On 7/4/2017 2:38 PM, Bob Friesenhahn wrote:
> This would not work for the common case of reporting an error when
> there is no TIFF handle.  There would need to be additional parameters
> added for the functions which open a a TIFF.
>
> If OS thread APIs can be used, then there is the option to use thread
> specific data without any API change, but a possible added library
> dependency.

Just thinking out loud here. If there was some way to get the client code to
provide a callback that returned a thread specific id, then libtiff could use
that as a key into a thread-specific data structure (eg. hash table). Then there
would not require an additional library dependency other than what the client
code decides that it wants to do. If no such function is provided, then we can
always just go back to the default current behaviour of not being thread-safe(?).

-Edward
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Olivier Paquet-2
2017-07-05 8:12 GMT-04:00 Edward Lam <[hidden email]>:

> On 7/4/2017 2:38 PM, Bob Friesenhahn wrote:
>> This would not work for the common case of reporting an error when
>> there is no TIFF handle.  There would need to be additional parameters
>> added for the functions which open a a TIFF.
>>
>> If OS thread APIs can be used, then there is the option to use thread
>> specific data without any API change, but a possible added library
>> dependency.
>
> Just thinking out loud here. If there was some way to get the client code to
> provide a callback that returned a thread specific id, then libtiff could use

And where do you store that callback? You're going round in circles here :-)

By the way, thread local storage is a poor solution as it does not
handle the case of several unrelated libraries using libtiff from
within the same thread. Giving TIFFOpen and a few other functions
which run without a TIFF handle the error handler is the proper way to
fix this problem. Either that or come with with a concept of "library
handle" which would do the same with more indirection and flexibility.
Perhaps it's time to design a new version of TIFFOpen as I'm fairly
certain I remember other issues with it. Or perhaps that was
TIFFClientOpen and windows handles.

Olivier
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Bob Friesenhahn
On Wed, 5 Jul 2017, Olivier Paquet wrote:
>
> And where do you store that callback? You're going round in circles here :-)
>
> By the way, thread local storage is a poor solution as it does not
> handle the case of several unrelated libraries using libtiff from
> within the same thread. Giving TIFFOpen and a few other functions
> which run without a TIFF handle the error handler is the proper way to
> fix this problem. Either that or come with with a concept of "library

Agreed.  We can not predict how libtiff may be used.  It may be that
the same thread has many tiff handles open at once and incrementally
accesses those handles.  Both an error handler and an optional opaque
pointer to data (which may be used by the error handler) are needed at
the time the handle is opened.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Paavo Helde
In reply to this post by Olivier Paquet-2

On 5.07.2017 15:40, Olivier Paquet wrote:
> By the way, thread local storage is a poor solution as it does not
> handle the case of several unrelated libraries using libtiff from
> within the same thread.

This can be made to work if the client library installs and restores the
handlers each time before/after any call to libTIFF. Yes that's tedious
but better than the current state. Better fixes would mean API changes,
like adding error/warning pointers to an enhanced version of
TIFFClientOpen().

Another hypothetical option would be to gather errors and warnings in
thread local storage inside libTIFF and provide special API calls for
retrieving these messages for the last libTIFF call in the current
thread. This means that there would be no need to install any
error/warning handlers; the client libraries which are not interested in
the messages would just not call these routines. The messages would be
automatically cleared in the start of the next libTIFF API call (in the
same thread).

This is basically the same what I do in our app when calling libtiff. I
gather the messages in the handlers (held in thread-local storage BTW
because I cannot trust the tif_clientdata pointer) and if the libtiff
call fails I will attach the gathered libtiff messages to my error message.

Paavo

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Olivier Paquet-2
2017-07-05 9:57 GMT-04:00 Paavo Helde <[hidden email]>:

>
> On 5.07.2017 15:40, Olivier Paquet wrote:
>> By the way, thread local storage is a poor solution as it does not
>> handle the case of several unrelated libraries using libtiff from
>> within the same thread.
>
> This can be made to work if the client library installs and restores the
> handlers each time before/after any call to libTIFF. Yes that's tedious
> but better than the current state. Better fixes would mean API changes,
> like adding error/warning pointers to an enhanced version of
> TIFFClientOpen().

Yes, it would require API changes but I think they are needed. I don't
think we should add a dependency to get thread local storage which
provides only half a fix when there is a better fix which does not
require it. Both require changes to client code anyway.

Besides, it would also break some stuff. I'm fairly certain our own
code sets the error handler only once and expects it to be used by
multiple threads. I doubt we're the only ones doing that. A new
version of TIFFClientOpen() would allow that to keep working.

The other thing to be decided is if we want to push the idea further
to have a library handle so we can also fix TIFFRegisterCODEC(),
TIFFSetTagExtender(), etc. Such a handle could hold all currently
global data and would be given to the new TIFFOpen. A little more
complexity but also potentially a more complete long term fix.

Olivier
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Paavo Helde

On 5.07.2017 18:02, Olivier Paquet wrote:
> The other thing to be decided is if we want to push the idea further
> to have a library handle so we can also fix TIFFRegisterCODEC(),
> TIFFSetTagExtender(), etc. Such a handle could hold all currently
> global data and would be given to the new TIFFOpen. A little more
> complexity but also potentially a more complete long term fix.

Do you mean two-step initialization, e.g. first creating an "engine" or
something, then using this to open TIFF files? That would be very nice
design! Different libraries (or a single library for that matter) could
then create multiple "engines" which would be totally independent.
Future extensions could be added by new API functions which modify the
engine object.

It should be clearly defined if and how this engine can be accessed from
multiple threads. If it can, then it must not require any external or
internal locking when processing the actual TIFF files - if the client
wants to read or write 20 different TIFF files in parallel, the library
should support that. Probably the simplest way to achieve that is to
require that the client does not access the engine in other threads when
calling mutator methods like TIFFRegisterCODEC().

Paavo


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: A bug in libtiff error/warning handling

Olivier Paquet-2
2017-07-05 14:51 GMT-04:00 Paavo Helde <[hidden email]>:
> Do you mean two-step initialization, e.g. first creating an "engine" or
> something, then using this to open TIFF files? That would be very nice
> design! Different libraries (or a single library for that matter) could
> then create multiple "engines" which would be totally independent.
> Future extensions could be added by new API functions which modify the
> engine object.

Yes, that's exactly the idea. It's an explicit container for what are
currently global variables. And indeed even a single library could use
multiple copies, for example to install a different error handler for
every open file. That would make sense for a C++ object wanting to
keep a list of its own errors.

> It should be clearly defined if and how this engine can be accessed from
> multiple threads. If it can, then it must not require any external or
> internal locking when processing the actual TIFF files - if the client
> wants to read or write 20 different TIFF files in parallel, the library
> should support that. Probably the simplest way to achieve that is to
> require that the client does not access the engine in other threads when
> calling mutator methods like TIFFRegisterCODEC().

I think a reasonable requirement is that nothing else is using it
while it is modified. TIFFOpen and everything else using the TIFF
handle should only have read-only access so multiple files can share
it. I think that's more or less how it is with current global
variables. And how you put it in your example.

Olivier
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/