Quantcast

started seeing breakage with libtiff

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

started seeing breakage with libtiff

jcupitt
Hello everyone,

Overnight I've started seeing problems with libtiff. It seems this
patch has just been rolled out to Ubuntu:

http://bugzilla.maptools.org/show_bug.cgi?id=2590

This issues a warning if some tags are not NULL-terminated. However,
libtiff itself can write tags which are not NULL-terminated, for
example:

$ tiffcp -c jpeg k2a.tif x.tif

(where k2a.tif is an old uncompressed RGB strip tif I had handy) works
fine. However I now can't read it back without a warning:

$ tiffcp x.tif y.tif
TIFFFetchNormalTag: Warning, ASCII value for tag "JPEGTables" does not
end in null byte. Forcing it to be null.
JPEGLib: Warning, Premature end of JPEG file.

This causes problems for some other libraries, such as openslide,
which seem to treat libtiff warnings as fatal.

Has anyone looked into this? What would be the best fix here? It looks
like libtiff perhaps shouldn't check null-termination on JPEGTables.

John
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

Even Rouault-2

On samedi 4 mars 2017 10:59:05 CET [hidden email] wrote:

> Hello everyone,

>

> Overnight I've started seeing problems with libtiff. It seems this

> patch has just been rolled out to Ubuntu:

>

> http://bugzilla.maptools.org/show_bug.cgi?id=2590

>

> This issues a warning if some tags are not NULL-terminated. However,

> libtiff itself can write tags which are not NULL-terminated, for

> example:

>

> $ tiffcp -c jpeg k2a.tif x.tif

>

> (where k2a.tif is an old uncompressed RGB strip tif I had handy) works

> fine. However I now can't read it back without a warning:

>

> $ tiffcp x.tif y.tif

> TIFFFetchNormalTag: Warning, ASCII value for tag "JPEGTables" does not

> end in null byte. Forcing it to be null.

> JPEGLib: Warning, Premature end of JPEG file.

 

I'm unable to reproduce with CVS HEAD. The warning about JPEGTables is weird, since this not supposed to be an ASCII tag.

 

>

> This causes problems for some other libraries, such as openslide,

> which seem to treat libtiff warnings as fatal.

>

> Has anyone looked into this? What would be the best fix here? It looks

> like libtiff perhaps shouldn't check null-termination on JPEGTables.

>

> John

> _______________________________________________

> Tiff mailing list: [hidden email]

> http://lists.maptools.org/mailman/listinfo/tiff

> http://www.remotesensing.org/libtiff/

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

jcupitt
On 4 March 2017 at 12:10, Even Rouault <[hidden email]> wrote:

> On samedi 4 mars 2017 10:59:05 CET [hidden email] wrote:
>> Overnight I've started seeing problems with libtiff. It seems this
>> patch has just been rolled out to Ubuntu:
>> http://bugzilla.maptools.org/show_bug.cgi?id=2590
>> $ tiffcp x.tif y.tif
>> TIFFFetchNormalTag: Warning, ASCII value for tag "JPEGTables" does not
>> end in null byte. Forcing it to be null.
>> JPEGLib: Warning, Premature end of JPEG file.
>
> I'm unable to reproduce with CVS HEAD. The warning about JPEGTables is
> weird, since this not supposed to be an ASCII tag.

Ah interesting. Perhaps it's an ubuntu or debian packaging problem.
I'll check there.

Thanks!

John
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

jcupitt
On 4 March 2017 at 13:08,  <[hidden email]> wrote:
>> I'm unable to reproduce with CVS HEAD. The warning about JPEGTables is
>> weird, since this not supposed to be an ASCII tag.
>
> Ah interesting. Perhaps it's an ubuntu or debian packaging problem.
> I'll check there.

For reference, the problem is in tiff_4.0.6-2ubuntu0.1, the version
currently being used in 16.10. Plain 4.0.6 seems to work, but one of
the (very many) patches Ubuntu is applying has broken stuff.

To reproduce:

http://packages.ubuntu.com/yakkety/libtiff5-dev

tar xf tiff_4.0.6-2ubuntu0.1.debian.tar.xz
tar xf tiff_4.0.6.orig.tar.gz
cd tiff-4.0.6
for i in ../debian/patches/*.patch; do patch -p1 < $i; done

I've reported a bug on launchpad.

John
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

Bob Friesenhahn
On Sat, 4 Mar 2017, [hidden email] wrote:

> On 4 March 2017 at 13:08,  <[hidden email]> wrote:
>>> I'm unable to reproduce with CVS HEAD. The warning about JPEGTables is
>>> weird, since this not supposed to be an ASCII tag.
>>
>> Ah interesting. Perhaps it's an ubuntu or debian packaging problem.
>> I'll check there.
>
> For reference, the problem is in tiff_4.0.6-2ubuntu0.1, the version
> currently being used in 16.10. Plain 4.0.6 seems to work, but one of
> the (very many) patches Ubuntu is applying has broken stuff.

They are one release behind.

Even Rouault has (again) made a great many fixes to the development
version.  After some testing of the development version by interested
parties, it would be good to release 4.0.8.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

Even Rouault-2
In reply to this post by jcupitt

On samedi 4 mars 2017 18:51:20 CET [hidden email] wrote:

> tar xf tiff_4.0.6-2ubuntu0.1.debian.tar.xz

> tar xf tiff_4.0.6.orig.tar.gz

> cd tiff-4.0.6

> for i in ../debian/patches/*.patch; do patch -p1 < $i; done

 

Actually to reproduce, you need to apply the patches in a precise order with

 

for i in `cat ../debian/patches/series`; do \

patch -p1 <../debian/patches/$i; done

 

I've then compared the patched libtiff/tif_dirread.c with the official one from CVS, and I understand now what happens in Debian/Ubuntu.

 

It appears that the following snippet

if( dp->tdir_count > 0 && data[dp->tdir_count-1] != '\0' )

{

TIFFWarningExt(tif->tif_clientdata,module,"ASCII value for tag \"%s\" does not end in null byte. Forcing it to be null",fip->field_name);

data[dp->tdir_count-1] = '\0';

}

 

that in official libtiff is applied in the TIFF_SETGET_C16_ASCII cases (line 5017 in HEAD) and in the TIFF_SETGET_C32_ASCII cases (line 5194 in CVS HEAD) has been wrongly applied in Debian in the TIFF_SETGET_C16_UINT8 case (line 5008) and TIFF_SETGET_C32_UINT8 case (line 5180)...

This explains the warning about the JPEGTables...

 

Unfortunately "make check" in the Debian patched libtiff still passes, so they have some excuse. Not so surprised since the test suite is rather small.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

jcupitt
On 4 March 2017 at 19:53, Even Rouault <[hidden email]> wrote:
> that in official libtiff is applied in the TIFF_SETGET_C16_ASCII cases (line
> 5017 in HEAD) and in the TIFF_SETGET_C32_ASCII cases (line 5194 in CVS HEAD)
> has been wrongly applied in Debian in the TIFF_SETGET_C16_UINT8 case (line
> 5008) and TIFF_SETGET_C32_UINT8 case (line 5180)...

Thank you Even, I'll fwd your mail to the debian/ubuntu bug page, I
hope that's OK.

John
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

Lee Howard-2
In reply to this post by jcupitt
On 03/04/2017 10:51 AM, [hidden email] wrote:
>
> For reference, the problem is in tiff_4.0.6-2ubuntu0.1, the version
> currently being used in 16.10. Plain 4.0.6 seems to work, but one of
> the (very many) patches Ubuntu is applying has broken stuff.

I wish package maintainers at the various distros wouldn't glamorize
patches as they seem to do.  They should be involved in the upstream
project development and seek to integrate as many patches as possible so
that the only patches they apply are truly distribution-specific.  Most
customization patches can still be upstreamed and enabled with
build-time flags.

Thanks,

Lee.
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

Bob Friesenhahn
On Mon, 6 Mar 2017, Lee Howard wrote:

> On 03/04/2017 10:51 AM, [hidden email] wrote:
>>
>> For reference, the problem is in tiff_4.0.6-2ubuntu0.1, the version
>> currently being used in 16.10. Plain 4.0.6 seems to work, but one of
>> the (very many) patches Ubuntu is applying has broken stuff.
>
> I wish package maintainers at the various distros wouldn't glamorize
> patches as they seem to do.  They should be involved in the upstream
> project development and seek to integrate as many patches as possible so
> that the only patches they apply are truly distribution-specific.  Most
> customization patches can still be upstreamed and enabled with
> build-time flags.

Distributions like Debian (and thus Ubuntu) seem to tie their hands
behind their back due to an apparent rule that it is forbidden to
solve security problems by updating to the current release known to
solve those problems.  Instead everything is handled via patches.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

Toby Thain-2
On 2017-03-06 10:46 AM, Bob Friesenhahn wrote:

> On Mon, 6 Mar 2017, Lee Howard wrote:
>
>> On 03/04/2017 10:51 AM, [hidden email] wrote:
>>>
>>> For reference, the problem is in tiff_4.0.6-2ubuntu0.1, the version
>>> currently being used in 16.10. Plain 4.0.6 seems to work, but one of
>>> the (very many) patches Ubuntu is applying has broken stuff.
>>
>> I wish package maintainers at the various distros wouldn't glamorize
>> patches as they seem to do.  They should be involved in the upstream
>> project development and seek to integrate as many patches as possible so
>> that the only patches they apply are truly distribution-specific.  Most
>> customization patches can still be upstreamed and enabled with
>> build-time flags.
>
> Distributions like Debian (and thus Ubuntu) seem to tie their hands
> behind their back due to an apparent rule that it is forbidden to
> solve security problems by updating to the current release known to
> solve those problems.  Instead everything is handled via patches.

That sounds like the same ultra-conservative policy that systems like
Solaris had to adopt - the goal being to have updates cause as little
customer breakage as possible?

I can sort of see points on both sides of this... The two modes are not
easy to reconcile?

--Toby


>
> Bob
>

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: started seeing breakage with libtiff

Lee Howard-2
On 03/06/2017 07:59 AM, Toby Thain wrote:

> On 2017-03-06 10:46 AM, Bob Friesenhahn wrote:
>> On Mon, 6 Mar 2017, Lee Howard wrote:
>>
>>> On 03/04/2017 10:51 AM, [hidden email] wrote:
>>>> For reference, the problem is in tiff_4.0.6-2ubuntu0.1, the version
>>>> currently being used in 16.10. Plain 4.0.6 seems to work, but one of
>>>> the (very many) patches Ubuntu is applying has broken stuff.
>>> I wish package maintainers at the various distros wouldn't glamorize
>>> patches as they seem to do.  They should be involved in the upstream
>>> project development and seek to integrate as many patches as possible so
>>> that the only patches they apply are truly distribution-specific.  Most
>>> customization patches can still be upstreamed and enabled with
>>> build-time flags.
>> Distributions like Debian (and thus Ubuntu) seem to tie their hands
>> behind their back due to an apparent rule that it is forbidden to
>> solve security problems by updating to the current release known to
>> solve those problems.  Instead everything is handled via patches.
> That sounds like the same ultra-conservative policy that systems like
> Solaris had to adopt - the goal being to have updates cause as little
> customer breakage as possible?
>
> I can sort of see points on both sides of this... The two modes are not
> easy to reconcile?

This is why the package maintainers need to be involved in the upstream
project development.  There they can argue for an upstream patch-level
release outside of a feature release.  It keeps them out of the patching
mess that the OP ran into, and it also satisfies the packaging guideline
policies that prohibit anything other than security fixes.

Obviously, at some point upstream will not be interested in releasing
outdated patch-level updates.  Hopefully that interest expires long
after the distro is interested in providing security updates.  But if
not, then they can do their patching thing.   It just seems to me that
patching should be the last resort rather than the first response.

Thanks,

Lee.
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Loading...