[PATCH] Windows CI support for GitHub and AppVeyor

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH] Windows CI support for GitHub and AppVeyor

Roger Leigh
Hi,

Attached are two patches to improve the Windows support for libtiff.

- 0001-cmake-Improve-Cygwin-and-MingGW-test-support.patch
   This patch makes the CMake build system support running the tests
   with MinGW or Cygwin

- 0002-ci-Add-AppVeyor-support.patch
   This patch adds a .appveyor.yml file to the top-level.  This allows
   one to opt in to having a branch built on Windows with Cygwin,
   MinGW and MSVC automatically when a branch is pushed to GitHub,
   GitLab, BitBucket or any other supported git hosting service.


How does it work?

Take a look at my GitHub branch for the above two patches:
https://github.com/rleigh-codelibre/libtiff/commits/ci-appveyor
then click on the green tick mark and it will take you to
https://ci.appveyor.com/project/rleigh-codelibre/libtiff/build/1.0.30
(at the time of writing).  This contains the build logs and status for
each combination of compiler and configuration options tested.

We can vary the number of combinations tested; more means it takes
longer to test.


Why do this?

The Windows test jobs hosted here
https://ci.openmicroscopy.org/view/Third-Party/job/TIFF-HEAD-win-cmake/ 
and
https://ci.openmicroscopy.org/view/Third-Party/job/TIFF-HEAD-win-nmake/ 
are likely to disappear in the future--we're replacing this CI
infrastructure entirely over the next few months, and most of the jobs
here will disappear unless reimplemented on the replacement machines.  I
wanted to provide an alternative which was overall an improvement.

This is something which anyone hacking on libtiff can opt into using (or
ignore).  You can sign up for an AppVeyor account using your
GitHub/BitBucket/GitLab credentials, and enable your libtiff git
repository, and after that any branch you push will be automatically
tested (it's free for open source projects).  If the GitHub fork
"vadz/libtiff" enabled it, then any CVS commit would also be
automatically tested.  The aim is to make it easy for anyone to get
better automated testing of their changes.

I'll follow up with another patch for Unix as well probably using Travis
CI; this is for the Windows side of things.


Regards,
Roger

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

0001-cmake-Improve-Cygwin-and-MingGW-test-support.patch (4K) Download Attachment
0002-ci-Add-AppVeyor-support.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Windows CI support for GitHub and AppVeyor

Roger Leigh
On 2017-06-07 23:30, Roger Leigh wrote:

> Hi,
>
> Attached are two patches to improve the Windows support for libtiff.
>
> - 0001-cmake-Improve-Cygwin-and-MingGW-test-support.patch
>   This patch makes the CMake build system support running the tests
>   with MinGW or Cygwin
>
> - 0002-ci-Add-AppVeyor-support.patch
>   This patch adds a .appveyor.yml file to the top-level.  This allows
>   one to opt in to having a branch built on Windows with Cygwin,
>   MinGW and MSVC automatically when a branch is pushed to GitHub,
>   GitLab, BitBucket or any other supported git hosting service.
Updated copies of the patch attached.  I also added support for testing
the nmake Makefile.vc build as well.

You can see this in action here:
https://ci.appveyor.com/project/rleigh-codelibre/libtiff/build/1.0.43


Regards,
Roger

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

0001-cmake-Improve-Cygwin-and-MingGW-test-support.patch (5K) Download Attachment
0002-ci-Add-AppVeyor-support.patch (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Roger Leigh
In reply to this post by Roger Leigh
On 2017-06-07 23:30, Roger Leigh wrote:

> I'll follow up with another patch for Unix as well probably using
> Travis CI; this is for the Windows side of things.

As promised, this patch does the same think for Unix/Linux as the
previous ones did for Windows.

You can find the branch here:

   https://github.com/rleigh-codelibre/libtiff/commits/ci-travis

and follow the build link to here:

   https://travis-ci.org/rleigh-codelibre/libtiff

where it shows builds with Autoconf and CMake for both the GCC and Clang
compilers.  Autoconf is tested with Make; CMake is tested with Make and
Ninja.

Together with the Windows patches, this provides pretty decent test
coverage for common platforms and compilers, and should pick up most
regressions in the codebase.


Regards,
Roger
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

0001-ci-Add-Travis-support-for-Linux-builds-with-Autoconf.patch (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Even Rouault-2

On jeudi 8 juin 2017 12:22:01 CEST [hidden email] wrote:

> On 2017-06-07 23:30, Roger Leigh wrote:

> > I'll follow up with another patch for Unix as well probably using

> > Travis CI; this is for the Windows side of things.

>

> As promised, this patch does the same think for Unix/Linux as the

> previous ones did for Windows.

>

> You can find the branch here:

>

> https://github.com/rleigh-codelibre/libtiff/commits/ci-travis

>

> and follow the build link to here:

>

> https://travis-ci.org/rleigh-codelibre/libtiff

>

> where it shows builds with Autoconf and CMake for both the GCC and Clang

> compilers. Autoconf is tested with Make; CMake is tested with Make and

> Ninja.

>

> Together with the Windows patches, this provides pretty decent test

> coverage for common platforms and compilers, and should pick up most

> regressions in the codebase.

 

All 3 patches committed. Thanks !

 

Even

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Roger Leigh
On 2017-06-08 21:46, Even Rouault wrote:

>> Together with the Windows patches, this provides pretty decent test
>> coverage for common platforms and compilers, and should pick up most
>> regressions in the codebase.
>
> All 3 patches committed. Thanks !

Thank you.  One problem I noticed is that the build/travis-ci script
does not have executable permissions, which is needed to build.  I think
with CVS that requires a chmod +x of the ,v file in the repository if
was not set when initially added.

I've also attached a new patch with a few small improvements and the
addition of MacOS X to the test matrix.  You can see the results here:

https://github.com/rleigh-codelibre/libtiff/commits/ci-fixes
https://travis-ci.org/rleigh-codelibre/libtiff/builds/241129101
https://ci.appveyor.com/project/rleigh-codelibre/libtiff/build/1.0.53



Regards,
Roger
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

0001-ci-Travis-script-improvements.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Even Rouault-2

On vendredi 9 juin 2017 11:54:53 CEST [hidden email] wrote:

> On 2017-06-08 21:46, Even Rouault wrote:

> >> Together with the Windows patches, this provides pretty decent test

> >> coverage for common platforms and compilers, and should pick up most

> >> regressions in the codebase.

> >

> > All 3 patches committed. Thanks !

>

> Thank you. One problem I noticed is that the build/travis-ci script

> does not have executable permissions, which is needed to build. I think

> with CVS that requires a chmod +x of the ,v file in the repository if

> was not set when initially added.

 

Arg... damned CVS. I do not have access to the server. I've asked an admin if he can do that (I tried to cvs delete, commit, do chmod +x, cvs add, commit, but that doesn't seem to hve an effect)

 

I'll defer applying your new patch until this is solved

 

>

> I've also attached a new patch with a few small improvements and the

> addition of MacOS X to the test matrix. You can see the results here:

>

> https://github.com/rleigh-codelibre/libtiff/commits/ci-fixes

> https://travis-ci.org/rleigh-codelibre/libtiff/builds/241129101

> https://ci.appveyor.com/project/rleigh-codelibre/libtiff/build/1.0.53

>

>

>

> Regards,

> Roger

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Bob Friesenhahn
In reply to this post by Roger Leigh
On Fri, 9 Jun 2017, [hidden email] wrote:
>
> Thank you.  One problem I noticed is that the build/travis-ci script does not
> have executable permissions, which is needed to build.  I think with CVS that
> requires a chmod +x of the ,v file in the repository if was not set when
> initially added.

This seems to be a shell script which wants to be run with /bin/sh.
As such, it could be executed like '/bin/sh build/travis-ci' (or
whatever is needed for the context in which it is used) without any
execute bits set.  This would also help for operating systems or
filesystems which have no notion of an execute bit.

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Roger Leigh


On 09/06/17 14:18, Bob Friesenhahn wrote:

> On Fri, 9 Jun 2017, [hidden email] wrote:
>>
>> Thank you.  One problem I noticed is that the build/travis-ci script
>> does not have executable permissions, which is needed to build.  I
>> think with CVS that requires a chmod +x of the ,v file in the
>> repository if was not set when initially added.
>
> This seems to be a shell script which wants to be run with /bin/sh. As
> such, it could be executed like '/bin/sh build/travis-ci' (or whatever
> is needed for the context in which it is used) without any execute bits
> set.  This would also help for operating systems or filesystems which
> have no notion of an execute bit.

If it's not possible to tweak the executable bit, I can certainly switch
to invoking the script by hand.  Since it's only ever going to be run on
Unix platforms via the .travis.yml, we don't need to worry about
filesystems with no execute bit.

Regards,
Roger
_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Even Rouault-2

On vendredi 9 juin 2017 22:10:49 CEST Roger Leigh wrote:

> On 09/06/17 14:18, Bob Friesenhahn wrote:

> > On Fri, 9 Jun 2017, [hidden email] wrote:

> >> Thank you. One problem I noticed is that the build/travis-ci script

> >> does not have executable permissions, which is needed to build. I

> >> think with CVS that requires a chmod +x of the ,v file in the

> >> repository if was not set when initially added.

> >

> > This seems to be a shell script which wants to be run with /bin/sh. As

> > such, it could be executed like '/bin/sh build/travis-ci' (or whatever

> > is needed for the context in which it is used) without any execute bits

> > set. This would also help for operating systems or filesystems which

> > have no notion of an execute bit.

>

> If it's not possible to tweak the executable bit, I can certainly switch

> to invoking the script by hand. Since it's only ever going to be run on

> Unix platforms via the .travis.yml, we don't need to worry about

> filesystems with no execute bit.

 

Yes please do so. The adminstrator has just informed me that they have issues to SSH connect to the server currently.

 

>

> Regards,

> Roger

> _______________________________________________

> Tiff mailing list: [hidden email]

> http://lists.maptools.org/mailman/listinfo/tiff

> http://www.remotesensing.org/libtiff/

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Roger Leigh
On 09/06/17 22:23, Even Rouault wrote:

> On vendredi 9 juin 2017 22:10:49 CEST Roger Leigh wrote:
>
>  > On 09/06/17 14:18, Bob Friesenhahn wrote:
>
>  > > On Fri, 9 Jun 2017, [hidden email] wrote:
>
>  > >> Thank you. One problem I noticed is that the build/travis-ci script
>
>  > >> does not have executable permissions, which is needed to build. I
>
>  > >> think with CVS that requires a chmod +x of the ,v file in the
>
>  > >> repository if was not set when initially added.
>
>  > >
>
>  > > This seems to be a shell script which wants to be run with /bin/sh. As
>
>  > > such, it could be executed like '/bin/sh build/travis-ci' (or whatever
>
>  > > is needed for the context in which it is used) without any execute bits
>
>  > > set. This would also help for operating systems or filesystems which
>
>  > > have no notion of an execute bit.
>
>  >
>
>  > If it's not possible to tweak the executable bit, I can certainly switch
>
>  > to invoking the script by hand. Since it's only ever going to be run on
>
>  > Unix platforms via the .travis.yml, we don't need to worry about
>
>  > filesystems with no execute bit.
>
> Yes please do so. The adminstrator has just informed me that they have
> issues to SSH connect to the server currently.
That doesn't sound good, hope they regain access soon.  Related to that,
is there any plan to move fully over to git?  The existing mirror works
pretty well; all my recent patches are generated directly with it.
Could we create a libtiff organisation on GitHub or GitLab?
Unfortunately, "libtiff" is taken on github, as is "tiff".  But libtiff
is available on GitLab.


I've attached the patches to invoke the script directly (0002) plus the
original fixes (0001).


Regards,
Roger


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

0001-ci-Travis-script-improvements.patch (6K) Download Attachment
0002-ci-Invoke-helper-script-via-shell.patch (654 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Aaron Boxer
+1 for GitLab. Very happy with my account there.

On Fri, Jun 9, 2017 at 6:00 PM, Roger Leigh <[hidden email]> wrote:
On 09/06/17 22:23, Even Rouault wrote:
On vendredi 9 juin 2017 22:10:49 CEST Roger Leigh wrote:

 > On 09/06/17 14:18, Bob Friesenhahn wrote:

 > > On Fri, 9 Jun 2017, [hidden email] wrote:

 > >> Thank you. One problem I noticed is that the build/travis-ci script

 > >> does not have executable permissions, which is needed to build. I

 > >> think with CVS that requires a chmod +x of the ,v file in the

 > >> repository if was not set when initially added.

 > >

 > > This seems to be a shell script which wants to be run with /bin/sh. As

 > > such, it could be executed like '/bin/sh build/travis-ci' (or whatever

 > > is needed for the context in which it is used) without any execute bits

 > > set. This would also help for operating systems or filesystems which

 > > have no notion of an execute bit.

 >

 > If it's not possible to tweak the executable bit, I can certainly switch

 > to invoking the script by hand. Since it's only ever going to be run on

 > Unix platforms via the .travis.yml, we don't need to worry about

 > filesystems with no execute bit.

Yes please do so. The adminstrator has just informed me that they have issues to SSH connect to the server currently.

That doesn't sound good, hope they regain access soon.  Related to that, is there any plan to move fully over to git?  The existing mirror works pretty well; all my recent patches are generated directly with it. Could we create a libtiff organisation on GitHub or GitLab? Unfortunately, "libtiff" is taken on github, as is "tiff".  But libtiff is available on GitLab.


I've attached the patches to invoke the script directly (0002) plus the original fixes (0001).


Regards,
Roger


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Even Rouault-2
In reply to this post by Roger Leigh

> That doesn't sound good, hope they regain access soon. Related to that,

> is there any plan to move fully over to git? The existing mirror works

> pretty well; all my recent patches are generated directly with it.

> Could we create a libtiff organisation on GitHub or GitLab?

> Unfortunately, "libtiff" is taken on github, as is "tiff". But libtiff

> is available on GitLab.

>

 

I've sent a query to github to see if they can get libtiff back for the project (as the existing account seems to have no recent activity since 3 years).

 

>

> I've attached the patches to invoke the script directly (0002) plus the

> original fixes (0001).

 

Applied

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Kemp Watson-2
I’ve made this offer to the list before, but here it is again:

We (Objective Pathology) work heavily with libTIFF for it’s BigTIFF features which are important
in our digital pathology field. We will be open-sourcing our own BigTIFF subspec/flavour designed
for interactive online viewing fairly soon – it’s already fairly widely in use.

We own Organization BigTIFF on GitHub, and domains tiff.photo, tiff.rocks, libtiff.online, and bigtiff…
   .net .ca .us .photo .info .rocks .asia .desi .eu .lat .online; bigtiff.org is owned by an associate company.

Maybe something could be done with one or several., like hosting libtiff under BigTIFF/libTIFF or some such.
No agenda other than a bit of credit for our company for supporting the project, not dissimilar to
the current support from Aware, OSGeo, Remote Sensing, Simple Systems, who knows what… all very confusing
now as the domains have been lost/hijacked several times.

We maintain close to 300 domains, and the site would be accelerated through a paid Cloudflare CDN account
with bandwidth paid by us if not on GitHub.

I do feel strongly that it’s time to move libtiff away from CVS and over to Git, and it’s my personal
opinion that GitHub offers the most value overall for open-source projects.

What’s the scoop with libtiff.org? That would certainly be the most sensible home.

W. Kemp Watson


Objective Pathology Services Limited

8250 Lawson Road
Milton, Ontario
Canada  L9T 5C6

www.objectivepathology.com
tel. +1 (416) 970-7284


From: <[hidden email]> on behalf of Even Rouault <[hidden email]>
Date: Friday, June 9, 2017 at 7:15 PM
To: Roger Leigh <[hidden email]>
Cc: <[hidden email]>
Subject: Re: [Tiff] [PATCH] Unix/Linux CI support for GitHub and Travis

> That doesn't sound good, hope they regain access soon. Related to that,

> is there any plan to move fully over to git? The existing mirror works

> pretty well; all my recent patches are generated directly with it.

> Could we create a libtiff organisation on GitHub or GitLab?

> Unfortunately, "libtiff" is taken on github, as is "tiff". But libtiff

> is available on GitLab.

>

 

I've sent a query to github to see if they can get libtiff back for the project (as the existing account seems to have no recent activity since 3 years).

 

>

> I've attached the patches to invoke the script directly (0002) plus the

> original fixes (0001).

 

Applied

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com

_______________________________________________ Tiff mailing list: [hidden email] http://lists.maptools.org/mailman/listinfo/tiff http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Roger Leigh
In reply to this post by Even Rouault-2


On 10/06/17 00:15, Even Rouault wrote:

>  > That doesn't sound good, hope they regain access soon. Related to that,
>
>  > is there any plan to move fully over to git? The existing mirror works
>
>  > pretty well; all my recent patches are generated directly with it.
>
>  > Could we create a libtiff organisation on GitHub or GitLab?
>
>  > Unfortunately, "libtiff" is taken on github, as is "tiff". But libtiff
>
>  > is available on GitLab.
>
>  >
>
> I've sent a query to github to see if they can get libtiff back for the
> project (as the existing account seems to have no recent activity since
> 3 years).
Super thanks.  I've asked them about the codelibre account in the past,
which was also "inactive".  But they can be accounts which only use
private repositories and have no public activity :(  Hopefully this
won't be the case for libtiff.

Related to git, I've created a list of CVS username → email mappings
(attached).  I got the email addresses from the ChangeLog and a little
searching for one or two not in there.  Does this look correct, or do
any of the addresses need updating?  I added a question about the "mike"
username; is my assumption there correct?

The list is for doing the CVS → git conversion with all the needed
information for the commit history; I'll also send the list to vadz for
his git mirror.

>  > I've attached the patches to invoke the script directly (0002) plus the
>
>  > original fixes (0001).
>
> Applied

Thanks.  You can see in https://github.com/vadz/libtiff/commits/master 
that this now shows a passing status for the travis and appveyor builds.


Regards,
Roger

_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

user-list.txt (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Unix/Linux CI support for GitHub and Travis

Even Rouault-2

 

> Super thanks. I've asked them about the codelibre account in the past,

> which was also "inactive". But they can be accounts which only use

> private repositories and have no public activity :( Hopefully this

> won't be the case for libtiff.

 

GitHub staff answered me that the account has recent private activity, but they accepted it to forward to the user a message on my behalf. Which I've just done. We'll see if the user is willing to donate his account.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Tiff mailing list: [hidden email]
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/