Ticket #1784 (closed enhancement: fixed)

Opened 7 years ago

Last modified 7 years ago

Add a Jenkins job that can release a new pom-scijava and update our Maven projects accordingly

Reported by: dscho Owned by: dscho
Priority: major Milestone: imagej-2.0.0
Component: Build System Version:
Severity: major Keywords:
Cc: curtis Blocked By:
Blocking: #1783

Description (last modified by dscho) (diff)

Just like  http://jenkins.imagej.net/job/Synchronize-and-deploy-IJ1/ synchronizes ImageJ 1.x fully automatically, commits, pushes and deploys, we need Jenkins jobs (most likely manually triggered, however) to help bumping up the version in one of our projects (pom-scijava, scijava-common, imglib2, imagej2, scifio, fiji).

The job needs to be parametrized (the Git project to bump).

It would be nice if this job could trigger all projects to be bumped to new pom-scijava versions.

This Jenkins job should avoid pushing updated master branches when builds fail, but simply mark the builds as failed.

For maximal transparency, the job should be implemented as a shell script in scijava-common's bin/ directory.

Change History

comment:1 Changed 7 years ago by dscho

  • Description modified (diff)

comment:2 Changed 7 years ago by dscho

  • Description modified (diff)

comment:3 Changed 7 years ago by dscho

There is now something in place:  https://github.com/scijava/scijava-common/blob/master/bin/bump-pom-scijava.sh and  http://jenkins.imagej.net/job/Bump-POM-SciJava/.

A little work is left to do:

  • should we use <pluginManagement> instead of properties to manage versions? That would allow us to use the maven-versions-plugin to bump the pom-scijava versions.
  • The Jenkins job currently only updates pom-scijava, but it needs to update scijava-common, imglib2, imagej2, scifio, fiji and any other repository we want to stay on the edge, too. Happily, this is easily accomplised with bump-pom-scijava.sh's --bump-parent option, but it needs thorough testing.

comment:4 Changed 7 years ago by dscho

One thing we have to be careful about: ImageJA. As it is never at a -SNAPSHOT version, we probably need to use bump-pom-scijava.sh --skip-commit --bump-parent in  http://jenkins.imagej.net/job/Synchronize-and-deploy-IJ1/.

comment:5 Changed 7 years ago by dscho

Curtis reported that the sed call does not work properly on MacOSX: the pom-scijava version is not incremented properly, most likely due to the great and incredibly powerful BSD sed.

To side-step that issue, and also to update all dependent major projects (IJ2, SCIFIO, etc), let's finalize the  http://jenkins.imagej.net/job/Bump-POM-SciJava/ job.

However, this requires a substantial change first: bump-pom-scijava.sh should learn to figure out the latest release versions itself and update the pom.xml accordingly, exiting silently if nothing needs to be changed.

comment:6 Changed 7 years ago by dscho

It appears that the <versioning> tag of maven-metadata.xml is not automatically correct when deploying with Maven 2 (which our Jenkins does).

So before bump-pom-scijava.sh could acquire a new, intelligent mode to update pom-scijava to upgrade to our latest release versions, I needed to

  • run Rebuild Metadata on our Nexus, and
  • change a couple of deploy calls used directly or indirectly by Jenkins to include -DupdateReleaseInfo=true

Sadly, it seems that the deploy plugin is not clever enough to skip interpreting the updateReleaseInfo property for -SNAPSHOT versions, so we cannot do it wholesale.

So far, I updated scijava-common's release-version.sh, bump-pom-scijava.sh, ImageJ's doc/release-steps.txt, ImageJA's jenkins-job.sh (on the tools branch) and the ImageJ-release-deploy Jenkins job to use said property.

It seems that the following Jenkins jobs have shell calls to mvn deploy (but want to deploy -SNAPSHOT versions most of the time):

  • Imagej-launcher-deploy
  • Saalfeld-MPICBG-Maven, and
  • TrakEM2-Maven.

The following Jenkins jobs have Maven build steps calling deploy (but also typically deploy -SNAPSHOT versions):

  • Bio-Formats-maven,
  • ImgLib,
  • LOCI,
  • LOCI-daily,
  • native-lib-loader, and
  • SLIM-curve.

So I left all of those jobs untouched. (FWIW I called grep 'mvn.*deploy' jobs/*/config.xml and grep 'targets.*deploy' jobs/*/config.xml in Jenkins' home directory.)

comment:7 Changed 7 years ago by dscho

I tested  http://jenkins.imagej.net/view/SciJava/job/Bump-POM-SciJava/ as well as I can without actually bumping the pom-scijava version. So far, it is only set up to take care of ImageJ2's and Fiji's master branch.

For ImageJA, I enhanced the sync-with-imagej.sh script (again, tested as well as possible without actually releasing a new ImageJ 1.x version).

I'll leave this ticket open until we get a chance to test it.

comment:8 Changed 7 years ago by dscho

  • Status changed from new to closed
  • Resolution set to fixed

The first test failed: a couple of issues prevented the dependencees to be updated correctly. But the issues have been resolved, the next time we need to bump the pom-scijava project should be just a "Build now" away...

comment:9 Changed 7 years ago by dscho

  • Summary changed from Add a Jenkins job that can release new versions of our Maven projects to Add a Jenkins job that can release a new pom-scijava and update our Maven projects accordingly

comment:10 Changed 7 years ago by dscho

Since IJ2 now depends on release versions of scijava-common, I changed Jenkins to stop triggering ImageJ builds after SciJava-common was built successfully.

comment:11 Changed 7 years ago by dscho

I changed the Jenkins job to be parameterized so you can switch on/off what gets bumped.

For example, in the use case that came up today (and for which we did not have parameters), a new scijava-common version was released that required some changes in imagej.git to go in before bumping the required pom-scijava version. In that case, we would want to uncheck BUILD_IMAGEJ and bump imagej.git's parent version later, when merging the branch with the required changes.

Note: See TracTickets for help on using tickets.