Ticket #1085 (closed defect: fixed)

Opened 7 years ago

Last modified 6 years ago

Build & deploy the ImageJ Launcher only when the relevant source code changed

Reported by: dscho Owned by: dscho
Priority: minor Milestone: imagej-2.0.0
Component: Build System Version:
Severity: cosmetic Keywords:
Cc: Blocked By: #832, #1722
Blocking: #1723

Description

To avoid having to download new snapshot versions of the ImageJ Launcher, let's only deploy, and actually let's only build, when the source of the Launcher changed.

We can use the Script Trigger for that:
 https://wiki.jenkins-ci.org/display/JENKINS/ScriptTrigger+Plugin

A simple script like this should work:

-- snipsnap --

OLD_HEAD=$(git rev-parse HEAD)

exit 0 # trigger upon first poll

# only trigger when anything in core/launcher changed
git pull origin master &&
test $(git rev-parse $OLD_HEAD:core/launcher) != $(git rev-parse HEAD:core/launcher)

Change History

comment:1 Changed 7 years ago by dscho

  • Milestone changed from imagej-2.0-beta2 to imagej-2.0-final

This is not really critical for users, and for now we can live with the incurred disk usage (it is not really much) and the mild issue that developers will download a new copy of the launchers every morning.

As there are more important issues, pushing to -final.

comment:2 Changed 7 years ago by dscho

Having thought about this for a little while, I think there is a much simpler way to do this: take advantage of the new Implementation-Build field in the manifest (thanks to the buildnumber-maven-plugin).

So instead of having a Maven build step, we would have a Shell build step that does this:

uptodate () {
    # extract some metadata from the pom
    artifactId=$(sed -n 's|.*<artifactId>\(.*\)</artifactId>.*|\1|p' < pom.xml | head -n 1)
    version=$(sed -n 's|.*<version>\(.*\)</version>.*|\1|p' < pom.xml | head -n 1)
    jar=$artifactId-$version.jar

    # if the .jar is not there, we need to build
    test -f target/$jar || return 1

    # if the manifest does not have the revision, we need to build
    revision=$(unzip -p target/$1-$2.jar META-INF/MANIFEST.MF |
        sed -n 's/^Implementation-Build: //p') || return 1

    # if there have been any changes since <revision>, we need to build
    git diff --quiet $revision.. -- . || return 1

    # no build required
    return 0
}

uptodate || mvn clean deploy

We could even put this into our scripts/ directory for easier re-use.

The same script could be enhanced a little for ticket #1086 so it traverses the directories with pom.xml files that do not have a <packaging>pom</packaging> line. It is a great pity that Maven does not have an easy way to just output that list (topologically sorted by dependency), but we could easily hack something on top of MiniMaven.

If nothing else works, we can always put some trivial parsing of the output of "mvn help:effective-pom" into place. Something like

# Maven resolves symlinks incorrectly
cd $(pwd -P)

# Maven uses DOS line endings even in Unix
for directory in $(mvn help:effective-pom |
    tr -d '\r' |
    sed -n 's|.*<sourceDirectory>\(.*\)</sourceDirectory>|\1|p')
do
    test -d $directory || continue
    (cd $directory && uptodate || mvn clean deploy)
done

should do it.

However, we have to take care to build anyway (but not necessarily deploy) as we want to catch breakages due to API changes in dependencies. So maybe instead of calling

uptodate || mvn clean deploy

it should read more like

if uptodate
then
    mvn compile
else
    mvn clean deploy
fi

Note: care has to be taken to error out properly if Maven fails, as errors in for loops are not fatal in POSIX shells, and neither are they propagated as exit values of the loop.

Last edited 7 years ago by dscho (previous) (diff)

comment:3 Changed 7 years ago by dscho

comment:4 Changed 6 years ago by bdezonia

  • Blocked By 1722 added

comment:5 Changed 6 years ago by bdezonia

  • Blocked By 1723 added

comment:6 Changed 6 years ago by bdezonia

  • Blocking 1723 added
  • Blocked By 1723 removed

comment:7 Changed 6 years ago by dscho

  • Status changed from new to closed
  • Resolution set to fixed

I just solved this (and verified it by adding the icon from logo/ImageJ.ico as icon to the Windows launchers by default; the Jenkins job was called as intended): since build 4191, the ImageJ job calls scijava-common's (just-enhanced) maven-helper to find out whether the launcher needs to be built and deployed and if so, triggers the ImageJ-launcher job via curl call.

Note: See TracTickets for help on using tickets.