Support xml and html reports at once

[Vampire]

I know that SpotBugs currently only supports one report at a time and that there is an open improvement request to change this, but I don't really expect this to happen anytime soon.

But as far as I have seen from a quick look, the HTML reporter just gets the same XML structure the XML reporter is generating and storing to a file (with messages enabled and minimal XML disabled) and then applies the configured XSL script to get the HTML report.

Knowing this, the Gradle plugin could either "if xml and html is enabled" forward the xml setting to SpotBugs and then apply the XSL itself to the XML report and "if xml is disabled but html enabled" either imlicitly enable the XML report with the tempdir as target and do the same to get the same result in both cases or in this case forward the HTML setting to SpotBugs itself.

This way you could generate a human readable report and a machine readable report in one go without having to do the very same thing manually in your build script.

[Vampire]

I can say it works fine. :-)
Here the relevant parts of the config for a manual workaround for anyone interested:

tasks.withType(SpotBugsTask) {
    reports {
        xml.withMessages true
        html.stylesheet resources.text.fromArchiveEntry(configurations.spotbugsStylesheets, 'fancy-hist.xsl')
    }

    task "${it.name}HtmlReport" {
        def input = reports.xml.destination
        inputs.file reports.html.stylesheet.asFile() withPropertyName 'spotbugsStylesheet' withPathSensitivity NONE
        inputs.files fileTree(input) withPropertyName 'input' withPathSensitivity NONE skipWhenEmpty()
        def output = file(input.absolutePath.replaceFirst(/\.xml$/, '.html'))
        outputs.file output withPropertyName 'output'
        doLast {
            def factory = TransformerFactory.newInstance('net.sf.saxon.TransformerFactoryImpl', getClass().classLoader)
            def transformer = factory.newTransformer(new StreamSource(reports.html.stylesheet.asFile()));
            transformer.transform(new StreamSource(input), new StreamResult(output))
        }
    }
    it.finalizedBy "${it.name}HtmlReport"
}

It cannot be a doLast action for the SpotBugs task as it would not run if there were findings and it cannot depend on it either due to the same reason.
But with this setup, anytime a SpotBugs task is run, after it the XML report will be transformed to an HTML report no matter whether successful or not.

[KengoTODA]

duplicates spotbugs/spotbugs#857

[Vampire]

I'd not exactly name it a duplicate, especially as I mentioned that issue, just not linked it.
My request is to work-around that SpotBugs limitation in the meantime as it should be quite easily possible in the Gradle plugin and could be implemented much faster probably than waiting for SpotBugs to implement that improvement request. :-)
When doing it in the plugin, there is most probably als no need for a separate task but it can and should be done in the actions of the SpotBugsTask.
Just for the work-around this is not possible, because if I would do it in a doLast action, it would not run if the analysis made the task fail already.
Due to this the ....HtmlReport task can also not depend on the SpotBugs task, because the same would be true while a finalizedBy task is always run, even if the respecting task failed.
When doing it in the SpotBugsTask, you can do it no matter whether the analysis failed or not and only fail the task after generating the HTML report.

[jpschewe]

@Vampire can you share a full gradle file? I tried using your example and first got errors about StreamSource not known and now I'm getting errors about spotbugsStylesheets being an unknown property.

[Vampire]

I'm on mobile right now, the posted part was only the essential part. In your case, you are missing the configuration spotbugsStylesheets where you add the spotbugs dependency of which to take the stylesheet

[Vampire]

The missing parts should be

import com.github.spotbugs.SpotBugsTask

import javax.xml.transform.TransformerFactory
import javax.xml.transform.stream.StreamResult
import javax.xml.transform.stream.StreamSource

import static org.gradle.api.tasks.PathSensitivity.NONE

plugins {
    id 'com.github.spotbugs' version '1.6.9'
}

spotbugs {
    toolVersion '3.1.11'
}

configurations {
    spotbugsStylesheets { transitive false }
}

dependencies {
    spotbugsStylesheets "com.github.spotbugs:spotbugs:$spotbugs.toolVersion"
}

[jpschewe]

Thanks, that helps a lot. Although now I'm getting

> Provider net.sf.saxon.TransformerFactoryImpl not found

[jpschewe]

Nevermind, I just removed the reference and I don't get anymore errors. However the report output doesn't look like the output that is normally produced by the HTML version. It just has counts, no details. I found that changing 'fancy-hist.xsl' to 'default.xsl' I get what I'm expecting.

[Vampire]

Better add the missing buildscript dependency on Saxon.
Just removing makes it use Xalan-J which is much worse, slower and most importantly an XSLT 1.0 processor while the stylesheets are 2.0.

[jpschewe]

I added

buildscript {
    dependencies {
        classpath group: 'net.sf.saxon', name: 'Saxon-HE', version: '9.4'
    }
}    

And switched back to the saxon transformer and now I get the following output

>./gradlew spotbugsMain                            
Starting a Gradle Daemon (subsequent builds will be faster)                    
                    
> Configure project :                                                 
POM relocation to an other version number is not fully supported in Gradle : xml-apis:xml-apis:2.0.2 relocated to xml-api
s:xml-apis:1.0.b2.                      
Please update your dependency to directly use the correct version 'xml-apis:xml-apis:1.0.b2'.
Resolution will only pick dependencies of the relocated element.  Artifacts and other metadata will be ignored.
                                                                              
> Task :spotbugsMainHtmlReport FAILED
Warning: at xsl:variable on line 348 column 56 of default.xsl:
  SXWN9001: A variable with no following sibling instructions has no effect
Warning: at xsl:variable on line 351 column 59 of default.xsl:                                 
  SXWN9001: A variable with no following sibling instructions has no effect
Error on line 71 of default.xsl:
  SEPM0009: Values of 'standalone' and 'omit-xml-declaration' conflict                                                                              
FAILURE: Build failed with an exception.
                                          
* Where:
Build file '/home/jpschewe/projects/fll-sw/working-dir/build.gradle' line: 396             
                                                                     
* What went wrong:                                                                             
Execution failed for task ':spotbugsMainHtmlReport'.
> net.sf.saxon.trans.XPathException: Values of 'standalone' and 'omit-xml-declaration' conflict
                                            
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with -
-scan to get full insights.                      
                                         
* Get more help at https://help.gradle.org
                             
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 2s
5 actionable tasks: 3 executed, 2 up-to-date

I tried adding

   configurations.classpath {
        resolutionStrategy {
            force 'xml-apis:xml-apis:1.4.01'
        }
    }

to the buildscript section and the error about switching versions of the xml-apis went away, but I still got the xsl errors.

[Vampire]

Maybe try not using an ancient version? 9.4 is from 2012, try 9.9.1-2 instead.

[Vampire]

Ah, no, while you still should, default.xsl shows the same error here, seems it is broken, it defines version 2.0 but is not compatible.
You should open a separate issue for that.

[jpschewe]

Not sure why I found the ancient version to start with. I just tried 9.9.1-2 and got the same error. So is default.xsl really a 1.0 XSLT and I should just go back to the default transformer? It seems to work and I don't mind it being slow.

[Vampire]

You should anyway open an issue about it.
The stylesheet declares it is 2.0 but is not valid, whether it works in 1.0 or not.

[chrismiceli]

PR opened to fix main error message in stylesheet: spotbugs/spotbugs#1000

[linusjf]

https://github.com/Fernal73/LearnJava/blob/master/spotbugs

Any particular reason why this script isn't generating XML output? Am I missing something obvious?

EDIT: Removed sortByClass flag. That fixes the output. The documentation needs to be updated.

[Vampire]

@Fernal73 Please so not abuse foreign issues to post absolutely unrelated questions. The question is not even in the correct project, as it is about spotbugs itself, not about the Gradle plugin.

[linusjf]

@Vampire Where do I post it? What's the best way to go about it? Create an issue that may be a non-issue?
When I search Google for something related to the matter, this post and the other was what I find.
I could go ahead and create an issue to update the documentation now but that's about it.
There's always StackOverflow, I guess, but users over there are very quick to downvote questions as either duplicate or outright silly. Does that mean that I should not seek solutions or answers? I didn't know what the solution was when I posted the question or whether I'd resolve it myself soon enough.
I'd be inclined to move on to the next thing rather than stubbornly try to break my head against it.

Can these posts not be deleted or marked as outdated?

[Vampire]

@Fernal73 These are not "posts" but comments to an issue, that's the whole point, because this is not a forum. You pollute the issue with unrelated information and may hide important information, you send notification emails to all involved people that are interested in this issue, but not in your question.

Of course some admin could delete the comments, but they usually have more important things to do, like actually fixing bugs. You can also delete your comments yourself. But whether it is possible to delete or not, it is rude and impolite and totally not helpful to wildly post at wing places. It disturbs others and it makes it much less likely you will get an answer.

Posting to a totally unrelated issue is like going to a statistics class and asking how top draw a triangle. You might get a answer, but the question is totally out of place and you mainly disturb anyone present.

And posting to the wrong project like you did additionally is like asking in a physics class (where physicians use math) how to draw a triangle.

Next time, a good head start, us to look at the website, where most projects tell you where to get support. In this case it is even right on the startpage: https://spotbugs.github.io/#support-or-contact

There both, the correct issue tracker and the mailing list for questions are linked. And yes, posting an own issue that maybe is just a question is always better than abusing an existing issue that is not at all related. Only comments or questions related to the issue should go to that issue.

[linusjf]

@Vampire ok. Done.

Hmm, I didn't notice that you'd converted a Github repo into a general purpose mailing list.
spotbugs/spotbugs#1038

Btw, are you really telling me that an email or two pertaining to a matter that is just slightly tangential to the issue you're discussing is so very disturbing? Or you didn't like my response to your 'Please'?

Frankly , if the documentation had been correct , I wouldn't have to post anything at all.

[mandrachek]

I'm attempting to do this from an external script. Currently it looks something like this:
main build.gradle:

buildscript {
    repositories {
        maven {
            url "https://plugins.gradle.org/m2/"
        }
        jcenter()
        mavenCentral()
    }
    dependencies {
        classpath "com.github.spotbugs:spotbugs-gradle-plugin:2.0.1"
        classpath group: 'net.sf.saxon', name: 'Saxon-HE', version: '9.9.1-5'
    }
}

// work around inability to import this in an external script
ext {
    SpotBugsTask = com.github.spotbugs.SpotBugsTask
}

apply from: 'spotbugs.gradle'

Then spotbugs.gradle:

apply plugin: 'com.github.spotbugs'
project.configurations.create("spotbugsStylesheets").transitive = false

project.dependencies.add("compileOnly","com.github.spotbugs:spotbugs-annotations:3.1.12")
project.dependencies.add("compileOnly","com.google.code.findbugs:jsr305:3.0.1")
project.dependencies.add("spotbugsPlugins","com.h3xstream.findsecbugs:findsecbugs-plugin:1.10.0")
project.dependencies.add("spotbugsPlugins","com.mebigfatguy.sb-contrib:sb-contrib:7.4.7")
project.dependencies.add("spotbugsStylesheets","com.github.spotbugs:spotbugs:3.1.12")

spotbugs {
    toolVersion = "3.1.12"
    effort="max"
    reportLevel="low"
}

import javax.xml.transform.TransformerFactory
import javax.xml.transform.stream.StreamResult
import javax.xml.transform.stream.StreamSource

import static org.gradle.api.tasks.PathSensitivity.NONE

tasks.withType(SpotBugsTask) {
    reports {
        xml.withMessages true
        html.stylesheet resources.text.fromArchiveEntry(configurations.spotbugsStylesheets, 'fancy-hist.xsl')
    }

    task "${it.name}HtmlReport" {
        def input = reports.xml.destination
        inputs.file reports.html.stylesheet.asFile() withPropertyName 'spotbugsStylesheet' withPathSensitivity NONE
        inputs.files fileTree(input) withPropertyName 'input' withPathSensitivity NONE skipWhenEmpty()
        def output = file(input.absolutePath.replaceFirst(/\.xml$/, '.html'))
        outputs.file output withPropertyName 'output'
        doLast {
            def factory = TransformerFactory.newInstance('net.sf.saxon.TransformerFactoryImpl', getClass().classLoader)
            def transformer = factory.newTransformer(new StreamSource(reports.html.stylesheet.asFile()));
            transformer.transform(new StreamSource(input), new StreamResult(output))
        }
    }
    it.finalizedBy "${it.name}HtmlReport"
}

So now I'm getting the dreaded > Provider net.sf.saxon.TransformerFactoryImpl not found message, despite including saxon on the buildscript classpath. Might this have something to do with the same reason we can't import/access SpotbugsTask in the separate script?

[mandrachek]

Indeed, it does appear to be related. If I make my build.gradle ext block, like so:

ext {
    SpotBugsTask = com.github.spotbugs.SpotBugsTask
    SaxonTransformer = net.sf.saxon.TransformerFactoryImpl
}

then in my spotbugs.gradle, I can replace:

def factory = TransformerFactory.newInstance('net.sf.saxon.TransformerFactoryImpl', getClass().classLoader)

with

def factory = SaxonTransformer.newInstance()

and it works. This isn't as clean as I was hoping for, but it does work.

[Vampire]

@mandrachek why don't you add Saxon to the build script class path of the script where you want to use it?

If you add it to the class path of one script and want to use it on another, it is quite understandable, that it is not present.

[mandrachek]

Actually, I'm converting it to a plugin, so I will have the plugin in the classpath, and saxon will be an implementation dependency of the plugin.

[daanschipper]

Using 4.0.5, SpotBugs seems to generate both xml and html report by default.

[jpschewe]

@daanschipper can you share the configuration that you are using? I have applied the spotbugs 4.0.5 plugin and I get a main.xml, but I don't get an HTML file.

[daanschipper]

Sorry my bad, the html report was of a previous run :(

[henrpe]

I propose another workaround built upon the saxon transformation idea by @Vampire. I use org.myire.munge Gradle plugin to parse the xml report generated by spotbugs to html. Tested with Gradle 6.4.1.

plugins {
    ..
    id 'com.github.spotbugs' version '4.3.0'
    id 'org.myire.munge' version '1.0'
}

configurations {
    ..
    spotbugsStylesheets { transitive false }
}

dependencies {
    ..
    spotbugsStylesheets 'com.github.spotbugs:spotbugs:4.0.2'
}

spotbugsMain.finalizedBy 'transform'

transform {
    saxon {
        source "$buildDir/reports/spotbugs/main.xml"
        template resources.text.fromArchiveEntry(configurations.spotbugsStylesheets, 'default.xsl')
        outputFile = "$buildDir/reports/spotbugs/main.html"
    }
}

Usage: ./gradlew spotbugsMain

[jpschewe]

@henrpe thanks for the alternate solution. This seems a little cleaner than what I've been using. I have made it generic for any number of spotbugs tasks
{{{
import com.github.spotbugs.snom.SpotBugsTask

import org.myire.munge.TransformTask

configurations {
spotbugsStylesheets { transitive = false }
}

dependencies {
spotbugsStylesheets("com.github.spotbugs:spotbugs:$project.spotbugs_version")
}

spotbugs {
toolVersion = "4.0.3"

// allow the build to continue, catch this in CI as warnings
ignoreFailures = true
effort = "max"
reportLevel = "low"
excludeFilter = file("spotbugs-filters.xml")

}

tasks.withType(SpotBugsTask) {
def stylesheetName = 'default.xsl'

def xmlOutput = it.outputs.files.singleFile
def htmlOutput = file(xmlOutput.absolutePath.replaceFirst(/\.xml$/, ".html"))

tasks.register("${it.name}HtmlReport", TransformTask) {
    saxon {
        source(xmlOutput)
        template(resources.text.fromArchiveEntry(configurations.spotbugsStylesheets, stylesheetName))
        outputFile = htmlOutput
    }       
}
it.finalizedBy("${it.name}HtmlReport")

}

[henrpe]

@jpschewe, that's an excellent point that transformation should work on multiple sourceSets (main/test atleast). Another way to achieve this would be by

1. making all tasks with type SpotBugsTask to be finalized by transform (it will only be called once even for multiple sourceSets).
2. configuring the transform task to transform all *.xml files (main.xml, test.xml etc) to *.html (main.html, test.html etc):

tasks.withType(com.github.spotbugs.snom.SpotBugsTask) {
    finalizedBy 'transform'
    ..
}

transform {
    saxon {
        sources ("$buildDir/reports/spotbugs") { include '*.xml' }
        template resources.text.fromArchiveEntry(configurations.spotbugsStylesheets, 'default.xsl')
        outputMapping {
            sourceFile, templateFile -> "$buildDir/reports/spotbugs/" + sourceFile.name.tokenize('.')[0] + '.html'
        }
    }
}

[jpschewe]

@henrpe you need to be careful to handle files with multiple dots in them. I think this will be more robust. This still has the assumption that the output directory of the spotbugs task hasn't changed. That's where it may be good to create the task inside the withType.

tasks.withType(com.github.spotbugs.snom.SpotBugsTask) {
    finalizedBy 'transform'
    ..
}

transform {
    saxon {
        sources ("$buildDir/reports/spotbugs") { include '*.xml' }
        template resources.text.fromArchiveEntry(configurations.spotbugsStylesheets, 'default.xsl')
        outputMapping {
            sourceFile, templateFile -> sourceFile.absolutePath.replaceFirst(/\.xml$/, ".html")
        }
    }
}

[UkonnRa]

So any update? Can the plugin integrate this feature?

[jscancella]

@UkonnRa Pull requests are always welcome. But I believe the long term fix is spotbugs/spotbugs#857

[github-actions]

🎉 This issue has been resolved in version 4.8.0 🎉

The release is available on:

• 4.8.0
• GitHub release

Your semantic-release bot 📦🚀