|
Hello all,
I would like to release the very first version of the Pitest plugin. PIT/Pitest is a "mutation testing" tool for Java. You can find a quick introduction on mutation testing on the wiki page of the plugin:
From a sonar point of view, this plugin is a rule engine. A violation is created on a line of code when an alteration on this line does not break the build. The plugin can execute Pitest or reuse existing reports.
A snapshot jar is available at the following location: Vote open for 72 hours. Look forward to get your feedback Alex
|
|
Hi Alex
First, congratulations for your plugin! :) I downloaded in a test sonar installation (3.0.1) , added the rule in a quality profile , and configured it in a short demo maven project ( 5 classes and 2 unit tests )
Sonar analysis took a few seconds and PiTest Sensor was correctly executed. I suppose that I should only see the violation Survived Mutant on classes that haven't survived the mutation. In my case I don't see any violations so I assume that all classes have survived? :)
I include also the last lines of plugin logging. Can you please explain what do they mean and if I can understand by reading them that there are no violations?
================================================================================ - Statistics ================================================================================ >> Generated 25 mutations Killed 5 (20%)
>> Ran 5 tests (0.2 tests per mutation) ================================================================================ - Mutators ================================================================================
> org.pitest.mutationtest.engine.gregor.mutators.ReturnValsMutator >> Generated 22 Killed 4 (18%) > KILLED 4 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0
> NO_COVERAGE 18 -------------------------------------------------------------------------------- > org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator >> Generated 3 Killed 1 (33%)
> KILLED 1 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 > NO_COVERAGE 2
Regards Patroklos
2012/6/5 Alexandre Victoor <[hidden email]> Hello all, |
|
Thanks a lot for your feedback
I guess the survived mutants are on uncovered lines. The plugin ignores the violations on code uncovered by unit tests. Only violations on covered code are taken in account. Let me know if you think does not make sens
Regards Alex On Tue, Jun 5, 2012 at 7:54 AM, Papapetrou P.Patroklos <[hidden email]> wrote: Hi Alex |
|
If I have correctly understood PiTest is reporting only existing tests :) I think it makes sense.
So in my case all my tests have survived mutation that's why i don't see any violations even though in log it seems that there are more? Patroklos
2012/6/5 Alexandre Victoor <[hidden email]> Thanks a lot for your feedback |
|
There is no visible violation because there is no alteration of the covered code that do no break the tests. It would be clear if the pitest violation could be reported in the test tab in the sonar UI... that would be great in the future if we could do that :)
Just to be sure, could you post the Pitest report located in "target/pit-reports" ?On Tue, Jun 5, 2012 at 8:06 AM, Papapetrou P.Patroklos <[hidden email]> wrote:
|
|
Yes Indeed!!
I attached mutations.xml Since the plugin works as expected here is my +1 !!
It's really an interesting topic and I can't wait to see some more features :) Regards Patroklos
2012/6/5 Alexandre Victoor <[hidden email]>
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
So far I was unable to run an analysis with Pitest launched by the Sonar plugin. I'm getting the following error: [ERROR] Failed to execute goal org.codehaus.mojo:sonar-maven-plugin:2.0:sonar (default-cli) on project java-test: Can not execute Sonar: No test library found on classpath. PIT requires either JUnit or TestNG
On Tue, Jun 5, 2012 at 8:47 AM, Papapetrou P.Patroklos <[hidden email]> wrote: Yes Indeed!! --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
Hi Dinesh
Maybe you need to find at least the root name of your packages ( i.e. org* or com* ) instead of just typing *
Regards Patroklos
2012/6/5 Dinesh Bolkensteyn <[hidden email]>
|
|
I meant ENTER and not find....
2012/6/5 Papapetrou P.Patroklos <[hidden email]> Hi Dinesh |
|
In fact the same error occurs even without those 2 properties (i.e. by using their default values), so I doubt it is directly related. Would it be possible for you to send me your sample project?
On Tue, Jun 5, 2012 at 10:02 AM, Papapetrou P.Patroklos <[hidden email]> wrote: I meant ENTER and not find.... |
|
Dinesh, could you try to add the surefire plugin in your pom? Le 5 juin 2012 10:07, "Dinesh Bolkensteyn" <[hidden email]> a écrit :
|
|
In reply to this post by Patroklos Papapetrou
I can confirm that using "*" is currently a very bad idea see
http://code.google.com/p/pitestrunner/issues/detail?id=25 |
|
I was able to run Pitest manually with Maven by adding the pitest-maven plugin. Indeed specifying <param>*</param> for targetClasses and targetTests lead to an exception (listed hereunder FYI, it relates to pitest itself and not the Sonar plugin).
But when specifying the exact values ("Adder" for targetClasses and "AdderTest" for targetTests in my case), it did work as expected. However, I was still not able, even after having added the surefire plugin to the pom file, to have the Sonar plugin launch pitest.
(note that I did *not* add anything to the CLASSPATH environment variable) Why other idea on how to make this work? The exception, relating to pitest itself and not the Sonar plugin:
stderr : 14:15:26 PIT >> INFO : Running 1 units
stderr : 14:15:26 PIT >> INFO : Finished 14:15:26 PIT >> WARNING : Error while watching child process java.util.concurrent.ExecutionException: org.pitest.PitError: null Please copy and paste the information and the complete stacktrace below when reporting an issue VM : Java HotSpot(TM) 64-Bit Server VM Vendor : Oracle Corporation Version : 23.0-b21 Uptime : 2517 Input -> 1 : -Xmx1500m 2 : -XX:MaxPermSize=64M 3 : -Dclassworlds.conf=C:\SonarSource\apache-maven-3.0.4\bin\m2.conf 4 : -Dmaven.home=C:\SonarSource\apache-maven-3.0.4 BootClassPathSupported : true at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111) at org.pitest.util.CommunicationThread.waitToFinish(CommunicationThread.java:60) at org.pitest.coverage.execute.CoverageProcess.waitToDie(CoverageProcess.java:34) at org.pitest.mutationtest.DefaultCoverageDatabase.gatherCoverageData(DefaultCoverageDatabase.java:181) at org.pitest.mutationtest.DefaultCoverageDatabase.calculateCoverage(DefaultCoverageDatabase.java:145) at org.pitest.mutationtest.DefaultCoverageDatabase.initialise(DefaultCoverageDatabase.java:102) at org.pitest.mutationtest.MutationCoverageReport.runReport(MutationCoverageReport.java:161) at org.pitest.mutationtest.MutationCoverageReport.run(MutationCoverageReport.java:83) at org.pitest.RunPitStrategy.execute(RunPitStrategy.java:80) at org.pitest.PitMojo.execute(PitMojo.java:215) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196) at org.apache.maven.cli.MavenCli.main(MavenCli.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352) Caused by: org.pitest.PitError: null Please copy and paste the information and the complete stacktrace below when reporting an issue VM : Java HotSpot(TM) 64-Bit Server VM Vendor : Oracle Corporation Version : 23.0-b21 Uptime : 2517 Input -> 1 : -Xmx1500m 2 : -XX:MaxPermSize=64M 3 : -Dclassworlds.conf=C:\SonarSource\apache-maven-3.0.4\bin\m2.conf 4 : -Dmaven.home=C:\SonarSource\apache-maven-3.0.4 BootClassPathSupported : true at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:22) at org.pitest.util.SafeDataInputStream.readString(SafeDataInputStream.java:46) at org.pitest.util.SafeDataInputStream.read(SafeDataInputStream.java:52) at org.pitest.mutationtest.CoverageCommunicationThread$Receive.handleTestEnd(CoverageCommunicationThread.java:89) at org.pitest.mutationtest.CoverageCommunicationThread$Receive.apply(CoverageCommunicationThread.java:81) at org.pitest.util.SocketReadingCallable.receiveResults(SocketReadingCallable.java:66) at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:37) at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:13) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.lang.Thread.run(Thread.java:722) Caused by: java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:197) at java.io.DataInputStream.readFully(DataInputStream.java:169) at org.pitest.util.SafeDataInputStream.readString(SafeDataInputStream.java:43) ... 9 more 14:15:26 PIT >> INFO : Calculated coverage in 0 seconds. 14:15:26 PIT >> INFO : Created 1 mutation test units 14:15:26 PIT >> INFO : Running 1 units 14:15:26 PIT >> INFO : Finished 14:15:26 PIT >> INFO : Completed in 1 seconds. Tested 1 classes. ================================================================================ - Timings ================================================================================ > scan classpath : < 1 second > coverage and dependency analysis : < 1 second > build mutation tests : < 1 second > run mutation analysis : < 1 second -------------------------------------------------------------------------------- > Total : 1 seconds -------------------------------------------------------------------------------- ================================================================================ - Statistics ================================================================================ >> Generated 2 mutations Killed 0 (0%) >> Ran 0 tests (0 tests per mutation) ================================================================================ - Mutators ================================================================================ > org.pitest.mutationtest.engine.gregor.mutators.MathMutator >> Generated 1 Killed 0 (0%) > KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 > NO_COVERAGE 1 -------------------------------------------------------------------------------- > org.pitest.mutationtest.engine.gregor.mutators.ReturnValsMutator >> Generated 1 Killed 0 (0%) > KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 > MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 > NO_COVERAGE 1 -------------------------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.997s [INFO] Finished at: Tue Jun 05 14:15:26 CEST 2012 [INFO] Final Memory: 8M/820M [INFO] ------------------------------------------------------------------------ On Tue, Jun 5, 2012 at 1:11 PM, henry <[hidden email]> wrote: I can confirm that using "*" is currently a very bad idea see |
|
Hi Dinesh
Do u still need my Sample project? Patroklos
2012/6/5 Dinesh Bolkensteyn <[hidden email]>
|
|
Hi Papapetrou, If you managed to analyse it by having the Sonar plugin launch pitest, then yes. If you used the reuseReport option, then no ;-) -- Dinesh Bolkensteyn www.SonarSource.com twitter.com/DBolkensteyn On Tue, Jun 5, 2012 at 5:40 PM, Papapetrou P.Patroklos <[hidden email]> wrote: Hi Dinesh |
|
Hi Dinesh
Please find attached my sample maven project. It uses Sonar plugin to launch pitest :) Regards
Patroklos
2012/6/5 Dinesh Bolkensteyn <[hidden email]>
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
In reply to this post by Dinesh Bolkensteyn-2
This sounds like this issue
http://code.google.com/p/pitestrunner/issues/detail?id=44 Which will be fixed in the latest 0.28 snapshot of PIT. |
|
In reply to this post by Patroklos Papapetrou
Hello all
Thanks for your tests so far. Dinesh, could you send us the sample project you were using to test the plugin. I would like to know why it fails on this project... Regards Alex On Tue, Jun 5, 2012 at 7:16 PM, Papapetrou P.Patroklos <[hidden email]> wrote: Hi Dinesh |
|
Thanks for the project Papapetrou. In fact I'm getting the same error, no JUnit nor TestNG in classpath, so this makes me think something might be required in the environment to make it work?
On Tue, Jun 5, 2012 at 8:55 PM, Alexandre Victoor <[hidden email]> wrote: Hello all --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
Hi Dinesh
I succesfully analysed your project. See attached screenshot showing the Mutation violation.
for the record I am using Sonar 3.0.1 , Maven 2.2.1 and I run the analysis with mvn clean install sonar:sonar Regards Patroklos
2012/6/6 Dinesh Bolkensteyn <[hidden email]>
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
| Powered by Nabble | Edit this page |
