Multimodule Project Question

[Jim Anderson]

I'm working on utilizing Sonar for an extremely large (1.4 million LOC), multi-module Java project.  The vision is that we'd use the multi-module support in Sonar Runner (or Ant task since we are an Ant shop), as the modules are already clearly defined, and it will allow us to have a high-level view of the product's quality, as well as set goals at the module level.

Obviously, running analysis on a project this large takes some time.  In my prototyping, after reaching a failure point after some time, I tried just updating the sonar.modules property to be one of the submodules so I could debug better, but then I noticed when it completed that all other submodules previously part of the parent project (specified with a name and key) were removed from the Sonar UI.  Is this expected?  Is there any way that I can configure sonar to allow me to add submodules to an existing project without specifying them all over again?

This may have an impact on how Sonar integrates with our current build process as well; we'd like to have a parent sonar project and "plug" submodules into it during some of our existing build processes, rather than run the whole analysis as a separate step.

Thanks,

-Jim Anderson

[Johannes Müller]

Hey Jim,

i also work on an multi modul project which is even larger (over 600 moduls ond over 1.5 LOC). Yes it needs real hard time to figure it out since the project is set "as it is now working". Can you explain to me how you managed to use the sonar.moduls to update your submoduls? I dont get it maybe some lines of code will help. I solved this in another way. All my needed moduls (very very variable) are copied for each buildung script in one folder (its already part of the building routine) and then i take them in a namelist and let the sonar.moduls property run. If the structure of the moduls is like org.projekt.base.core with the subfolder src/ it will work as shown below.

ANT:
namelist looks like: org.projekt1.base.core,org.projekt1.base.server,org.projekt1.base.integration and so on.
<property name="sonar.sources" value="src/" />
<property name="sonar.modules" value="${namelist}" />

only problem is that i need a variable basedir property and this does not exist in sonar. I will try to code the basedir property in the sonar core / ant task plugin because it´s not working for me to take the project name ="!!!" basedir="." to be my actual basedir. I need a generic basedir option as long my scripts are all also generic. A variable option from the sonar team would be nice but this will not happen as i read in the mailing list :/.

Regards Johannes

2012/10/30 Jim Anderson <[hidden email]>

I'm working on utilizing Sonar for an extremely large (1.4 million LOC), multi-module Java project.  The vision is that we'd use the multi-module support in Sonar Runner (or Ant task since we are an Ant shop), as the modules are already clearly defined, and it will allow us to have a high-level view of the product's quality, as well as set goals at the module level.

Obviously, running analysis on a project this large takes some time.  In my prototyping, after reaching a failure point after some time, I tried just updating the sonar.modules property to be one of the submodules so I could debug better, but then I noticed when it completed that all other submodules previously part of the parent project (specified with a name and key) were removed from the Sonar UI.  Is this expected?  Is there any way that I can configure sonar to allow me to add submodules to an existing project without specifying them all over again?

This may have an impact on how Sonar integrates with our current build process as well; we'd like to have a parent sonar project and "plug" submodules into it during some of our existing build processes, rather than run the whole analysis as a separate step.

Thanks,

-Jim Anderson

[Jim Anderson]

Hi Johannes,

The project I work on does have the benefit of being highly modularized, though these modules can be variable.  What I am doing is looking up the modules from a registry file that defines the modules, and then building the sonar.modules up from that.  The source for each module may be different, so the <moduleName>.sonar.sources property is also generated depending upon what source exists for a given module.

Thanks,

-Jim

On Fri, Nov 2, 2012 at 6:11 AM, Johannes Müller <[hidden email]> wrote:

Hey Jim,

i also work on an multi modul project which is even larger (over 600 moduls ond over 1.5 LOC). Yes it needs real hard time to figure it out since the project is set "as it is now working". Can you explain to me how you managed to use the sonar.moduls to update your submoduls? I dont get it maybe some lines of code will help. I solved this in another way. All my needed moduls (very very variable) are copied for each buildung script in one folder (its already part of the building routine) and then i take them in a namelist and let the sonar.moduls property run. If the structure of the moduls is like org.projekt.base.core with the subfolder src/ it will work as shown below.

ANT:
namelist looks like: org.projekt1.base.core,org.projekt1.base.server,org.projekt1.base.integration and so on.
<property name="sonar.sources" value="src/" />
<property name="sonar.modules" value="${namelist}" />

only problem is that i need a variable basedir property and this does not exist in sonar. I will try to code the basedir property in the sonar core / ant task plugin because it´s not working for me to take the project name ="!!!" basedir="." to be my actual basedir. I need a generic basedir option as long my scripts are all also generic. A variable option from the sonar team would be nice but this will not happen as i read in the mailing list :/.

Regards Johannes

2012/10/30 Jim Anderson <[hidden email]>

I'm working on utilizing Sonar for an extremely large (1.4 million LOC), multi-module Java project.  The vision is that we'd use the multi-module support in Sonar Runner (or Ant task since we are an Ant shop), as the modules are already clearly defined, and it will allow us to have a high-level view of the product's quality, as well as set goals at the module level.

Obviously, running analysis on a project this large takes some time.  In my prototyping, after reaching a failure point after some time, I tried just updating the sonar.modules property to be one of the submodules so I could debug better, but then I noticed when it completed that all other submodules previously part of the parent project (specified with a name and key) were removed from the Sonar UI.  Is this expected?  Is there any way that I can configure sonar to allow me to add submodules to an existing project without specifying them all over again?

This may have an impact on how Sonar integrates with our current build process as well; we'd like to have a parent sonar project and "plug" submodules into it during some of our existing build processes, rather than run the whole analysis as a separate step.

Thanks,

-Jim Anderson

[Ann Campbell]

This probably isn't helpful but we long ago broke up our builds (and therefore analyses) into individual units. 

---

G. Ann Campbell

Sr. Systems Engineer, IS Production Systems - Shop Floor Systems

Shaw Industries Inc,

201 S. Hamilton St.

Dalton Ga 30720

On Fri, Nov 2, 2012 at 9:06 AM, Jim Anderson <[hidden email]> wrote:

Hi Johannes,

The project I work on does have the benefit of being highly modularized, though these modules can be variable.  What I am doing is looking up the modules from a registry file that defines the modules, and then building the sonar.modules up from that.  The source for each module may be different, so the <moduleName>.sonar.sources property is also generated depending upon what source exists for a given module.

Thanks,

-Jim

On Fri, Nov 2, 2012 at 6:11 AM, Johannes Müller <[hidden email]> wrote:

Hey Jim,

i also work on an multi modul project which is even larger (over 600 moduls ond over 1.5 LOC). Yes it needs real hard time to figure it out since the project is set "as it is now working". Can you explain to me how you managed to use the sonar.moduls to update your submoduls? I dont get it maybe some lines of code will help. I solved this in another way. All my needed moduls (very very variable) are copied for each buildung script in one folder (its already part of the building routine) and then i take them in a namelist and let the sonar.moduls property run. If the structure of the moduls is like org.projekt.base.core with the subfolder src/ it will work as shown below.

ANT:
namelist looks like: org.projekt1.base.core,org.projekt1.base.server,org.projekt1.base.integration and so on.
<property name="sonar.sources" value="src/" />
<property name="sonar.modules" value="${namelist}" />

only problem is that i need a variable basedir property and this does not exist in sonar. I will try to code the basedir property in the sonar core / ant task plugin because it´s not working for me to take the project name ="!!!" basedir="." to be my actual basedir. I need a generic basedir option as long my scripts are all also generic. A variable option from the sonar team would be nice but this will not happen as i read in the mailing list :/.

Regards Johannes

2012/10/30 Jim Anderson <[hidden email]>

I'm working on utilizing Sonar for an extremely large (1.4 million LOC), multi-module Java project.  The vision is that we'd use the multi-module support in Sonar Runner (or Ant task since we are an Ant shop), as the modules are already clearly defined, and it will allow us to have a high-level view of the product's quality, as well as set goals at the module level.

Obviously, running analysis on a project this large takes some time.  In my prototyping, after reaching a failure point after some time, I tried just updating the sonar.modules property to be one of the submodules so I could debug better, but then I noticed when it completed that all other submodules previously part of the parent project (specified with a name and key) were removed from the Sonar UI.  Is this expected?  Is there any way that I can configure sonar to allow me to add submodules to an existing project without specifying them all over again?

This may have an impact on how Sonar integrates with our current build process as well; we'd like to have a parent sonar project and "plug" submodules into it during some of our existing build processes, rather than run the whole analysis as a separate step.

Thanks,

-Jim Anderson

**********************************************************

Privileged and/or confidential information may be contained in this message. If you are not the addressee indicated in this message (or are not responsible for delivery of this message to that person) , you may not copy or deliver this message to anyone. In such case, you should destroy this message and notify the sender by reply e-mail.

If you or your employer do not consent to Internet e-mail for messages of this kind, please advise the sender.

Shaw Industries does not provide or endorse any opinions, conclusions or other information in this message that do not relate to the official business of the company  or its subsidiaries.

**********************************************************

[Johannes Müller]

As so did we. Two generic scripts that build many different plugin constellation of projects (like tier1, client...) in our Jenkins. So also sometimes different dir constructions are nessessary with no optional setting for a basedir we can not use sonar on generic purpose. And even with one set basedir i got some strange behavior on some projects with many plugins to read in (i will post this separately).

Regards Johannes 

2012/11/2 Ann Campbell <[hidden email]>

This probably isn't helpful but we long ago broke up our builds (and therefore analyses) into individual units. 

---

G. Ann Campbell

Sr. Systems Engineer, IS Production Systems - Shop Floor Systems

Shaw Industries Inc,

201 S. Hamilton St.

Dalton Ga 30720

On Fri, Nov 2, 2012 at 9:06 AM, Jim Anderson <[hidden email]> wrote:

Hi Johannes,

The project I work on does have the benefit of being highly modularized, though these modules can be variable.  What I am doing is looking up the modules from a registry file that defines the modules, and then building the sonar.modules up from that.  The source for each module may be different, so the <moduleName>.sonar.sources property is also generated depending upon what source exists for a given module.

Thanks,

-Jim

On Fri, Nov 2, 2012 at 6:11 AM, Johannes Müller <[hidden email]> wrote:

Hey Jim,

i also work on an multi modul project which is even larger (over 600 moduls ond over 1.5 LOC). Yes it needs real hard time to figure it out since the project is set "as it is now working". Can you explain to me how you managed to use the sonar.moduls to update your submoduls? I dont get it maybe some lines of code will help. I solved this in another way. All my needed moduls (very very variable) are copied for each buildung script in one folder (its already part of the building routine) and then i take them in a namelist and let the sonar.moduls property run. If the structure of the moduls is like org.projekt.base.core with the subfolder src/ it will work as shown below.

ANT:
namelist looks like: org.projekt1.base.core,org.projekt1.base.server,org.projekt1.base.integration and so on.
<property name="sonar.sources" value="src/" />
<property name="sonar.modules" value="${namelist}" />

only problem is that i need a variable basedir property and this does not exist in sonar. I will try to code the basedir property in the sonar core / ant task plugin because it´s not working for me to take the project name ="!!!" basedir="." to be my actual basedir. I need a generic basedir option as long my scripts are all also generic. A variable option from the sonar team would be nice but this will not happen as i read in the mailing list :/.

Regards Johannes

2012/10/30 Jim Anderson <[hidden email]>

I'm working on utilizing Sonar for an extremely large (1.4 million LOC), multi-module Java project.  The vision is that we'd use the multi-module support in Sonar Runner (or Ant task since we are an Ant shop), as the modules are already clearly defined, and it will allow us to have a high-level view of the product's quality, as well as set goals at the module level.

Obviously, running analysis on a project this large takes some time.  In my prototyping, after reaching a failure point after some time, I tried just updating the sonar.modules property to be one of the submodules so I could debug better, but then I noticed when it completed that all other submodules previously part of the parent project (specified with a name and key) were removed from the Sonar UI.  Is this expected?  Is there any way that I can configure sonar to allow me to add submodules to an existing project without specifying them all over again?

This may have an impact on how Sonar integrates with our current build process as well; we'd like to have a parent sonar project and "plug" submodules into it during some of our existing build processes, rather than run the whole analysis as a separate step.

Thanks,

-Jim Anderson

**********************************************************

Privileged and/or confidential information may be contained in this message. If you are not the addressee indicated in this message (or are not responsible for delivery of this message to that person) , you may not copy or deliver this message to anyone. In such case, you should destroy this message and notify the sender by reply e-mail.

If you or your employer do not consent to Internet e-mail for messages of this kind, please advise the sender.

Shaw Industries does not provide or endorse any opinions, conclusions or other information in this message that do not relate to the official business of the company  or its subsidiaries.

**********************************************************