How much m2e test suites cover of Maven ?

[Mickael Istria]

Hi all,

In my intent to improve m2e, I realize that one key criteria of
sustainability for m2e project is the ability to interact well with Maven
project (interacting more with the community, trying snapshots, identifying
breaking or major issues, consolidating the non-CLI usage of Maven as
APIs...). And, also, the other way round: Maven needs more feedback from
downstream integrations like IDE or CI engines provide to make sure it
remains easy to write tools or other bricks for it.

I've set up m2e code coverage for that locally (and submitting related
patches to Gerrit and GitHub to the m2e project), and have included in the
coverage report the plain Maven packages to measure how much m2e test suite
covers Maven.
I build the HTML reports with Ant:
https://github.com/mickaelistria/m2e-covering-maven/blob/master/coverage-report-build.xml
against the version of Maven that's used by m2e (currently 3.5.3).
And here is the output: https://mickaelistria.github.io/m2e-covering-maven/

We can see that m2e tests roughly covers ~35% of Maven, and in a different
story as the CLI.
I think this results are interesting for both m2e and Maven and I let you
all have a look. In the future, I'd like to see those produced more
frequently by m2e and -if it makes sense- aggregated with plain Maven test
coverage data when m2e tries a snapshot or staging; with the goal of
identifying parts of the code that are less reliable or critical.

Cheers,

[Robert Scholte]

Hi,

during the vote for 3.6.0 we were made aware of issues with different
IDEs, which did indeed raise the question: how can we verify that Maven
changes are still compatible with IDEs? [1]
Even though I consider the IDE as a wrapper around a tool like Maven (and
IDEs should follow the tool), IDEs are nowadays way to important to
ignore. Hence we are still looking for a way to verify these changes and
their effects on IDEs.

thanks,
Robert

[1]

Post by Mickael Istria
Hi all,
In my intent to improve m2e, I realize that one key criteria of
sustainability for m2e project is the ability to interact well with Maven
project (interacting more with the community, trying snapshots, identifying
breaking or major issues, consolidating the non-CLI usage of Maven as
APIs...). And, also, the other way round: Maven needs more feedback from
downstream integrations like IDE or CI engines provide to make sure it
remains easy to write tools or other bricks for it.
I've set up m2e code coverage for that locally (and submitting related
patches to Gerrit and GitHub to the m2e project), and have included in the
coverage report the plain Maven packages to measure how much m2e test suite
covers Maven.
https://github.com/mickaelistria/m2e-covering-maven/blob/master/coverage-report-build.xml
against the version of Maven that's used by m2e (currently 3.5.3).
https://mickaelistria.github.io/m2e-covering-maven/
We can see that m2e tests roughly covers ~35% of Maven, and in a different
story as the CLI.
I think this results are interesting for both m2e and Maven and I let you
all have a look. In the future, I'd like to see those produced more
frequently by m2e and -if it makes sense- aggregated with plain Maven test
coverage data when m2e tries a snapshot or staging; with the goal of
identifying parts of the code that are less reliable or critical.
Cheers,

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-***@maven.apache.org
For additional commands, e-mail: dev-***@maven.apache.org

[Mickael Istria]

Hi,

Hi Robert,

how can we verify that Maven changes are still compatible with IDEs?
[...] Hence we are still looking for a way to verify these changes and
their effects on IDEs.

I think https://bugs.eclipse.org/bugs/show_bug.cgi?id=542555 can be a
solution for m2e. I'll keep you in touch when this is done.