Parallel execution of "Statement coverage" check does not terminate

[ThomasErgin]

Check Name
This issue is related to the check "Statement Coverage" and likely to other unit test checks, such as "Procedure Coverage" and "Branch Coverage"

Actual Behavior
I want to measure the unit test coverage of a large set of unit tests. This would allow me to assess, the completeness of the unit tests from my developers in a large software project. Therefore, I schedule a scan run with transaction SCI using the check "Statement Coverage". I notice that the job starts forking away parallel processes with packages of objects to scan. These workprocesses will terminate after a while. However, the main workprocess keeps running for days without finishing. It seems to be WAITing in source LSPTA_NEWF02, line 550 (method lcl_parallel_processor->run( ) ) without beeing able to terminate properly. I recon, that failures in the execution of unit tests are somewhere not properly caught, what causes the parallel workprocesses to terminate, without messaging back to the main batchjob, that they have been terminated.

Expected Behavior
I would expect, that when a unit test fails or causes the unit test execution framework to stop, that still the scan run will terminate and record the results so far. I appears, that errors from the unit test execution framework are not properly caught.

I've already opened an incident with the SAP AG, but SAP claimed, this is coming from the CodePal and they do not feel responsible to solve that.

I know, this description is quite vague. I dind'nt found the particular unit test yet, that causes a scan process terminate in such a way.

[bjoern-jueliger-sap]

Figuring out whether a bug is in the SAP standard or in CodePal isn't always easy, but let's try to narrow it down:

1. There should be an SAP-delivered check "ABAP Unit" in the "Dynamic Tests" category. It doesn't measure coverage, but it also executes unit tests. Does this also happen if you run a check variant that contains this check instead of the CodePal coverage check?
2. If you run ABAP Unit normally (e.g. on each development package via the context menu in SE80 oder ADT), are there any unusual terminations or endless loops?

[ThomasErgin]

Hi Björn,
I scanned the same set of objects with the "ABAP Unit" check. I saw several short dumps coming from various faulty unit tests (such as COMPUTE_INT_ZERODIVIDE or DYNPRO_SEND_IN_BACKGROUND) but the scan job finally terminated. This behavior of the SCI is acceptable: I can review the Unit test results and can contact the developers to have their unit tests fixed.

However, the behavior of the CodePal unit test checks seems wrong to me: The scan job must terminate at a certain point of time and not run forever. (The DURATION-clause inside of an unit test class should prevent endless execution of a unit test, anyway).

A possible reason for the problem: The ABAPunit framework seems to struggle with Dynpro-interaction. For example if a unit test triggers a popup, this will interrupt the unit test execution, instead of letting the unit test fail. On my request a while ago the SAP AG introduced the parameter ABAP_UNIT_SE80_MODE that should suppress dialog activity when executing unit tests (Check ). In order to execute unit tests during CHARM/CALM-release (SolutionManager) I need to set the parameter ABAP_UNIT_SE80_MODE at my user to 'I' (Cancel when runtime errors occur).

The unit test execution in Y_UNIT_TEST_COVERAGE->get( ) (CodePal) seems to be triggered differently than in CL_SAUNIT_LEGACY_CI_CHECK->RUN( ) (SAP standard).

Perhaps you have an idea on that?

(Maybe) see you on the ABAPCONF on thursday - Schöne Grüße aus Südafrika :-)

Thomas