how I ride the bullet - how I set up a testing environment

[cmb]

Christoph, do you know what this is?

Ah, yes, that test would need to be rewritten, but the necessary CURLFile class is only available since PHP 5.5.0. The cleanest solution would be duplicating the test logic (if (class_exists(...), or something), but it might be acceptable to just silence the test with an @ operator. I'll look into that.

phing coverage doesn't write anything.

That might be due to unsufficient permissions to create the folder tests/coverage. Please create this folder manually, and try again. If that works, it might be best to add the (empty) folder to the repo and ignore its contents only.

PS:

phing all-tests seem to stop after unit-tests

That doesn't happen to me.

[manu]

Please create this folder manually, and try again.

same result:

Code: Select all

C:\xampp\htdocs\CMSimpleDev>phing coverage -logger phing.listener.DefaultLogger
Buildfile: C:\xampp\htdocs\CMSimpleDev\build.xml

CMSimple_XH > coverage:

PHPUnit 4.4.0 by Sebastian Bergmann.

Configuration read from C:\xampp\htdocs\CMSimpleDev\coverage.xml

...............................................................  63 / 560 ( 11%)

............................................................... 126 / 560 ( 22%)

.............
BUILD FINISHED

Total time: 21.3330 seconds


C:\xampp\htdocs\CMSimpleDev>

tests/coverage/ remains empty.

That doesn't happen to me.

And this is the log of phing all-tests:

Code: Select all

C:\xampp\htdocs\CMSimpleDev>phing all-tests -logger phing.listener.DefaultLogger

Buildfile: C:\xampp\htdocs\CMSimpleDev\build.xml

CMSimple_XH > unit-tests:

  [phpunit] Testsuite: AdminfuncsTest
  [phpunit] Tests run: 3, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.02000 s
  [phpunit] Testsuite: AdminMenuTest::testShowsItem
  [phpunit] Tests run: 19, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.57000 s
  [phpunit] Testsuite: AdminMenuTest::testShowsPluginsInColumns
  [phpunit] Tests run: 4, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.13000 s
  [phpunit] Testsuite: AdminMenuTest
  [phpunit] Tests run: 29, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.85000 s
  [phpunit] Testsuite: BackupTest::testIsContentBackup
  [phpunit] Tests run: 10, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.15000 s

C:\xampp\htdocs\CMSimpleDev>

[cmb]

Please create this folder manually, and try again.

same result:
[ console output ]
tests/coverage/ remains empty.

Do you have Xdebug installed?

That doesn't happen to me.

And this is the log of phing all-tests:

Code: Select all

[ ... ]
  [phpunit] Testsuite: BackupTest::testIsContentBackup
  [phpunit] Tests run: 10, Failures: 0, Errors: 0, Incomplete: 0, Skipped: 0, Time elapsed: 0.15000 s

C:\xampp\htdocs\CMSimpleDev>

Ah! Apparently the unit tests fail early, probably during BackupTest::testFailedBackupReportsError() where I had to turn down error reporting temporarily (can't remember why; should have documented the hack ). I'll investigate.

PS: During investigating the issues I found a new one: the last test of SearchTest::testSearch() fails, if the intl extension is disabled. <del>I'll fix this test ASAP.</del><ins>I've fixed this test with r1426.</ins>

Furthermore I've noticed that there are new versions of XAMPP (I'm still using 1.8.2 and occassionally 1.8.3). Which version do you use, manu?

PPS: When searching for "&" in edit mode three results are found; amongst them "News02" which I find pretty strange. Not a real problem, but something that should be investigated.

[manu]

My XAMPP Version number is 5.5.19 (same as PHP Version).
Revision 1426? You probably have to push this onto the server. In my log the latest revision is 1425.

[cmb]

My XAMPP Version number is 5.5.19 (same as PHP Version).

Thanks. Anyhow, in the meantime I have been able to reproduce the silently failing test suite (including the failing coverage report) with XAMPP 1.8.3. I'll see to it this evening.

Revision 1426? You probably have to push this onto the server. In my log the latest revision is 1425.

It is there for roughly an hour. You may have to update your working copy.

[cmb]

Ah! Apparently the unit tests fail early, probably during BackupTest::testFailedBackupReportsError() where I had to turn down error reporting temporarily (can't remember why; should have documented the hack ). I'll investigate.

The reason for temporarily disabling error_reporting is that the test checks for a failing backup, and we can't use a try-catch statement in this case (otherwise e() won't be called, what is checked by the test spy). I've added a respective comment (r1427). Alternatively, we could use the @ operator here, what is simpler, but not as explicit. When I've written the code back then, I tried to avoid @ at all cost, because I generally consider it bad practise, but for unit-tests we might have to live with it (actually, disabling the error_reporting explicitly is not better).[1]

Anyhow, just during `$this->_subject->execute();` we need a function nesting level of more than 100 (when the tests are run by Phing), but that is prohibited by Xdebug's default settings. The solution is to add the following to php.ini:

Code: Select all

xdebug.max_nesting_level = 200

I have documented this in the Wiki article just now. (Probably 110 would be sufficient, but better safe than sorry -- and I don't expect any problems with this rather high value.)

PS: [1] The cleanest solution might be to use exception handling in the core, but that's not an option for now (PHP 4 support).

[manu]

aah, all targets run now as expected, even the coverage target, I suppose because of the max_nesting_level. To get the CSSFAttack run I added CURL_SAFE_UPLOAD option (see Revision Log 1428). Christoph please check if this is correct.

[cmb]

aah, all targets run now as expected, even the coverage target, I suppose because of the max_nesting_level.

Yes (at least that was so for me). Thanks for the confirmation.

To get the CSSFAttack run I added CURL_SAFE_UPLOAD option (see Revision Log 1428). Christoph please check if this is correct.

Great! I've already forgotten that issue. Using CURL_SAFE_UPLOAD is a very good idea, but I suppose it will fail for PHP 5.6, where the meaning of the option has been reversed , according to the RFC. The manual page of curl_setopt() doesn't list the new option. I'll try to verify the issue, but unfortunately, there is no runkit.dll available for PHP 5.6.

[cmb]

The manual page of curl_setopt() doesn't list the new option.

That has already been reported a while ago: https://bugs.php.net/bug.php?id=66677.

there is no runkit.dll available for PHP 5.6.

The zenovich fork doesn't even build on PHP 5.6, but the adrianguenter fork does.

Using CURL_SAFE_UPLOAD is a very good idea, but I suppose it will fail for PHP 5.6, where the meaning of the option has been reversed , according to the RFC.

I have misread the description in the RFC, so r1428 is fine.

[cmb]

Well, I just run `phing attack-tests` under PHP 5.4.19[1] and that failed because CURLOPT_SAFE_UPLOAD has been introduced only in PHP 5.5.0. I wanted to fix that with a conditionally set option, but I've noticed that setting the option to TRUE doesn't work as intended -- instead of uploading hack.txt, this simply sends a parameter with the filename (prepended by @). That makes the test bogus.

The option would have to be set to FALSE (i.e. allow unsafe file uploads), but that triggers a deprecation warning. So the clean solution would be to use the CURLFile class for PHP >= 5.5.0. For now (r1440) I've added an @ operator, and a respective @todo. I recommend to fix that (and other @ operators in the tests) ASAP.

[1] I suggest to keep the tests and other supporting development scripts compatible to all supported PHP versions, at least.