Enable generation of html report of integration tests, and upload to dokuwiki. #570

kdexd · 2016-05-24T16:32:01Z

This PR introduces a set of pytest functions which would be complaint with pytest-html. These fixtures introduce a layer of customization over default html report provided by plugin.

As mentioned in README of pytest-html, to generate a report, an extra argument has to be included as:

--html=path/to/file.html

This command line option no longer has to be specified, due to invocation of pytest_configure method, before test execution. It assigns a tempfile path to config.option.filepath for writing html report.
Another method pytest_unconfigure, serves as a global teardown. It uploads the test report to dokuwiki.
The command line argument --slow-tests-data has been renamed to --integration-tests and it accepts a path to YAML configuration for running slow tests. All other related fixtures have been modified. For now the schema of configuration file is:

reference: "/absolute/path/to/reference/dir"

dokuwiki:
  username: "username"
  password: "password"

This configuration file is parsed as a dict in a fixture created in conftest.py.

The checklist here maintains the status of work on this PR:

Show commit hash of the build on which current test run is being executed.
Make pytest-html write the html report to a tempfile, and --html may no longer be required.
Upload report to dokuwiki and leave no trace of tempfiles in filesystem.
Accept configuration for integration test run as YAML instad of individual fixtures.

- This method is automatically picked up and called by pytest during initialization. - To generate an html report, --html flag was used, filepath was provided through command line but this flag isn't necessary anymore as filepath is added in pytest_configure(). - Naming is done in accordance to tardis.__githash__

- This method is included in top level conftest and called after the tests are executed. It establishes a dokuwiki connection and uploads test report to dokuwiki. Naming is done as per githash. - Local file containing html test report is finally deleted.

kdexd · 2016-05-29T16:35:14Z

The resources referred were:
https://pytest.org/dev/plugins.html
https://pytest.org/dev/_modules/_pytest/hookspec.html
https://www.github.com/davehunt/pytest-html

Test execution looks like this:

Here is the report for fafa9e5, which is specified in the screenshot:
http://opensupernova.org/~karandesai96/integration2/doku.php?id=reports:fafa9e5

wkerzendorf · 2016-05-30T09:28:14Z

tardis/conftest.py

+def pytest_unconfigure(config):
+    # Html report created by pytest-html plugin is read here, uploaded to
+    # dokuwiki and finally deleted.
+    try:


you should do a try and except statement at the beginning with then just having dokuwiki_available boolean variable.

I'll do it that way then.

Done.

the try-except block should only contain things that fail. Move this block up to the import statements and only use if here.

wkerzendorf · 2016-05-31T11:54:52Z

tardis/conftest.py

+            # Upload report on dokuwiki. Temporary link due to prototyping purposes.
+            doku_conn.pages.append("reports:{0}".format(githash[:7]), report_content)
+        except gaierror, dokuwiki.DokuWikiError:
+            pass


again - stick the things that generate the error in the try except block and nothing else.

wkerzendorf · 2016-05-31T11:56:01Z

tardis/conftest.py

+
+try:
+    import dokuwiki
+    dokuwiki_available = True


same goes here - the things that raise the error need to be in the try, but nothing else.

- Remove all print statements from pytest_unconfigure (they might be included in debug log afterwards.) - Reorder import statements and add block comments.

kdexd · 2016-05-31T12:58:13Z

tardis/conftest.py

+            "Test executed on commit "
+            "[[https://www.github.com/tardis-sn/tardis/commit/{0}|{0}]]\n\n"
+            "{1}".format(githash, report_content)
+        )


@wkerzendorf These four lines are good to be here or in else block ? I spoke to myself: try to make a connection, except these two errors, else upload the report we have. Hence, I kept the report ready beforehand (compromising the time of execution of these lines if connection isn't at all made.)

@karandesai-96 it's not about execution time it's about making the code easy to understand. What really happens when the if is True, what happens when the if is False. I think its fine the way it is for now.

kdexd force-pushed the html-test-report branch from 1c2f7c3 to cce7c66 Compare May 29, 2016 04:14

Karan Desai added 3 commits May 29, 2016 21:07

Use tempfile module for writing html reports.

a4a8023

kdexd force-pushed the html-test-report branch from 9b29d5e to fafa9e5 Compare May 29, 2016 15:44

kdexd changed the title ~~[wip] Enable generation of html report of integration tests.~~ Enable generation of html report of integration tests, and upload to dokuwiki. May 29, 2016

wkerzendorf reviewed May 30, 2016
View reviewed changes

kdexd force-pushed the html-test-report branch 2 times, most recently from 3d81297 to 846952d Compare May 31, 2016 10:47

wkerzendorf reviewed May 31, 2016
View reviewed changes

kdexd force-pushed the html-test-report branch from 846952d to 0566d39 Compare May 31, 2016 11:55

wkerzendorf reviewed May 31, 2016
View reviewed changes

Karan Desai added 3 commits May 31, 2016 18:11

Refactor try-except of pytest_unconfigure, do cosmetic changes.

e0adcaf

- Remove all print statements from pytest_unconfigure (they might be included in debug log afterwards.) - Reorder import statements and add block comments.

Create integration-tests marker and fixture in conftest.py

33cc033

Modify fixtures for TestW7 to accept from YAML.

59429af

kdexd force-pushed the html-test-report branch from 135d08c to 59429af Compare May 31, 2016 12:46

kdexd reviewed May 31, 2016
View reviewed changes

wkerzendorf merged commit 84c3329 into tardis-sn:master May 31, 2016

kdexd deleted the html-test-report branch May 31, 2016 16:19

kdexd mentioned this pull request Jun 5, 2016

Shift slow test related pytest hooks in lower level conftest.py #578

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable generation of html report of integration tests, and upload to dokuwiki. #570

Enable generation of html report of integration tests, and upload to dokuwiki. #570

kdexd commented May 24, 2016 •

edited

Loading

kdexd commented May 29, 2016

wkerzendorf May 30, 2016

kdexd May 30, 2016 •

edited

Loading

wkerzendorf May 31, 2016

wkerzendorf May 31, 2016

wkerzendorf May 31, 2016

kdexd May 31, 2016

wkerzendorf May 31, 2016

Enable generation of html report of integration tests, and upload to dokuwiki. #570

Enable generation of html report of integration tests, and upload to dokuwiki. #570

Conversation

kdexd commented May 24, 2016 • edited Loading

kdexd commented May 29, 2016

wkerzendorf May 30, 2016

Choose a reason for hiding this comment

kdexd May 30, 2016 • edited Loading

Choose a reason for hiding this comment

wkerzendorf May 31, 2016

Choose a reason for hiding this comment

wkerzendorf May 31, 2016

Choose a reason for hiding this comment

wkerzendorf May 31, 2016

Choose a reason for hiding this comment

kdexd May 31, 2016

Choose a reason for hiding this comment

wkerzendorf May 31, 2016

Choose a reason for hiding this comment

kdexd commented May 24, 2016 •

edited

Loading

kdexd May 30, 2016 •

edited

Loading