Example.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `ipytest` Summary\n",
    "\n",
    "`ipytest` allows you to run [pytest](https://pytest.org) in Jupyter notebooks. `ipytest` aims to give access to the full `pytest` experience to make it easy to transfer tests out of notebooks into separate test files.\n",
    "\n",
    "To get started install `ipytest` via"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -U ipytest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use `ipytest`, import it and configure the notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import ipytest\n",
    "\n",
    "ipytest.autoconfig()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In most cases, running `ipytest.autoconfig()` will result in reasonable defaults:\n",
    "\n",
    "- register the `%%ipytest` magic to execute tests\n",
    "- register the `pytest` assert rewriter with the notebook to get nice assert messages \n",
    "\n",
    "For more control, pass the relevant arguments to `ipytest.autconfig()`. For details, see the reference in the readme."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Execute tests\n",
    "\n",
    "To execute test, just decorate the cells containing tests with the `%%ipytest` magic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m.\u001b[0m\u001b[32m                                                                                            [100%]\u001b[0m\n",
      "\u001b[32m\u001b[32m\u001b[1m1 passed\u001b[0m\u001b[32m in 0.02s\u001b[0m\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "%%ipytest\n",
    "\n",
    "# define the tests\n",
    "\n",
    "\n",
    "def test_my_func():\n",
    "    assert my_func(0) == 0\n",
    "    assert my_func(1) == 0\n",
    "    assert my_func(2) == 2\n",
    "    assert my_func(3) == 2\n",
    "\n",
    "\n",
    "def my_func(x):\n",
    "    return x // 2 * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To execute tests without IPython magics use the `ipytest.run` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m.\u001b[0m\u001b[32m                                                                                            [100%]\u001b[0m\n",
      "\u001b[32m\u001b[32m\u001b[1m1 passed\u001b[0m\u001b[32m in 0.02s\u001b[0m\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<ExitCode.OK: 0>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ipytest.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using pytest fixtures\n",
    "\n",
    "Common pytest features, such as fixtures and parametrize, are supported out of the box:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m                                                                                        [100%]\u001b[0m\n",
      "\u001b[32m\u001b[32m\u001b[1m5 passed\u001b[0m\u001b[32m in 0.05s\u001b[0m\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "%%ipytest\n",
    "\n",
    "import pytest\n",
    "\n",
    "\n",
    "@pytest.mark.parametrize(\n",
    "    \"input,expected\",\n",
    "    [\n",
    "        (0, 0),\n",
    "        (1, 0),\n",
    "        (2, 2),\n",
    "        (3, 2),\n",
    "    ],\n",
    ")\n",
    "def test_parametrized(input, expected):\n",
    "    assert my_func(input) == expected\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def my_fixture():\n",
    "    return 42\n",
    "\n",
    "\n",
    "def test_fixture(my_fixture):\n",
    "    assert my_fixture == 42"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selecting tests"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[pytest](https://pytest.org) offers a [extensive options](https://docs.pytest.org/en/latest/how-to/usage.html#specifying-which-tests-to-run) to select subsets of tests to run. They can be also used in conjunction with `ipytest`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m                                                                                           [100%]\u001b[0m\n",
      "\u001b[32m\u001b[32m\u001b[1m2 passed\u001b[0m, \u001b[33m2 deselected\u001b[0m\u001b[32m in 0.03s\u001b[0m\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "%%ipytest -k feature1\n",
    "\n",
    "\n",
    "def test_feature1_test1():\n",
    "    assert True\n",
    "\n",
    "\n",
    "def test_feature1_test2():\n",
    "    assert True\n",
    "\n",
    "\n",
    "def test_feature2_test1():\n",
    "    pytest.fail(\"expected failure\")\n",
    "\n",
    "\n",
    "def test_feature2_test2():\n",
    "    pytest.fail(\"expected failure\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tests can also be selected based on node ids. The notebook can be referenced via the special `{MODULE}` name. In addition, it is possible to generate node ids for tests inside the notebook via the `{test_name}` shorthand. For example `{test_feature1_test1}` references the corresponding test defined in the notebook: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m.\u001b[0m\u001b[32m                                                                                            [100%]\u001b[0m\n",
      "\u001b[32m\u001b[32m\u001b[1m1 passed\u001b[0m\u001b[32m in 0.03s\u001b[0m\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "%%ipytest {test_feature1_test1}\n",
    "\n",
    "\n",
    "def test_feature1_test1():\n",
    "    assert True\n",
    "\n",
    "\n",
    "def test_feature1_test2():\n",
    "    assert pytest.fail(\"expected failure\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Deselection](https://docs.pytest.org/en/7.1.x/example/pythoncollection.html#deselect-tests-during-test-collection) works as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m.\u001b[0m\u001b[32m                                                                                            [100%]\u001b[0m\n",
      "\u001b[32m\u001b[32m\u001b[1m1 passed\u001b[0m, \u001b[33m1 deselected\u001b[0m\u001b[32m in 0.03s\u001b[0m\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "%%ipytest --deselect {test_feature1_test2}\n",
    "\n",
    "\n",
    "def test_feature1_test1():\n",
    "    assert True\n",
    "\n",
    "\n",
    "def test_feature1_test2():\n",
    "    pytest.fail(\"expected failure\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Debugging failed tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [debugging functionality of pytest](https://docs.pytest.org/en/latest/how-to/failures.html) can be used as well. For example, to debug the first failed test (and then stop the pytest run) use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "nbval-skip"
    ]
   },
   "outputs": [],
   "source": [
    "%%ipytest -x --pdb\n",
    "\n",
    "\n",
    "def test_example():\n",
    "    for i in range(10):\n",
    "        if i == 5:\n",
    "            raise ValueError(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [
     "NBVAL_RAISES_EXCEPTION"
    ]
   },
   "source": [
    "## Checking notebooks automatically\n",
    "\n",
    "`ipytest` itself does not support validating notebooks in a programmatic fashion. For this task, the [`nbval` package](https://nbval.readthedocs.io/en/latest) can be used. In its default configuration, `ipytest` will not raise lead to execution failures, but only display the exception. While this behavior is helpful during interactive development, it will prevent `nbval` from catching errors. To raise errors if any test fails, configure `raise_on_error=True`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "nbval-raises-exception"
    ]
   },
   "outputs": [],
   "source": [
    "%%ipytest\n",
    "# ipytest: raise_on_error=True\n",
    "\n",
    "\n",
    "def test():\n",
    "    raise ValueError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}