AssertionError triggered by `pytest --override-ini='pythonpath=foo'`

[miccoli]

Description

The option --override-ini='pythonpath=foo' may trigger an AssertionError.

Minimal example

Running pytest --override-ini='pythonpath=foo' in an empty directory fails with

Traceback (most recent call last):
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1495, in getini
    return self._inicache[name]
           ~~~~~~~~~~~~~~^^^^^^
KeyError: 'pythonpath'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/miccoli/.pyenv/versions/_pytest_bug/bin/pytest", line 8, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 189, in console_main
    code = main()
           ^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 147, in main
    config = _prepareconfig(args, plugins)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 328, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_hooks.py", line 433, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_callers.py", line 133, in _multicall
    teardown[0].send(outcome)
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
    config: Config = outcome.get_result()
                     ^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_result.py", line 108, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1075, in pytest_cmdline_parse
    self.parse(args)
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1425, in parse
    self._preparse(args, addopts=addopts)
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1327, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_hooks.py", line 433, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_callers.py", line 155, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_result.py", line 108, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/python_path.py", line 15, in pytest_load_initial_conftests
    for path in reversed(early_config.getini("pythonpath")):
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1497, in getini
    self._inicache[name] = val = self._getini(name)
                                 ^^^^^^^^^^^^^^^^^^
  File "/Users/miccoli/.pyenv/versions/3.11.4/envs/_pytest_bug/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1540, in _getini
    assert self.inipath is not None
           ^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Output of pip list and enviornment

Package    Version
---------- -------
iniconfig  2.0.0
packaging  23.1
pip        23.2.1
pluggy     1.2.0
pytest     7.4.0
setuptools 65.5.0

Run on macOS-13.2.1-x86_64-i386-64bit, but same problem also on linux.

Context

I stumbled in this error while building a python package for Arch: instead of using

check(){
    cd "$_name-$pkgver"
    PYTHONPATH=src pytest
}

I tried

check(){
    cd "$_name-$pkgver"
    pytest -o pythonpath=src
}

which fails with the above traceback.

I replicated the problem on a MacOs stripped down venv just to be sure that it is not related to the ArchLinux build env, which is quite complicated.

[bluetech]

Thanks for the report.

Problem

This happens because:

1. Running pytest in a directory without a config file (or in one of the parents) means config.inipath is None.

2. -o pythonpath uses a type="paths" ini option.

3. type="paths" ini options are defined to take paths relative to inipath.

4. The code does not handle 1 + 3 at the same time:

   pytest/src/_pytest/config/__init__.py

   Lines 1544 to 1545 in 73d754b

   	# TODO: This assert is probably not valid in all cases.
   	assert self.inipath is not None

   (note: similar assert is in legacypath plugin for type="pathlist" options)

Possible solutions

Option 1: raise an error in case of trying to use a type="paths" option without an ini file.

Option 1': same as 1 but only for relative paths, so absolute paths are OK.

Option 2: use some other path instead if the inipath is not defined, most likely the current working directory/invocation path.

Since this basically can only happen when using -o on the command line, I think Option 2 is sensible, but would be interested in other's opinion.

[miccoli]

I see: I made the wrong assumption that PYTHONPATH=src pytest and pytest -o pythonpath=src should always be equivalent, but from your discussion I undestand now that the second one has a different meaning!

So I would suggest option 1 in order to enforce the override ini meaning of -o

Brief explanation of the cognitive pitfall

Probably I didn't read the docs with sufficent attention, but my error here was to assume that every variable in the pytest configuration options has a default value, and thus can be overridden. But when a config file is missing, some variables are undefined, and there is no meaningful way to override them.

Another not so easy fact to undestand was that a pyproject.toml file defines the self.inipath only if it contains a possibly empty [tool.pytest.ini_options] table. I was confused from this fact, not being able to undestand why sometimes the -o pythontpath=src option didn't trigger an error.

As what regards option 2, I would therefore suggest to consider also the directory containing pyproject.toml etc. as the self.inipath, even if the file does not contain any pytest configuration.

[nicoddemus]

As what regards option 2, I would therefore suggest to consider also the directory containing pyproject.toml etc. as the self.inipath, even if the file does not contain any pytest configuration.

This is actually a good point, because pyproject.toml is now the defacto standard to mean the root of a Python project.