Circular python import of SparsePoints

[agoscinski]

On the python side if one imports something from the rascal.models before importing rascal.representations one gets the following error message.

ImportError                               Traceback (most recent call last)
<ipython-input-3-939eeea548d7> in <module>
----> 1 from rascal.models import Kernel
~/lib/librascal/build/rascal/models/__init__.py in <module>
----> 1 from .sparse_points import SparsePoints
      2 from .krr import train_gap_model, KRR
      3 from .kernels import Kernel
~/lib/librascal/build/rascal/models/sparse_points.py in <module>
----> 1 from ..utils import BaseIO
      2 from ..lib._rascal.models import kernels
      3 from ..neighbourlist import AtomsList
      4 
      5 # names of existing pseudo points implementation on the pybinding side.
~/lib/librascal/build/rascal/utils/__init__.py in <module>
      5 from .io import (BaseIO, to_dict, from_dict, get_current_io_version,
      6                  get_supported_io_versions, dump_obj, load_obj)
----> 7 from .cur import CURFilter
      8 from .scorer import get_score, print_score
      9 
~/lib/librascal/build/rascal/utils/cur.py in <module>
      5                                   convert_selected_global_index2perstructure_index)
      6 
----> 7 from ..models.sparse_points import SparsePoints
      8 import numpy as np
      9 from scipy.sparse.linalg import svds
ImportError: cannot import name 'SparsePoints'

There is probably a circular import of SparsePoint happening somewhere in rascal.models which gets resolved by rascal.representations

[max-veit]

I think this is the same issue as #273, which was fixed by PR #274... are you sure you're using the latest version of the Python bindings?

[agoscinski]

just tried in master import rascal.models after cleaning my build, still does not work

[max-veit]

I've tracked down at least one import loop, which starts in rascal.models's __init__.py:
https://github.com/cosmo-epfl/librascal/blob/6dca371be05d025a43a28fe41f3b97a49d4813c8/bindings/rascal/models/__init__.py#L1
which of course goes to import SparsePoints, which requires BaseIO from the utils submodule:https://github.com/cosmo-epfl/librascal/blob/6dca371be05d025a43a28fe41f3b97a49d4813c8/bindings/rascal/models/sparse_points.py#L1
but the utils submodule's own __init__.py imports lots of things, like the CUR filter:https://github.com/cosmo-epfl/librascal/blob/6dca371be05d025a43a28fe41f3b97a49d4813c8/bindings/rascal/utils/__init__.py#L14
which in turn requires SparsePoints:https://github.com/cosmo-epfl/librascal/blob/6dca371be05d025a43a28fe41f3b97a49d4813c8/bindings/rascal/utils/cur.py#L9
completing the loop. And of course, the sparse_points module at this point has only been partially initialized, so it fails with the above-mentioned "circular import" error.

This issue perfectly illustrates the danger of having too many "convenience" imports in __init__.py files. It might make sense for a few high-level classes, like Kernel or CURFilter, but add them indiscriminately just to save time and you end up with tricky, loopy dependency graphs like we have now.

I've got a minimal fix in 1912b24 (which fixes another loop as well - though it does break one or two examples using the "convenience" import from rascal.models import SparsePoints rather than from rascal.models.sparse_points import SparsePoints). In general, though, I think we should try to pare down the __init__ imports to the bare minimum -- definitely nothing that's required by multiple other modules and could cause a messy dependency loop like this one.