Skip to content

Changes in requirements to run Studio on Python 3.8#2236

Closed
jredrejo wants to merge 1 commit intolearningequality:developfrom
jredrejo:studio_python3.8
Closed

Changes in requirements to run Studio on Python 3.8#2236
jredrejo wants to merge 1 commit intolearningequality:developfrom
jredrejo:studio_python3.8

Conversation

@jredrejo
Copy link
Copy Markdown
Member

@jredrejo jredrejo commented Sep 8, 2020

Description

Upgrade some python packages to be able to run Studio on python 3.8

Steps to Test

  • Using different versions of Python:
    pip install -r requirements.txt
    make prodceleryworkers

  • Do current tests pass?

Reviewers

If you are looking to assign a reviewer, here are some options:

  • Jordan jayoshih (full stack)
  • Aron aronasorman (back end, devops)
  • Micah micahscopes (full stack)
  • Kevin kollivier (back end)
  • Ivan ivanistheone (Ricecooker)
  • Richard rtibbles (full stack, Kolibri)
  • Radina @radinamatic (documentation)

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 8, 2020

Codecov Report

Merging #2236 into develop will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #2236   +/-   ##
========================================
  Coverage    80.80%   80.80%           
========================================
  Files          289      289           
  Lines        14022    14022           
========================================
  Hits         11331    11331           
  Misses        2691     2691

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3814abe...3e2fae2. Read the comment docs.

@DXCanas
Copy link
Copy Markdown
Member

DXCanas commented Sep 23, 2020

Getting some errors:

$ yarn run devsetup
yarn run v1.22.5
$ cd contentcuration && python manage.py setup --settings=contentcuration.dev_settings
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 206, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 40, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/david/Projects/studio/contentcuration/contentcuration/management/commands/setup.py", line 22, in <module>
    from contentcuration.utils.db_tools import *
  File "/home/david/Projects/studio/contentcuration/contentcuration/utils/db_tools.py", line 22, in <module>
    from contentcuration.utils.files import duplicate_file
  File "/home/david/Projects/studio/contentcuration/contentcuration/utils/files.py", line 27, in <module>
    from pressurecooker.images import create_image_from_pdf_page
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/pressurecooker/images.py", line 17, in <module>
    import matplotlib.pyplot as plt
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/pyplot.py", line 31, in <module>
    import matplotlib.colorbar
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/colorbar.py", line 32, in <module>
    import matplotlib.artist as martist
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/artist.py", line 16, in <module>
    from .path import Path
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/path.py", line 21, in <module>
    from . import _path, rcParams
ImportError: numpy.core.multiarray failed to import
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

@DXCanas
Copy link
Copy Markdown
Member

DXCanas commented Sep 23, 2020

Also submitted a PR for some missing apt packages on Ubuntu 20, which is the reason I'm able to test this so thoroughly :)
jredrejo#1

@jredrejo
Copy link
Copy Markdown
Member Author

jredrejo commented Sep 23, 2020

Getting some errors:

$ yarn run devsetup
yarn run v1.22.5
$ cd contentcuration && python manage.py setup --settings=contentcuration.dev_settings
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 206, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 40, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/david/Projects/studio/contentcuration/contentcuration/management/commands/setup.py", line 22, in <module>
    from contentcuration.utils.db_tools import *
  File "/home/david/Projects/studio/contentcuration/contentcuration/utils/db_tools.py", line 22, in <module>
    from contentcuration.utils.files import duplicate_file
  File "/home/david/Projects/studio/contentcuration/contentcuration/utils/files.py", line 27, in <module>
    from pressurecooker.images import create_image_from_pdf_page
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/pressurecooker/images.py", line 17, in <module>
    import matplotlib.pyplot as plt
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/pyplot.py", line 31, in <module>
    import matplotlib.colorbar
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/colorbar.py", line 32, in <module>
    import matplotlib.artist as martist
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/artist.py", line 16, in <module>
    from .path import Path
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/path.py", line 21, in <module>
    from . import _path, rcParams
ImportError: numpy.core.multiarray failed to import
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

@DXCanas this is my numpy installation:

(venv) jose@jose-xps:/datos/le/studio$ pip show numpy
Name: numpy
Version: 1.15.4
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /datos/le/studio/venv/lib/python3.8/site-packages
Requires: 
Required-by: wordcloud, pressurecooker, matplotlib

can you check which version do you have installed in your venv?

@DXCanas
Copy link
Copy Markdown
Member

DXCanas commented Sep 23, 2020 •
edited
Loading

Getting some errors:

$ yarn run devsetup
yarn run v1.22.5
$ cd contentcuration && python manage.py setup --settings=contentcuration.dev_settings
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 206, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/django/core/management/__init__.py", line 40, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/david/Projects/studio/contentcuration/contentcuration/management/commands/setup.py", line 22, in <module>
    from contentcuration.utils.db_tools import *
  File "/home/david/Projects/studio/contentcuration/contentcuration/utils/db_tools.py", line 22, in <module>
    from contentcuration.utils.files import duplicate_file
  File "/home/david/Projects/studio/contentcuration/contentcuration/utils/files.py", line 27, in <module>
    from pressurecooker.images import create_image_from_pdf_page
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/pressurecooker/images.py", line 17, in <module>
    import matplotlib.pyplot as plt
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/pyplot.py", line 31, in <module>
    import matplotlib.colorbar
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/colorbar.py", line 32, in <module>
    import matplotlib.artist as martist
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/artist.py", line 16, in <module>
    from .path import Path
  File "/home/david/.virtualenvs/studio/lib/python3.8/site-packages/matplotlib/path.py", line 21, in <module>
    from . import _path, rcParams
ImportError: numpy.core.multiarray failed to import
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

@DXCanas this is my numpy installation:

(venv) jose@jose-xps:/datos/le/studio$ pip show numpy
Name: numpy
Version: 1.15.4
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /datos/le/studio/venv/lib/python3.8/site-packages
Requires: 
Required-by: wordcloud, pressurecooker, matplotlib

can you check which version do you have installed in your venv?

Sure! I have the same as you:

$ pip show numpy
Name: numpy
Version: 1.15.4
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /home/david/.virtualenvs/studio/lib/python3.8/site-packages
Requires: 
Required-by: wordcloud, pressurecooker, matplotlib

Seems like I'm getting the same error with make prodceleryworkers now too

@rtibbles
Copy link
Copy Markdown
Member

rtibbles commented Nov 4, 2020

Note that numpy should now be removed from the Studio dependencies.

@jredrejo jredrejo mentioned this pull request Nov 11, 2020
Merged
@rtibbles
Copy link
Copy Markdown
Member

rtibbles commented Feb 1, 2021

As noted from the discussion in #2533 it is probably desirable that we have a single Python version target for Studio. Perhaps we can upgrade that target to Python 3.8 when we upgrade the Django version, but for now Python 3.6 is what we are using. Closing for now.

@rtibbles rtibbles closed this Feb 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kollivier kollivier Awaiting requested review from kollivier

@DXCanas DXCanas Awaiting requested review from DXCanas

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jredrejo @DXCanas @rtibbles