We are not pure python anymore

[aronasorman]

Branch: 0.15.x & develop

While trying to get KA Lite running on the Sandisk, we noticed that we tried compiling PyCrypto. Unlike other libraries PyCrypto does not have a pure python fallback, and we fail the installation.

I believe the trojan horse that introduced said library was django-dbbackup. Can you confirm @benjaoming?

Temp solution: fork django-dbbackup, find a way to remove the pycrypto dependency. Failing that, make the backup feature opt-out, and add another config for sandisk which disables this.

[rtibbles]

@aronasorman Would this help at all?

https://cloud.google.com/appengine/docs/python/tools/libraries?csw=1#PyCrypto

[benjaoming]

@aronasorman not a dependency for django-dbbackup that I'm familiar of:

https://github.com/django-dbbackup/django-dbbackup/blob/master/requirements.txt

Could you clarify:

we noticed that we tried compiling PyCrypto

When did this happen?

[aronasorman]

Actually this is really only a problem for the sandisk, which we don't support yet -- the other platforms have installers. Maybe this is a good time to discuss whether we can relax the pure python requirement.

/cc @jamalex @benjaoming @MCGallaspy @cpauya

[aronasorman]

@benjaoming happens when we do a pip install -r requirements.txt So at least one of those deps have PyCrypto in them.

[benjaoming]

@aronasorman which system is the sandisk running?

ka-lite-raspberry-pi depends on these packages from the ubuntu repo:

https://github.com/learningequality/installers/blob/master/debian/debian/control#L27

[aronasorman]

It's a very minimal Linux machine. There's no package manager whatsoever, and we have to cross compile python for it. It also only has 128 MB of RAM.

[MCGallaspy]

Maybe this is a good time to discuss whether we can relax the pure python requirement.

Can we easily install on our target platforms? There's nothing inherently special about pure python, the goal as I understand it is to be able to run on a large variety of platforms. If we can achieve that, we're okay.

[MCGallaspy]

BTW, it's able to build on the pi with raspbian no problem if you have the python-dev package, for the Python.h header.

[aronasorman]

@MCGallaspy did you have to install build-essential on it? I believe that pulls in gcc as well. python-dev might've depended on it.

re: large variety of platforms -- if we don't have an installer for it, the additional requirements for installing KA Lite are the python headers, and a working C compiler. This might be a good time to look a the stats that @jamalex has to see what platforms are actually running KA Lite.

[benjaoming]

@aronasorman I see the problem:

Collecting pysftp (from django-dbbackup==2.0.4->-r ka-lite/requirements.txt (line 9))
Collecting paramiko>=1.7.7 (from pysftp->django-dbbackup==2.0.4->-r ka-lite/requirements.txt (line 9))
  Downloading paramiko-1.15.3-py2.py3-none-any.whl (166kB)
    100% |████████████████████████████████| 167kB 607kB/s 
Collecting ecdsa>=0.11 (from paramiko>=1.7.7->pysftp->django-dbbackup==2.0.4->-r ka-lite/requirements.txt (line 9))
  Using cached ecdsa-0.13-py2.py3-none-any.whl
Collecting pycrypto!=2.4,>=2.1 (from paramiko>=1.7.7->pysftp->django-dbbackup==2.0.4->-r ka-lite/requirements.txt (line 9))
  Using cached pycrypto-2.6.1.tar.gz
Building wheels for collected packages: pycrypto
  Running setup.py bdist_wheel for pycrypto
  Stored in directory: /home/benjamin/.cache/pip/wheels/96/b0/e6/03e439d41cb2592b5c4c9c77873761d6cbd417b332076680cd
Successfully built pycrypto

We pinned dbbackup==2.0.4. Should be 2.3.1 which is 2 days old.