`pip install kivy` fails with `'../include/config.pxi' not found`

[sietse]

I am attempting to install kivy in a virtualenv, following the official instructions. However, when I run pip install kivy I get the following error (full traceback and steps to reproduce below). I am running Ubuntu 16.04 LTS 'Xenial'. Where should config.pxi come from, and what should it contain? Am I missing an installation step, or is this due to an assumption in Kivy's setup.py?

   [...]
   File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 474, in included_files
     include_path = self.context.find_include_file(include, None)
   File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 274, in find_include_file
     error(pos, "'%s' not found" % filename)
   File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Compiler/Errors.py", line 177, in error
     raise InternalError(message)
 Cython.Compiler.Errors.InternalError: Internal compiler error: '../include/config.pxi' not found

The installation process looks for config.pxi in these locations:

• /tmp/pip-build-MB2bZR/include/config.pxi
• /home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/include/config.pxi

Investigation

The problem is consistent across versions

The problem occurs both with pip install kivy==1.10.0 and with pip install git+https://github.com/kivy/kivy.git@master.

Execution context:

The problem occurs in find_include_file in Cython 0.25's Main.py.

Relevant bits from strace

(Thanks to Julia Evans for her zine explaining strace!)

utimes("/tmp/pip-build-PQ2L9G/kivy/kivy/graphics/tesselator.pxd", [{1485276501, 0}, {1485276501, 0}]) = 0
stat("/tmp/pip-build-PQ2L9G/kivy/kivy", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
mkdir("/tmp/pip-build-PQ2L9G/kivy/kivy/graphics", 0777) = -1 EEXIST (File exists)
open("/tmp/pip-build-PQ2L9G/kivy/kivy/graphics/instructions.pxd", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 5
fstat(5, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fstat(5, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
write(5, "include \"../include/config.pxi\"\n"..., 3564) = 3564
[...]
stat("/tmp/pip-build-PQ2L9G/kivy/kivy/graphics/../include/config.pxi", 0x7ffc556c92a0) = -1 ENOENT (No such file or directory)
getcwd("/tmp/pip-build-PQ2L9G/kivy", 1024) = 27
getcwd("/tmp/pip-build-PQ2L9G/kivy", 1024) = 27
stat("./../include/config.pxi", 0x7ffc556c89c0) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Includes/../include/config.pxi", 0x7ffc556c89c0) = -1 ENOENT (No such file or directory)

Full traceback

(kivy) vagrant@vagrant:~$ pip install kivy==1.10.0
Collecting kivy==1.10.0
  Using cached Kivy-1.10.0.tar.gz
    Complete output from command python setup.py egg_info:
    fatal: Not a git repository (or any of the parent directories): .git
    Using distutils
    
    Detected Cython version 0.25
    Using this graphics system: OpenGL
    GStreamer found via pkg-config
    WARNING: A problem occurred while running pkg-config --libs --cflags sdl2 SDL2_ttf SDL2_image SDL2_mixer (code 1)
    
    Package sdl2 was not found in the pkg-config search path.
    Perhaps you should add the directory containing `sdl2.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'sdl2' found
    Package SDL2_ttf was not found in the pkg-config search path.
    Perhaps you should add the directory containing `SDL2_ttf.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'SDL2_ttf' found
    Package SDL2_image was not found in the pkg-config search path.
    Perhaps you should add the directory containing `SDL2_image.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'SDL2_image' found
    Package SDL2_mixer was not found in the pkg-config search path.
    Perhaps you should add the directory containing `SDL2_mixer.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'SDL2_mixer' found
    
    
    running egg_info
    creating pip-egg-info/Kivy.egg-info
    writing requirements to pip-egg-info/Kivy.egg-info/requires.txt
    writing pip-egg-info/Kivy.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/Kivy.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/Kivy.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/Kivy.egg-info/SOURCES.txt'
     Traceback (most recent call last):
       File "<string>", line 1, in <module>
       File "/tmp/pip-build-JTLGTF/kivy/setup.py", line 1060, in <module>
         ] if not skip_cython else [])
       File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
         dist.run_commands()
       File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
         self.run_command(cmd)
       File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
         cmd_obj.run()
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 280, in run
         self.find_sources()
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 295, in find_sources
         mm.run()
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 526, in run
         self.add_defaults()
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 562, in add_defaults
         sdist.add_defaults(self)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
         self._add_defaults_ext()
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
         build_ext = self.get_finalized_command('build_ext')
       File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
         cmd_obj.ensure_finalized()
       File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
         self.finalize_options()
       File "/tmp/pip-build-JTLGTF/kivy/setup.py", line 251, in finalize_options
         retval = build_ext.finalize_options(self)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Distutils/build_ext.py", line 19, in finalize_options
         self.distribution.ext_modules)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 809, in cythonize
         aliases=aliases)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 714, in create_extension_list
         kwds = deps.distutils_info(file, aliases, base).values
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 590, in distutils_info
         return (self.transitive_merge(filename, self.distutils_info0, DistutilsInfo.merge)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 600, in transitive_merge
         node, extract, merge, seen, {}, self.cimported_files)[0]
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 605, in transitive_merge_helper
         deps = extract(node)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 581, in distutils_info0
         externs = self.cimports_and_externs(filename)[1]
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Utils.py", line 44, in wrapper
         res = cache[args] = f(self, *args)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 491, in cimports_and_externs
         for include in self.included_files(filename):
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Utils.py", line 44, in wrapper
         res = cache[args] = f(self, *args)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 474, in included_files
         include_path = self.context.find_include_file(include, None)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 285, in find_include_file
         error(pos, "'%s' not found" % filename)
       File "/home/vagrant/.virtualenvs/kivy/local/lib/python2.7/site-packages/Cython/Compiler/Errors.py", line 177, in error
         raise InternalError(message)
     Cython.Compiler.Errors.InternalError: Internal compiler error: '../include/config.pxi' not found

Steps to reproduce

Because the steps to reproduce include sudo apt-get install, I have placed
them in a Vagrantfile. Vagrant makes it trivial to create and destroy virtual
machines that have the same environment every time. This is useful for
reproducible development environments, but also to demonstrate bugs.

If you do not want to use Vagrant, look for the code below.

  config.vm.provision "shell", inline: <<-SHELL
    # ... this is the code you need ...
  SHELL

Vagrant usage

• Install Vagrant from https://www.vagrantup.com
• Create a directory, and place the contents below in a file called Vagrantfile
• cd to the directory and run vagrant up
• The problem mentioned above will occur at the end of the provisioning.
• You can use vagrant ssh to SSH into the virtual machine
• You can use vagrant destroy + vagrant up to recreate the machine from scratch

Vagrantfile

Call this file Vagrantfile. See the previous section for usage.

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|

  config.vm.box = "bento/ubuntu-16.04"

  config.vm.synced_folder "data", "/vagrant_data"

  config.vm.provision "shell", inline: <<-SHELL
    # apt-get update

    sudo apt-get install -y \
      build-essential \
      mercurial \
      git \
      virtualenv \
      virtualenvwrapper \
      python2.7 \
      python-dev \
      libfreetype6-dev \
      libjpeg-dev \
      libav-tools \
      libsdl-image1.2-dev \
      libsdl-mixer1.2-dev \
      libsdl-ttf2.0-dev \
      libsmpeg-dev \
      libsdl1.2-dev \
      libportmidi-dev \
      libswscale-dev \
      libavformat-dev \
      libavcodec-dev \
      libfreetype6-dev \
      unzip \
      zlib1g-dev

    . /usr/share/virtualenvwrapper/virtualenvwrapper.sh
    mkvirtualenv kivy
    kivypip=$HOME/.virtualenvs/kivy/bin/pip

    $kivypip install cython==0.25
    $kivypip install kivy==1.10.0
  SHELL
end

[sietse]

Blast, I submitted this to the wrong project. I am very sorry!