Install 3rd party Python packages into QGIS

When you write plug-in for QGIS, you may need to rely on additional python packages to work with. However, I find no clue from qgis.org and forum search. After trial and error, here is the approach I used to install 3rd party python packages to QGIS.

Configuration

In my Windows XP, I have 2 copy of Python 2.5.2. One is the original stock version downloaded from www.python.org and installed at c:\python25. Another of course is the one come with QGIS. If you issue

import sys
sys.path

in stock Python console and QGIS Python console respectively, you will get two different results:

Result from stock Python console:

>>> import sys
>>> sys.path
['', 'C:\\windows\\system32\\python25.zip', 'c:\\python25\\DLLs', 'c:\\python25\\lib', 'c:\\python25
\\lib\\plat-win', 'c:\\python25\\lib\\lib-tk', 'c:\\python25', 'c:\\python25\\lib\\site-packages']

Result from QGIS Python Console:

>>> import sys
>>> sys.path
['C:/PROGRA~1/QUANTU~1/apps/qgis/./python', 'C:/Documents and Settings/19/.qgis/python', 'C:/Documents and Settings/19/.qgis/python/plugins', 'C:/PROGRA~1/QUANTU~1/apps/qgis/./python/plugins', 'C:\\PROGRA~1\\QUANTU~1\\apps\\gdal-17\\pymod', 'C:\\Program Files\\Quantum GIS Copiapo', 'C:\\PROGRA~1\\QUANTU~1\\bin\\python25.zip', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\DLLs', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib\\plat-win', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib\\lib-tk', 'C:\\PROGRA~1\\QUANTU~1\\apps\\qgis\\bin', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib\\site-packages', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib\\site-packages\\win32', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib\\site-packages\\win32\\lib', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib\\site-packages\\Pythonwin', 'C:\\PROGRA~1\\QUANTU~1\\apps\\Python25\\lib\\site-packages\\wx-2.8-msw-unicode', 'C:\\PROGRA~1\\QUANTU~1\\apps\\qgis\\python\\plugins\\fTools\\tools']

The above info tell us that Python package install in c:\python25 cannot be shared by Python in QGIS.

easy_install

If you want to install easy_install into QGIS, the setuptool will just tell you that it can only find one copy of Python in c:\python25 in registry:

To let setuptools find QGIS's python copy, go to Registry and look for HKEY_LOCAL_MACHINE\SOFTWARE\Python and export the selected branch to python25.reg beofore any modification. The exported python25.reg will look like this:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Python]

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore]

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5]

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\Help]

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\Help\Main Python Documentation]
@="C:\\Python25\\Doc\\Python25.chm"

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\InstallPath]
@="C:\\Python25\\"

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\InstallPath\InstallGroup]
@="Python 2.5"

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\Modules]

[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\PythonPath]
@="C:\\Python25\\Lib;C:\\Python25\\DLLs;C:\\Python25\\Lib\\lib-tk"

In the registry, there is only information of the Python installed in c:\python25 folder. Select HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonPath and change the path to QGIS one. That is:

replace c:\python25 to C:\Program Files\Quantum GIS Copiapo\apps\Python25

After change, export the Python branch again and save it as qgispython.reg for future use.

Also, you also need to add C:\Program Files\Quantum GIS Copiapo\bin and C:\Program Files\Quantum GIS Copiapo\apps\Python25 into PATH. I removed c:\python25 from the PATH when I add the above items into PATH. After installation, I revert the change by adding back c:\python25 and remove the two items.

If you haven't install MinGW, install it and at c:\mingw\bin into PATH. You may also need to tweak the cgwincompiler. Check here for detail.

After the above setups, setuptools should be able to install in QGIS folder:

However, if go to C:\Program Files\Quantum GIS Copiapo\apps\Python25\Scripts and execute easy_install, you will get no responds. The program just return nothing. In this case, you can use easy_install-script.py to install packages. For example, to install Jinja package,

C:\Program Files\Quantum GIS Copiapo\apps\Python25\Scripts>python easy_install-2.5-script.py pyodbc
Traceback (most recent call last):
File "easy_install-2.5-script.py", line 5, in
from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

C:\Program Files\Quantum GIS Copiapo\apps\Python25\Scripts>python eas_install-script.py jinja
python: can't open file 'eas_install-script.py': [Errno 2] No such file or directory

C:\Program Files\Quantum GIS Copiapo\apps\Python25\Scripts>python easy_install-script.py jinja
Searching for jinja
Reading http://pypi.python.org/simple/jinja/
Reading http://wsgiarea.pocoo.org/jinja/
Reading http://jinja.pocoo.org/
Best match: Jinja 1.2
Downloading http://pypi.python.org/packages/2.5/J/Jinja/Jinja-1.2-py2.5-win32.egg#md5=27b0804a126c2c
0ebd4a9dacffb9dcbb
Processing Jinja-1.2-py2.5-win32.egg
creating c:\progra~1\quantu~1\apps\python25\lib\site-packages\Jinja-1.2-py2.5-win32.egg
Extracting Jinja-1.2-py2.5-win32.egg to c:\progra~1\quantu~1\apps\python25\lib\site-packages
Adding Jinja 1.2 to easy-install.pth file

Installed c:\progra~1\quantu~1\apps\python25\lib\site-packages\jinja-1.2-py2.5-win32.egg
Processing dependencies for jinja
Finished processing dependencies for jinja

If you can issue "import jinja" in QGIS Python console without error, it means the installation is done.

Hope the above can help.

p.s. after install the 3rd party package, I will revert the registry (by double click python25.reg saved before )and edit the PATH (add c:\python25 and remove QGIS entries)

Recent search find another approach:
http://forum.qgis.org/viewtopic.php?f=2&t=8201

Use of Python in TEMS Investigation

TEMS Investigation is widely used in celluar network operators to carry out drive test activities. It supports Python to decode TEMS logfiles to user specific format.

Python Environment Setup

Currently, I am using TEMS Investigation 11.0.5. If you follow the manual of TEMS Invesitgaton to setup Python environment (PYTHONPATH for TI and PATH for Python25), you may still hit the following error:

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'TEMS.DataModel, Version=11.0.0.0, Culture=neutral, PublicKeyTok
en=148637998f549f7b' or one of its dependencies. The system cannot find the file specified.
File name: 'TEMS.DataModel, Version=11.0.0.0, Culture=neutral, PublicKeyToken=148637998f549f7b'

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

It seems there is a bug with the TEMS.pyd in the Application folder of TI. Copy the python.exe in c:\python25 to C:\Program Files\Ascom\TEMS Products\TEMS Investigation 11.0\Application should be able to solve the problem.

Also, make sure your Python version is 2.5x. Manual of TEMS Investigation clearly stated:

Please note that precisely this version (2.5.x) is required, not any earlier or later version

Reference

Useful links and reference that I visited frequently.

QGIS

QGIS
QGIS API
OSGeo.org QGIS forum
QGIS forum
PyQGIS Developer Cookbook

Python

Python
Python Packages
pysqlite usage guide
Python Online Doc

Spatialite

Spatialite
Spatialite tutorial

PyQt

Qt Online Reference Documentation

Compile Python 2.5 Packages with MinGW

In Python 2.5, default compiler is Visual Studio 2003. If you don't have the complier, you may end up with the following error message:

error: Python was built with Visual Studio 2003;
extensions must be built with a compiler than can generate compatible binaries. Visual Studio 2003 was not found on this system. If you have Cygwin installed, you can try compiling with MingW32, by passing "-c mingw32" to setup.py.

I don't have Visual Studio 2003 and therefore I try to follow the msg to download MingW32. Here are the steps:

1. Download MinGW from http://sourceforge.net/projects/mingw/ and install. It seems we better install the complier in default folder location c:\MinGW.

2. During the installation, I selected "C++ Compiler", "MSYS Basic System" and "MinGW Developer ToolKit".



3. Update your PATH: Add the C:\MinGW\bin directory to the system PATH. You may need to log-off and log-in again to reflect the added path.

4. Go to C:\Python25\Lib\distutils, create a file distutils.cfg and add the following lines:

[build]
compiler=mingw32

5. If you compile the package at this stage with "Python setup.py install", you may get the following error:

File "C:\Python25\lib\distutils\cygwinccompiler.py", line 84, in __init__
get_versions()
File "C:\Python25\lib\distutils\cygwinccompiler.py", line 424, in get_versions

ld_version = StrictVersion(result.group(1))
File "C:\Python25\lib\distutils\version.py", line 40, in __init__
self.parse(vstring)
File "C:\Python25\lib\distutils\version.py", line 107, in parse
raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number '2.20.51.20100613'

6. It says that your compiler version is invalid. Go to C:\Python25\Lib\distutils and backup a copy of cygwinccompiler.py. Open cygwinccompiler.py in text editor (e.g. notepad or IDLE). In def get_versions(): section you will find the following line:

result = re.search('(\d+\.\d+(\.\d+)*)',out_string)

Replace all "result = re.search('(\d+\.\d+(\.\d+)*)',out_string)" with the following code:

result = re.search('(\d+(\.\d+)?(\.\d+)?)',out_string)

It will tell Python to check the first digit only.

7. Try to compile again and it should work now.

8. Since I am working with Python 2.5.2, I am not sure 2.6 or 2.7 have that kind of issue or not.

Please tell me if it works for you are not.

Reference:
http://bugs.python.org/issue2234

About this blog

Welcome to Python code blog. Most likely, I think you reach this blog by searching info related to QGIS or Python. I hope you will find the blog useful.

Background

This is a blog about programming in Python. Or I should say it is about the problems that I encountered in creating QGIS Plugin with Python script. I am working in a mobile network company as an RF Engineer. My team started a project to port an old in-house GIS program written in Borland C++ to QGIS. As a telecom engineer, I try to use my limited programming skill to start the project.

In general, radio network planning team will have coverage planning tools such as Mentum and Atoll to assist their planning activities. The tools are quite expensive and license controlled. If we just want to display site information on GIS platform, Mapinfo is usually used. Mapinfo info is good and easy to use, but it is not free as well. Therefore, we try to find a suitable open-source platform to replace our old GIS tools.

Googling the web, we found several suitable GIS tools : MapWindow, Udig and QGIS. While all 3 GIS apps support programming or plug-ins, they are walking in different paths. MapWindow goes in dot net approach. If you are familiar with dot net and ActiveX, MapWindow may suit your need. Or if you are a Java programmer, you should consider Udig seriously. Since I am not good at programming, I choose an easier path of Python scripting approach.

After kick-off, I am almost submerged in the sea of QGIS, Python, PyQt, GIS file format and of course debugging process. It is quite difficult to remember everything without write it down clearly, especially to a "part-time programmer" like me. And I think it would be good to share the info to someone who needs to write QGIS plug-in in Python.

Without formal training or programming background, most of the codes and info shared are learnt by googling the web and wild-guess. So please accept my ignorance and your comments will be highly appreciated.

So you may ask what is the function of plug-in you are going to write? Screen captures are attached to give you some ideas:



Display of site inform on a map




Display signal level of drive test result


MP