Building pyexiv2 on Windows

Version: 19 August 2008.

Before I launch into the details here, I'd like to thank Rob Wallace in New Zealand for cooperating in the preparation of these notes. Rob was especially helpful in the development of the cross-plaform SConscript file required to build pyexiv2 on different platforms. Thank you, Rob.

You'll have to build and install the following:

1. Active Python http://www.activestate.com/Products/activepython
   or build your own python: http://www.python.org/download/
2. bjam http://www.boost.org/
3. boost http://www.boost.org/
4. libexpat http://sourceforge.net/projects/expat/
5. libexiv2 http://www.exiv2.org/
6. scons http://www.scons.org/
7. pyexiv2 http://tilloy.net/dev/pyexiv2/
8. Installation and Testing
9. Document/Code History

You're going to have to put the code somewhere to work on it. I use c:\gnu. The name c:\gnu is not important and it doesn't need to be a top level directory. However, you are going to have a build tree with the following directories:

c:\gnu>dir
 Volume in drive C has no label.
 Volume Serial Number is BCA7-C689

 Directory of c:\gnu

04/02/2008  10:06    <DIR>          exiv2
04/02/2008  08:26    <DIR>          expat-2.0.1
04/22/2008  22:37    <DIR>          pyexiv2
04/02/2008  14:32    <DIR>          Python-2.5.2
04/01/2008  20:16               285 hello.cpp
               1 File(s)            285 bytes

I'll explain in a few minutes how to obtain these directories.

1 Active Python (or build your own python)

You'll have to install python. I obtained Active Python from www.activestate.com (ActivePython-2.5.1.1-win32-x86.msi). I installed it into c:\Python25

Alternatively you can download, build and install python 2.5 from source from sourceforge.net. I used Python-2.5.2.tar and unzipped into the directory: C:\gnu\Python-2.5.2. You'll probably have to do this, even if you're using ActivePython. You're going to need some headers from the Python code base to compile the pyexiv2 code.

2 Build or install bjam

Before you can build the boost-python libraries, you'll need to download/install bjam. This is the 'make' utility of boost. I found a prebuilt version on the net (boost-jam-3.1.16-1-ntx86.zip), however you can also download and build it from sourceforge.net. You should put bjam.exe on your path.

And now we're into chickens and eggs, right? We can't build bjam with bjam. Can we build it with ./configure && make ? NO! use build.bat. I haven't tried this, I download the pre-built version and put it on my path. You should also update your environment with INCLUDE and BOOST_ROOT.

In the following console log, pathy is a little perl script to pretty print environment strings. Which is an MKS utility to locate an executable on the path.

c:\>set | grep -i boo
BOOST_ROOT=c:\boost_1_35_0
INCLUDE=c:\boost_1_35_0

c:\>pathy path
c:\robin\com             <---- this is where I add private scripts and stuff
c:\python25
c:\python25\scripts
... lots of stuff deleted ....

c:\>which bjam
c:\robin\com\bjam.exe    <---- bjam's on the path
c:\>

3 Install and build BOOST

I installed and built boost 1.35.0. This builds with very little effort. The on-line documents which come with the code are very good.

C:\boost_1_35_0>bjam release
.... loads of warning and stuff
msvc.link.dll bin.v2\libs\re

Creating library bin.v2\llibs\regex ...
...updated 221 targets...

C:\boost_1_35_0>

The built object that's useful to us is the boost-python library:

c:\boost_1_35_0\bin.v2\libs\python\build\msvc-8.0\release\threading-multi\boost_python-vc80-mt-1_35.dll
04/01/2008  20:43           258,048 boost_python-vc80-mt-1_35.dll

Once you have boost built. Build and run this little test program hello.cpp: click here.

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

c:\>call "\Program Files\Microsoft Visual Studio 8\VC\bin\vcvars32.bat"

c:\>"c:\Program Files\Microsoft Visual Studio 8\Common7\Tools\vsvars32.bat"
Setting environment for using Microsoft Visual Studio 2005 x86 tools.

c:\>cd gnu

c:\gnu>dir
 Volume in drive C has no label.
 Volume Serial Number is BCA7-C689

 Directory of c:\gnu

04/02/2008  10:06    <DIR>          exiv2
04/02/2008  08:26    <DIR>          expat-2.0.1
04/01/2008  20:16               285 hello.cpp
04/22/2008  22:37    <DIR>          pyexiv2
04/02/2008  14:32    <DIR>          Python-2.5.2
               1 File(s)            285 bytes

C:\gnu>type hello.cpp
#include <boost/lambda/lambda.hpp>
#include <iostream>
#include <iterator>
#include <algorithm>

int main()
{
    using namespace boost::lambda;
    typedef std::istream_iterator<int> in;

    std::for_each(
        in(std::cin), in(), std::cout << (_1 * 3) << " " );
}

c:\gnu>nmake -a hello.exe CPPFLAGS=/EHsc

Microsoft (R) Program Maintenance Utility Version 8.00.50727.762
Copyright (C) Microsoft Corporation.  All rights reserved.

        cl /EHsc hello.cpp
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

hello.cpp
Microsoft (R) Incremental Linker Version 8.00.50727.762
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:hello.exe
hello.obj

c:\gnu>hello
2
6 10
30 q

c:\gnu>

4 libexpat - dependent library

libexiv2 has dependancies on expat to read XML files. As with boost, it builds very easily. Read the documentation that comes with expat - it's good.

I downloaded expat-2.0.1.tar.gz, decompressed and expanded it to the directory: c:\gnu\expat-2.0.1. I opened the file expat.dsw in Visual Studio and clicked 'OK' to upgrade the project files. Build/Batch build. Deselect All. Select expat-release and expat-debug. Build. Rattle and role and it builds in about 1 minute.

Expat can build dynamic and static (debug and release) libraries in unicode and ascii version. There are 14 build targets - however we only require the Dynamic MT and MTd ascii libraries (libexpat.dll).

5 libexiv2 - dependent library

Now you'll need to build libexiv2 for windows. I pulled down and built libexiv2-0.17.1. I extracted the directory and put it into the build tree as c:\gnu\exiv2.

Then I encountered lots of difficults getting the code to build! Here are some of the issues:

• Can't get debug and release to link
• Compilation errors
• The debug and release builds are overwritting each other
• The linkage is incorrect - a mixture of static and dynamic linking
• Unsatisfied externals from the linker on the release build
• Having to manually copy files from the expat build
• ... and probably some other pain I've forgotton

I believe I've fixed this on Visual Studio .Net 2003 (VC7.1 and up) by modifying the following files:

exiv2\msvc\exiv2\exiv2.vcproj
exiv2\msvc\exiv2lib\exiv2lib.vcproj
exiv2\msvc\xmpsdk\xmpsdk.vcproj

You can download the replacements: click here.

Replace the files in exiv2 and open exiv2\msvc\exiv2.sln with Visual Studio. If you're on 2005 and later, the project update wizard will step in and upgrade your projects. When he's done, select Build/Batch Build/ and select exiv2-debug and exiv2-release / Rebuild. You'll get thousands of warnings, however the code should build completely and successfully.

Do not follow the directions in exiv2\README to use the "Librarian/Link Library Dependencies" - I don't believe that is necessary. And do not copy anything from the libexpat-2.0.1 directory - the modified vcproj files will do this on your behalf.

After I built libexiv2 and exiv2.exe, I executed some tests with exiv2.exe to make sure it was working OK. It's in the directory exiv2\msvc\exiv2\{Release|Debug}\exiv2.exe. You can look in the build notes for the Mac to see the little tests I ran on on exiv2.exe. click here.

If you wish to build exiv2.lib to use the static C run time library, you should change the code generation switches in the project exiv2, exiv2lib and xmpsdk.

At this time there is no DLL version of exiv2.

6 scons

Scons is "Software Constructor" and is a tool to replace make. It's written in Python. You'll need this to build pyexiv2.

I downloaded the windows installer: scons-0.98.0.win32.exe and installed that. I believe this installed some python support code and the script c:\python25\scripts\scons.bat which is on my path.

In preparing these notes, I ran the installer again and got the message "No Python installation found in the Registry". I'm not going to investigate this further - I know scons was successfully installed on the machine a few months ago and is working OK. I hope you don't see the "No Python ... Registry" message!

7 pyexiv2

And now we're finally (almost ready) to build pyexiv2! I used pyexiv2 0.1.2 (February 2008): pyexiv2-0.1.2.tar.bz2 which I decompressed and expanded to c:\gnu\pyexiv2.

To modify the scons build, you have to modify the src/SConscript file which is read by scons. This is a Python program. Because SConscript is Python, it can read from the command line, the environment strings or anything else accessible to Python (cool, isn't it?). So SConscript sets up build flags and search paths based on both the platform and the environment. I like this a lot - it's very elegant.

Here's my version of c:\gnu\pyexiv2\src\SConscript click here. The windows specific code does the following:

1. Modifies the compiler options
2. Modifies the compiler include path
3. Modifies the library search path
4. Adds a post processing step to build the manifest into libpyexiv2.dll

#!/usr/bin/python
# -*- coding: utf-8 -*-

import sys
import os
import fnmatch

def locate(pattern, root=os.curdir):
  ''' Locate all files matching supplied filename pattern in and below supplied root directory. '''
  for path, dirs, files in os.walk(os.path.abspath(root)):
      for filename in fnmatch.filter(files, pattern):
          yield os.path.join(path, filename)
def locate_last(pattern, root=os.curdir):
  for file in locate(pattern, root):
      ret = file
  return ret

env = Environment()

# Figure out the library and link paths for the platform
# Include directories to look for 'Python.h' in
if sys.platform[:3] != "win":
  python_inc_path = os.path.join(sys.prefix, 'include', 'python' + sys.version[:3])
  env.Append(CPPPATH=[python_inc_path])
  env.Append(LIBPATH=['/usr/local/lib/'])

  boost_path = os.environ.get('BOOST_ROOT')
  if boost_path:
    env.Append(CPPPATH=[boost_path])

  libs = ['boost_python', 'exiv2']

else:
  # windows

  boost_path = os.environ.get('BOOST_ROOT')

  if os.environ.get('WindowsSdkDir') != None:
     winsdk_path = os.environ.get('WindowsSdkDir')
  else:
     winsdk_path = os.environ.get('VCINSTALLDIR')

  python_path = os.path.split(sys.executable)[0]
  python_ver = sys.version[0]+sys.version[2]

  exiv2_path = os.path.abspath(os.getcwd() + '\\..\\..\\exiv2')
  expat_path = os.path.abspath(os.getcwd() + '\\..\\..\\expat-2.0.1')

  python_inc_path = python_path + '\\include'
  exiv2_inc_path = exiv2_path + '\\msvc\\include'
  exiv2_src_path = exiv2_path + '\\src'
  expat_inc_path = expat_path + '\\include'
  boost_inc_path = boost_path

  env.Append(CPPPATH=[python_inc_path, exiv2_inc_path, exiv2_src_path,expat_inc_path, boost_inc_path])
  env.Append(CPPFLAGS='/EHsc /D_DLL')

  # Libraries to link against for win
  boost_lib = locate_last('boost_python*mt*.lib',boost_path +
      '\\bin.v2\\libs\\python\\build')
  expat_lib = expat_path + '\\win32\\bin\\release\\libexpat.lib'
  exiv2_lib = exiv2_path + '\\msvc\\exiv2lib\\Release\\exiv2.lib'
  python_lib = python_path +'\\libs\\python'+ python_ver +'.lib'
  uuid_lib = winsdk_path + '\\lib\\uuid.lib'
  kernel_lib = winsdk_path + '\\lib\\kernel32.lib'

  libs = [python_lib, exiv2_lib, boost_lib, expat_lib, '/NODEFAULTLIB:LIBCMT.lib']
  if os.environ.get('WindowsSdkDir') != None: # MSVC Express needs extra libs
     libs = libs + [uuid_lib, kernel_lib]

if sys.platform[:6] == 'darwin':
  # MacOS X
  # link the Python framework
  env['FRAMEWORKS'] += ['Python']

# Libraries to link against
env.Append(LIBS=libs)

# Build shared library libpyexiv2
cpp_sources = ['libpyexiv2.cpp', 'libpyexiv2_wrapper.cpp']
libpyexiv2 = env.SharedLibrary('libpyexiv2', cpp_sources)

# on windows, add a post-build step to embed the manifest using mt.exe
if sys.platform[:3] == "win":
  # The number at the end of the line indicates the file type (1: EXE; 2:DLL)
  env.AddPostAction(libpyexiv2,
      'cd build && "' + winsdk_path +
      '\\bin\\mt.exe" -nologo -manifest libpyexiv2.dll.manifest '
      '-outputresource:libpyexiv2.dll;#2'
      )

# Install the shared library and the Python module, invoked using
# 'scons install'. If DESTDIR is specified on the command line when invoking
# scons, it will be prepended to each installed target file. See
# http://www.gnu.org/prep/standards/html_node/DESTDIR.html for reference.
python_lib_path = os.path.join(sys.prefix,
 'lib', 'python' + sys.version[:3], 'site-packages')
dest_dir = ARGUMENTS.get('DESTDIR')
if (dest_dir is None) or (not os.path.isabs(dest_dir)):
  install_dir = python_lib_path
else:
  install_dir = os.path.join(dest_dir, python_lib_path[1:])
env.Install(install_dir, [libpyexiv2, 'pyexiv2.py'])
env.Alias('install', install_dir)

8 Installation and testing

At the end of the day, you'll have to install libpyexiv2.dll (renamed libpyexiv2.pyd) and pyexiv2.py, boost, and libexpat.dll into the site-packages of your python. These are:

04/01/2008  20:43           258,048 boost_python-vc80-mt-1_35.dll
04/02/2008  11:28           122,880 libexpat.dll
04/22/2008  22:38           860,160 libpyexiv2.pyd
02/18/2008  14:32            26,561 pyexiv2.py

Because you've built and installed libexiv2 and all the dependent libraries, you should now be able to run gps.py. Follow the instructions in the ReadMe.txt.

9 History of code and documentation

Most recent change at top of document (YYYYMMDD)

• 20080819 : Added cross-platform SConscript.
• 20080813 : Added vcprojects.zip file and directions for its use.
• 20080812 : Considerable revision.
• 20080811 : Initial version of notes.
• 20080422 : Binary code in archive for download.

This is my first open source code project. Very interesting. It seems that open source developers are very Linux oriented and the Windows build environment is treated with less concern. I hope you find it possible to follow all the steps. Your feedback is welcome.

I'm very happy to accept comments, feedback and suggestions for any of my articles. I'm always happy to hear you - especially if you have constructive suggestions. And I'm particularily pleased if you can let me know about corrections.