[an error occurred while processing this directive]

Building pyexiv2 on Mac OS X

Version: 19 August 2008.

I was hoping to be able to build universal binaries on my iMac. However I eventually gave that up and built the native versions only. I build for intel on my iMac and for ppc on my elderly PowerBook. In the build script (click here), I used lipo to combine the two flavors into a universal binary:

set t=libexiv2.2.1.0.dylib && lipo -arch ppc ../ppc/$t -arch i386 ../intel/$t -create -output $t

You'll have to build and install the following:

  1. Active Python http://www.activestate.com/Products/activepython
    or build your own python: http://www.python.org/download/
  2. bjam http://www.boost.org/
  3. boost http://www.boost.org/
  4. libexiv2 and dependent libraries http://www.exiv2.org/
    GNU open source at sourceforge: http://sourceforge.net/
  5. scons http://www.scons.org/
  6. pyexiv2 http://tilloy.net/dev/pyexiv2/
  7. Installation and Testing

I did the job in about 4 hours (on ppc, and another 4 on intel), including all the dependancies and stuff. So it's an afternoon's work. Hopefully with these notes, it'll be 30 minutes for you. And of course if you already have boost installed and built - that's the biggest part of the job. I also suspect it'd be even easier if I used DarwinPorts. I live and learn.

The biggest time waster however in this process was to try and build the universal binaries on my iMac. I ended up frustrated by that. 2 days wasted!

And of course writing these notes has taken about 4 hours. I hope you find them useful.

1 Active Python (or build your own python)

You'll have to install python
I couldn't work with the version of python installed by Apple in /usr/bin/python. I think it's basically too old - especially on Tiger. So I obtained Active Python from www.activestate.com and used that. I also changed /usr/bin/python to be a link to Active Python:

148 /usr/bin> ls -alt python
lrwxr-xr-x  1 root  wheel  21 Mar 31 20:14 python -> /usr/local/bin/python
149 /usr/bin>

Alternatively you can download, build and install python 2.5 from source from sourceforge.net.

2 Build or install bjam

Before you can build the boost-python libraries, you'll need to download/install bjam. This is the 'make' utility of boost. I found a prebuilt version on the net, however you can also download and build it from sourceforge.net

And now we're into chickens and eggs, right? We can't build bjam with bjam. Can we build it with ./configure && make ? NO! build.sh (or build.bat on Windows) is our saviour.

181 /Users/rmills/gnu/boost-jam-3.1.16> build.sh
###
### Using 'darwin' toolset.
###
rm -rf bootstrap
mkdir bootstrap
cc -o ... stuff deleted ...
... stuff stuff stuff ...
...updated 1 target...
182 /Users/rmills/gnu/boost-jam-3.1.16> find . -name bjam
./bin.macosxx86/bjam
183 /Users/rmills/gnu/boost-jam-3.1.16> sudo cp ./bin.macosxx86/bjam /usr/local/bin

Alternatively, you simply sudo cp bjam /usr/local/bin if you've downloaded bjam alread built.

3 Install boost libraries

I installed boost into BOOST_ROOT=/usr/local/boost_1_34_1/ (I used 1_35_0 on the ppc). You'll need to set an environment string permanently in ~/.MacOSX/environment.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList.dtd">
<plist version="0.9">
<dict>
        <key>BOOST_ROOT</key>
        <string>/usr/local/boost_1_34_1/</string>
        <key>CPLUS_INCLUDE_PATH</key>
        <string>/usr/local/boost_1_34_1/</string>
</dict>
</plist>

After you've installed boost, and modified environment.plist, you should restart your computer (there's probably something easier/quicker command line way to to this, such as sudo killall -s HUP WindowServer, however I found it simpler to just reboot.

After the reboot, test your installation of boost. Build and run this little test program hello.cpp: click here.

115 /Users/rmills/Projects/HelloBoost> cat hello.cpp
#include <boost/lambda/lambda.hpp>
#include <iostream>
#include <iterator>
#include <algorithm>

int main()
{
    using namespace boost::lambda;
    typedef std::istream_iterator<int> in;

    std::for_each(
        in(std::cin), in(), std::cout << (_1 * 3) << " " );
}
117 /Users/rmills/Projects/HelloBoost> make -B hello
g++     hello.cpp   -o hello
118 /Users/rmills/Projects/HelloBoost> ./hello
2
6 4
12 q
119 /Users/rmills/Projects/HelloBoost> 

Now you're ready to build the boost-python libraries. Remarkably, the configure script has Windows line-endings. I had to change this to UNIX line endings (with TextWrangler) to be able to run ./configure

cd /usr/local/boost_1_34_1
./configure
make
.... rattle and roll
You can go for lunch - even on a fast machine this can take about an hour.

You should be able to build universal binaries with bjam.

./configure CFLAGS="-arch ppc -arch i386" LDFLAGS="-arch ppc -arch i386"
bjam --user-config=user-config.jam --toolset=darwin release

Although I was able to build universal boost-python libraries, I had problems with building other libraries (libexiv2 or one of its dependences), so I gave up and built the native platform.

Now copy the boost-python libraries to /usr/local/lib.

sudo cp /usr/local/boost_1_34_1/bin.v2/libs/python/ \
  build/darwin/release/link-static/threading-multi/libboost_python-mt-1_34_1.a \
  /usr/local/lib/libboost_python.a

4 libexiv2 and dependent libraries

libexiv2 has dependancies on other open source projects. You can use darwin-ports to download, build and install everything. Personally, I have a directory ~/gnu which I use to hold opensource projects.

102 /Users/rmills/gnu> ls -alt
drwxr-xr-x@  33 rmills  staff   1122 Apr 14 17:51 gettext-0.17
drwxr-xr-x@  28 rmills  staff    952 Apr 14 17:20 expat-2.0.1
drwxr-xr-x   17 rmills  staff    578 Apr  9 20:32 exiv2-0.16
drwxr-xr-x@ 111 rmills  staff   3774 Apr  6 15:14 gawk-3.1.6
drwxr-xr-x@  71 rmills  staff   2414 Apr  4 12:11 libusb-0.1.12
drwxr-xr-x@ 339 rmills  staff  11526 Apr  4 11:43 gpsbabel-1.3.4
drwxr-xr-x@  12 rmills  staff    408 Mar 28 20:26 pyexiv2
drwxr-xr-x@  17 rmills  staff    578 Mar 28 17:51 scons-0.97
drwxr-xr-x@ 105 rmills  staff   3570 Mar 28 16:39 boost-jam-3.1.16
drwxr-xr-x@   5 rmills  staff    170 Mar  4 21:05 wxjs
drwxr-xr-x  158 rmills  staff   5372 Feb 11 15:17 nmap-4.53
drwxr-xr-x   50 rmills  staff   1700 Feb  2 15:58 curlftpfs-0.9.1
drwxr-xr-x   60 rmills  staff   2040 Feb  2 15:42 pkg-config-0.23
drwxr-xr-x   77 rmills  staff   2618 Feb  2 15:40 glib-2.0.7
drwxr-xr-x   40 rmills  staff   1360 Feb  2 15:27 curl-7.18.0
drwxr-xr-x   64 rmills  staff   2176 Feb  2 15:15 openssl-0.9.8e
drwxr-xr-x   38 rmills  staff   1292 Feb  2 15:06 libssh2-0.17
drwxr-xr-x    6 rmills  staff    204 Sep 15  2007 ImageMagick-6.3.5
103 /Users/rmills/gnu> 

So, when I download a new library from sourceforge.net, I unzip the directory and add it to ~/gnu. Then I use the conventional build mechanism

cd ~/gnu/expat-2.0.1
make clean
./configure
make
sudo make install

This usually works fine.

To build libexiv2 correctly on PPC on Tiger, I had to use the commands:
configure MACOSX_DEPLOYMENT_TARGET=10.4
make MACOSX_DEPLOYMENT_TARGET=10.4
sudo make install

When I didn't, I had linking errors - doubly defined entry points from the c++ libraries.

You can build universal versions for many libraries using make 'CXXFLAGS=-arch ppc -arch i386' 'CFLAGS=-arch ppc -arch i386' 'LDFLAGS=-arch ppc -arch i386'. For example, the wonderful expat library builds quickly and easily this way.

200 /Users/rmills/gnu/expat-2.0.1> make clean
... stuff deleted ...
201 /Users/rmills/gnu/expat-2.0.1> make ...
... 'CXXFLAGS=-arch ppc -arch i386' 'CFLAGS=-arch ppc -arch i386' 'LDFLAGS=-arch ppc -arch i386'
/bin/sh ./libtool --silent ...  -c lib/xmlparse.c
... stuff deleted ...
202 /Users/rmills/gnu/expat-2.0.1> find . -name "*.dylib" -exec lipo -info {} ";"
Architectures in the fat file: ./.libs/libexpat.1.5.2.dylib are: ppc7400 i386 
Architectures in the fat file: ./.libs/libexpat.1.dylib are: ppc7400 i386 
Architectures in the fat file: ./.libs/libexpat.dylib are: ppc7400 i386 
203 /Users/rmills/gnu/expat-2.0.1> 

Sadly, I haven't found this to be totally reliable. You can end up with linking errors and all sorts of stuff. If you forget about CXXFLAGS and use the conventional mechanism, you'll end up with a build for your machine (i386 or ppc).

In the case of libexiv2, you have dependances on expat and gettext. So you should build and install them before building libexiv2

  When you build and installed libexiv2, you also build a command line program exiv2 which can be used to read and write exif data in an image file. It's worth testing this before trying to build pyexiv2.

30 /Users/rmills/gnu/exiv2-0.16> exiv2 -pt Diana.jpg
Exif.Image.ImageDescription                  Ascii      32
Exif.Image.Make                              Ascii       5  SONY
... bla bla ...
Exif.Thumbnail.DateTime                      Ascii      20  2008:01:11 11:31:02
Exif.Thumbnail.JPEGInterchangeFormat         Long        1  0
Exif.Thumbnail.JPEGInterchangeFormatLength   Long        1  16035

31 /Users/rmills/gnu/exiv2-0.16> cat zz.txt
# add Exif.Image.GPSTag                            290 
add Exif.GPSInfo.GPSVersionID                    2 2 0 0 
add Exif.GPSInfo.GPSLatitudeRef                  'N' 
add Exif.GPSInfo.GPSLatitude                     37/1 15/1 57/1 
add Exif.GPSInfo.GPSLongitudeRef                 'W'
add Exif.GPSInfo.GPSLongitude                    121/1 58/1 5/1
add Exif.GPSInfo.GPSAltitudeRef                  0
add Exif.GPSInfo.GPSAltitude                     78/1
add Exif.GPSInfo.GPSTimeStamp                    22/1 29/1 14/1
add Exif.GPSInfo.GPSMapDatum                     Ascii      'WGS-84'
add Exif.GPSInfo.GPSProcessingMethod             Undefined  65 83
add Exif.GPSInfo.GPSDateStamp                    2003:05:24

32 /Users/rmills/gnu/exiv2-0.16> exiv2 -m zz.txt Diana.jpg

33 /Users/rmills/gnu/exiv2-0.16> exiv2 -pt Diana.jpg | grep GPS
Exif.Image.GPSTag                            Long        1  2533
Exif.GPSInfo.GPSVersionID                    Byte        4  2 2 0 0 
Exif.GPSInfo.GPSLatitudeRef                  Ascii       2  North
Exif.GPSInfo.GPSLatitude                     Rational    3  37deg 15' 57" 
Exif.GPSInfo.GPSLongitudeRef                 Ascii       2  West
Exif.GPSInfo.GPSLongitude                    Rational    3  121deg 58' 5" 
Exif.GPSInfo.GPSAltitudeRef                  Byte        1  Above sea level
Exif.GPSInfo.GPSAltitude                     Rational    1  78 m
Exif.GPSInfo.GPSTimeStamp                    Rational    3  22:29:14
Exif.GPSInfo.GPSMapDatum                     Ascii       7  WGS-84
Exif.GPSInfo.GPSProcessingMethod             Undefined  18  65 83
Exif.GPSInfo.GPSDateStamp                    Ascii      11  2003:05:24

34 /Users/rmills/gnu/exiv2-0.16>

5 scons

Scons is "Software Constructor" and is a tool to replace make. It's written in Python. You'll have to download, build and install that.

To modify the scons build, you have to modify the src/SConscript file which is read by scons. This is a Python program. Because SConscript is Python, it can read from the command line, the environment strings or anything else accessible to Python (cool, isn't it?). So SConscript sets up build flags and search paths based on both the platform and the environment. I like this a lot - it's very elegant.

6 pyexiv2

And finally we're ready to build pyexiv2.

I modified pyexiv2/src/SConscript. You'll see the 'darwin' specific code adds Python to the linked Frameworks.

To build pyexiv2, simply type scons:

138 /Users/rmills/gnu/pyexiv2> scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o build/libpyexiv2.os ... stuff deleted ...
g++ -o build/libpyexiv2_wrapper.os ... stuff deleted ...
g++ -o build/libpyexiv2.dylib -dynamiclib ... stuff deleted ...
scons: done building targets.
139 /Users/rmills/gnu/pyexiv2> find . -name "*.dylib" -exec ls -alt {} ";"
-rwxr-xr-x  1 rmills  staff  524176 Apr 16 16:39 ./build/libpyexiv2.dylib
140 /Users/rmills/gnu/pyexiv2> 

I believe you can also do sudo scons install, however that's not quite correct for the Mac. The shared library which has been built is in build/libpyexiv2.dylib. This should be libexiv2.so

Here's the code in pyexiv2/src/SConscript: click here. I would particularily like to acknowledge the help of Rob Wallace in New Zealand in the development Windows part of this file. Thanks, Rob!

#!/usr/bin/python
# -*- coding: utf-8 -*-

import sys
import os
import fnmatch

def locate(pattern, root=os.curdir):
  ''' Locate all files matching supplied filename pattern in and below supplied root directory. '''
  for path, dirs, files in os.walk(os.path.abspath(root)):
      for filename in fnmatch.filter(files, pattern):
          yield os.path.join(path, filename)
def locate_last(pattern, root=os.curdir):
  for file in locate(pattern, root):
      ret = file
  return ret

env = Environment()

# Figure out the library and link paths for the platform
# Include directories to look for 'Python.h' in
if sys.platform[:3] != "win":
  python_inc_path = os.path.join(sys.prefix, 'include', 'python' + sys.version[:3])
  env.Append(CPPPATH=[python_inc_path])
  env.Append(LIBPATH=['/usr/local/lib/'])

  boost_path = os.environ.get('BOOST_ROOT')
  if boost_path:
    env.Append(CPPPATH=[boost_path])

  libs = ['boost_python', 'exiv2']

else:
  # windows

  boost_path = os.environ.get('BOOST_ROOT')

  if os.environ.get('WindowsSdkDir') != None:
     winsdk_path = os.environ.get('WindowsSdkDir')
  else:
     winsdk_path = os.environ.get('VCINSTALLDIR')

  python_path = os.path.split(sys.executable)[0]
  python_ver = sys.version[0]+sys.version[2]

  exiv2_path = os.path.abspath(os.getcwd() + '\\..\\..\\exiv2')
  expat_path = os.path.abspath(os.getcwd() + '\\..\\..\\expat-2.0.1')

  python_inc_path = python_path + '\\include'
  exiv2_inc_path = exiv2_path + '\\msvc\\include'
  exiv2_src_path = exiv2_path + '\\src'
  expat_inc_path = expat_path + '\\include'
  boost_inc_path = boost_path

  env.Append(CPPPATH=[python_inc_path, exiv2_inc_path, exiv2_src_path,expat_inc_path, boost_inc_path])
  env.Append(CPPFLAGS='/EHsc /D_DLL')

  # Libraries to link against for win
  boost_lib = locate_last('boost_python*mt*.lib',boost_path +
      '\\bin.v2\\libs\\python\\build')
  expat_lib = expat_path + '\\win32\\bin\\release\\libexpat.lib'
  exiv2_lib = exiv2_path + '\\msvc\\exiv2lib\\Release\\exiv2.lib'
  python_lib = python_path +'\\libs\\python'+ python_ver +'.lib'
  uuid_lib = winsdk_path + '\\lib\\uuid.lib'
  kernel_lib = winsdk_path + '\\lib\\kernel32.lib'

  libs = [python_lib, exiv2_lib, boost_lib, expat_lib, '/NODEFAULTLIB:LIBCMT.lib']
  if os.environ.get('WindowsSdkDir') != None: # MSVC Express needs extra libs
     libs = libs + [uuid_lib, kernel_lib]

if sys.platform[:6] == 'darwin':
  # MacOS X
  # link the Python framework
  env['FRAMEWORKS'] += ['Python']

# Libraries to link against
env.Append(LIBS=libs)

# Build shared library libpyexiv2
cpp_sources = ['libpyexiv2.cpp', 'libpyexiv2_wrapper.cpp']
libpyexiv2 = env.SharedLibrary('libpyexiv2', cpp_sources)

# on windows, add a post-build step to embed the manifest using mt.exe
if sys.platform[:3] == "win":
  # The number at the end of the line indicates the file type (1: EXE; 2:DLL)
  env.AddPostAction(libpyexiv2,
      'cd build && "' + winsdk_path +
      '\\bin\\mt.exe" -nologo -manifest libpyexiv2.dll.manifest '
      '-outputresource:libpyexiv2.dll;#2'
      )

# Install the shared library and the Python module, invoked using
# 'scons install'. If DESTDIR is specified on the command line when invoking
# scons, it will be prepended to each installed target file. See
# http://www.gnu.org/prep/standards/html_node/DESTDIR.html for reference.
python_lib_path = os.path.join(sys.prefix,
 'lib', 'python' + sys.version[:3], 'site-packages')
dest_dir = ARGUMENTS.get('DESTDIR')
if (dest_dir is None) or (not os.path.isabs(dest_dir)):
  install_dir = python_lib_path
else:
  install_dir = os.path.join(dest_dir, python_lib_path[1:])
env.Install(install_dir, [libpyexiv2, 'pyexiv2.py'])
env.Alias('install', install_dir)

7 Installation and testing

At the end of the day, you'll have to install libpyexiv2.so and pyexiv2.py into the site-packages of your python. This is:

sudo cp build/libpyexiv2.dylib \
     /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/libpyexiv2.so
sudo cp src/pyexiv2.py \
     /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pyexiv2.py

Because you've built and installed libexiv2 and all the dependent libraries, you should now be able to run gps.py. Follow the instructions in the ReadMe.txt.

Comments?

I'm very happy to accept comments, feedback and suggestions for any of my articles. I'm always happy to hear you - especially if you have constructive suggestions. And I'm particularily pleased if you can let me know about corrections.


Home ......... About

Page design © 1996-2008 Robin Mills / webmaster@clanmills.com

Updated Tuesday August 19, 2008