Mac dylib rpath framework bundle explanation

The purpose of this article is to teach you how to properly configure a dylib / framework on the mac and the application linking against it. If you don’t do this properly, you could be in for a long time wasting ordeal. In this article we will describe, in enough detail, how the linking process works in order to get things working and hopefully teach you enough to configure things on your own if you decide to do something in a non-standard way. We will also give you the tools and knowledge required in order to debug issues related to library loading, particularly with rpaths.

The subject
When you load an application, the dynamic linker (it’s actually an application on the mac called dyld I believe), will look at all of the references (symbols) in the application and try to resolve them (perhaps on load, perhaps not, who knows these days). In order to do this, it needs to know what libraries this application depends upon, where the libraries are on the hard drive, and also, what libraries those libraries depend upon so it can load them too, recursively. If the dynamic linker can’t find these libraries, it can’t load them, bottom line. You’ll end up with an error such as “bundle is corrupt, invalid, or missing resources” or some error like that.

The question
How do you tell the application where to find libraries to load? What if they’re system libraries? What if they’re custom libraries or third party libraries? What if you don’t want to install them to /usr/bin or some standard place? What do you do when a library from a third party (such as Intel), isn’t configured properly for loading? When you’re finished reading this article, you should be able to answer all of these questions.

How to configure your stuff
The things you need to set up are: the run path search path, the install directory, and the install name. There are two parts you need to configure properly, the application using the library/framework and the library/framework itself.

The runpath search path:
This is a setting within XCode that you set. You basically put one or more paths in here. The paths tell the dynamic linker, when it loads your application, where to load for dynamic libraries. Some people try to get away with using an environment variable called LD_LIBRARY_PATH or other environment variables. Do not use these. These are old and can cause issues. Using an environment variable basically means they didn’t configure their libraries properly. Normally you’ll see something like “@executable_path; @executable_path/../Frameworks”. @executable_path automatically expands to the path of the executable. If you are building a library, you would use @loader_path, which expands to the path of the library you’re building (at run time). This allows you to specify libraries relative to the library you’re building. For example, if you have a package of libraries that depend upon each other.

The install name:
Inside of a dylib, it has something called an “install name”. The install name is a path to the location of where the library is to be installed. This seems like a stupid idea at first, but I will explain further in the article. For system libraries, the install path might be something like this: /usr/bin/zlib.1.dylib. This is great for system libraries, but what about for libraries you create that are packaged inside of an app bundle or might be moved around to a location the user can specify? In this situation, the install name should be “@rpath/libName.dylib”.

The install directory:
The install directory is a setting within XCode, also a system path. I’m specifically talking about the setting that is to be set for the library. In this case, it should be “@rpath”. When you set this setting, the “install name” setting should say “@rpath/libName.dylib” automatically. If you set it to “$(DYLIB_INSTALL_NAME_BASE:standardizepath)/$(EXECUTABLE_PATH)” it should fill this field out for you and expand it to “@rpath/libName.dylib”.

How it works
When you build an application, the linking phase places symbols and references inside of the executable. The same goes when you build a library. On the mac, when you build an application/library and link against a dynamic library, it will also store the install name of the library it is linking against. This is good because now it knows where to find it. If the install name is /usr/bin/libz.1.dylib, then it knows the exact path to load that dylib. If the install path is @rpath/libName.dylib, then it tries each of the “runpath search paths” one by one and substitutes it in for “@rpath” until it finds the library it’s looking for. runpath search paths are specified when you build the executable or library(an xcode setting). So the runpath search path would expand to “@executable_path/libName.dylib”, which @executable_path then expands to wherever the executable is executing from. You should use @loader_path when you build a library, and @executable_path when building an executable. In both cases, it substitutes in the current path to that binary file.

An example
Let’s assume that “Application” is a binary object that contains the following pieces of data:
Application x86_64
runpath_search_paths: @executable_path, @executable_path/../Frameworks, @executable_path/../someplace_else
referenced libraries: "/usr/bin/blah.dylib"
void doSomething()
{
libraryFunction();
}

Let’s also assume that this dylib is a binary object that contains the following pieces of data:
Dylib x86_64
install_name: "/usr/bin/blah.dylib"
referenced libraries: "/usr/bin/blah2.dylib; /usr/bin/blah3.dylib"
void libraryFunction()
{
// Do something
libraryfunc2();
libraryfunc3();
}

This set up is very common, so we will describe the details more:
1) We have an application that links against /usr/bin/blah.dylib and it calls a function called “libraryFunction()”.
2) This tells us that, it’s going to search for the library blah.dylib inside of /usr/bin. If it doesn’t find it, it will fail to load. “referenced libraries” is filled out when the application is built from xcode. This is grabbed from the dylib it’s linked against (it’s install name).
3) Notice that /usr/bin/blah.dylib also is linked against two other libraries: /usr/bin/blah2.dylib and /usr/bin/blah3.dylib. This is important because if these libraries are not found, this library will fail to load and you may not know why. So it’s important that these libraries are also configured properly.

These details tell us a lot, so let’s look at real terminal commands that give us this information:

This will tell you if it’s built as a 64-bit x86 or whatever you want architecture.
file libname.dylib
Example:
Todds-MacBook-Pro:lib Todd$ file ./libmkl_intel_thread.dylib
./libmkl_intel_thread.dylib: Mach-O universal binary with 2 architectures
./libmkl_intel_thread.dylib (for architecture i386): Mach-O dynamically linked shared library i386
./libmkl_intel_thread.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64

This information is very useful. I’ve been in a few situations where I was linking against an architecture that was for 32-bit and not 64-bit.

This should display something like @rpath/libname or @rpath/blah.framework/Versions/A/blah”
otool -D libname.dylib

Example:
Todds-MacBook-Pro:lib Todd$ otool -D ./libmkl_intel_thread.dylib
./libmkl_intel_thread.dylib:
@rpath/libmkl_intel_thread.dylib

In this example, we see the install name is “@rpath/libmkl_intel_thread.dylib”. The second line.

This shows all of the libraries that this library is dependent on. If any of these are incorrect, loading will fail.
otool -L libname.dylib

Example:
Todds-MacBook-Pro:lib Todd$ otool -L ./libmkl_intel_thread.dylib
./libmkl_intel_thread.dylib:
@rpath/libmkl_intel_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libiomp5.dylib (compatibility version 5.0.0, current version 5.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

This shows that the library that “libmkl_intel_thread.dylib” has references to three libraries, one of which is a system library with an absolute path. This is just as expected. You will have to make sure that those other two libraries are also configured properly as well.

This shows you all of the rpaths. That’s a lower case “L”, not a number 1 or an “i”.
Run this on an executable or a library.
otool -l executableName

Example:
// ... Lots of other stuff ...
Load command 43
cmd LC_RPATH
cmdsize 80
path @executable_path/../Frameworks (offset 12)
// .. some more stuff ...

Towards the bottom of what’s displayed, you should see “LC_RPATH” entries. Each one is a runpath search path that was specified in the executable’s runpath search path entries.

Two other tools you can use to assist your adventure are: Mach-O-View and MacDependency. They are both useful, but MacDependency allows you to drill down into libraries that have references to other libraries, which allows you to see if one of them is not configured properly.

Reconfiguring libraries and Executables
Sometimes a shipped library isn’t configured properly and you need to reconfigure it. The things you’ll want to change are 1) it’s install name, 2) all the referenced libraries.

This changes the install name for a library.
install_name_tool -id "@rpath/libName.dylib" ./libName.dylib

If you change the install name for a library, and you have other libraries that reference that library, you’ll have to go through each library that was linked against the library you’ve changed and adjust the referenced install name, since you change it. They need to match. It might be best to create a script to do this.

This changes the referenced install name to use the new @rpath/libName.dylib install name. This is assuming that the old install name used to be “libName.dylib”, whereas the new install name is “@rpath/libName.dylib”.
install_name_tool -change libName.dylib @rpath/libName.dylib ./libThatUsesLibName.dylib

Windows comparison
Windows uses something called an “import library” that you link against if you want to use a dll. You can also load a dll at runtime but that’s less fun. So it doesn’t use rpaths or runpath search paths at all. Generally, it searches for libraries using the executable path, and if that fails, it uses the PATH environment variable to search for dlls.

References:
mikeash.com
dyld Documentation
dribin.org
wincent.com

Comments are closed.