Version 6 (modified by martin, 7 years ago)

--

Summary

This page describes the intricate steps necessary to achieve a monolithic statically linked (no DLLs) OSG-based executable on the Win32 platform. Various aspects of the Win32 development environment make this more complicated than it might first seem, and certainly more complex than on *nix platforms. Some of these steps may be obsoleted by future integration of this process into the core OSG project.

Overview:

  1. Recompile necessary third-party libraries to eliminate DLL dependencies and ensure proper threading/debug configuration.
  2. Compile OpenThreads as a static library, using non-DLL standard libraries.
  3. Compile Producer as a static library, using non-DLL standard libraries.
  4. Compile core OSG as static libraries, using non-DLL standard libraries.
  5. Compile all required osg plugins as static libraries, using non-DLL standard libraries.
  6. Determine C++ mangled names of registration function in each required plugin.
  7. Compile your application, to use OSG staticly (#define OSG_LIBRARY_STATIC).
  8. Setup linker force symbol includes for plugins using C++ mangled names from above (to defeat /OPT:REF which would ignore/discard them).
  9. Link your application with all of the libs generated above.
  10. Profit!

1. Recompile necessary third-party libraries to eliminate DLL dependencies and ensure proper threading/debug configuration.

Many of the libraries used by various OSG plugins and supplied in pre-compiled form in the 3rdparty archive are compiled to use the DLL-based C standard library (MSVCR *.DLL). Most Windows systems have the earlier runtime DLLs already installed, but the VC++ 7.x runtime DLLs (MSVCR71.DLL) are not pre-installed and you application will not run without them if you link to libraries that require them. I know of no way to determine what runtime a .lib is compiled for, so I simply recompiled ALL third-party libs I knew I would need for my plugins. This also gave me the opportunity to ensure I had the latest versions of the libraries like zlib that have had buffer-overrun exploits discovered in the past. Finally, I built both debug and release versions of these libraries to make linking cleaner and debugging easier. I also made sure to compile these libs with the multithreading-safe runtime.

My archive of pre-compiled static release and debug 3rdparty libs for ZLIB, PNG, JPEG, TIFF, UnGIF and FreeType? under MSVC++7.1/2003 is here: Attach:3rdparty-static.zip (2.6Mb). Thanks to Mike Weiblen for helping me build these from his package of OSG thirdparty libs.

If you need more 3rdparty support libs, you'll need to compile them yourself.

2. Compile OpenThreads as a static library, using non-DLL standard libraries.

  • Select the project configuration Release Static and build it.
  • You will probably want to build Debug Static configuration too, so that you can link to those libraries when debugging.

3. Compile Producer as a static library, using non-DLL standard libraries.

  • Select the project configuration Release Static and build it.
  • You will probably want to build Debug Static configuration too, so that you can link to those libraries when debugging.

4. Compile core OSG as static libraries, using non-DLL standard libraries.

  • Select the solution file Solution OpenSceneGraph in the Solution Explorer of your Visual Studio. Save it as OpenSceneGraph_s.sln, so that the following changes affect to this new solution file, and OpenSceneGraph.sln remains intact for optionally building OSG DLLs.
  • Select the project configuration Release Static.
  • You have to remove the dependencies of each "Core" project (named Core * * * * *), which are only needed when building OSG DLLs. For example, the project Core osgGA depends on Core osg and Core osgUtil. These dependencies would make the linker include code from osg_s.lib and osgUtil_s.lib into osgGA_s.lib when needed, making it unnecessarily bigger and repeating symbols among libraries. To avoid it, you have to remove those dependencies for each core project.
  • Build the core projects but for Core osgIntrospection.
  • You will probably want repeat the build for Debug Static configuration too.

5. Compile all required osg plugins as static libraries, using non-DLL standard libraries.

  • osgText requires the freetype plugin.
  • Most 3D object loaders (3DS, LWO, etc) will require some of the image loaders (JPEG, PNG, etc)
  • Create the plugins you need following the steps below:
    • Remove the plugin project dependencies on the core projects (same explanation that Step 4).
    • Build the Release Static configuration of the plugins projects you need (and the Debug Static if you need it).
    • Even though some plugins require linking with another static libraries, don't bother adding it to the link step here, just add it to your application's own link phase (Step 9, below).

6. Determine C++ mangled names of registration function in each required plugin.

There are two main goals one might want from static monolithic linking. One is ease of distribution (single file, no DLL versioning issues). The other is a reduction in total distribution size (from not shipping an entire DLL full of mostly-unused code just because you need one function). To achieve the second goal, we must instruct the linker to use function-level linking instead of object-level linking. In object-level linking, the entire contents of a .lib are included into the output EXE, even if only one item was referenced from the calling application code. In function-level linking, the linker only pulls in the methods/functions/objects/data that are actually referenced by the calling application. It must recursively resolve references to items not referred to by the main caller, but are referenced by code the main application called. Under MSVC++, the linker option for function-level linking is /OPT:REF (can be visually changed through your project's Properties/Linker/Optimization/References).

Using function-level linking instead of object level linking, significant size savings are possible. My application went from 4+Mb of EXE (692k) and DLLs (3.45Mb) to a 2.2Mb EXE. The EXE got bigger by incorporating the necessary contents of the DLLs, but the overall package got much smaller because large portions of the DLLs weren't used/referenced, and were omitted by the linker.

Unfortunately, this technique is at odds with the clever automatic-registration technique OSG uses to register available plugins. This technique relies on the DLL instantiating all global objects within the plugin at the moment the plugin is forcibly loaded into memory by the osgDB code. Other than the call to load the DLL, the osgDB never directly refers to any data/code members of the DLL. As a result, when you compile the plugins as static libraries, you eliminate the explicit DLL load action. As well, since there are no direct references from osgDB (or your main application) to any data or code members of the plugin, the linker cleverly decides it is not necessary and omits all of the plugin from your final EXE= Not what you want.

The first step in solving this problem is to determine the name of the specific "registration object" that each plugin normally would rely on executing at DLL load time. We must determine the C++ mangled name for this registration object, in order to tell the linker to forcibly include it (and all everything it refers to, recursively) in step 8. There are various ways to do this. The easiest one that I know of is to use the dumpbin tool supplied by Microsoft (per Ben Crossman's osgusers message of Oct 11th, 2004):

dumpbin /linkermember:1 libraryname.lib

The dumpbin tool can usually be found in the VC/bin directory of your Visual Studio installation, though it will probably crash if you haven't run previously the batch file vcvars32.bat in that same directory.

This will spew out a listing of the members within the .lib. You're looking for a particular one. It will be a permutation of the name of the registration object as is it found in the plugin source. For example, in ReaderWriter?3DS.cpp, you will see a line like this:

osgDB::RegisterReaderWriterProxy?<ReaderWriter?3DS> g_readerWriter_3DS_Proxy;

The name of the object is g_readerWriter_3DS_Proxy. It is a template of the osgDB::RegisterReaderWriterProxy class. Different plugins use different naming conventions for this object. To locate it for a given plugin, search through the plugin source for usage of the RegisterReaderWriterProxy template (beware: might or might not be prefixed with osgDB::=). Find the actual object name (in this case g_readerWriter_3DS_Proxy). Now, search through the output of the dumpbin command for a symbol that is a permutation of g_readerWriter_3DS_Proxy:

dumpbin /linkermember:1 osgdb_3dsstatic.lib | find "g_readerWriter_3DS_Proxy"

This should produce some output that looks like:

?g_readerWriter_3DS_Proxy@@3V?$RegisterReaderWriterProxy?@VReaderWriter3DS@@@osgDB@@A

For the other plugins I compiled, here are the symbols:

?g_readerWriter_FreeType_Proxy@@3V?$RegisterReaderWriterProxy?@VReaderWriterFreeType@@@osgDB@@A
?g_readerWriter_PNG_Proxy@@3V?$RegisterReaderWriterProxy?@VReaderWriterPNG@@@osgDB@@A
?g_readerWriter_JPEG_Proxy@@3V?$RegisterReaderWriterProxy?@VReaderWriterJPEG@@@osgDB@@A
?g_lwoReaderWriterProxy@@3V?$RegisterReaderWriterProxy?@VReaderWriterLWO@@@osgDB@@A

As far as I can tell, name mangling is identlical between debug and release builds, but may be different between different major versions of the compiler. These are from MSVC++7/2003.

Note these somewhere (copy & paste, don't try to transcribe that madness=) because we'll need them in step 8, below.

7. Compile your application, to use OSG staticly (#define OSG_LIBRARY_STATIC).

  • Make new debug and release static targets for your application, ensuring you use the proper Multithread, non-DLL debug/release MS standard libs. Ensure RTTI is on.
  • All targets that will use OSG/OP/OT statically should define OSG_LIBRARY_STATIC, OT_LIBRARY_STATIC and PR_LIBRARY_STATIC globally in the project settings. This tells the OSG header files to drop the declspec(dllimport) qualifier on all OSG/OP/OT library function prototypes.
  • Compile= (Linking is done in step 9, below...)

8. Setup linker force symbol includes for plugins using C++ mangled names from above (to defeat /OPT:REF which would ignore/discard them).

We now need to list all those registration-object symbol names for the linker to forcibly bind them in, even though it doesn't think they're referenced or used from anywhere.

  • In VC++7, this is in the Project properties, Linker section, Input subsection. Add each mangled symbol name (space delimited) on the line next to Force Symbol References. Alternately, open the multiline editor with the ... button on the right and enter each on its own line.
  • Ben Crossman notes that you can also accomplish this by adding a #pragma line to one of your application's main source modules: #pragma comment(linker, "/include:?g_readerWriter_3DS_Proxy`3V?$RegisterReaderWriterProxy@VReaderWriter3DS`@osgDB`A")
  • Add Force Symbol References for the registration object for every plugin you intend to use. If you do not, no errors will occur during compiling or linking, but at runtime your plugin will not be found and your file(s) will not load.

9. Link your application with all of the libs generated above.

  • Add the proper static OpenThreads (step 2) and Producer (step 3) .libs to your build, on the Linker section, Input subsection, Additional Dependencies line.
  • Add all of the core OSG static libs (step 4) you need. It probably is safe to just add them all and let the /OPT:REF option ignore the ones you don't need, but this probably will increase your link time and increase the likelihood that the linker will do something dumb you don't intend. So, be selective, only add .libs that you know you need. If you don't know, try leaving it out and see if the linker complains.
  • Remember that ProducerStatic.lib and osgProducerstatic.lib are two different libs, and if you're using Producer, you need them both.
  • You will probably need to add opengl32.lib and glu32.lib, as they are not linked in the OSG libs.
  • Add all of the plugin .libs (step 5) you require.
  • Add all of the static libs your plugins might require (step 1).
  • Make sure you use the debug .lib for debug targets and the release .lib for relase targets, or the compiler will be angry.
  • Link=

10. Profit!

Enjoy your compact easy-to-distribute and update OSG application. Revel in your power.

Notes:

If things don't work at this stage, study the error message(s) you get carefully. This is a complex process involving LOTS of files and settings. It is very easy to miss or botch a step. It does work if everything is done right. Go back and check all your steps. Get rid of compiled objects and recompile everything from scratch. It is important that the changes to OT, OP and OSG be done in that order. If you build OSG before making the OP or OT changes, you'll have to go back and rebuild OSG once the changes are made, as OSG and OP each include headers from their dependent libraries further up the tree.

You cannot mix DLL plugins and static plugins. If you're using static linking, ALL of your plugins must be statically/monolithically linked into the EXE. I could try to explain why this is, but it would probably be a longer document than this entire how-to.

Attachments