The C and C++ developer tools at the HMS

Last updated on March 10, 1998

C and C++ compilers are a veritable mess on the HMS's machines right now, thanks to the different processor architectures and IRIX versions. I hope to shed some light on the issues related to different compiler and operating system versions in this document. Understanding the issues involved can help you to speed up your programs by 30% or even more!

Feedback on this document is very welcome. Please e-mail me at cvogler@gradient.cis.upenn.edu.
 

Table of contents

  1. Background: The different ABIs
  2. What to use: The bottom line
  3. Compiling and linking with the o32 ABI
  4. Compiling and linking with the n32 ABI
  5. Creating libraries
  6. Debugging
  7. C++-specific topics
    1. Template instantiation
    2. Creating class libraries
    3. Gotchas
  8. Other useful documentation

1. Background: The different ABIs

Originally, the MIPS architecture that powers the SGIs in the HMS was a 32 bit architecture. This changed with the advent of the MIPS R4xxx processor series, which is a full-fledged 64 bit processor. Most applications do not need the extra power that comes with 64 bit architectures as opposed to 32 bit, so there is rarely a reason for you to be concerned with the implications of 64 bit compiling and linking.

However, the changes in the MIPS R4xxx and subsequent processor revisions also carry some significant benefits for everyday 32 bit code. These are:

But there is a gotcha: The operating system and runtime libraries have to support these changes. Otherwise, the application cannot take advantage of the extended register sets. That is why there are now three different application binary interfaces (or ABI for short). They are: See man 5 abi for more information.

Unfortunately, the n32 and o32 ABIs are incompatible. It is impossible to link n32 object files with o32 object files (including libraries) and vice versa. For the standard libraries, this is usually not a problem, because we have both o32 and n32 versions for virtually every standard library. Even if the n32 version is not available, it is usually easy to obtain one within a short time.

Programs compiled and linked with the n32 ABI will work fine on all O2s, the Octane, and the Origin, but they will not work on our Indigo 2s and aladdin, which all still have IRIX 5.3 installed. At some point we will supposedly be upgrading every HMS machine to IRIX 6.4, so this problem will disappear in the future. But please don't quote me on that. :-)

But there is a worse problem: The source code for Transom Jack 1.x does not even compile with the n32 ABI. The reason is that some of Jack's code conflicts with the latest C++ language features that are activated under an n32 compile. This means that if your code is part of Jack, as of now, you are stuck with the o32 ABI with all the performance penalties that come with it. The new Jack toolkit, however, reportedly works fine with n32. Hopefully, it will not take too long before we get a new Jack version based on the new toolkit. Then this problem will also disappear.

All this leads to the bottom line:

2. What  to use: The bottom line

For C++ code there is another important benefit: The latest feature of the language, such as exceptions, namespaces, member templates, Run Time Type Information, and others are supported only if you use the n32 ABI. See the section on C++ for more info.

Unless your code conflicts with some of the new C++ language features, or you use Transom Jack 1.x, switching over to n32 is usually as simple as recompiling and relinking the code, with all the speedup. How is that for a free lunch?

Next follows how to tell the compiler and linker which ABI to use.

3. Compiling and linking with the o32 ABI

There is not much to say about this topic, because this is the default on every machine except graphics and buzz. On the latter two you can force both the compiler and the linker to use the o32 ABI by supplying the -32 command line switch.
You can invoke the C compiler and linker with cc, and the C++ compiler and linker with CC.

The compiler switch -O2 turns on compiler optimizations. The switch -mips2 uses the R4xxx instruction set, which is somewhat faster than the default, but will not work on any machines older than the Indigo2s. Note that -mips2 implies -32.
The switch -O3 optimizes better than -O2 and is safe to use, but it cannot be used generate object files, which makes it more or less useless for larger projects.

See man cc or man CC for more information.
 

4. Compiling and linking with the n32 ABI

Unless you compile on buzz or graphics, you have to force the compiler to use the n32 ABI. In addition, it has to be supported by the compiler (this is the case for the compiler on the O2s, the Octane, and the Origin). You can force compiling and linking with the n32 ABI by supplying the -n32 command line switch. That is, for C programs use cc -n32  in the place of simply cc. For C++ programs, use  CC -n32.

If the linker gives you problems, you might be able to solve them by setting this environment variable:
setenv LD_LIBRARYN32_PATH /usr/lib32:/lib32
If you see error messages from the linker in the likes of  "elf.c assertion failed," this is a sure sign that you mixed o32 and n32 object files and libraries. Make sure that all files have been compiled with the -n32 option.

Be careful when you strip your executables. By default, /pkg/bin/strip is executed, which works only with o32 executables. It will happily clobber n32 executables without comment. Instead, execute /bin/strip on the machine on which you linked the code.

The switch -O2 forces the maximum level of safe optimizations. In contrast, -O3 encompasses aggressive optimizations that may sacrifice floating point precision for the sake of performance. Unlike under the o32 ABI, -O3 can be used to generate object files.

The switch -mips3, which is the default for -n32, uses the R4xxx instruction set. The switch -mips4 uses the full R5xxx/R8xxx/R10xxx instruction set and should be faster on those machines than the default. If  your program needs to run only on the O2s, Octane, or Origin, it is a good idea to set it.

See man cc or man CC for more information.
 

5. Creating libraries

For creating a static library, use ar with the switches -cr. Be careful with n32 object files, because ar defaults to /pkg/bin/ar, which cannot handle n32 objects. Use /bin/ar instead. The n32 ar versions already encompass the functionality of ranlib, so it is not necessary to run it explicitly.

If you create a C library, this is all that you have to do after compiling (but not linking) the individual object files. If you create a C++ class library, there is some extra work necessary. See the section on C++ for details.

I have never created a shared library for IRIX, so I cannot explain the steps that are necessary. Anyone willing to step in? Then please contact me.
 

6. Debugging

The primary debugger for SGI applications is cvd, which comes with a graphical user interface. In order to debug programs with it, compile and link programs with the -g switch. Note that the -g switch turns off all compiler optimizations, so it does not make sense to specify both -g and -O2.

The older versions of cvd do not support n32 debugging. Only the machines that have the n32 compiler installed also support debugging n32 executables.

In addition, for debugging C++ programs it is a good idea to turn off inlining. See man CC for details.

There is also a text-only debugger available. Its name is dbx.
 

7. C++-specific topics

In general, you will make your life a lot easier by using the n32 ABI. Many features of the language are not supported under the o32 ABI. That said, the following sections explain a few things peculiar to the MIPSPro C++ environment.

7.1. Template instantiation

The MIPSPro C++ compiler does not instantiate templates at compile time. It merely creates a directory called ii_files/ under the current working  directory and uses it to keep track of the compile time options for each source file. Templates are instantiated in a special prelinking step that is executed right before the object files are linked. During this step, the C++ compiler makes a second pass over the relevant source files, using the options that previously have been stored in the ii_files/ directory. In fact, you might not see some syntax error messages until the prelinking step takes place because of this scheme.

Usually, the prelinking step is invoked automatically by the linker and requires no further action by the programmer. But watch out for Gotcha #3 below.

7.2. Creating a class library

Because of the MIPSPro compiler's template instantiation mechanism, creating a statically linked C++ library requires more work than creating an equivalent C library. The problem is that in creating a static library, the linker is not invoked at all. Instead, the object files are passed directly to ar. Hence, the prelinker is not invoked, and none of the template instantiations that are needed in the class library take place.

Sometimes this problem just results in a lot of linker warnings about redefined symbols when you link a program with this static class library. These warnings are usually harmless, and they are the result of the prelinker performing multiple instantiations. But what if you distribute the class library in object form and the source code is not available for the prelinker anymore? Then the linker will not be able to resolve references to the template instantiations. Oops.

The solution is calling the prelinker explicitly before calling ar on the object files. Here is how it works. Suppose that there are three source files, called a.cc, b.cc, and c.cc that are to be linked into a class library called libdemo.a. Furthermore, suppose that a.o, b.o, and c.o depend on symbols in the libraries library1, library2, ... libraryn. Then the step that combines the files a.o, b.o, and c.o into the file libdemo.a consists of these two commands:

Note that absolutely no symbols from library1 through libraryn are actually exported to the libdemo library. The only reason why they need to be specified is that the prelinker has to determine, whether any of the necessary template instantiations already exist in any of the other libraries. It will instantiate only those templates that do not already exist in other libraries. This method will prevent linker warnings about multiple symbol definitions.

In view of this mess, there are two silver linings on the cloud: First, creating a shared library seems to be actually easier than creating the static library, because the former requires calling the CC linker. Hence, it will do the prelinking work automatically. Second, in the MIPSPro C++ 7.2 release (currently not available on the HMS systems), the CC linker is able to create archives. Hence, the call to ar will be replaced by a call to CC, and again, it will take care of everything automatically.

7.3. Gotchas

Using C++ has some  gotchas on the HMS machines as of now. Most of them can be circumvented, but it is necessary to be aware of them, so as to avoid unpleasant surprises later on.

Gotcha #1: Different compiler versions

As C++ is still an evolving standard, it matters a lot with which compiler the code is generated. We currently have the MIPSPro 7.1, 7.0 and some older versions floating around on the HMS machines. (The current version distributed by SGI is 7.2.) In addition, most new C++ language features are supported with the n32 ABI only.
Version 7.1 is installed on the Octane, the Origin and almost all O2s. It supports many, but by far  not all of the features of the official C++ standard. However, to get some new functionality such as namespaces and member templates, you have to compile with the switch -experimental. In practice, I have found it to be reasonably safe to use, even though it is officially documented as experimental.

Version 7.0 is installed on bandit. It does not support the new features that come with the -experimental switch of 7.1, and it has more limited template support for the o32 ABI than 7.1. In particular, the STL will not compile with 7.0 for the o32 ABI.

The older compilers are installed on the Indigo2s, the Onyx, the Indys and the Indigos. They work fine for C programs, but unless there is some legacy code to support, it might not be a good idea to use them for C++.

It is possible to find out what version a machine is running with the command versions | grep C++.

Gotcha #2: Using the STL

An older version of the STL has been installed with the MIPSPro 7.x standard libraries, but it is badly out of date. SGI distributes a freely available thread-safe version (3.0) of the STL that mostly conforms to the standard. It is generally considered to be the most advanced STL available. It is available at http://www.sgi.com/Technology/STL/, or in /home2/cvogler/include/stl/.

Compiling the SGI  STL 3.0 with the 7.0 or 7.1 compilers for the n32 ABI is no problem. Compiling it for the o32 ABI  requires the 7.1 compiler. Furthermore, you cannot use buzz for compiling the STL for the o32 ABI, because of the way the linker is set up (more on that under the topic of template instantiation following next).

Gotcha #3: Template instantiation

Usually the prelinking step is executed automatically by the linker and is transparent to the programmer. However, this scheme sometimes fails after making changes to templates. Apparently, the prelinker gets confused as to which template instantiations are up to date and which ones have to be recompiled. In my experience, it has been safest to remove both the *.o files and the ii_files/ directory and recompiling all files after editing templates.
 The one situation when the prelinking step is not transparent comes up with statically linked class libraries. MIPSPro 7.2 is reported to have solved this problem, but for now it is necessary to perform an extra step manually.

When compiling o32 executables on buzz, watch out. The prelinker is set up incorrectly on this machine and always attempts to use the n32 ABI. The result is, of course, that the linker fails to generate the executable. Workaround: Compile on one of the O2s that have MIPSPro 7.1 installed.

Gotcha #4: Exceptions

Although exceptions are supported by the MIPSPro 7.x compilers for both the o32 and the n32 ABI, our IRIX 5.3 machines do not seem to have the necessary runtime library support for it. In other words, programs compiled with exception support enabled fail miserably on them at startup. But they do work on the 6.3 systems without problems, albeit slower than the equivalent n32 executables.

Exception handling for the n32 ABI seems generally to be working fine, except when used in conjunction with the pthreads library. There is a known bug in the MIPSPro 7.1 runtime system that causes a multithreaded program to crash if it throws an exception. Until the compilers are upgraded to 7.2, there are four possible workarounds, although they are less than satisfactory:

8. Other useful documentation

Send feedback to Christian Vogler - 03/10/98