template hell!

topic posted Sat, July 15, 2006 - 4:53 AM by 
Share/Save/Bookmark
Advertisement

Hello, all ...

I've been doing templates for years (either using others' templates or making my own), but this is the first time I need to make a library that implements a template class. Here is my sample:

template1.h

// class templates
#include <iostream>

template < class T > class pair
{
private:
T a, b;
public:
pair (T first, T second);
T getmax ();
};

template_total.cpp

// class templates
#include <iostream>
#include "templates1.h"

template < class T > pair< T >::pair( T first, T second )
{
a=first; b=second;
}

template < class T > T pair< T >::getmax ()
{
T retval;
retval = a>b? a : b;
return retval;
}

int main ()
{
pair <int> myobject ( 100, 75 );
std::cout << myobject.getmax() << std::endl;
return 0;
}

... now when I compile this with gcc4.0.1 (on Mac OSX 10.4.7), this is what I get:

PugsleyButt:~/devstuff/c++/templates jmzorko$ c++ templates_total.cpp -o templates_total
PugsleyButt:~/devstuff/c++/templates jmzorko$ ./templates_total
100

... which is correct. Now I want to make a library with the template class and implementation, and use that library:

PugsleyButt:~/devstuff/c++/templates jmzorko$ cat templates1.h
// class templates
#include <iostream>

template < class T > class pair
{
private:
T a, b;
public:
pair (T first, T second);
T getmax ();
};

PugsleyButt:~/devstuff/c++/templates jmzorko$ cat templates1lib.cpp
// class templates
#include <iostream>
#include "templates1.h"

template < class T > pair< T >::pair( T first, T second )
{
a=first; b=second;
}

template < class T > T pair< T >::getmax ()
{
T retval;
retval = a>b? a : b;
return retval;
}

PugsleyButt:~/devstuff/c++/templates jmzorko$ c++ -c templates1lib.cpp -o templates1lib.o
PugsleyButt:~/devstuff/c++/templates jmzorko$ ar rcs libtemplates1.a templates1lib.o
PugsleyButt:~/devstuff/c++/templates jmzorko$ ll
total 44
0 drwxr-xr-x 9 jmzorko jmzorko 306 Jul 15 04:40 .
0 drwxr-xr-x 48 jmzorko jmzorko 1632 Jul 15 04:29 ..
4 -rw-r--r-- 1 jmzorko jmzorko 2192 Jul 15 04:40 libtemplates1.a
4 -rw-r--r-- 1 jmzorko jmzorko 156 Jul 15 04:11 templates1.cpp
4 -rw-r--r-- 1 jmzorko jmzorko 149 Jul 14 17:51 templates1.h
4 -rw-r--r-- 1 jmzorko jmzorko 244 Jul 15 04:10 templates1lib.cpp
4 -rw-r--r-- 1 jmzorko jmzorko 1980 Jul 15 04:40 templates1lib.o
20 -rwxr-xr-x 1 jmzorko jmzorko 20040 Jul 15 04:34 templates_total
4 -rw-r--r-- 1 jmzorko jmzorko 340 Jul 15 04:34 templates_total.cpp
PugsleyButt:~/devstuff/c++/templates jmzorko$ nm libtemplates1.a

libtemplates1.a(templates1lib.o):
00000094 s EH_frame1
0000007c s __GLOBAL__I_templates1lib.cpp_1FD5A050_803EC32C
00000100 s __GLOBAL__I_templates1lib.cpp_1FD5A050_803EC32C.eh
00000024 s __Z41__static_initialization_and_destruction_0ii
000000d8 s __Z41__static_initialization_and_destruction_0ii.eh
U __ZNSt8ios_base4InitC1Ev
U __ZNSt8ios_base4InitD1Ev
00000140 b __ZSt8__ioinit
U ___cxa_atexit
U ___dso_handle
U ___gxx_personality_v0
0000013c S ___i686.get_pc_thunk.bx
00000000 t ___tcf_0
000000b0 s ___tcf_0.eh
PugsleyButt:~/devstuff/c++/templates jmzorko$

... well, I don't see anything that indicates that the template is in the library (libtemplate1.a), but i'll try making code that uses it anyway and see what happens:

PugsleyButt:~/devstuff/c++/templates jmzorko$ cat templates1.cpp
// class templates
#include <iostream>
#include "templates1.h"

int main ()
{
pair <int> myobject (100, 75);
std::cout << myobject.getmax();
return 0;
}
PugsleyButt:~/devstuff/c++/templates jmzorko$ c++ templates1.cpp -L. -ltemplates1 -o templates1
/usr/bin/ld: Undefined symbols:
pair<int>::getmax()
pair<int>::pair(int, int)
collect2: ld returned 1 exit status
PugsleyButt:~/devstuff/c++/templates jmzorko$

... and that's the problem -- when I put my template class and implementation in a library, I can't use them from anywhere else. I know that templates are difficult for a C++ compiler, but how do I solve this? BTW, this code fails the same way on Windows, so once I know what the real problem is and how to fix it on *nix, i'll find the MS equivalent way to fix it ...

Has anyone else run into this? I know it's a common problem ...

Regards,

John

Falling You - exploring the beauty of voice and sound
www.fallingyou.com
posted by:
Advertisement
Advertisement
  • Re: template hell!

    Sat, July 15, 2006 - 8:24 AM
    Template definitions go in header files, to be #included.

    When you put your getmax() "implementation" in the template_total.cpp file, well, it wasn't an implementation at all - it was a template. It compiled into nothing. It goes in the h file.
    • Re: template hell!

      Sat, July 15, 2006 - 9:37 AM
      Oh and BTW if everything you're creating is a template, then there's no point in creating a library for it, because there's nothing to link to. (Since it's just #included).
    • Re: template hell!

      Sat, July 15, 2006 - 9:59 AM

      Scott,

      I don't quite understand, so maybe I should rephrase my question. I use libraries with template implementations all of the time -- the C++ standard library (for vectors, lists, maps, dynamic_cast, etc.), Boost (for lexical_cast, scoped_ptr, threads and locks, etc.), ACE and others. I'm trying to make my own library with a template implementation, one which I can use for anything (i'm using simple ints in my sample above, but eventually I need to make it work for my own classes), one which I can compile into a library (static or dynamic) and use the template implementation from code which links to the library. This has to work on Mac OSX as well as Windows, though i'm sure if I can find the solution on one platform, I can find the analog on the other.

      Regards,

      John

      Falling You - exploring the beauty of voice and sound
      www.fallingyou.com
      • Re: template hell!

        Sat, July 15, 2006 - 10:27 AM
        When you defined the template, that's basically all you did - define it. Now it's up to some other code somewhere to cause the compiler to generate something.

        This:

        template<T> class vector;

        is not compiled into object code, it is basically a directive to the compiler explaining what to do when it finds an instantiation of this template. Like this:

        vector<int> myvector;

        This causes code to be generated where there was no code before, the kind stuffed into a library or a dynamic library, or an executable. There is no object code (compiled code) for vector<T>, there is only object code for vector<int>, vector<double>, vector<char>, etc.

        I can also write this:
        template <class T> MyTemplate {

        void ThisMethodDoesSomething(T obj) {
        CallMyRealFunction();
        }
        };

        When I instantiate this template,

        MyTemplate<int> HereItIs;

        The linker will yell at me if I don't link to whatever library contains "CallMyRealFunction". That's why you need to link to the boost library. (BTW I used the boost library a few years ago and I never had to link to anything - it just depends on what you are using in it).
        • Re: template hell!

          Sat, July 15, 2006 - 10:32 AM
          Let me clarify this:

          >> That's why you need to link to the boost library.

          The reason you need to link to a boost library is because part of its implementation is not template based. But it depends on what templates you are using. If you are just using shared_ptr or the graph library or tuples, you don't have to link to anything at all. If you're using the python library, then there's stuff you have to link to.
      • Re: template hell!

        Sat, July 15, 2006 - 11:00 AM
        I thought of one more thing that may be a source of confusion.

        This line:

        T getmax ();

        says to the compiler that this method (a template) is defined somewhere. But when you're compiling your application, the compiler never sees that definition because you put it in the cpp file.

        This line:
        pair<int> myobject;

        causes the compiler to think that a pair<int>::getmax exists somewhere, but it doesn't! The compiler needs to see the definition of pair<T>::getmax in order to generate the code.

        I can't tell if I'm making any sense here.
        • Re: template hell!

          Sat, July 15, 2006 - 11:24 AM

          Scott,

          Firstly, thanks for the help :-) I understand part of what you're saying e.g. vector<int> myVector creates code, though vector< class T > doesn't (because T isn't defined yet). The part i'm not understanding is this: in my short sample up there, the template implementation was obviously defined and code was generated when everything was built as one executable, because the program ran and gave me the result I expected. When I tried to move the implementation to a library and attempt to link to it, though, the linker can't find it, so something about making it a library broke it. I'm not sure what that is ...

          Since code often speaks for itself, can you show me source for a sample template implementation that lives in a library, and code which uses that implementation linking to that library? That's the point of my sample -- to try to get the smallest possible thing working to better understand it. Maybe that will help me understand what i'm really doing wrong.

          Regards,

          John, no this is not a homework question <smile> ... i've just never had to develop a template-based implementation that lives in a library before

          Falling You - exploring the beauty of voice and sound
          www.fallingyou.com
          • Re: template hell!

            Sat, July 15, 2006 - 11:43 AM
            >> can you show me source for a sample template implementation that lives in a library,

            Nope, can't do it! Not possible. It must absolutely must must must without exception be in a header file or somewhere where the compiler will see the definition before it sees your instantiations. There is no exception to this!

            The first sample you ran compiled because the compiler encountered your getmax() definition before it saw pair<int>. The goal is to get the compiler to see that definition. To do that, you just move it into the header file. You don't need a library, because at this point the library will be empty anyway. Really!

            Change this:

            template < class T > class pair
            {
            private:
            T a, b;
            public:
            pair (T first, T second);
            T getmax ();
            };

            to this:

            template < class T > class pair
            {
            private:
            T a, b;
            public:
            pair (T first, T second) { a = first; b = second; }
            T getmax () { return a > b ? a : b; }
            };

            Then your life will begin to be wonderful and you will be in template heaven.
            • Re: template hell!

              Sat, July 15, 2006 - 11:50 AM

              Scott,

              Yes, I understand that that will fix the problem, i'm not just understanding why simply putting that code in a library breaks it. I do appreciate the help, maybe I should go read more, because I really can't fathom why the linker can see an implementation when it's part of the same executable and not see it when the exact same implementation is in a library that the executable links with ... wow, I used to consider myself fairly good at C++, too :-(

              Regards,

              John

              Falling You - exploring the beauty of voice and sound
              www.fallingyou.com

              • Re: template hell!

                Sat, July 15, 2006 - 12:19 PM
                John,

                When moving the implementation from the header to the library, the compiler will not see the template definition for your template member functions when compiling other translation units.
                ==> the translation unit you are compiling will not have the instantiated function.
                Your library will not have the function eiher: the translation unit(s) used to compile the library did not need to generate code for the functions as there was no instantiation over "int" in the library code.

                there are basically three way to instantiate a template ( e.g. actualy generate code for the functions in the object file) :
                1. explicit template instantation :
                template class MyTemplate<int>;
                2. implicit instantiation : the compiler decided an instantiation is need.
                3. template specialization : you specialize a template definition

                One possible solution would be to explicitly instantiate the template(s) in the library code.
                ==> However this will still fail if the user of your library tries to instantitate over a type that was not explicitly instantiated.

                One way to look at templates would be to consider them preprocessor macros, ==> unless you use the macro in your code there will be no code generated for it ==> it will not be in the object code.

                For some legaleze :
                check :
                www.csci.csusb.edu/dick/c++...temp.inst

                Dirk
              • Re: template hell!

                Sat, July 15, 2006 - 12:27 PM
                The compiler, when it encounters pair<int>, automatically generates the functions as specified in the templates you defined - key word here is "defined". In your first sample, they are generated because T pair<T>::getmax() is defined directly above your main()! Because of that, in your main(), when it encounters pair<int>::getmax(), the compiler generates it. In the other example, it CAN'T be filled out because all it has is a declaration, so the compiler creates a class that looks more or less like this, with nothing else:

                class pair<int> {
                public:
                pair(int a, int b);
                int getmax(int a, int b);
                };

                How can the compiler possibly generate pair<int>::getmax() if you don't tell it how? The compiler cannot create an implementation of those functions from nothing. And how, when compiling your templates.cpp in the library, would it know to generate an implementation of pair<int>? There's no way it can know.

                Linkers can only link functions, global variables, and constants, to the references to those functions, global variables, and constants, by name. Templates are not real functions, they are templates for real functions. You can't have a "library of templates" (the object file/lib file kind), only a library of functions, global variables, and constants.
                • Re: template hell!

                  Sat, July 15, 2006 - 2:11 PM

                  Scott, Dirk ...

                  Yeah, I finally understand it now. Basically, the compiler treats templates much more like the preprocessor treats macros e.g. the linker has nothing to do with templates. So, making a library with a template implementation won't generate any code for that template implementation, because the type hasn't been defined yet e.g. the compiler has no base to generate code from.

                  So yeah, every time i've used a template implementation that I thought was in a library, it was actually in a header file. I get it now.

                  One thing, though -- things like dynamic_cast, static_cast, const_cast, etc. -- I don't think these live in a header file, and they seem to be templates (or at least, syntactically similar to templates), and they work on run-time data and use types that I can define myself. How do _they_ work?

                  Regards,

                  John

                  Falling You - exploring the beauty of voice and sound
                  www.fallingyou.com
                  • This is the maximum depth. Additional responses will not be threaded.

                    Re: template hell!

                    Sat, July 15, 2006 - 2:38 PM
                    dynamic_cast,... are part of the language (expression) itself, not the library.
                    ==> the compiler has builtin knowledge about these constructs.
                • Re: template hell!

                  Fri, September 15, 2006 - 4:43 PM
                  Actually, it's *possible* for the C++ compiler to find template method definitions outside of the compilation unit (e.g. source file) template instantiations are made. The C++ Language Referece says that you can mark a template method function definition as 'export' and it will be recognized in other source files (see sections 9.2.3 and 13.7). But guess what? Almost no compiler supports this keyword (that I'm aware of). For example, gcc is nice enough to give the following warning if you try to use the 'export' keyword on your template method definitions:

                  warning: keyword 'export' not implemented, and will be ignored

                  I hear this might be supported in the latest version of gcc (4.1.1), but haven't tried it yet.

                  Just an FYI...

                  DJ Fitz
                  • This is the maximum depth. Additional responses will not be threaded.

                    Re: template hell!

                    Mon, September 18, 2006 - 12:01 PM
                    See this paper for more info about gcc and the export keyword:

                    www.gccsummit.org/2005/2005...word%22%22
                    • Re: template hell!

                      Mon, September 18, 2006 - 1:37 PM
                      And here's Herb Sutter's paper on why he thinks the 'export' keyword should be removed from the language (and some history of EDG's implementation of the 'export' keyword):

                      std.dkuug.dk/jtc1/sc22/w...20C%2B%2B%22

                      One of the main points of this paper is that this keyword probably doesn't do what you want:
                      - It doesn't remove the restriction of needing the source code for the template member definition at compile-time. The template instantiation can't refer to object code, it needs the source code to work properly. 'export' just removes the need for the programmer to explicitly include it; the compiler will still need to. So, you still can't just have a separate header/source file paradigm where you ship object code for your template definition source files, and ship a header file for those that want to use them.
                      - There's lots of nasty dependencies and behavior changes that are necessitated by this keyword (ODR and unnamed namspace overloading issues).

                      The paper is good reading whether you're interested in this issue or not.

                      doug
                      • Re: template hell!

                        Mon, September 18, 2006 - 1:46 PM
                        I'll add this to my list of reasons why C++ is doomed.
                        • Re: template hell!

                          Mon, September 18, 2006 - 9:07 PM

                          I was at Apple's WWDC last month, and during one of the XCode sessions, the devtools evangelist who was chairing the session asked the audience what language they used. He started with C++, and 75% or so of the attendees raised their hands. He then asked about ObjC, and maybe 20% responded. ObjC++ accounted for the rest, at which point he said "well, it looks like we have a majority C++ crowd here." Maybe it was me, but I thought I detected a bit of disappointment in his voice.

                          Well, C++ may not be perfect, but it's pretty much the only choice for those of us who want / need to write code that runs on multiple platforms (I target Win2K / XP and Mac OSX PPC / Intel with the C++ code I write at work, and regularly build / test on both platforms) that isn't encumbered by the need to download a VM or something. C++ code often results in small / fast executables that don't take long to download, and they can interface to the OS directly when they need to. The gcc has been ported to about every platform known to man, and it is more-or-less at parity with MSVC. MS may want us to use C#, Apple may want us to use ObjC, but if our code has to run on both (and even more in the future -- i'm looking into what it would take to build and run our stuff on PS2 / PS3s, GC / Wii, DS / PSP, 360s and others), and we need to be small, fast and light, then ... well, C / C++ really is the only choice for writing code with a high degree of portability, esp. when libraries like Boost / ACE / etc. are available.

                          Regards,

                          John

                          Falling You - exploring the beauty of voice and sound
                          www.fallingyou.com