Basically, FBconfigs and PBuffers allow you to do off-screen rendering with OpenGL. The OSMesa interface does basically the same thing, but fbconfigs and pbuffers are supported by more vendors. PBuffer rendering may also be hardware accelerated.
PBuffers are getting more use nowadays, though they've actually been around for a long time on IRIX systems and other workstations.
The GL_SGIX_fbconfig and GL_SGIX_pbuffer extensions describe the functionality. More recently, these extensions have been promoted to ARB extensions (on Windows at least).
The Mesa/progs/xdemos/ directory has some useful code for working with pbuffers:
Mesa 4.1 and later support GL_SGIX_fbconfig and GL_SGIX_pbuffer (software rendering only).