Update shared libraries without restarting processes

Go To StackoverFlow.com

6

If my process is loading a .so library and if a new version of the library is available is it possible to switch to the new library without doing a process restart ? Or the answer depends on things like whether there is a parameter change to one of the existing functions in the library ?

I am working in a pretty big system which runs 100s of processes and each loading 10s of libraries. The libraries provide specific functionality and are provided by separate teams. So when one of the library changes (for a bug fix lets say) ideal thing would be to publish it under-the-hood without impacting the running process. Is it possible ?

EDIT Thanks! In my case when a new library is available all the running processes have to start using it. Its not option to let them run with the old version and pick-up the new one later. So it looks like the safer option is to just reload the processes.

2012-04-03 20:37
by Manohar
Unless you control all executables, hot updating so files is not possible without ugly hacks like ptracing processes. Also, linux does not use dll - BatchyX 2012-04-03 20:49


5

You cannot upgrade a linked library on the fly with a process running. You could even try to, but if you succed (and you'll not fail with a "text file is in use" error message), you'll have to restart the process to make it mapping the new library into memory.

You can use lsof command to check which libraries are linked in (runtime or linktime):

lsof -p <process_pid> | grep ' mem '
2012-04-03 21:25
by dAm2K
Instead of lsof -p <pid>, you can also use pldd <pid> - Mikel 2012-04-05 20:44
pldd: command not found. apt-cache search pldd: none found.. - dAm2K 2012-04-06 11:52
It was only added to glibc in March this year. Your distribution may not include it yet - Mikel 2012-04-06 14:13
it's also possible to use pmap to check for linked in libraries in a running proces - dAm2K 2017-04-15 22:31
Clarification: Do you mean you cannot upgrade a statically linked library on the fly? If so, please update your answer as this is presently a bit unclear. dlopen/dlclose exist for the purpose of being able to achieve this very thing with dynamically linked libs - Engineer 2017-05-23 06:58


5

One interesting technique, although it is somewhat prone to failure in the checkpoint restore step, is to do an invisible restart.

Your server process or whatever it is, saves all its necessary information into disk files. Including the file descriptor numbers and current states. Then, the server process does an exec system call to execute itself, replacing the current version of itself. Then it reads its state from the disk files and resumes serving its file descriptors as if nothing happened.

If all goes well, the restart is invisible and the new process is using all of the updated libraries.

2012-04-03 23:46
by Zan Lynx
A great example is irssi and its /upgrade command - R.. 2012-04-04 01:37


3

At the very least, you have to make sure that the interface of the library does not change between versions. If that is assured, then I would try looking into dynamically loading the libraries with dlopen/dlsym and see if dlclose allows you to re-load.

I've never done any of this myself, but that's the path I'd pursue first. If you go this way, could you publish the results?

2012-04-04 12:14
by Arkadiy


2

Linux provides several dynamic loader interfaces, and process can load dynamic librarys when running. dlopen and dlsysm provided by linux may solve your problem.

2013-03-18 09:22
by FCBusquest


1

If you expect libaries to change on a fairly regular basis, and you expect to maintain up-time, I think that your system should be re-engineered so that such libraries actually become loosely coupled components (e.g. services).

Having said that, my answer to the question is yes: under certain circumstances, it possible to update shared libraries without restarting processes. In most cases I expect it is not possible, for instance when the API of your library changes, when the arrangement of your data segment changes, when the library maintains internal threads. The list is quite long.

For very small bug fixes to the code, you can still make use of ptrace to write to the process memory space, and from there redo what /lib/ld-linux.so does in terms of dynamic linking. Honestly, it is an extremely complex activity.

2012-04-03 22:28
by SquareRootOfTwentyThree


0

ldd the binary of your process is one way to find out. although it is theoretically possible, it is not advisable to tinker with the running process, although i am sure utilities exist such as ksplice that tinker with the running linux kernels.

you can simply upgrade and the running process will continue with the old version, and pick up the new version when it restarts, assuming that your package management system is good and knows what is comptible to install.

2012-04-03 20:57
by johnshen64


0

You might want to learn about shared library versioning and the ld -h option.

One way to use it is as follows:

You maintain a version counter in your build system. You build the shared library with:

ld ..... -h mylibrary.so.$VERSION

however, you put it in your dev tree's lib as just plain mylibrary.so. (There's also a hack involving putting the entire .so into a .a file).

Now, at runtime, processes using the library look for the fully-versioned name. To roll out a new version, you just add the new version to the picture. Running programs linked against the old version continue to use it. As programs are relinked and tested against the new one, you roll out new executables.

2012-04-03 21:07
by bmargulies


0

Sometimes you can upgrade an in-use .so, and sometimes you cannot. This depends mostly on how you try to do it, but also on the safety guarantees of the kernel you're running on.

Don't do this: cat new.so > old.so ...because eventually, your process may try to demand page something, and find that it's not in the correct spot anymore. It's a problem because the addresses of things may change, and it's still the same inode; you're just overwriting the bytes in the file.

However, if you: mv new.so old.so You'll be OK on most systems, because your running processes can hold onto a now-unnamed inode for the old library, while new invocations of your processes get the new file. BUT, some kernels don't like to let you mv an in-use .so, perhaps out of caution, perhaps for their own simplicity.

2012-04-03 22:05
by user1277476
I don't think that "cat new.so > old.so" has any effect on a running program. Since the kernel knows that code segment in memory is mapped to the storage where old.so is, it will either refuse to write ("text busy") or even go ahead and set the sectors aside until the program is finished - SquareRootOfTwentyThree 2012-04-04 12:54