How does Python distinguish callback function which is a member of a class?

Go To StackoverFlow.com

24

Please look at the simple example:

class A:
  def __init__(self, flag):
    self.flag = flag

  def func(self):
    print self.flag

a = A(1)
b = A(2)
callback_a = a.func
callback_b = b.func

callback_a()
callback_b()

The result is:

1
2

It runs as expected. But I have a question. In C, the callback function is passed as a pointer. In Python, it should have a similar way to do this, so the caller knows the address of the function. But in my example, not only the function pointer is passed, but also the parameter (self) is passed, because the same method of the same class prints different results. So my questions are:

  1. Does such a method in Python only has one copy in memory? My meaning is that the code of any method only has one copy, and in my example the method won't be cloned itself. I think it should have only one copy, but here I still make this question in order to get more inputs.

  2. I remember everything in Python is an object. So in my example, are there two function instances with different parameters but only one copy of code?

2012-04-04 01:31
by flypen
I would think it creates a closure - Aram Kocharyan 2012-04-04 01:34
You should be using newstyle classes. Subclass everything from object or something that subclasses it - aaronasterling 2012-04-04 01:36
@aaronasterling: Though I am not sure it has too much impact on the question at hand - jdi 2012-04-04 01:41
@jdi There's always an opportunity to learn something new - aaronasterling 2012-04-04 01:48
@aaronasterling it doesn't even make sense to recommend it in this scenario, as there is no specific version of python specified. Old style classes are gone in python 3.x, and new styles are the default - Darthfett 2012-04-04 01:57
@Darthfett. Observe the print statements - aaronasterling 2012-04-04 01:58
@flypen, Greg's response is probably the better one and has a lot of useful information in the comments so it should probably be the accepted answer... - aaronasterling 2012-04-04 19:40
@aaronasterling ah, my bad. Must've missed that - Darthfett 2012-04-06 05:35


15

Specific to CPython, there is only one copy of the function object. During instance creation, the class wraps the unbound functions in its namespace as bound methods. But they all wrap the same function.

Here's your example expanded to show what's going on.

class A(object):
  def __init__(self, flag):
    self.flag = flag

  def func(self):
    print self.flag

a = A(1)
b = A(2)

callback_a = a.func
callback_b = b.func

print "typeof(callback_a) = {0}".format(type(callback_a))
print "typeof(callback_b) = {0}".format(type(callback_b))

print "typeof(callback_a.__func__) = {0}".format(type(callback_a.__func__))
print "typeof(callback_b.__func__) = {0}".format(type(callback_b.__func__))

print "'callback_a.__func__ is callback_b.__func__'  is {0}".format(callback_a.__func__ is callback_b.__func__)

callback_a()
callback_b()

This code outputs

typeof(callback_a) = <type 'instancemethod'>
typeof(callback_b) = <type 'instancemethod'>
typeof(callback_a.__func__) = <type 'function'>
typeof(callback_b.__func__) = <type 'function'>
'callback_a.__func__ is callback_b.__func__'  is True

You can clearly see, using the is operator, that both instancemethod classes are sharing the same function object.

2012-04-04 01:42
by aaronasterling
I'd like to know why this got downvoted. If there's something objectively wrong, then I'd like to know - aaronasterling 2012-04-04 02:03
What I find really interesting is that this print "'callback_a is callback_b' is {0}".format(callback_a is callback_b) prints false. Coming from Javascript and Perl I find this behavior really surprising. Are there Python documents that say more about it - Kevin G. 2014-01-31 00:39
callback_a is callback_b is False because each is an instancemethod that wraps both the function AND the instance with which to call that method. Both share the same __func__ (as shown in the initial post), but each has a different self object instance with which that function is called. Make sense - PfhorSlayer 2014-12-15 12:44


19

In Python, the callback is not simply a reference to a member function. Instead, it is "bound" to the object that it refers to when it was created. So a.func creates a callable that is bound to a, and b.func creates a callable that is bound to b.

Python only needs one implementation of func() in memory, but it will probably create one or more "trampoline" functions at runtime to accomplish the binding (I'm not certain of the internal details on this, and it would differ between Python implementations anyway).

If you print id(callback_a) and id(callback_b) you will get different results, showing that they are indeed different callable objects.

2012-04-04 01:34
by Greg Hewgill
And if you do id(b.func) == id(a.func) you find that they point to the same memor - jdi 2012-04-04 01:38
@jdi: Can you tell me why id(a.func) gives different result each time - mshsayem 2012-04-04 01:42
@mshsayem id is implemented as the memory address of the object in CPython. So it's getting a different location in memory each time the program runs - aaronasterling 2012-04-04 01:43
@mshsayem Because we have to store the self pointer somewhere (at least conceptually but also in reality - for cpython). Basically it's a struct Py_Object *self, *func and we obviously can't share the struct because self is always different. If you want the actual function object you can do Class.func (which you can also call, eg try A.func(a) in your code) - the function object should be the same (at least in cpython! - Voo 2012-04-04 01:44
@jdi: that's purely an implementation detail and isn't guaranteed, though. For example, I just tried it on pypy and ironpython and they weren't equal - DSM 2012-04-04 01:45
@DSM: Ah good to kno - jdi 2012-04-04 02:02
@jdi Try a_func = a.func; b_func = b.func; id(a_func) == id(b_func). a.func and b.func are different bound method objects. In CPython bound method objects are constructed on attribute access and not cached. When you do id(a.func) a new bound method is constructed, you get its id, then it is immediately garbage collected since there are no more references to it. When you then do id(b.func) the new bound method is constructed in the same piece of memory, so your equality is true by coincidence - Peter Graham 2012-04-04 02:59