tushman.io

The musings of an insecure technologist

Describing Descriptors Descriptively

Is it ironic that the documentation for descriptors is not very descriptive

Descriptors are one of my favorite Python features — but it took me too long to discover them. The documentation and tutorials that I found were too complex for me. So I would like to offer a different approach, a code first approach

Agenda

  • Definition
  • A Problem that Descriptors Can Solve
  • CODE!! Solution to the Problem
  • Reflection on the Code, and explanation on how we used Descriptors

Definition

In general, a descriptor is an object attribute with “binding behavior”, one whose attribute access has been overridden by methods in the descriptor protocol. Those methods are __get__(), __set__(), and __delete__(). If any of those methods are defined for an object, it is said to be a descriptor.

Read that and stash it in your brain for a few minutes. By the end of this article you’ll grok this

A Problem that Descriptors Can Solve

Imagine that you need to consume a 3rd party API that returns json documents. Often solutions to this problem look like this …

1
2
3

response = requests.get(USER_API_URI, id=111)
print(response.to_json()[0]['results'][0]['name'])
print('barf')

I dislike this solution, Its concise, but it breaks separation-of-concerns. The code consuming the API should not be concerned about the exact path and location of the data element in the json document.

Proposed Solution with Descriptors:

Our goal is to write code like this:

1
2
3

user = UserAPI.get_by_id(111)
print(user.name)
print(user.street_address)

CODE!! Solution to the Problem

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# Hey guys this is a descriptor -- Woot!!        
class Extractor(object):

    def __init__(self, *path)
        self.path = path

    def __get__(self, instance, owner):
        return _extract(instance.json_blob, *self.path)

    def _extract(doc, *keys):
        """
        digs into an dict or lists, if anything along the way is None, then simply return None
        """
        end_of_chain = your_dict
        for key in keys:
            if isinstance(end_of_chain, dict) and key in end_of_chain:
                end_of_chain = end_of_chain[key]
            elif isinstance(end_of_chain, (list, tuple)) and isinstance(key, int):
                end_of_chain = end_of_chain[key]
            else:
                return None

        return end_of_chain

Now look how elegant our code can look

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

class JSONResponse(object):

    def __init__(self, json_blob):
        self.json_blob = json_blob


class User(JSONRepsonse):

    name = Extractor('result','username')
    street_address = Extractor('result','address', 'street')
    status = Extractor('result','status')


class UserAPI(object)

    @class_method
    def get_by_id(cls, id):
        response = requests.get(USER_GET_ENDPOINT, id=id)
        return User(response.json())

And now are code is warm and fuzzy:

1
2
3

user = UserAPI.get_by_id(111)
print(user.name)
print(user.street_address)

Reflection on the Code

The real magic of descriptors happens with the signatures of __get__(), __set__(), and __delete__():

  • object.__get__(self, instance, owner)
  • object.__set__(self, instance, value)
  • object.__delete__(self, instance)

Each of these signatures contains a reference to instance, which is the instance of the owner’s class. So in our example:

instance will be an instance of the User class owner will be the User class self is the instance of the Descriptor, which in our case holds the path attribute.

Let’s take a look at our example where we made a descriptor Extractor. – user = UserAPI.get_by_id(111)

Here we get an instance of a User object, which has the json_blob stored on it from the GET request

  • print(user.name)

Now we call name on that object, which we defined: name = Extractor('result','username'). At this point when we call name it is going to use the Extractor descriptor to extract the value from the json_blob.

The concern of extracting data from a json blob is nicely contained in our Descriptor I think this is one of many great ways to use descriptors to DRY up your code.

Hope this is helpful!

Additional Resources

Comments