RegisterFields in Django

A common (in my opinion) complex pattern in Django is storing a field that identifies what logic should be used. This tends to happen when you have logic that’s similar about how it’s called and what it does, but how they function is significantly different.

An example of this may be a notification system where it can send an SMS or an email. They both require message content and a recipient, but the mechanisms for the actual sending along with generating the complete payload are dramatically different. With Django Cairn this pattern may be necessary to fetch content from different sources. A blog with an RSS feed has a different API than a YouTube channel, but both have lists of content and some way to find the new content. This post seeks to explain this concept and provide some sample code to use in your project.

In an ideal world, we would have a model field that when accessed returns an instance of a custom class¹.

from django.db import models


class SourceBackend:
    """This is an interface for defining how each Source backend should work"""
    key: str

    def __init__(self, source: Source):
        self.source = source

    def update(self):
        """
        Every subclass should define an update class.

        An abstract base class is probably the better choice here.
        """
        raise NotImplementedError


class RSSBackend(SourceBackend):
    key = "rss"

    def update(self):
        # Do something based on source.url and RSS
        pass

class YouTubeBackend(SourceBackend):
    key = "youtube"

    def update(self):
        # Do something based on source.url and YouTube's API
        pass

# Define a mapping to allow look ups for each backend
backend_register = Register()
backend_register[RSSBackend.key] = RSSBackend
backend_register[YouTubeBackend.key] = YouTubeBackend

class Source(models.Model):
    """This is an example model for Django Cairn for fetching data"""
    url = models.URLField()
    backend_key, backend = RegisterField.with_property(register=backend_register, max_length=64)

    
# This is what we're hoping to achieve:
source = Source.objects.first()
assert source.backend_key == "rss"
assert isinstance(source.backend, RSSBackend)
source.backend.update()

You probably noticed the RegisterField class that’s not defined. Good catch. That’s what we’ll be talking about for the most part, but let me finish setting up our scenario.

The class SourceBackend is an interface that defines how it should be called. It’s fairly straightforward only containing the __init__ and update methods. Effectively, as long as you can get it a Source instance the backend will be able to fetch the data for it. Since each content source could require an entirely different set of models to store a new post/video/[content], update needs to stay generic. The key attribute will eventually be used to identify this class in the database and go into backend_key. We’ll talk about it more later though, but you may already see how everything will come together.

The RSSBackend through the magic of handwaving is super simple. I didn’t define the update method here because it’s irrelevant to what we’re doing. We can make the assumption it’ll do whatever it’s supposed to do when it’s called. You can see that RSSBackend.key is set to "rss" and we’re using inheritance to define the __init__ method.

There is another custom class called Register. This is effectively a dictionary with a helper function. I’ll share the code for that later. For now, we add our backend classes to a specific instance of Register and pass that to RegisterField.

This brings us to the sample code at the end of the block that demos what we’re hoping to achieve. The goal is to have a Source instance that has a field called backend_key whose value will match the relevant SourceBackend subclass’ key attribute. In other words, if isinstance(source.backend, RSSBackend) then source.backend_key == "rss". Hopefully, you can see the power of this approach. The class SourceBackend could define many methods rather than just two. You also avoid having to litter your project with various hook mappings to access the functionality you want.

Implementation details

Let’s talk about Register and RegisterField. These are both utility classes that go hand-in-hand. Register is straightforward; it has a single helper function from_value:

class Register(dict):
    def from_value(self, value):
        for key, v in self.items():
            if v == value:
                return key
        raise ValueError("Value not found: %r" % value)

If you have a large amout of elements you may want to consider changing Register’s implementation to inherit from collections.UserDict and maintain a separate value to pair data structure. However, if you are going to have less than 100 elements, the above is fine.

The concept of a register is fairly common. It’s a lookup to determine what value (in our case functionality) to use based on a key. The register needs to be populated at the project level. You should not add items to the register dynamically.

Let’s assume this is our project structure.

myproject
├── config
│   ├── settings.py
│   ├── urls.py
│   ├── asgi.py
│   └── wsgi.py
│
├── content
│   ├── backends
│   │   ├── rss.py      # RSSBackend is defined here
│   │   └── youtube.py  # YouTubeBackend is defined here
│   │
│   ├── apps.py
│   ├── backend.py      # SourceBackend, backend_register are defined here
│   ├── register.py     # Register, RegisterField are defined here
│   ├── models.py
│   ├── views.py
│   └── urls.css
│
├── README.md
└── manage.py

The challenge now becomes how to register the various source backends into the backend_register instance. We could define a decorator similar to how the Django administration site can register ModelAdmin classes or how signal receivers are defined. However, let’s start by keeping it simple and registering the backends in myproject.content.apps.ContentAppConfig.ready():

# myproject/content/backend.py

from .register import Register


backend_register: [str, SourceBackend] = Register()

class SourceBackend:
    """This is an interface for defining how each Source backend should work"""
    key: str

    def __init__(self, source):
        self.source = source

    def update(self):
        """
        Every subclass should define an update class.

        An abstract base class is probably the better choice here.
        """
        raise NotImplementedError

# myproject/content/apps.py

from .backend import backend_register
from .backends.rss import RSSBackend
from .backends.youtube import YouTubeBackend


class ContentAppConfig(AppConfig):
    name = "myproject.content"

    def ready(self):
        backend_register.update({
            RSSBackend.key: RSSBackend,
            YouTubeBackend.key: YouTubeBackend,
        })

Cool, our backend_register now will contain records for each of our backend integrations whenever we access it.

RegisterField explanation

The next step is to use this register in a field class to be used on a model. Here’s all the code. The explanations are in the comments. My hope is that everything is clear enough except for with_property. If something is confusing, let me know.

# myproject/content/register.py

from django.db import models


class Register(dict):
    ...


class RegisterField(models.CharField):
    """Simple key-value serialization and storage in the database."""

    # A RegisterField must have an option selected. If you wish to have a
    # no-op option, then you'll need to create a sentinel value.
    empty_strings_allowed = False

    def __init__(self, /, *args, register: Register, **kwargs):
        # Require a new keyword argument named register.
        # Store it on the instance for later use.
        self.register = register
        super().__init__(*args, **kwargs)

    @classmethod
    def with_property(cls, *args, register: Register, **kwargs):
        """
        Construct the field and create a wrapping property.
        
        This allows us to define the register key as the true CharField
        but also provide the register value as a property on the model.
        
        There may be another way to do this, but I found this works well
        enough.
        
        Usage:
        
            class MyModel(models.Model):
                key, instance = RegisterField.with_property(register=SomeRegister, max_length=64)

        """
        field = cls(*args, register=register, **kwargs)

        @property
        def prop(self):
            key = getattr(self, field.name)
            return register[key] if key else None

        @prop.setter
        def prop(self, value):
            setattr(self, field.name, register.from_value(value))

        return field, prop

    def deconstruct(self):
        name, path, args, kwargs = super().deconstruct()
        # Remove dynamic choices to avoid constantly generating
        # new migrations.
        kwargs.pop("choices", None)
        return name, path, args, kwargs

    def formfield(self, **kwargs):
        # Get the choices immediately before getting the form field
        # This allows the dynamic registration of elements in the register.
        self.choices = self.register.items()
        return super().formfield(**kwargs)

Assuming the __init__, deconstruct and formfield all make sense, let’s move onto with_property. This class method is designed to replace the typical usage of my_field = models.CharField(). The end result is adding a register key field and a register value property to your model instance.

For example, if we have:

class Source(models.Model):
    backend_key, backend = RegisterField.with_property(register=backend_register, max_length=64)

When we fetch a Source instance, we’ll be able to get both the register key (“rss” or “youtube”) as instance.backend_key and the SourceBackend instance as instance.backend.

Expressed in code the following would be true:

instance = Source.objects.first()
assert instance.backend_key in ["rss", "youtube"]
assert (
    isinstance(instance.backend, RSSBackend)
    or isinstance(instance.backend, YouTubeBackend)
)

The really neat thing about this is that if we want to change the backend of an instance we can do so through either backend_key or backend:

instance.backend_key = "rss"
instance.backend = RSSBackend()

This is because of the inline prop. The value when you access it is looked up from the register each time. This implies that if you change backend_key behind it, it will still return the SourceBackend instance reflecting that change. Then because we have:

@prop.setter
def prop(self, value):
    setattr(self, field.name, register.from_value(value))

We can change backend_key by assigning a different SourceBackend instance to instance.backend. It will look up the key from the given SourceBackend instance and set backend_key to that key.

So what does this all mean?

In short, it means we can do this:

source = Source.objects.first()
source.backend.update()