Bit Strings

Postgres has a bit string type, which is exposed by django-postgres as BitStringField and the BitStringExpression helper (aliased as django_postgres.B). The representation of bit strings in Python is handled by the python-bitstring library (a dependency of django-postgres).


Given the following

from django.db import models
import django_postgres

class BloomFilter(models.Model):
    name = models.CharField(max_length=100)
    bitmap = django_postgres.BitStringField(max_length=8)

You can create objects with bit strings, and update them like so:

>>> from django_postgres import Bits
>>> from models import BloomFilter

>>> bloom = BloomFilter.objects.create(name='test')
INSERT INTO myapp_bloomfilter
  (name, bitmap) VALUES ('test', B'00000000')

>>> print bloom.bitmap
>>> bloom.bitmap |= Bits(bin='00100000')
>>> print bloom.bitmap

UPDATE myapp_bloomfilter SET bitmap = B'00100000'
 WHERE = 1;

Several query lookups are defined for filtering on bit strings. Standard equality:

>>> BloomFilter.objects.filter(bitmap='00100000')
SELECT * FROM myapp_bloomfilter WHERE bitmap = B'00100000';

You can also test against bitwise comparison operators (and, or and xor). The SQL produced is slightly convoluted, due to the few functions provided by Postgres:

>>> BloomFilter.objects.filter(bitmap__and='00010000')
SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap & B'00010000') > 0
>>> BloomFilter.objects.filter(bitmap__or='00010000')
SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap | B'00010000') > 0
>>> BloomFilter.objects.filter(bitmap__xor='00010000')
SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap # B'00010000') > 0

Finally, you can also test the zero-ness of left- and right-shifted bit strings:

>>> BloomFilter.objects.filter(bitmap__lshift=3)
SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap << 3) > 0
>>> BloomFilter.objects.filter(bitmap__rshift=3)
SELECT * FROM myapp_bloomfilter WHERE position(B'1' IN bitmap >> 3) > 0

Bit String Fields

class django_postgres.BitStringField(max_length=1[, varying=False, ...])

A bit string field, represented by the Postgres BIT or VARBIT types.

  • max_length – The length (in bits) of this field.
  • varying – Use a VARBIT instead of BIT. Not recommended; it may cause strange querying behavior or length mismatch errors.

If varying is True and max_length is None, a VARBIT of unlimited length will be created.

The default value of a BitStringField is chosen as follows:

  • If a default kwarg is provided, that value is used.
  • Otherwise, if null=True, the default value is None.
  • Otherwise, if the field is not a VARBIT, it defaults to an all-zero bit string of max_length (remember, the default length is 1).
  • Finally, all other cases will default to a single 0.

All other parameters (db_column, help_text, etc.) behave as standard for a Django field.

Bit String Expressions

It’s useful to be able to atomically modify bit strings in the database, in a manner similar to Django’s F-expressions. For this reason, BitStringExpression is provided, and aliased as django_postgres.B for convenience.

Here’s a short example:

>>> from django_postgres import B
>>> BloomFilter.objects.filter(id=1).update(bitmap=B('bitmap') | '00001000')
UPDATE myapp_bloomfilter SET bitmap = bitmap | B'00001000'
 WHERE = 1;
>>> bloom = BloomFilter.objects.get(id=1)
>>> print bloom.bitmap
class django_postgres.BitStringExpression(field_name)

The following operators are supported:

  • Concatenation (+)
  • Bitwise AND (&)
  • Bitwise OR (|)
  • Bitwise XOR (^)
  • (Unary) bitwise NOT (~)
  • Bitwise left-shift (<<)
  • Bitwise right-shift (>>)