Home » Odeon Blogs » Stefan Talpalaru, CTO »

integrating pygments with Django

integrating pygments with Django

In the python world pygments is the library of choice when it comes to syntax highlighting. This is an example of integration with Django by creating a custom template filter. First, let's make a generic function in the file misc/pygmentify.py ('misc' is a Django app):

  1. import warnings
  2. warnings.simplefilter('ignore')
  3. from pygments.lexers import LEXERS, get_lexer_by_name
  4. warnings.resetwarnings()
  5. from pygments import highlight
  6. from pygments.formatters import HtmlFormatter
  7. from lxml import html


  10. def pygmentify_text(text):
  11.     default_lang = 'text'

  13.     # a tuple of known lexer names
  14.     lexer_names = reduce(lambda a,b: a + b[2], LEXERS.itervalues(), ())

  16.     # default formatter
  17.     formatter = HtmlFormatter(encoding='utf-8', linenos='inline')

  19.     html_node = html.fragment_fromstring(text, create_parent='div')
  20.     new_html_node = html_node
  21.     for code_node in html_node.findall('pre'):
  22.         if not code_node.text:
  23.             continue
  24.         lang = code_node.attrib.get('lang', default_lang)
  25.         if lang not in lexer_names:
  26.             lang = default_lang
  27.         lexer = get_lexer_by_name(lang, stripall=True)
  28.         new_code_node = html.fragment_fromstring(highlight(code_node.text, lexer, formatter))
  29.         new_html_node.replace(code_node, new_code_node)

  31.     return html.tostring(new_html_node, encoding=unicode, method='xml')[5:-6] # need to strip the enclosing div


The whole warning disabling at the top is to work around an annoying bug in the pkg_resources module used by pygmentify's plugin system. We are using<pre> tags for our code with the custom attribute "lang=something_known_to_pygments" to specify the programming language of the snippet.

Notice the use of lxml for parsing and generating (X)HTML. The fast C module comes in handy for the type of on-the-fly highlighting that we do.

Now the custom template filter in misc/templatetags/pygmentify.py:

  1. from django import template
  2. from django.template.defaultfilters import stringfilter
  3. from misc.pygmentify import pygmentify_text

  5. register = template.Library()

  7. # truncate after a certain number of characters
  8. @register.filter
  9. @stringfilter
  10. def pygmentify(value):
  11.     try:
  12.         res = pygmentify_text(value)
  13.     except Exception, e:
  14.         print e
  15.         print 'value="%s"' % value
  16.         res = value

  18.     return res


The exception handling comes in handy when debugging because Django's trace-back for template exceptions is of no use.


Our filter would be used like this in a template: 

  1. {% load pygmentify %}

  3. {% autoescape off %}
  4. {{ entry.item.post|pygmentify|linebreaks }}
  5. {% endautoescape %}


The only thing missing is the CSS. We can generate the default theme from the python console: 

  1. from pygments.formatters import HtmlFormatter
  2. print HtmlFormatter().get_style_defs('.highlight')

You might want to control the overflow and impose a color for the line numbers so add this:

  1. .highlight  { background: #f8f8f8; overflow: auto; }
  2. .highlight .lineno { color: #000000 }

Some useful strings recognized by pygments:

- "python"

- "pycon" - python console

- "pytb" - python traceback

- "html+django" - Django template



Leave a Comment :

(required)


(required)




(required)




(required)





Page generated in: 0.26s