Automatic document generation is a common need of many applications, even if a lot of operations do not require printed paper anymore.
Invoices, tickets, leaflets, diplomas, documentation, books… All these documents are read and used on paper, but also on electronical readers, on smartphones, on computers. PDF is a great format to store and display them in a reliable way, with pagination.
Using HTML and CSS to generate static and paged content can be strange at first glance: browsers display only one page, with variable dimensions, often in a very dynamic way. But paged media layout is actually included in CSS2, which was already a W3C recommendation in 1998.
Other well-known tools can be used to automatically generate PDF documents, like LaTeX and LibreOffice, but they miss many advantages that HTML and CSS offer. HTML and CSS are very widely known, by developers but also by webdesigners. They are specified in a backwards-compatible way, and regularly adapted to please the use of billions of people. They are really easy to write and generate, with a ridiculous amount of tools that are finely adapted to the needs and taste of their users.
However, the web engines that are used for browsers were very limited for pagination when WeasyPrint was created in 2011. Even now, they lack a lot of basic features. That’s why projects such as wkhtmltopdf and PagedJS have been created: they add some of these features to existing browsers.
Other solutions have beed developed, including web engine dedicated to paged media. Prince, Antennahouse or Typeset.sh created original renderers supporting many features related to pagination. These tools are very powerful, but they are not open source.
Building a free and open source web renderer generating high-quality documents is the main goal of WeasyPrint. Do you think that it was a little bit crazy to create such a big project from scratch? Here is what Simon Sapin wrote in WeasyPrint’s documentation one month after the beginning:
We still need however to implement the whole CSS box model and visual rendering. This is a lot of work, but we feel we can get something useful much quicker than “Let’s build a rendering engine!” may seem.
Simon is often right.
Python is a really good language to design a small, OS-agnostic parser. As it is object-oriented, it gives the possibility to follow the specification with high-level classes and a small amount of very simple code.
Speed is not WeasyPrint’s main goal. Web rendering is a very complex task, and following the Zen of Python helped a lot to keep our sanity (both in our code and in our heads): code simplicity, maintainability and flexibility are the most important goals for this library, as they give the ability to stay really close to the specification and to fix bugs easily.
Dive into the Source
This chapter is a high-level overview of WeasyPrint’s source code. For more details, see the various docstrings or even the code itself. When in doubt, feel free to ask!
Much like in web browsers, the rendering of a document in WeasyPrint goes like this:
The HTML document is fetched and parsed into a tree of elements (like DOM).
CSS stylesheets (either found in the HTML or supplied by the user) are fetched and parsed.
The stylesheets are applied to the DOM-like tree.
The DOM-like tree with styles is transformed into a formatting structure made of rectangular boxes.
These boxes are laid-out with fixed dimensions and position onto pages.
For each page, the boxes are re-ordered to observe stacking rules, and are drawn on a PDF page.
Metadata −such as document information, attachments, embedded files, hyperlinks, and PDF trim and bleed boxes− are added to the PDF.
Not much to see here. The
HTML class handles step 1 and
gives a tree of HTML elements. Although the actual API is different, this
tree is conceptually the same as what web browsers call the DOM.
In addition to the actual parsing, the
modules do some pre-processing:
Unknown and unsupported declarations are ignored with warnings. Remaining property values are parsed in a property-specific way from raw tinycss2 tokens into a higher-level form.
Shorthand properties are expanded. For example,
Hyphens in property names are replaced by underscores (
margin_top). This transformation is safe since none of the known (not ignored) properties have an underscore character.
Selectors are pre-compiled with cssselect2.
After that and still in the
css package, the cascade
(that’s the C in CSS!) applies the stylesheets to the element tree.
Selectors associate property declarations to elements. In case of conflicting
declarations (different values for the same property on the same element),
the one with the highest weight wins. Weights are based on the stylesheet’s
!important markers, selector
specificity and source order. Missing values are filled in through
inheritance (from the parent element) or the property’s initial value,
so that every element has a specified value for every property.
These specified values are turned into computed values in the
css.computed_values module. Keywords and lengths in various units are
converted to pixels, etc. At this point the value for some properties can be
represented by a single number or string, but some require more complex
objects. For example, a
Dimension object can be either an absolute length
or a percentage.
The final result of the
css.get_all_computed_styles function is a big dict
where keys are
(element, pseudo_element_type) tuples, and keys are style
dict objects. Elements are ElementTree elements, while the type of
pseudo-element is a string for eg.
::first-line selectors, or
for “normal” elements. Style dict objects are dicts mapping property names to
the computed values. (The return value is not the dict itself, but a
style_for function for accessing it.)
The visual formatting model explains how elements (from the ElementTree tree) generate boxes (in the formatting structure). This is step 4 above. Boxes may have children and thus form a tree, much like elements. This tree is generally close but not identical to the ElementTree tree: some elements generate more than one box or none.
Boxes are of a lot of different kinds. For example you should not confuse
block-level boxes and block containers, though block boxes are both. The
formatting_structure.boxes module has a whole hierarchy of classes to
represent all these boxes. We won’t go into the details here, see the module
and class docstrings.
formatting_structure.build module takes an ElementTree tree with
associated computed styles, and builds a formatting structure. It generates the
right boxes for each element and ensures they conform to the models rules
(eg. an inline box can not contain a block). Each box has a
attribute containing the style dict of computed values.
The main logic is based on the
display property, but it can be overridden
for some elements by adding a handler in the
This is how
<td colspan=3> are currently implemented,
This module is rather short as most of HTML is defined in CSS rather than in Python, in the user agent stylesheet.
formatting_structure.build.build_formatting_structure function returns
the box for the root element (and, through its
children attribute, the
Step 5 is the layout. You could say the everything else is glue code and this is where the magic happens.
During the layout the document’s content is, well, laid out on pages. This is when we decide where to do line breaks and page breaks. If a break happens inside of a box, that box is split into two (or more) boxes in the layout result.
According to the box model, each box has rectangular margin, border, padding and content areas:
box.style contains computed values, the used values are set as
attributes of the
Box object itself during the layout. This include
resolving percentages and especially
auto values into absolute, pixel
lengths. Once the layout done, each box has used values for margins, border
width, padding of each four sides, as well as the
the content area. They also have
absolute coordinates of the top-left corner of the margin box (not the
content box) from the top-left corner of the page.1
Boxes also have helpers methods such as
that give other metrics that can be useful in various parts of the code.
The final result of the layout is a list of
Stacking & Drawing
In step 6, the boxes are reordered by the
stacking module to observe
stacking rules such as the
z-index property. The result is a tree of
Next, each laid-out page is drawn onto a PDF page. Since each box has absolute coordinates on the page from the layout step, the logic here should be minimal. If you find yourself adding a lot of logic here, maybe it should go in the layout or stacking instead.
The code lives in the
Finally (step 7), the