The Publishing Process

This brief article focuses on only one part of using Pelican as a static site generation tool - the day to day usage once the set up is established. There are loads of really good articles on how to get started with Pelican and I am not about to add to that growing pile; there is just no need.

I don't use the default make tool that comes with Pelican, because having a Python programming background made me feel much more comfortable with utilising the ever excellent Fabric tool.


Some may call this overkill, but I actually maintain three repositories for this site; two private ones and a public one where this site is hosted. One private repository is a store for all the site in the build phase. This contains all the .rst files, config files, draft articles and build script; it is basically the site in a raw form. The public repo is where the site is published, in HTML form. The second private repo functions purely as a complete backup of the public repo. As a final note all commits are signed, using git commit -S, with my identity.

The Process

I have boiled down my article creation process to four basic, but not necessarily linear, steps: write, read, back up and publish.


Articles are created in the reStructuredText format mostly using the superb Sublime Text 3 editor; if you have not tried it you should definitely give it a go, and it works on Linux, Mac and even Windows. The articles are stored in category specific sub-directories as required.


For local viewing prior to publishing I use Python's SimpleHTTPServer. All configuration values for the local Pelican build are handled by a file in the build/configs directory.

Back Up

Whilst writing I frequently make back up commits to the private build repo. Any changes I make to the build process are also committed to that repo. As nothing I ever write gets finished first time, these first three steps of write, read and back up are performed in an almost infinite loop.


This is the point at which the article to be published has passed my somewhat meagre level of quality control and it is ready to be launched onto the unsuspecting masses, such as yourself. In a nutshell, the fabric file below takes care of clearing out the deploy directory and adding the newly built HTML and theme sundries. All draft articles are removed, because I do not want them on the live site, and the site changes are committed to the public repository. In addition, as described above, a private back up is also made.

File and Directory Structure

Here's a slightly truncated view of the directory layout for the complete site creation and maintenance process.

├── build
│   ├── .git (Back up repo)
│   ├── cache
│   ├── configs
│   │   ├──
│   │   ├──
│   │   └──
│   │
│   └── content
│       ├── extras
│       ├── pages
│       └── [...]
├── deploy
│   ├── .git ( repo)
│   ├── [...]
│   └── theme
│       ├── css
│       ├── fonts
│       └── js
└── local
    ├── [...]
    └── theme
        ├── css
        ├── fonts
        └── js


This directory is where any editing and configuration takes place. This is backed up by the fab backup target to this repo. Following are some important points about the sub-directories:

Herein is contained a file - this is here to enable fab to import a file for setting a few useful environmental variables. A file also lives in here for when the fab publish target is run.
This is where the site articles are stored and edited - primarily as reStructuredText documents.


The local directory is where the site is published, via the fab serve target, and evaluated locally.


The deploy directory is itself a git repo and is where the site is published, using the fab publish fabric command, prior to committing and pushing into the repo etc. As mentioned before, all drafts are deleted prior to the git commit.

The Fabric File

Below is a copy of my current fabric script. It has turned into a bit of a monster but I am quite happy that there is nothing in there that does not get at least some regular use. The targets (apologies, but I am in the habit of calling the defs below targets from my days spent using Apache Ant) I mostly use are serve, backup and publish. You may notice that both the targets concerned with committing code to git repos support optional commit messages, something I thought was quite a nice touch.

import time

from fabric.api import local, settings, env, cd
import fabric.contrib.project as project
import os

# There must be at least a file in the configs directory.
import configs.local_settings as conf

# Local path configuration (can be absolute or relative to fabfile) - they are
# all added to env for convenience & abuse.

# Repo settings - from
env.git_url = conf.URL1
env.git_backup_url = conf.URL2
env.host_repo = conf.PUBLISH_REPO
env.build_repo = conf.BUILD_REPO
env.host_backup_repo = conf.PUBLISH_BACKUP_REPO

# Local file paths
env.config_path = './configs'
env.local_settings_file = ''
env.deploy_settings_file = ''
env.project_path = '../build'
env.deploy_path = '../deploy'
env.local_path = '../local'
env.content_path = './content'

def clean():
    Cleans and recreates local copy directory
    if os.path.isdir(DEPLOY_PATH):
        local('rm -rf {local_path}'.format(**env))
        local('mkdir {local_path}'.format(**env))

def kill():
    Kills errant SimpleHTTPServer processes - always outputs an error but works nevertheless
    with settings(warn_only=True):
        local('ps aux | grep SimpleHTTPServer | awk \'{print $2}\' | xargs kill -9')

def build():
    Runs local build with debug output - pushes out content to local copy directory, using local settings
    local('pelican -D {content_path} -o {local_path} -s {config_path}/{local_settings_file}'.format(**env))

def backup(commit_msg=False):
    Backs up this local copy to bitbucket with an optional " delimited commit message.
    with lcd(env.project_path):
        if commit_msg is False:
            command = 'git commit -S -m "Back up {0}"'
            command = 'git commit -S -m "Back up {0} - {1}"'

        local('git add -A .'.format(**env))
            time.strftime("%d %b %Y %H:%M:%S", time.localtime()), commit_msg, **env
        local('git push {git_backup_url}/{build_repo} master'.format(**env))

def rebuild():
    Cleans and pushes content to local copy directory, running clean() and build()

def remove_drafts():
    Removes drafts from deploy_path/drafts as we don't want publicly accessible
    with lcd(env.deploy_path):
        local('rm -fr drafts'.format(**env))

def regenerate():
    Regenerates site content
    local('pelican -r -s {local_settings_file}'.format(**env))

def serve():
    Serves content from local copy directory, calling clean() and build() beforehand
    with lcd(env.local_path):
        local('python -m SimpleHTTPServer'.format(**env))

def pub_deploy():
    Prepares for publishing - runs non debug build to deployment path output
    with lcd(env.deploy_path):
        local('rm -fr *'.format(**env));
    local('pelican {content_path} -o {deploy_path} -s {config_path}/{deploy_settings_file}'.format(**env))

def publish(commit_msg=False):
    Makes a build over to the deploy path, using deployment settings, cleaning up beforehand with pub_deploy() with an optional " delimited commit message.
    with lcd(env.deploy_path):
        if commit_msg is False:
            command = 'git commit -S -m "Publication {0}"'
            command = 'git commit -S -m "Publication {0} - {1}"'

    # Publish to deployment directory
    # Remove draft htmls - don't want them in a public repo
    # Git add and commit
    with lcd(env.deploy_path):
        local('git add -A .'.format(**env))
            time.strftime("%d %b %Y %H:%M:%S", time.localtime()), commit_msg, **env
        local('git push {git_url}/{host_repo} master'.format(**env))
        local('git push {git_backup_url}/{host_backup_repo} master'.format(**env))

So there you have it, not quite as brief as I intended... feel free to pass comment, suggest improvements etc.


comments powered by Disqus