My Pelican Publishing Process ############################# :title: My Pelican Publishing Process :date: 2014-10-04 :category: Blog :tags: blogging, python :slug: my-pelican-publishing-process :author: Chris Ramsay :status: published :language: en :show_source: True .. role:: bash(code) :language: bash .. contents:: The Publishing Process ---------------------- .. PELICAN_BEGIN_SUMMARY This brief article focuses on only one part of using `Pelican`_ as a static site generation tool - the day to day usage once the set up is established. There are loads of really good articles on how to get started with Pelican and I am not about to add to that growing pile; there is just no need. I don't use the default `make tool`_ that comes with Pelican, because having a `Python`_ programming background made me feel much more comfortable with utilising the ever excellent `Fabric`_ tool. .. PELICAN_END_SUMMARY Repositories ~~~~~~~~~~~~ Some may call this overkill, but I actually maintain three repositories for this site; two private ones and a public one where this site is hosted. One private repository is a store for all the site in the build phase. This contains all the .rst files, config files, draft articles and build script; it is basically the site in a raw form. The public repo is where the site is published, in HTML form. The second private repo functions purely as a complete backup of the public repo. As a final note all commits are signed, using :bash:`git commit -S`, with my `keybase.io identity`_. The Process ~~~~~~~~~~~ I have boiled down my article creation process to four basic, but not necessarily linear, steps: *write*, *read*, *back up* and *publish*. Write ^^^^^ Articles are created in the `reStructuredText`_ format mostly using the superb `Sublime Text 3 editor`_; if you have not tried it you should definitely give it a go, and it works on Linux, Mac and even Windows. The articles are stored in category specific sub-directories as required. Read ^^^^ For local viewing prior to publishing I use Python's `SimpleHTTPServer`_. All configuration values for the local Pelican build are handled by a :bash:`local_settings.py` file in the :bash:`build/configs` directory. Back Up ^^^^^^^ Whilst writing I frequently make back up commits to the private build repo. Any changes I make to the build process are also committed to that repo. As nothing I ever write gets finished first time, these first three steps of write, read and back up are performed in an almost infinite loop. Publish ^^^^^^^ This is the point at which the article to be published has passed my somewhat meagre level of quality control and it is ready to be launched onto the unsuspecting masses, such as yourself. In a nutshell, the fabric file below takes care of clearing out the :bash:`deploy` directory and adding the newly built HTML and theme sundries. All draft articles are removed, because I do not want them on the live site, and the site changes are committed to the public repository. In addition, as described above, a private back up is also made. File and Directory Structure ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here's a slightly truncated view of the directory layout for the complete site creation and maintenance process. .. code-block:: bash . ├── build │   ├── .git (Back up repo) │   ├── cache │   ├── configs │   │   ├── __init__.py │   │   ├── deploy_settings.py │   │   └── local_settings.py │   │    │   └── content │      ├── extras │      ├── pages │      └── [...] ├── deploy │   ├── .git (user.io repo) │   ├── [...] │   └── theme │   ├── css │   ├── fonts │   └── js └── local    ├── [...] └── theme ├── css ├── fonts └── js build ^^^^^ This directory is where any editing and configuration takes place. This is backed up by the :bash:`fab backup` target to this repo. Following are some important points about the sub-directories: configs Herein is contained a :bash:`__init__.py` file - this is here to enable *fab* to import a :bash:`local_settings.py` file for setting a few useful environmental variables. A :bash:`deploy_settings.py` file also lives in here for when the :bash:`fab publish` target is run. content This is where the site articles are stored and edited - primarily as reStructuredText documents. local ^^^^^ The local directory is where the site is published, via the :bash:`fab serve` target, and evaluated locally. deploy ^^^^^^ The deploy directory is itself a git repo and is where the site is published, using the :bash:`fab publish` fabric command, prior to committing and pushing into the github.io repo etc. As mentioned before, all drafts are deleted prior to the git commit. The Fabric File ~~~~~~~~~~~~~~~ Below is a copy of my current fabric script. It has turned into a bit of a monster but I am quite happy that there is nothing in there that does not get at least some regular use. The targets (apologies, but I am in the habit of calling the defs below targets from my days spent using `Apache Ant`_) I mostly use are :bash:`serve`, :bash:`backup` and :bash:`publish`. You may notice that both the targets concerned with committing code to git repos support optional commit messages, something I thought was quite a nice touch. .. code-block:: python import time from fabric.api import local, settings, env, cd import fabric.contrib.project as project import os # There must be at least a local_settings.py file in the configs directory. import configs.local_settings as conf # Local path configuration (can be absolute or relative to fabfile) - they are # all added to env for convenience & abuse. # Repo settings - from local_settings.py env.git_url = conf.URL1 env.git_backup_url = conf.URL2 env.host_repo = conf.PUBLISH_REPO env.build_repo = conf.BUILD_REPO env.host_backup_repo = conf.PUBLISH_BACKUP_REPO # Local file paths env.config_path = './configs' env.local_settings_file = 'local_settings.py' env.deploy_settings_file = 'deploy_settings.py' env.project_path = '../build' env.deploy_path = '../deploy' env.local_path = '../local' env.content_path = './content' def clean(): """ Cleans and recreates local copy directory """ if os.path.isdir(DEPLOY_PATH): local('rm -rf {local_path}'.format(**env)) local('mkdir {local_path}'.format(**env)) def kill(): """ Kills errant SimpleHTTPServer processes - always outputs an error but works nevertheless """ with settings(warn_only=True): local('ps aux | grep SimpleHTTPServer | awk \'{print $2}\' | xargs kill -9') def build(): """ Runs local build with debug output - pushes out content to local copy directory, using local settings """ local('pelican -D {content_path} -o {local_path} -s {config_path}/{local_settings_file}'.format(**env)) def backup(commit_msg=False): """ Backs up this local copy to bitbucket with an optional " delimited commit message. """ with lcd(env.project_path): if commit_msg is False: command = 'git commit -S -m "Back up {0}"' else: command = 'git commit -S -m "Back up {0} - {1}"' local('git add -A .'.format(**env)) local(command.format( time.strftime("%d %b %Y %H:%M:%S", time.localtime()), commit_msg, **env ) ) local('git push {git_backup_url}/{build_repo} master'.format(**env)) def rebuild(): """ Cleans and pushes content to local copy directory, running clean() and build() """ clean() build() def remove_drafts(): """ Removes drafts from deploy_path/drafts as we don't want publicly accessible """ with lcd(env.deploy_path): local('rm -fr drafts'.format(**env)) def regenerate(): """ Regenerates site content """ local('pelican -r -s {local_settings_file}'.format(**env)) def serve(): """ Serves content from local copy directory, calling clean() and build() beforehand """ rebuild() with lcd(env.local_path): local('python -m SimpleHTTPServer'.format(**env)) def pub_deploy(): """ Prepares for publishing - runs non debug build to deployment path output """ with lcd(env.deploy_path): local('rm -fr *'.format(**env)); local('pelican {content_path} -o {deploy_path} -s {config_path}/{deploy_settings_file}'.format(**env)) def publish(commit_msg=False): """ Makes a build over to the deploy path, using deployment settings, cleaning up beforehand with pub_deploy() with an optional " delimited commit message. """ with lcd(env.deploy_path): if commit_msg is False: command = 'git commit -S -m "Publication {0}"' else: command = 'git commit -S -m "Publication {0} - {1}"' # Publish to deployment directory pub_deploy() # Remove draft htmls - don't want them in a public repo remove_drafts() # Git add and commit with lcd(env.deploy_path): local('git add -A .'.format(**env)) local(command.format( time.strftime("%d %b %Y %H:%M:%S", time.localtime()), commit_msg, **env ) ) local('git push {git_url}/{host_repo} master'.format(**env)) local('git push {git_backup_url}/{host_backup_repo} master'.format(**env)) So there you have it, not quite as brief as I intended... feel free to pass comment, suggest improvements etc. .. _`Pelican`: http://blog.getpelican.com/ .. _`make tool`: http://www.gnu.org/software/make/ .. _`Python`: https://www.python.org/ .. _`Fabric`: http://www.fabfile.org/ .. _`keybase.io identity`: https://keybase.io/chrisramsay .. _`reStructuredText`: http://docutils.sourceforge.net/rst.html .. _`Sublime Text 3 editor`: http://www.sublimetext.com/3 .. _`SimpleHTTPServer`: https://docs.python.org/2/library/simplehttpserver.html .. _`Apache Ant`: http://ant.apache.org/