Tool Obsession: MKDocs

I wanted to take a bit to talk about my latest and greatest software tool I’m using to get stuff done: MKDocs. If you know me personally, you’ll be well aware that I don’t take software tools lightly. I love dabbling with new stuff, trying to squeeze out all the functionality I possibly can in order to push things to their limits.

If you’d like to skip to me just talking about MKDocs already, feel free to jump ahead!

Motivation

As I’ve been very focused on my writing projects lately (which you can hear more about over on my Twitter), I’ve been trying to optimize my bag of writing tools. During National Novel Writing Month 2017, I picked up Scrivener to take care of business. It was great, though I found it tough to just pick up and play with on such a tight timeline. It allowed me to write snippets and re-organize them, though, which was a tremendous help. I felt like, for the most part, Scrivener was really excellent at staying out of my way and letting me work.

However, I am impossible to satisfy, so I kept looking around after NaNoWriMo was done. What I really wanted was a single-user wiki to manage all my crazy fantasy world details. I had been using WikidPad for my tabletop notes, and that was working just fine, but something about it felt archaic. I found those notes difficult to back up and transfer because some of the files were more than just text. Don’t get me started on the lack of phone-based options for updating those notes!

Of course, rejecting WikidPad on the basis that it “feels archaic” is pretty silly, considering I now use plaintext for everything, but I digress.

I tried DokuWiki, which was nice for customization options. Unfortunately, it requires PHP, and I didn’t want to install PHP on this server. I could edit DokuWiki pages from my phone, but the web interface was pretty lackluster.

Enter MKDocs

Due to my obsession with Markdown, I searched for a Markdown-based wiki solution. What popped up was MKDocs: a Markdown-based static wiki generator written in Python.

I expect folks will get limited use out of this due to one sticky little keyword there: static. This means that you can’t edit pages without being able to push files to the webserver, which isn’t a tremendous obstacle for me (I update my whole website by pushing with git), but is a bit too tech-y for many. For those who want to follow along, I’ll write a bit about how to get that set up in the setting up MKDocs deployment section.

The chief draw of MKDocs is that you are given freedom to just write in Markdown. There is no odd system for linking between pages; you just make a standard markdown link to another page and MKDocs figures it out. This means that your links are clickable both within the editor and on the generated webpage. This feature is key, I find, because you’ll no doubt find yourself wanting to quickly hop into a linked page to edit. MKDocs tries to stay out of your way.

This seems so basic, but it is so great that MKDocs is just a regular folder structure filled with regular files. I have had no shortage of headaches migrating my notes between all these different tools, so just having a sane collection of files feels extremely freeing. If I ever get around to making my own writing tool, you can bet it will work off the same philosophy and accept the same folder structure.

When you want to preview your changes, MKDocs does its best to accommodate smoothly. It’s written in Python, so if you’re a developer, you’re likely to already have the tools you need to run it! Install it via pip install mkdocs and you’re ready to roll. mkdocs new wiki will make a new wiki in the “wiki” folder, and if you cd into that folder and enter mkdocs serve, mkdocs will give you a nice preview site at localhost:8000 that auto-refreshes when you change a page. Super painless.

Did I mention that MKDocs is gorgeous?

I use the “Material for MKDocs” style for my wikis because I absolutely love the clean look it provides. You can grab that theme for yourself by installing it with pip install mkdocs-material. The theme’s page has instructions on how to set it up.

“Andrew,” I hear you ask, “you’ve mentioned phone editing a few times now; how does MKDocs address this?” Good question, reader! It absolutely doesn’t, but, Android has some great apps for editing Markdown and using Git. I use Markor (FOSS) for editing and Pocket Git (paid) myself.

Pocket Git makes it very easy to set up multiple remotes for your Git projects. personally, I store my drafts in a private GitHub repo, and have a separate “deploy” remote which lives on my webserver. But we’ll talk about that in more detail below! An additional feature I like, and this is more an endorsement of Android, is that you can use Pocket Git to browse the files in your Git repo and set Markor as the default app for opening .md files. You practically never have to “leave” the Pocket Git app if you use it this way. What a convenient workflow!

How to Deploy MKDocs with Git

The second half of this article is a tutorial on setting up deployment with Git. If you’re not interested in the nitty gritty, feel free to skip to the end!

Alright, so you’ve gotten this far and you’ve decided to try it out. Or maybe you just want to see the details of actually putting MKDocs to use. Either way, we’ll take a look at my easy-to-use setup for automatically deploying an MKDocs wiki with Git.

This setup requires you to have SSH access to your webserver. I also use Ubuntu on my webserver, so your mileage may vary with this. I don’t receive anything for saying that I host with DigitalOcean, but they’ve been nothing but reliable and painless for me, so I’m happy to recommend them!

Setting Up the Server

In general, I like to set up my Git repos with a “dev/” folder which holds some scripts to get up and running on a new machine. For my MKDocs repo, I have a server_init.sh and an init.sh for server and client, respectively.

#!/bin/bash -l

# Setup for the webserver, assuming there is NO repo endpoint setup yet
GIT_REPO=$HOME/notes.git
SITE_ROOT=/var/www/andrewdys.art
NOTES_SUBDIR=$SITE_ROOT/public_html/notes

echo --------------------------------------------
echo Creating git repo and site dir...
mkdir $NOTES_SUBDIR
mkdir $GIT_REPO
pushd .
cd $GIT_REPO
git --bare init
popd
echo Done.

# Install mkdocs basics needed to build the site
echo --------------------------------------------
echo Installing MkDocs and plugins...
INITSCRIPT=./init.sh
chmod a+x $INITSCRIPT
$INITSCRIPT
echo Done.

# Set up the hooks folder and symlink
echo --------------------------------------------
echo Setting up server hooks...
HOOKS_FOLDER=$GIT_REPO.hooks
mkdir $HOOKS_FOLDER
rm -rf $GIT_REPO/hooks
ln -s $HOOKS_FOLDER $GIT_REPO/hooks

# Ensure hooks are made executable
cp -r hooks/* $HOOKS_FOLDER
find $HOOKS_FOLDER -type f -exec chmod +x {} \; \
   -exec stat --printf='%n - %U:%G\n' {} \;
echo Done.

#!/bin/bash -l

# Setup for new environments which wish to use the mkdocs
# dev server
pip install -r requirements.txt

init.sh uses a requirements file to tell pip what modules to install.

mkdocs

I have a couple of plugins/extensions in my requirements.txt which I haven’t listed here. All you really need is mkdocs, but you may like to tweak your installation a bit. Obviously, if you don’t want to use extensions, you can alter your init.sh to just pip install mkdocs.

The last two scripts needed are post-receive and buildscript.sh.

#!/bin/bash -l

GIT_REPO=$HOME/notes.git
TMP_GIT_CLONE=$HOME/tmp/notes
HOOKS_PATH=$HOME/notes.git.hooks

# Clone the repo to a temporary working directory
echo --------------------------------------------
echo Cloning MkDocs site...
git clone $GIT_REPO $TMP_GIT_CLONE
echo Done.

# Copy the hooks to the HOOKS_PATH
cp -r $TMP_GIT_CLONE/dev/hooks/* $HOOKS_PATH
find $HOOKS_PATH -type f -exec chmod +x {} \; \
   -exec stat --printf='%n - %U:%G\n' {} \;

# Call the build script from the Git repo
while read oldrev newrev refname
do
   pushd .
   branch=$(git rev-parse --symbolic --abbrev-ref $refname)

   # Checkout the branch
   pwd
   # For some reason, this is necessary?
   unset GIT_DIR
   (cd $TMP_GIT_CLONE && git checkout $branch)

   BUILDSCRIPT=$TMP_GIT_CLONE/dev/buildscript.sh
   if test -f "$BUILDSCRIPT"; then
      echo --------------------------------------------
      echo "$BUILDSCRIPT exists; executing..."
      cd $TMP_GIT_CLONE
      chmod a+x $BUILDSCRIPT
      $BUILDSCRIPT
      echo Done.
   fi

   popd
done

# Cleanup
rm -Rf $TMP_GIT_CLONE
exit

#!/bin/bash -l

SITE_ROOT=/var/www/andrewdys.art
NOTES_SUBDIR=$SITE_ROOT/public_html/notes

# Perform the MkDocs build
echo --------------------------------------------
echo Building MkDocs wikis...
pushd .
cd wikis
for d in */ ; do
   pushd .
   cd "$d"
   mkdocs build --clean
   popd
done
popd
echo Done.

echo --------------------------------------------
echo Cleaning existing wiki...
rm -rf $NOTES_SUBDIR/*
echo Done.

echo --------------------------------------------
echo Copying site build over...
pushd .
cd wikis
for d in */ ; do
   pushd .
   cd "$d"
   mkdir $NOTES_SUBDIR/$d
   cp -r site/* $NOTES_SUBDIR/$d
   popd
done
popd
echo Done.

echo --------------------------------------------
echo Copying top-level index...
cp dev/index.html $NOTES_SUBDIR
echo Done.

What Does it Do?

Okay! So what have we actually done with these scripts? Running server_init.sh will do several things for you. It handles:

• Setting up a bare Git repo
• Ensuring the destination subdirectory exists (NOTES_SUBDIR)
• Deleting the hooks folder
• Creating an external hooks folder for us to symlink to
• Creating the hooks symlink
• Copying the hooks from dev/hooks into the external hooks folder
• Ensuring the hooks are executable

At the end of this all, your server should be ready to receive a push, at which point it will run the post-receive script. What that script does is fairly complicated-looking, but it can be boiled down to this: after a push is received, the script will checkout any branch that was pushed and run the buildscript.sh contained within. This lets you have a different build procedure for different branches—for example, you may wish to have a drafts branch that builds your wiki and puts it in a private, password-protected section of your site.

The buildscript.sh will build every wiki contained within the “wikis” folder in a temporary staging folder. I like to have a bunch of separate wikis for my different projects so there’s no cross-contamination, and this setup makes it painless to add new wikis whenever I feel like it. Then, it cleans out the NOTES_SUBDIR (to get rid of any pages you may have deleted), and copies all the new files from the staging area to the NOTES_SUBDIR. Finally, it copies over a file from the dev folder to serve as an index for your NOTES_SUBDIR. Mine redirects to one of my wikis, like so:

<meta http-equiv="Refresh" content="0;url=general/">

Just like that, your wiki is published.

Setting Up the Local Git Repo to Deploy

Even though your server is ready to receive the push, that doesn’t mean you’re ready to send one. There’s still one more crucial step to set up your local Git repo. You have to add the webserver Git repo as another remote. The form of the remote url will be name@domain:GIT_REPO, where name is the username you use to administer your server, domain is the domain name pointed to the server, and GIT_REPO is the directory where your repo is stored on the server. Mine is andrew@dysart.dev:~/notes.git.

You’ll add this remote by typing git remote add deploy name@domain:GIT_REPO from your local Git repo. From then on, you can git push as normal, and when you want to push your changes to the live server, you’ll git push deploy master instead (or whichever branch name you choose). It’s that easy!

In Conclusion

I hope you’ve enjoyed this overlong discussion of why I like MKDocs and how to put it to use yourself. If you have any questions, comments, or corrections, feel free to contact me in the comments below or through one of my many contact info icons on the about page!

Interested in what makes me tick? Do you like odd programming journals, or perhaps epic fantasy? Maybe some tabletop? Consider subscribing to my mailing list to get notified about my new postings!

Cheers,
Andrew