Connor Lydon

Building A Developer Blog: From Start to Finish

I often visit other people’s blogs and wonder how they built them. While it may be a simple topic, if have asked the question, then someone else has as well.

TOC:

  1. Making the Site
  2. Hosting the Site

catch the conversation on X

Making the Content

There’s a lot of cool web development you can do on the web, but my goal was to have a simple blog. I wanted to have a portfolio section and a posts/projects section. While I could do this with some Javascript framework for me it was: overly complex, heavyweight, and not something I wanted to spend my Javascript hours on.

I decided to use a static site generator called hugo. It takes a directory of markdown files and renders them into a static site that is navigable. Yeah markdown is clunky at first, but really nice once you get used to the formatting.

There are a lot of options, but this was the most reccomended one. Analysis paralysis is pretty easy to fall into, but the easiest thing to do is to just pick a relatively popular one and switch if you find limitations.

After following the hugo quickstart, I picked a basic theme hugo-bearcub. The content/ directory holds all the articles that you write, _index.md in each directory will be the default page. When visiting lydon.fyi/projects/ the content/projects/_index.md file will render. To create a new article you use hugo new content blog/_index.md (or any other file name you’d like) on the command line. Rendering the site in development mode is done with hugo server --buildDrafts which will output a localhost:1313 address for you to see live updates.

Images and other linkable items are in the static/ directory.

To enable html elements in the ‘About’ section I enabled unsafe markdown rendering in the hugo config file hugo.toml. I also wanted newlines to be normal, so I enabed hardWraps. Here is what the hugo config snippet looks like, append this to the end of that file. As far as I understand, unsafe is disabling proper markdown rendering.

Unsafe rendering helps display images in any resolution I want instead of just using the existing resolution.

[markup]
    [markup.goldmark]
        [markup.goldmark.renderer]
            unsafe = true
            hardWraps = true

I can use in the +++ (or ---) block.

Here is a workflow for creating a new article in the blog and also creating a new article there.

After initializing a new hugo project

hugo new content blog/post.md

Fill in the content in the article. Mark the draft false

hugo --gc --minify to build the static into the public/ dir.

Great! now you have your static content in the public/ directory. Now lets host it

Hosting the Site

I’ve since abandoned this method. While a good proof of concept, using git is much more convenient and much less error prone.


To host the site I use a rented Linux server from Linode. Really, any VM or SSH-enabled server will do. To make your life easier, make sure it has an IPv4 address. IPv6 works as well, but IPv4 is easier to learn with.

With this server I set up the Caddy web server. It is a reverse-proxy/web-server like NGINX, but it is much simpler and will manage the HTTPS certs by itself. NGINX will work here too, but you will need to set up certbot to keep your SSL/TLS certs up to date. If you don’t have these certs, it will go through http and most browsers will disallow access. Follow the quick start guide and then you can manage incoming requests for your site like https://lydon.fyi to serve from a directory of files on your remote machine.

Now, how do you get files to your remote machine"?
I evaluated 3 ways to do this:

  1. Use a Docker image from a registry.
  2. Use Git repo webhooks.
  3. Sync files using Rsync to the target directory.

Evaluations for each:

  1. This is great, however I think its a bit complex for a simple static site. For a project that runs as a service, this would be how I would do it
  2. This is another great option, it could work for static sites or for live services. I could use this with what I chose if I did something more complex.
  3. I ended up picking the third option. This was the most straight forward and I was able to do it with just my local machine and my remote server.

The deployment method I chose has a few moving parts. Let me explain.
My local device would run the following script to send the files to a tmp/ directory on the server.

deploy.bash

# build site
rm -rf public/
echo "ensure all posts are drafts=false"
hugo --gc --minify

# make tmp dir
ssh my_ssh_alias 'mkdir -p ~/tmp/lydonfyi-static'
rsync  -avz --delete -mkpath public my_ssh_alias:~/tmp/lydonfyi-static

It builds the static site, builds the remote directory, and then uses rsync to send the files to the server.

This Caddy serves out of the /var/www directory, so I need to move the files from ~/tmp to /var/www/<dir>. Also, I need to allow the caddy group permissions on the files I am serving. so that means adding it as a group owner and adding correct read permissions on the file. chown -RLv caddy:caddy <my_dir> and then adding read permissions chmod -R 740 <my_dir>.

I can’t run this via an automated script because I need to use sudo to move the files into the privelaged directory. There are ways to do this in a remote script by running the command with ssh -t my_ssh_alias "sudo <command>. This will allow you, a remote user, to run screen based programs on the remote server without opening an actual ssh session. This method is great, BUT if you want to allow unprivelaged users to host content on your server this isn’t a good idea. By giving many people sudo power, they can destroy your server. Also, you’d need to enter your ssh password everytime you log in. I have an alternative…

Instead of using entering my sudo password everytime, I decided to write a small system service to move files from my user’s folder (~/tmp) to a priveleged root folder (/var/www). This also opens up the possibility of allowing other people to serve content on your server without getting sudo priveleges. I wrote the following script that I run as a systemctl service. Systemctl is a way to run scripts on boot and keep them running. Think of them as basic basic docker containers. The script will listen to a certain directory, move it to a new directory, and apply permissions to that directory once moved. Here it is below.

watch-directory.bash

#!/bin/bash

# Initialize default values for variables
SOURCE_DIR=""
DESTINATION_DIR=""
NEW_OWNER=""
NEW_GROUP=""

# Function to show usage
usage() {
    echo 'Usage: $0 -s "<source_dir>" -d "<destination_dir>" -o "<owner>" -g "<group>"'
    echo 'owner and group are used to change the owner to allow for the load balancer to work properly.'
    exit 1
}

# Parse command-line options
while getopts "s:d:o:g:" opt; do
    case $opt in
        s) SOURCE_DIR=$OPTARG ;;
        d) DESTINATION_DIR=$OPTARG ;;
        o) NEW_OWNER=$OPTARG;;
        g) NEW_GROUP=$OPTARG;;
        *) usage ;;
    esac
done

# Check if the directories are set
if [ -z "$SOURCE_DIR" ] || [ -z "$DESTINATION_DIR" ] || [ -z "$NEW_OWNER" ] || [ -z "$NEW_GROUP" ]; then
    usage
fi

# Add `/` to the end if it doesn't exist
SOURCE_DIR="${SOURCE_DIR%/}/"
DESTINATION_DIR="${DESTINATION_DIR%/}/"

# Make directories if they don't exist
echo "Creating directories if they don't exist: ${SOURCE_DIR} & ${DESTINATION_DIR}"
mkdir -p ${SOURCE_DIR}
mkdir -p ${DESTINATION_DIR}

# Clean source directory
echo "Cleaning source directory ${SOURCE_DIR}*"
rm -rf  ${SOURCE_DIR}*

# Monitor for file changes
inotifywait -m "$SOURCE_DIR" -e create -e moved_to |
    while read path action file; do
        echo "The file '$file' appeared in directory '$path' via '$action>"
	echo "Waiting 10s for file transfers, DOWNTIME"
        sleep 10 # let files transfer
	echo "Done waiting"
	rm -rf ${DESTINATION_DIR}*
	# Move the file to the destination directory
        mv "${path}${file}" "$DESTINATION_DIR"
	chown -RLv ${NEW_OWNER}:${NEW_GROUP} ${DESTINATION_DIR}
	chmod -R 770 ${DESTINATION_DIR}*
    	rm -rf ${SOURCE_DIR}*
    done

I included a sleep because it may take some time to transfer files into the source directory on the wire. To avoid this you can have another tmp directory that you transfer to over the wire, then the original soruce directory will be on the same disk so the transfer will be fast.

And the systemctl ExecStart command looks like this:
/usr/bin/watch-directory.bash -s "/home/user/tmp/lydonfyi-static/" -d "/var/www/html/" -o "caddy" -g "caddy"


If I did this again
I wouldn’t do the whole watch-directory.bash script. Now I can easily allow hosting for other projects and other people who’d like to host static files. I’m going to do another project on how to set up a static file hosting for developers with ssh. This was the typical example of a programmer working too long on a simple problem.


thanks for reading

catch the conversation on X