In this blog post, I will walk you through my journey of establishing a compact k3s (Kubernetes) cluster using Raspberry Pis. From assembling the initial hardware to configuring the software, you’ll see how I brought my Raspberry Pi Kubernetes cluster to life. We’ll begin with an overview of the hardware arrangement, followed by an in-depth look at the software setup process.
I’m going to leave you to the task of getting the RPis out of their boxes and connecting them up to the PoE switch, it’s pretty much plug and play.
First, let’s go over the process of installing Ubuntu on our Raspberry Pis using the Raspberry Pi Imager tool.
Scroll through the list and choose the version of Ubuntu you’d like to install (In this case, it’s recommended to choose Ubuntu Server 22.04 LTS (64 bit) for Raspberry Pi Zero/2W/4/400).
In the advanced options menu we can set the host name of our Pi which otherwise defaults to raspberrypi. This is also a good time to enable SSH and set a username and password to login to the Pi. Once happy with the settings click “SAVE”
The Imager tool will now write the Ubuntu image to the SD card, and once it’s finished, you can insert the SD cards into your Raspberry Pis connect them to your switch and power them on.
Once the Raspberry Pi’s have started up, you may need to give them a little time to finish their first boot process, we can find out their IP addresses which we’ll need to ssh
onto them and continue the cluster setup.
Couple of ways you can do this, firstly you could log onto your router and have a look at the network. Most routers will show you the IP address next to devices hostname.
If that’s not an option you can ssh
onto the Pi using the username and password you picked in the imaging step.
Establishing an SSH connection on a Mac is incredibly simple. Open your Terminal and run the following command:
ssh username@hostname-or-ip
To SSH into your Raspberry Pi on a Windows computer, you can use the PuTTY program. Follow these steps:
Once we’ve ssh’d onto our Pi we can run ifconfig
which will display the network adapters and associated IPs. Assuming you are using ethernet to connect the IP address will be listed under eth0
on the second line next to inet
and will look similar to inet 192.168.178.x netmask 255.255.255.0
.
Once you have Ubuntu up and running on your Raspberry Pis, if you forgot to set the hostnames or don’t like the one you picked you can change them in Ubuntu with the hostnamectl
command. I’ve gone with an Avengers theme; capt
(master node), thor
and hulk
(worker nodes).
Log in to each Raspberry Pi via SSH using the username and password you picked in the imaging step.
ssh username@hostname-or-ip
Using the hostnamectl command give the Pi a new name.
sudo hostnamectl set-hostname new-name
Next edit the /etc/hosts file, you could use nano or vi to do this and replace any existing occurance of the existing pi name with your new hostname
sudo nano /etc/hosts
Save the changes and reboot the pi
sudo reboot
To avoid having the PoE HATs fans constantly spinning, it’s essential (unless you like the noise) to configure their settings. If you’re not using PoE HATs on your Raspberry Pis, feel free to skip this step.
On each of the raspberry pi’s you’ll need to first ssh onto them and then edit the /boot/firmware/usercfg.txt
file, you may see other articles that direct you to /boot/firmware/config.txt
but if you are running Ubuntu it’s the usercfg.txt
file you need to update.
sudo nano /boot/firmware/usercfg.txt
Add the following to the end of the file and save and exit ctl+X
then Y
and enter.
# PoE Hat Fan Speeds
dtparam=poe_fan_temp0=50000
dtparam=poe_fan_temp1=60000
dtparam=poe_fan_temp2=70000
dtparam=poe_fan_temp3=80000
You’ll need to reboot each Pi after the change, but when they restart you should be greeted with the sound of silence, unless the Pi is under load.
To Ansible, or not to Ansible: that is the question
Manual install vs. Ansible - 2023
As we travel further on our kubernetes homelab journey and look to install K3s on the Rasberry Pi’s, there are two options to choose from: But fear not, intrepid homelab installer! We’ve got you back, in this section we will delve into the pro and cons of each approach, ensuring you can make an informed choice for your Raspberry Pi Kubernetes journey.
For those comfortable with getting down and dirty, the manual approach to K3s installation allows you to dive into the nitty-gritty of setting up a cluster. K3s provides an easy-to-use installation script that sets up the necessary services on systemd or OpenRC-based systems.
Armed with the script from https://get.k3s.io, you can bring your Raspberry Pi Kubernetes cluster to life using the following command (remember to ssh onto your Pi first):
curl -sfL https://get.k3s.io | sh -
Is that it you ask? For getting your master node up and running it may well be (assuming you like all the defaults) once the script is finished you’ll have the following:
kubectl
, crictl
, ctr
, k3s-killall.sh
, and k3s-uninstall.sh
will be at your disposal.But wait, there’s more! At least there is if you want to setup a 3 node cluster like me. With the master node sorted, we’ll need to ssh onto the worker(agent) nodes and recruit them to the cause.
curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -
You’ll need to change the K3S_URL
to match your master node (server) for me that’s https://capt:443
using the hostname we set earlier. The value for K3S_TOKEN
can be found lurking at /var/lib/rancher/k3s/server/node-token
on your master node (server).
Once your agents/worker nodes are signed up with the master/server you have a running K3s cluster. Steps to connect to it from your computer can be found after the Ansible instructions, see you there.
If you’re not a fan of manual installation, you might want to consider using Ansible for setting up your Raspberry Pi Kubernetes cluster. Ansible offers an automated way of installing our cluster using a playbook, and gives us a nice repeatable way to deploy our cluster.
By using Ansible for your K3s deployment, you can easily reset your cluster to its initial state whenever you feel like trying new configurations or applying upgrades. With this automated approach, you can sit back and relax while Ansible handles the complex tasks for you.
On Mac this is pretty simple if you have Homebrew installed
brew install ansible
If you don’t have it installed visit brew.sh and follow the instructions.
If you are on Windows I suggest you take a look at the ansible docs it’s pretty simple but involves installing Python 3 first than then using it’s package manager pip to install Ansible.
Head over to the GitHub repo for K3s and ansible and either clone or fork it. If you fork it a copy will be created in your account which you can then clone.
Navigate to a folder that you want to clone the code to I have repos
folder in my user directory I use, when you clone use HTTPS unless you have an SSH key setup on your GitHub account. This will create a new folder under repos
called k3s-ansible
, switch to this folder and continue.
cd repos/
git clone https://github.com/k3s-io/k3s-ansible.git
cd k3s-ansible
Create a new directory based on the sample
directory within the inventory
directory. I named mine pi-cluster.
k3s-ansible
└── inventory
└── pi-cluster
│ ├── hosts.ini
│ └── group_vars
│ └── all.yaml
└── sample
├── hosts.ini
└── group_vars
└── all.yaml
Edit the inventory/pi-cluster/hosts.ini
to match you IP addresses. This file communicates the K3s masters and nodes to Ansible during the K3s installation process, mine looks like:
[master]
192.168.178.50
[node]
192.168.178.51
192.168.178.52
[k3s_cluster:children]
master
node
Modify the inventory/pi-cluster/group_vars/all.yml
file and update the ansible_user to the one you set earlier in my case it’s mike
.
---
k3s_version: v1.28.2+k3s1
ansible_user: mike
systemd_dir: /etc/systemd/system
master_ip: ""
extra_server_args: ""
extra_agent_args: ""
With the config done we can kick off provisioning the cluster with the following command (-K prompts you for the Pi sudo passsword):
ansible-playbook site.yml -i inventory/pi-cluster/hosts.ini -K
Once the cluster has been installed, you’ll be able to move onto the next section and connect to it using kubectl
.
In the event that you need to uninstall the cluster you can use Ansible for that as well. Simply run the following command.
ansible-playbook reset.yml -i inventory/pi-cluster/hosts.ini -K
You can also use Ansible to shutdown the cluster using.
ansible all -i inventory/pi-cluster/hosts.ini -a "shutdown now" -b -K
So, which path will you choose for your Raspberry Pi Kubernetes adventure? Whether you prefer to go hands-on with K3s installation scripts or let Ansible work its magic, the choice is yours. Armed with knowledge and guidance, you’re now ready to dive into the exciting world of Raspberry Pi Kubernetes clusters. And always remember, when it comes to choosing between Manual and Ansible, there’s no wrong answer – it’s all about finding the approach that fits you best.
Before we can use the K3s cluster we are going to need to install kubectl
which provides a way of administering our cluster from our computers.
Homebrew to the rescue again, installing on Mac with homebrew is as simple as.
brew install kubectl
We can test kubectl is up-to-date with the following command.
kubectl version --client
We can use Chocolatey or winget to install kubectl.
choco install kubernetes-cli
winget install -e --id Kubernetes.kubectl
We can test kubectl is up-to-date with the following command.
kubectl version --client
Now we have the installation of kubectl
on our computer we can finally connect, well we can almost connect. First we need to ssh
onto our master node and run the following command.
sudo cat /etc/rancher/k3s/k3s.yaml > kubeconfig.txt
On Mac, we can use SCP to copy the config from the master node to our computer.
scp username@master-node:~/kubeconfig.txt ~/.kube/config
Once copied, we need to edit the file and change the 127.0.0.1 IP to the IP of the master node (e.g. 192.168.178.50).
sudo nano ~/.kube/config
After making the change, save and exit Ctrl+X
then Y
and Enter
. We are now ready to connect.
First lets setup the .kube directory to store the kubernetes config.
Navigate to your home directory and create the .kube
directory
cd ~
mkdir .kube
Change to the .kube
directory we have created and create a new config file
cd .kube
New-Item config -type file
Now that you’ve created the config file, use a file transfer tool like WinSCP to copy the kube config from the master node to your local .kube folder.
.kube
directory that you created earlier.kubeconfig.txt
file from the remote pane to the local .kube folder. After the transfer is complete, rename the file from kubeconfig.txt
to config
in the local pane.Once the file is copied and renamed, you’ll need to edit it and change the 127.0.0.1
IP to the IP address of your master node (e.g., 192.168.178.50
). You can use a text editor like Notepad++ to make the changes.
Save the changes and exit. We are now ready to connect.
With the kubectl setup we can now test that everything is configured properly. On your computer.
kubectl get nodes
Which should return something like.
NAME STATUS ROLES AGE VERSION
capt Ready control-plane,master 2d1h v1.28.2+k3s1
thor Ready <none> 2d1h v1.28.2+k3s1
hulk Ready <none> 2d1h v1.28.2+k3s1
K3s comes pre-packaged with Klipper LoadBalancer (KlipperLB) aka StandardLB, it’s a bit limiting, it’s fine for small scale clusters but if you wanted to put this into production it’s not a great choice. MetalLB is a common replacement which works well in homelab setups. For the purposes of this guide we are going to remove KlipperLB and replace with MetalLB.
MetalLB is a useful tool for Kubernetes clusters in bare metal or on-premises environments like our Raspberry Pi cluster, where automatic service exposure through external load balancers is not available, Kubernetes assumes that your cloud provided will provide this component. MetalLB assigns IP addresses on the local network to Kubernetes services, making them easily accessible within the network. MetalLB uses Layer 2 (ARP/NDP) or BGP mode to dynamically provide and announce IP addresses, ensuring efficient load balancing and better fault tolerance for your services.
A good understanding of your home network is essential to get MetalLB to work correctly. First we need to workout our network address space, on my router under the home network settings we can see the IPv4 address and subnet mask which when written in CIDR notation is 192.168.178.0/24
. The IP address of the router is 192.168.178.1
your network may vary but should be similar.
The key bits of information we need are what IP addresses are available in our network and which are free for us to allocate to MetalLB. I find a good tool for this is the Visual Subnet Calculator from Sargasso if you enter the IPv4 address into the network address box and set the mask bits to 24
and click update, it’ll let you know what the address should be 192.168.178.0
and you can check the netmask (select the option) against what your router is reporting.
Assuming everything matches we can now see our usable IPs which for us are 192.168.178.1-192.168.178.254
giving us a total of 254 IPs for the home network. We can see on the router that it is using 192.168.178.100-192.168.178.200
for DHCP (the IPs it hands out when a device joins the network). This means the pool of IPs we have available to MetalLB are:
I’m going to use 192.168.178.201-192.168.178.254
for MetalLB, which I’ll make a note of and come back to later.
We can’t have two Load Balancers so KlipperLB needs to be disabled, thankfully this is pretty easy to do.
Update the /etc/systemd/system/k3s.service
file around line 10 after ExecStart=/usr/local/bin/k3s server
you need to add --disable servicelb
flag as per below.
[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
After=network-online.target
[Service]
Type=notify
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s server --disable servicelb --data-dir /var/lib/ran>
KillMode=process
...
Ctrl+X
then Y
and Enter
. sudo systemctl daemon-reload && sudo systemctl restart k3s
Lets install MetalLB and get it assigning IP addresses for services.
We can apply a manifest file for MetalLB to install version 0.13.11 which was the latest at the time of this article.
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.11/config/manifests/metallb-native.yaml
Once that’s run we can confirm that MetalLB is running by checking the pods in the metallb-system
namespace.
kubectl get pods -n metallb-system
Which should return something similar to.
NAME READY STATUS RESTARTS AGE
controller-7d56b4f464-2bvdg 1/1 Running 2 (2d ago) 2d2h
speaker-nqk44 1/1 Running 2 (2d ago) 2d2h
speaker-6kshp 1/1 Running 2 (2d ago) 2d2h
speaker-clw8w 1/1 Running 4 (2d ago) 2d2h
The controller is responsible for allocating IP addresses and the speakers broadcast which node will respond to the request on that IP.
We need to tell MetalLB which IPs (which we identified eariler) it can use, to do this we’ll create a manifest file as per below.
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: default
namespace: metallb-system
spec:
addresses:
- 192.168.178.201-192.168.178.254
Save as a yaml file called metallb-ips.yaml
and apply it to the cluster.
kubectl apply -f metallb-ips.yaml
Once applied it’ll now be assigning external IPs on our network for any new Kubernetes Services.
Having an external IP is only half of the battle, next we need a way of linking the IP with the service that is running on a node. We are going to use the Layer 2 config option which respondes to ARP requests on our network to give the MAC address of a node to clients to route the request.
Lets create another manifest file called metallb-l2.yaml
with the below contents.
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: l2
namespace: metallb-system
Lets apply it to the cluster
kubectl apply -f metallb-l2.yaml
We now have everything we need to test our setup which we’ll do next.
To test MetalLB actually works I’m going to deploy Nginx and expose it with a LoadBalancer.
kubectl create deploy nginx --image nginx
kubectl expose deploy nginx --port 80 --type LoadBalancer
To find out the address that Nginx is being exposed on we can use kubectl to get more information about the service.
kubectl describe service nginx
Name: nginx
Namespace: default
Labels: app=nginx
Annotations: metallb.universe.tf/ip-allocated-from-pool: default
Selector: app=nginx
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.43.78.21
IPs: 10.43.78.21
LoadBalancer Ingress: 192.168.178.202
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 31390/TCP
Endpoints: 10.42.1.33:80
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal IPAllocated 6s metallb-controller Assigned IP ["192.168.178.202"]
Normal nodeAssigned 6s metallb-speaker announcing from node "hulk" with protocol "layer2"
We can see that the IP 192.168.178.202
as part of a IPAllocated
event from metallb-controller
and following that metallb-speaker
running on hulk
announced it.
We can can check if the output by visiting http://192.168.178.202
using a browser on our computer or simply use curl.
curl 192.168.178.202
Which should return the Nginx welcome page
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
There we have it, a K3s Raspberry Pi cluster with MetalLB configured and serving traffic on our local network. Hope you’ve enjoyed setting this up, and if you’ve gotten to the end of this post well done :)
]]>Are you eager to learn Kubernetes and wondering, “Is Kubernetes easy to learn?” or “How long will it take to learn Kubernetes?” You’re in the right place! We’ve designed a comprehensive 12-week study guide to help you master Kubernetes quickly and efficiently, and prepare you for the Certified Application Developer (CKAD) exam.
If you’re asking yourself, “How do I start learning Kubernetes?” or “What is the fastest way to learn Kubernetes?”, our guide offers a structured approach that covers essential topics, from container fundamentals and Kubernetes basics to advanced networking and service meshes. Designed for developers with experience in JavaScript and C#, this guide combines video lessons, practical exercises, and resources from popular platforms like A Cloud Guru and LinkedIn Learning, as well as free materials to enhance your understanding of Kubernetes.
By following this 12-week journey, you will not only develop a strong foundation in container orchestration and Kubernetes but also gain the confidence to take on the CKAD exam and boost your career in the world of cloud-native development. So, embark on this exciting learning journey and unlock the full potential of Kubernetes!
docker
) to run a simple container, such as Nginx or Redis, and access the application inside the container. Example: docker run -d -p 80:80 nginx
docker build
. Reference: Dockerizing a Node.js Web Appdocker start
, docker stop
, and docker rm
. Inspect the state of running containers using docker ps
and view logs using docker logs
.docker network
commands. Connect containers to the custom networks and experiment with container-to-container communication.kubectl
command-line tool.kubectl
. For example, run kubectl run my-nginx --image=nginx
to create a Deployment with an Nginx container. Practice scaling the Deployment using kubectl scale
.kubectl proxy
to interact with the Kubernetes API and explore the API objects using a REST client, such as Postman or curl. Reference: Access Clusters Using the Kubernetes APIkubectl
commands to inspect the state of your cluster resources, such as Nodes, Pods, Deployments, and Services. Practice using kubectl get
, kubectl describe
, and kubectl logs
to gather information about your resources.kubectl delete
to clean up your cluster and prevent unwanted resource consumption.kubectl
kubectl apply
to create them. Practice updating the application version, rolling back updates, and managing replicas. Reference: Deploymentskubectl delete
to clean up your cluster and prevent unwanted resource consumption.kubectl apply
to create them. Understand the differences between Deployments and StatefulSets, and observe how Pods are named and managed in a StatefulSet. Reference: StatefulSetskubectl apply
to create them. Observe the behavior of Jobs, such as their completion status and Pod management. Reference: Jobs Run to Completionkubectl apply
to create them. Understand the relationship between CronJobs and Jobs, and observe how CronJobs manage scheduling and execution of Jobs. Reference: CronJobskubectl delete
to clean up your cluster and prevent unwanted resource consumption.kubectl delete
to clean up your cluster and prevent unwanted resource consumption.By the end of this 12-week study guide, you will have developed a strong foundation in Kubernetes and be prepared for the Certified Kubernetes Application Developer (CKAD) exam. Remember to practice regularly and review any areas where you may need improvement. Good luck on your journey to becoming a Kubernetes expert!
]]>Disclaimer: This content was generated using ChatGPT, an AI language model, and has not yet been verified by an expert. While we strive to provide accurate information, we cannot guarantee the accuracy, completeness, or suitability of this content for any particular purpose. Please consult with a qualified professional or conduct your own research before acting on the information provided in this study guide.
At the start of the talk Brian signposted a previous talk from Ignite that went into a bit more detail (being 47 mins long vs this 30 min session) ARM Ignite 19.
In this post I’ll be giving a brief overview of the topics he discussed with links to further reading.
Deployment scripts let you perform custom steps with Powershell or Bash scripts that you can’t do in ARM templates. Brian said the intention was for them to cover the last mile in a deployment. They support both Azure Powershell and AZ CLI. Now in public preview for you to try out.
read more about ARM deployment scripts
Arm-ttk is a set of tests that will check your ARM template code or sets of templates for best practices. You can extend it and add your own tests, in all it looks like a good addition to a developers toolkit and I can see it being useful in pipelines to spot potential errors before deployments.
Currently in preview, what-if lets you see what changes your ARM templates will trigger should you deploy your templates. Sounds a lot like Terraform Plan, with the advantage that rather than being run against a state file, it is looking at the current resources in Azure. Brian mentioned that you’ll be able to output the response of what-if to a json file, which opens up the possibility of running Unit Tests against it using Pester or your favourite testing framework.
read more about ARM template what-if operation
ARM template specs are new and allow you to bundle together ARM templates and their supporting files into another ARM resource that can be re-used within an ARM template private registry and shared with others. If this sounds a lot like Blueprints you’d be right, Brian said the intention long-term was to migrate people from blueprints to template specs. Details on this are pretty light at the moment, expect documentation to follow in the coming weeks.
The final bit of news was pretty big, there are plans afoot to revise the ARM language as we know it. After much feedback on how using JSON as a programming language really wasn’t cutting it, when you compared with HashiCorp’s HCL as used in Terraform. The most common complaints being about modularising code for maximum re-use and struggling working with multiple files.
To address this the new language will have first class support for modularisation and working with multiple files. It will have transparent abstraction in Azure, they don’t want it to get too far away from a native experience. JSON is being ditched, and no YAML isn’t going to replace it. The new language won’t be copying Terraform and using a state file, being a native solution it doesn’t need it. The what-if operation will continue to serve that purpose.
The initial 0.1 release is expected in 2-3 months so keep your eye out for it. Short term this new language will exist alongside legacy ARM, it’ll likely be many years before they drop support for the JSON format. All in all, exciting times for ARM template developers.
]]>For the guys that can’t afford these prices, they often make do with the likes of WKhtmlToPDF or the many wrappers for it like Rotativa in .Net or Java WKhtmlToPDF Wrapper. But could XSL-FO be a better option?
When should we be using XSL-FO over other tools? This is a good question to ask yourself, if you are doing a simple 1 page document with some variables, then XSL-FO is probably a sledge hammer to crack a nut. But anything more complex than that and it’s still the best solution in my book for data driven transaction documents. I’ve used it for some of the following:
So assuming your still with me and thinking of giving XSL-FO a go for your next documents project, how is the eco-system doing? Well it’s alive and kicking, the main vendors are still updating and supporting their XSL-FO rendering engines. With the three main players all having an update in the last 12 months. In fact Antenna House updated their formatter only 2 days ago.
Last updated in October 2018 with the release of XEP 4.28 prices currently start at $4750 USD.
Last updated in May 2018 with the release of Apache FOP 2.3 and is still free.
Last updated in April 2019 with the release of Formatter V6.6 MR5 prices currently starts at $5000 USD.
I’d recommend Apache FOP as the starting point, and if you need more advanced features take a look at the commercial vendors. I think you’ll be surprised at what it can do.
So how do I go about getting starting with XSL-FO today. Well not so long ago I’d have directed you to w3schools, but they’ve taken the XSL-FO content down and the old Dave Pawson site also seems to have gone. RenderX still have some good content on their site, see the RenderX XSL-FO Tutorial, and there is still the books on Amazon:
Of the two books, “Definitive XSL-FO” is the easiest to follow. I’ll be adding some getting started blog posts to show XSL-FO in the modern era in the coming months, so watch this space.
XSL-FO still looks to be the best fit for complex data driven documents, give it a try today.
If you have a project that you need some expert help on, I’m available to provide consultancy and development services. Drop me an email on hello@rockweb.co.uk or tweet @rockwebuk
]]>Reports of my death have been greatly exaggerated
XSL-FO - 2014, 2015, 2016, 2017, 2018, 2019
There have been a number of reports that XSL-FO is a dead language and that CSS Paged Media is ready to usurp it from it’s role as the industry standard for printed output.
As both a web and print developer this topic is of great interest to me, and as much as I have liked working with XSL-FO in the past I would love to be able to double down on a single language for both websites and printed media. So for this article I’m planning to see what the current state of affairs for both XSL-FO and CSS Page Media to see which is the best for building a new publishing solution.
New article on the current state of the XSL-FO eco-system
The biggest advantage is that you can use it today without having to pay for it, there is an open source implementation of the spec in the form of the Apache Formatting Objects Processor which is very good. I’ve used it for a number of commercial projects and it’s more than up to the job. It’s primary output format is PDF but it also supports PostScript, PCL, AFP, AWT, PNG, RFT & TXT. It also enables you to embed SVG and regular images and fonts, allowing you to create pretty much any document you can imagine.
There’s also some commercial vendors such as RenderX and Antenna House which are more targeted towards the advanced requirements of commercial publishers. Both offer Java and .Net implementations of their processors, whilst Apache is only available in Java.
XSL-FO is part of the XSL specification from the W3C, its companion language XSLT is used for XML transformation and XSL-FO is used for formatting, specifically for paged media (print). When it was first released it was intended that XSL-FO would for print what CSS is for screens. For this task it is more than up to the job.
XSL-FO is a good fit for an XML based publishing work-flow, when combined with XSLT it creates a powerful way of generating documents from an XML source. I’ve used it to create transactional documents such as Invoices, Schedules & Letters and have used it for entire book series. It’s very powerful and gives excellent results.
My customers are getting real-world problems solved with XSL-FO 1.1.
It is often said that XSL-FO is hard to learn, that it should be left to an elite FO coder that does nothing else, that mere mortal developers will turn to stone just by gazing upon the complexity of the code. This of course not true, XSL-FO is actually relatively simple once you get going. Here’s an example block element with some styles applied:
<fo:block font-family="Arial" font-size="14pt" color="red">a simple FO block</fo:block>
The same thing in HTML:
<div style="font-family: Arial; font-size: 14pt; color:red;">a simple HTML div</div>
When we combine XSL-FO with XSLT we can even have something akin to CSS classes, called xsl-attribute-sets
<fo:block xsl:use-attribute-sets="block-style">a simple FO block</fo:block>
...
<xsl:attribute-set name="block-style">
<xsl:attribute name="font-family">Arial</xsl:attribute>
<xsl:attribute name="font-size">14pt</xsl:attribute>
<xsl:attribute name="color">red</xsl:attribute>
</xsl:attribute-set>
HTML & CSS Equivalent
<div class="block-style">a simple HTML div</div>
...
.block-style {
font-family: Arial;
font-size: 14pt;
color: red;
}
As a developer I’ve found it easy to switch between CSS and XSL-FO and in many cases especially before the introduction of CSS pre-processors like LESS or SASS I’ve missed the added capabilities of xsl-attribute-sets which allow you to extend existing sets (classes) leading to more modular code.
My examples are a bit basic, if you want to learn more about XSL-FO there are a couple of good books available which are still as relevant to today as they where 10 years ago, don’t let the age put you off (Not something you’d hear me say about any other technology book).
Of the 2 books, this is the easiest to follow, and I have found myself using it more often for it’s bullet listed style which makes it a great reference book. Here’s the Amazon link in case you want to get it. It was £26.57 at the time I wrote this article.
This was the first book I ever read on XSL-FO, and whilst it can take a while to get into it’s very good. It shows the stylesheets used to actually create the book using XSL-FO which gives you an idea of the power of the language. It’s also full of best practices and if you can stick with it you’ll end up writing better code, I certainly did. Here’s the Amazon link, it was £23.01 at time of writing.
Some links for reference:
Free open source implementation of XSL-FO 1.1.
Dollars | Pounds | Euros | |
---|---|---|---|
Server License | $0 | £0 | €0 |
Developer License | $0 | £0 | €0 |
Commercial implementation of XSL-FO 1.1, also available as a cloud service.
Dollars | Pounds | Euros | |
---|---|---|---|
Server License | $4750 | £2838 | €3489 |
Developer License | $400 | £239 | €239 |
Commercial implementation of XSL-FO 1.1 with CSS 3 support, the most fully featured and expensive option for XML printing.
Dollars | Pounds | Euros | |
---|---|---|---|
Server License | $7000 | £4183 | €5142 |
Developer License | $1750 | £1750 | €1285 |
The case for XSL-FO is pretty strong, you can use it for free or pay for a commercial solution, it’s stable and is an industry standard. What’s not to like?
We have closed the Working Group because not enough people were taking part.
Liam R. E. Quin (W3C XML Activity Lead) - November 2013
One of the arguments for not using XSL-FO for a new project is that the W3C Working Group that looked after the specification is no longer active and is no longer maintained. It could however be spun the other way to say that it’s now good enough and doesn’t need further revision or extension:
in a few short specs it was nailed pretty well and it does what 90% of the people want right now for print. There is really little to add to it to cover the full intention of what it should be – a standard for the representation of Formatted (print) output. … XSL FO will survive for a long time for those that require true print output and for a long time it will be the only Industry Standard way of doing it.
Kevin Brown, RenderX
Perhaps the ultimate nail in the coffin for XSL-FO is the lack of experienced developers. There are many more experienced web developers out there than XSL-FO developers. It will be easier (and cheaper) to hire a web developer and teach them CSS paged media, building on what they already know, than to find an experienced XSL-FO developer. Although they are out there I’ve trained at least 6 developers in the art.
I think another issue maybe for non Java developers there isn’t many free options to get started with XSL-FO. In the .Net world for example there are some ports of Apache but they are based on the 0.20.5 edition released in 2003 and are missing a lot of the later releases features.
With the advent of eReaders, eInk devices and Tablets has come the EPUB format, which is based on HTML, CSS and JavaScript. Given the ever growing popularity of EPUB it makes sense to prioritise it’s place in our work-flow, which throws a bit of a spanner into the works for XSL-FO. Current XSL-FO processors don’t support EPUB, and to be honest shouldn’t as it has very different use cases to fixed layout print.
In this new world we will now need two work-flows one for EPUB and one for PDF etc, or do we? We could conceivably use an XHTML document for a source file as XHTML is a subset of XML. Then using XSLT we could add the EPUB classes for one work stream and transform to XSL-FO for the others, the only real problem with this is that in many cases XHTML is to loose a specification for building a publishing platform on.
Now we come to O’Reilly a publisher of technical books whose publishing platform originally used XSL-FO, but has now moved to XHTML & CSS.
O’Reilly replaced XSL-FO with CSS Paged Media and introduced an extra step into their work-flow in the form of AsciiDoc (looks like markdown, AsciiDoc is a shorthand replacement for DocBook (XML Book Format))
Dynamic Elements such as; TOC, Index, Cross References are easier to do in XML/XSL than HTML which is why they haven’t gone straight from AsciiDoc to HTML. The new work-flow allows authors to write docs in asciidoc on their Atlas web application (in-house application for creating the documents) or via a text editor.
They are using Antenna House for the HTML & CSS to PDF conversion allowing them to migrate existing titles without loosing functionality thanks to Antenna Houses CSS extended properties.
In the future they plan to move to their own HTML based spec called HTMLBook - O’Reilly’s own spec for books (like DocBook) HTMLBook = HTML & CSS no new stuff, just more structured.
This would allow the creation of a WYSIWYG editor that creates valid markup just for the use with HTMLBook, allowing them to remove the first two steps from their work-flow
If you want to know more about O’Reilly’s solution check out this video titled Single Source Publishing and HTML by Nellie McKesson.
Yes, I believe that within the next few years CSS will supplant XSL-FO in the world’s publishing houses as it has at O’Reilly.
But as of today you can’t use CSS Paged Media in a browser, according to Mozilla only basic support is currently available.
You will therefore need to use a PDF processor to transform your XHTML and CSS into a PDF.
Free XHTML & CSS processor
Dollars | Pounds | Euros | |
---|---|---|---|
Server License | $0 | £0 | €0 |
Developer License | $0 | £0 | €0 |
As well as XSL-FO it also takes CSS3 and provides a couple of hundred CSS extended properties to fill the gap between CSS and XSL-FO. See previous section for full details.
PDF Processor for HTML5 & CSS it also has a number of extensions to fill the gaps in the CSS Paged Media spec.
Dollars | Pounds | Euros | |
---|---|---|---|
Server License: | $3800 | £2271 | €2791 |
Developer License: | $495 | £296 | €363 |
The CSS to PDF processors aren’t cheap and with the number of vendor specific extended properties I’d have some concern about code portability and vendor lock-in, the current situation reminds of when we moved from Internet Explorer 6 to other browsers.
XSL-FO isn’t dead yet and is still more than up to the task of being the basis of an XML based publishing system today. If you are only interested in printed media and have no plans to support EPUB I would recommend using XSL-FO without reservation.
If however you need to also support EPUB, CSS should figure in your plans, its evolving fast and the paged media spec is now a working draft which should mean browser vendors start implementing it.
All in all it’s an interesting time to be involved in print development.
If you have a project that you need some expert help on, I’m available to provide consultancy and development services. Drop me an email on hello@rockweb.co.uk or tweet @thisismikekelly
]]>The meeting space at Spaceport was full to bursting on a humid night in the centre of Manchester. The hosts had laid on beer and soft drinks for everyone and the event sponsors Evolution Recruitment provided Domino’s pizza for all, which was a nice touch.
The guest speaker tonight was Macs Dickenson, who was engaging and enthusiastic about React.js for ASP.Net MVC developers. He walked us through some basic code examples before showing how it fitted into the .Net workflow with Visual Studio.
Good talk, very useful for getting your head round the basic concepts of ReactJS and how to use with ASP.net MVC and the new ASP.net Core. I for one will likely be trying it in my next project.
I was most disappointed I didn’t win the raffle with the first prize being a Xbox One(sy), oh well maybe next time.
26th July 2016 @ 6:30 - 9pm
SpaceportX
Talk details and slides
Github with code samples from the talk
]]>Some of you may be surprised to learn that machine learning, big data and the other terms that go with it are all just calculus or if you prefer statistics. It is applied in a very clever manner that means a computer can be transformed with enough data into an intelligent (albeit not Einstein intelligent) entity that can predict a wide range of behaviours, such as what a price of a house may be worth to what I might like to watch next on Netflix.
This is an emerging and exciting field and as the cost of computing continues to drop we are going to see this becoming ever more present in the web sites we visit, to interacting with the internet of things, such as toaster trying to take over the world… or lights that know when to turn them selves off.
This blog series will concentrate on the foundations of machine learning, assuming no knowledge (as that is where I am starting) and delving into the what machines can do for you.
]]>We celebrated our 6 month anniversary last night with 9 programmers in attendance.
The format for the meetup was lean coffee; where we all came up with some topics to discuss, voted on them and then talked through them over coffee and biscuits.
Simon has read about Mob programming and was interested in seeing if any of us had tried it, and/or what our thoughts on it were
I (Mike) was interested in other developers techniques for improving and retaining knowledge of programming languages. In martial arts they have Kata, a set of moves that are practised to perfection to re-enforce techniques and to commit them to muscle memory. Is there something similar for programmers?
Nick was interested in our experiences using TDD commercially
Dan is learning it at the moment via Coursera and University of Washington
Sean is rewriting an old web-forms app with .Net core
Graham is using Docker for a new product he’s developing to make deployments easier
I (Mike) was interested in peoples views on Mobile apps build with web tech vs native applications and what issues, if any people have faced with both approaches.
Simon was interested on how we all saw Brexit with regards to how we get work in the future
Link to the Trello board with the cards on, if you want to add any details or links please feel free.
Another good meetup with 4 new faces, good variety of topics discussed and a lot of further reading to do.
All future meetups will be hash tagged on twitter with #prestonprogrammers if you want to keep up to date outside of Meetup.
]]>The meetup hosts where
The meetup was really 2 talks, the first being the Marc and Kev show, giving us an overview of the pre Code Garden retreat that Marc attended and Code Garden itself. The second was from Dan Booth showing his new God mode package.
Here are the highlights
Dan presented his new package for Umbraco 7.4+ God Mode here are the highlights
Dan shared some tips on how to create your own packages using Angular 1.2 and Web api within Umbraco
Perhaps the most important bit for those of us that didn’t get to go to Code Garden is the videos of the talks.
In summary was a good meetup, lots of information to take in. Lets hope the next one is soonish.
]]>Talk will be in 2 parts lasting about 20 mins each followed with a break for coffee in the middle. Afterwards we’ll be doing an extended lean coffee session on compiler related things.
Venue Cotton Court, Church Street, Preston, PR1 3BY
Directions Just off Church Street by the Blue Bell Pub, opposite the Ceragem store.
Date and Time 14th June at 6pm
]]>