Building something

So I thought I’d try to jot down a few words on how my latest side project came to be, what it is, and where it is at the moment, since future posts will probably touch on this topic. Whenever those will be written. As per usual regarding my, and probably many other developers personal projects, the subtitle, or whatever you’d call it, should probably be something like “Building something absolutely no one asked for” or perhaps “I got this idea, so I had to do it”. It is what it is, at worst I will burn a little money on cheap Hetzner VPSes, but still probably learn a lot, which will still be a win in my book. And I will of course have had quite some fun in the process.

What is it? Why? Where am I?

It is both the where and the how this blog is built, deployed and hosted. It is a web hosting service, a PaaS, Platform as a Service, if you will. spwnr.io. The name: because Elixir, a programming language with a lovely community that I have taken to. In Elixir, processes are spawned. It’s a thing. Hence, spwnr. It obviously is the dominant language in this project, not the only one though. More on that later.

Why build your own web hosting service? A question you might ask. I know I have. Well, why not? To see if I could? For fun? A reason as good as any. It isn’t because there is a lack of hosting providers. We have Vercel, Render, Fly.io, Heroku, Azure and all the other big ones, and a bunch of others too I’m sure. Sometimes they’re free, sometimes kind of free, for a while. Sometimes your free plan isn’t so free at all anymore, then you realize, maybe too late, that you gave them your credit card info. And you rarely have any control over how and where your data is handled, which is a thing I have become interested in over the last year or so. These are thoughts that have been with me for a while, so there may be a little vague ‘plan’ behind this project, other than obsessive building.

What do we have here?

Let’s run through the parts of this project on a high level and start with the first thing I built, which is the build agent. A cool name I wish I actually used, it is named build_receiver, which is just plain wrong, since it does more things than receive. I realized this a little late. Anyway, it is an Elixir application that operates on a build server running Debian Linux, it exposes an HTTPS endpoint that accepts authenticated build requests, fetches additional data from a PostgreSQL DB, pulls source code from a GitHub repo connected to the project, builds a Docker image and deploys it to a host server. I am skipping/simplifying a few steps here, mainly with the ‘deploys it on a host server’, but this is just a high level overview. Maybe more on this in another post.

And that brings us to the next part of the project, a host server. It’s essentially a web server, also running Debian, using Docker for running the hosted web apps in containers and Traefik as a reverse proxy to route incoming traffic. No custom applications running here yet, just a bunch of config work, as with the other servers, but let’s keep that for another post.

A project (a web app to be deployed on spwnr) is created, maintained and deployed from the control_plane. This is a Phoenix application, Phoenix Framework being the standard web framework for Elixir. This web application also runs on a VPS with Debian and is the user interface for the users (me). We have the basic stuff needed here: create account, log in, create project, connect a GitHub account to pull source code from private repos, manual build/deploy trigger and some basic stats. Also some pretty basic, but still a little more advanced stuff: set up environment variables/secrets and live logs from the build/deploy pipeline.

There is also an infrastructure repository with Ansible for automating the setup of servers, which is pretty neat and something that is a first for me. A bit of a hassle to set up and get right initially, but very satisfying when it’s working. It basically means we can spin up new identical servers at the press of a button, and since there can be quite a lot of setup for a server this is pretty cool. I will probably also want to dive into this in a future post.

State right now

  • Basic web landing page with account creation/login and project management
  • GitHub app integration to fetch private (or public) repos
  • Basic environment variables/secrets management
  • Choose build and host server per project (Hetzner VPSes located in Helsinki for now)
  • Manual build/deploy trigger
  • Live streaming build/deploy logs
  • Sample repo to clone and test the build/deploy pipeline with

What next?

There are a few things on my wish list worth mentioning, first of all I’m not really happy being solely dependent on GitHub for fetching source code. But if you are going to start somewhere, GitHub is the place most people, including me, keep code. I would like to have a simple CLI to trigger a build/deploy from a local environment, but I’m guessing that will be a non-trivial undertaking, even if I try to make it as simple as possible. And it will likely be written in Go, the little investigating I have done on building CLIs points to that being the best suited for the job. And I have not coded in Go yet, though I would like to learn it. It could maybe be a cool open source project, if anyone is interested, ping me somewhere.

A few more things on the wish list:

  • Simple managed database service
  • Basic traffic stats per project
  • Custom domain support
  • Pre-build security check
  • Application logs in UI

Another thing on my nice-to-have list would be using a Swedish/local data center for the VPSes, I actually have a very local operator in my hometown, Glesys. They are probably a much better option than Hetzner in all sorts of ways but since this is a hobby project not generating any income, the cheapest option wins for now.

Lastly, onboarding a test user or two would be pretty cool. However, I’m not sure there’s any real need or interest for another web hosting service. The web is full of options, so this will most likely remain a learning project, and I’m fine with that. What this project possibly does have going for it is the EU-centric stance on transparent data handling and a focus on simplicity. We’ll see. Whatever happens, I’ve got a few posts planned to document the journey and learnings. Thanks for reading =)

Things mentioned above