Following on from my last post, I’ve been working to track and control the conditions in my tea storage boxes. Initially, I got some cheap units from Amazon. They were basically a humidity/temperature sensor, an LCD display, and a battery. But since I don’t just sit in the basement staring at my tea shelf, I wanted to upgrade to a solution that let me transmit that data to view it more easily. ...
I’m a big fan of DockerHub’s Automated Builds: they let you tie a git repo to a DockerHub image, so that updating the repo kicks off a fresh build of the image. It further lets you define Repository Links, so that if a parent image is updated, child images rebuild themselves. In my case, I maintain a base image with the OS and package updates, and then a tree of images based on that. I want to rebuild that image every so often to get updated packages, but that feature is missing from DockerHub. So I wrote an AWS Lambda to handle it. ...
I’ve been using Terraform for managing my AWS account for a while. It’s pretty snazzy, but there are still a couple of things that Terraform doesn’t fully handle. For example, making an IAM access key in Terraform stores the secret key in the statefile. They’ve added support to store the secret key encrypted with a GPG key, but I’d much prefer to not have it end up in the statefile at all. ...
A while back, I wanted to do a couple quick things w/ the Slack API. The script I was writing would only end up being run a handful of times, all from my local computer, and I hate having multiple distinct credentials stored in the same place with the same perms, so I hatched a plan: piggyback on the existing creds my browser was using to access Slack. ...
At some point while working VM images and containers, I ended up wanting some custom Arch packages. It started with a desire for lighter packages and to really understand what was going into my system, and then turned into something of an obsession. As of today, I publish 92 Archlinux packages, most of them custom builds of common Linux tools. And because otherwise I’d be drowning in manual work, I’ve automated the hell out of the process. ...
I spend a decent amount of time thinking about init systems. Most of the time, that means s6, but for more complex or user-interactive systems, I’d go for systemd. This puts me squarely opposed with a decent-sized group of loud people, it seems. One of the complaints that is occasionally brought up is that sysvinit was great and init scripts are great, etc etc. My hypothesis: for people whose primary init system interaction is writing and using initscripts, systemd unit files are so amazingly easier to read and write and use that it is without a doubt the better choice. ...
Now that tax day has officially passed, it occurred to me that the best way to celebrate would be to plan for next year. A few of my friends semi-seriously keep tabs on a lightweight challenge: try to break as close to zero on tax day. I don’t know how much actual effort most of them put into this challenge, but I’d average it’s low, because I’ve not historically put effort into it. But the goal is fairly sound: don’t give Uncle Sam an interest free loan of your hard-earned cash. It’s also dawned on me that, complex though tax code may be, I have a computer and a marginal understanding of math. As such, I’ve set out to code my way to victory. ...
It seems like every day a new project is released for managing datacenters full of containers, all networked together and serving content to users. I enjoy that aspect of containers as much as the next sysadmin, but I’ve found one of the coolest use cases for them to be repeatable/isolated software builds. Over time I’ve collected a decent list of codebases I want to utilize, and in the past I would pull and build them on the systems I planned to use them on. I’ve already talked at length about how poorly that scales, but now I’d like to focus on one specific area of the solution: using Docker containers to perform and share compiled software packages. ...
To say I’m addicted to GitHub is an understatement. But I’ve attempted to focus my addiction towards productive goals, and so I decided that I wanted to process GitHub streak data programmatically. To my dismay, streak data isn’t exposed as part of their API, and my request that they add it was met with polite neutrality. So I set out to see how their site built the streak chart on the user page. When I began this adventure, I knew enough JavaScript to shoot myself in the foot and I’d never dealt with large existing JavaScript codebases, but I nonetheless dove in to Chrome’s Developer Tools to dissect how the page created that chart. I got my first win from Jon Chen, who identified the source of the data: a JSON array of dates and scores served at https://github.com/users/{username}/contributions_calendar_data (a URL that no longer works, which we’ll get to). This gave me the raw score data, and I got to work building a module around that. ...
One of the natural consequences of using lightweight Docker hosts and running everything in containers is that I tend not to interact with the actual VM’s system as much. I SSH straight to a container for IRC, I do my kernel builds in containers, etc. With this, it’s made sense to strip out components out of the VM that were used solely for user interaction. Some of those components were easy to identify and pull out: Stopped cloning my dotfiles and scripts repos to the VM image Removed packages like vim-minimal and zsh But the big change I wanted to make was simplifying the init process itself. ...
Building on my previous work, I’ve been working to solve some pain points for deploying and managing VMs. There were a couple big ones that I specifically targetted: Speed. It turns out that building a kernel and an Arch environment from scratch on each VM takes a long time. In-place upgrades. All upgrades were full redeployments. Coupled with the speed problem, this got pretty annoying Secure data. When all the data on the VM is generated on-site and sourced from public GitHub repos, passing secure data (think SSL keys or API keys) becomes problematic. Thankfully, after some thought, I realized that these problems are all facets of what was, in hindsight, a poor design choice: generating the components on the VM itself. In round two, I set out to find a better route. ...
Linux containers have recently been in the spotlight, in no small part due to Docker’s work in providing a user-friendly interface to LXC. Lightweight containers allow us to reimagine how we handle deployments, separation of services, and infrastructure management. A system that can be spun up in seconds, configured on the fly, and spun down just as fast presents the next step in a trend begun by the virtual private server. In a world where all the action happens inside containers, it seemed wasteful for the “host” system to be a full-service Linux system of its own. I went on a mission to strip down an Arch system as far as I could, to the point where I had a read-only system with only essential services from which to run Docker. ...
It’s pretty clear these days that {thing}-as-a-service is a powerful concept. That said, it involves a certain level of trust in whoever is providing the service. Part of that is the trust that they’ll keep providing you the service, but on the tin-foil-hat side, you trust that they won’t use the information you give them to your detriment. This is especially true for storage/hosting service providers, where you’re quite literally handing your data to a 3rd party. I love Dropbox, but I’m not sure I’m ready to trust them with my secret plans for world domination. As such, I’ve decided that I’d like to add a layer of encryption on top of my Dropbox storage, as well as other similar providers. I looked around, and decided to work with Dropbox, Copy (Update: the Copy service has shut down), and Google Drive. As a disclaimer, the Dropbox link includes my referral code so I get extra space if you use it. ...