Tutorial: Create Your Own Package With Homebrew and Python

A few weeks ago, I wrote a script to manage AWS EC2 spot instances. It was the most complex BASH script I'd ever written. I was proud, and while riding that pride on my wave of success I confidently decided to extend my script into a full-blown package I could share with the world.

Two weeks and an uncountable number of hours later, I've succeeded. But I now recognize that my confidence was borderline arrogance. It turns out that turning your project into a package isn't hard - you really just need to create an extra file or two. However, learning WHAT those files should contain, WHERE your project files will be installed to, and the WHY behnd those two items is extremely difficult. I couldn't find a single resource that covered all of those items.

So I've created one. This tutorial is intended for people who've never created a package before, and are unfamiliar with how Linux handles terminal commands and executable files. It will walk you through HOW to create a package step by step, and WHY each step is taken. I hope this saves you time, hours of pain, and enables you to share something cool with the world. Here we go!

Contents

  • What is a package?
  • Overview of Homebrew
  • Basic Package Requirements
  • Steps to Create Package
  • Example
    • Example Package
    • Linux Commands
    • Formula Content
  • Closing Remarks

What is a package?

A package is simply a set of files that do something. It could be a full-blown application with its own modules and libraries, or simply a handful of scripts. Some of the most useful packages are command line utilities, such as wget and awscli.

Overview of Homebrew

Homebrew is a package manager for OS X. It's gimmick is that it 'brews' and installs packages for you. It's typically used to install command line utilities and other general packages. And it can be used for packages written in any language.

Let's walk through the steps Homebrew takes when you install a package. First, there are 4 terms you should know:

  • Formula: A ruby file, <package>.rb. It contains a description of the package, as well as instructions for how to 'brew' it. Specifically:
    • Where to find the package files.
    • Steps for installing and setting up the package (e.g. compiling code, installing dependencies, running tests, etc)
    • Where package files should be installed on your computer.
  • Tap: Directory or GitHub repo that contains Formulas.
  • Keg: Local directory where the 'brewed' package files are stored.
  • Cellar: Local directory with Kegs of 'brewed' packages.

Note that the path for the Cellar is usr/local/Cellar, and the path for a keg is usr/local/Cellar/<package>/<version_number>.

Here's what installing a package looks like:

  • You run the command brew install <package>.
  • Homebrew searches its Taps for the corresponding Formula.
  • Homebrew then reads the Formula's instructions and:
    • Downloads the package files.
    • Creates a Keg in your computer's Cellar for the package.
    • 'Brews' the package.

Basic Package Requirements

A Homebrew package requires 4 things:

  1. A tarball of the package files - e.g. awspot-0.1.tar.gz. *This is sometimes referred to as the source tarball.
  2. A GitHub repo containing the tarball.
  3. A Formula file for the package - e.g. awspot.rb.
  4. A Tap containing the package formula.

You should keep your package files and their tarball in the same repo. And, if you're planning on sharing your package with anyone, I'd recommend creating a repo for your Tap.

Here's the repo containing our example formula.

Package Creation Steps

At a high level, here are the steps you need to take to create your own package:

1. Compress your package files.

Compress your project files using tarball. Name your file using this pattern: <project name>-<version>.tar.gz, e.g. hworld-1.0.tar.gz

Here's an example command:

tar --exclude='./.git' --exclude='./README.md' -zcvf "hworld-1.0.tar.gz" .

2. Setup your Tap.

If you don't have an existing Tap, you should create a directory or repo for your package's formula file. You can then add your tap with the command: brew tap <path to tap>, e.g. brew tap https://github.com/rob-dalton/homebrew-tap.

3. Create your package Formula.

Run brew create <link to tarball>, e.g. brew create https://github.com/rob-dalton/hworld. This will automatically generate a formula file with the appropriate link and SHA256 hash value.

4. Fill out your package Formula.

Define the installation instructions and any dependencies your package may have.

Example

Let's examine the details of a formula using an example.

Hworld

I've created a simple command line utility, Hworld. It prints "Hello world!" to the console when the command hworld is run.

It's stored in this GitHub repo.. The tap containing its formula is located here.

You can install it yourself by running brew tap https://github.com/rob-dalton/homebrew-tap and brew install hworld.

Linux Commands

In order to understand how Hworld works, we need to understand how Linux handles commands. It's pretty simple: when you run a command, Linux searches for an executable file that matches the command. It looks for this file in the locations specified in the environment variable PATH (a list of directories).

For example, when you run the command, brew install hworld, Linux will do the following:

  1. Search through the directories listed in PATH, in order.
  2. Look for an executable file named brew.
  3. If the file brew is found, execute the file and pass the entire command and its args to it.
  4. If no matching file is found, return a command not found error.

Note that Linux will execute the FIRST match it finds.

For our package, the hworld command will execute a single script file, also named hworld:

#!/bin/bash

echo "Hello world!"

Hworld Formula

The formula for hworld is shown below. You can see that it simply extends Homebrew's Formula class. Let's take a look at it and expand on several key attributes and methods:

class Hworld < Formula
  desc "Simple hello world script."
  homepage "https://github.com/rob-dalton/hworld"
  url "https://github.com/rob-dalton/hworld/raw/master/hworld-1.0.tar.gz"
  sha256 "8443118e257c4c109332ae58df932da99f3bd1291a67b8a8a0283f529bc4f48e"
  version "1.0"

  def install
    # install hworld script, create symlink to script in /usr/local/bin
    bin.install "hworld"
  end

  test do
    # test script output
    assert_equal %x('#{bin}/hworld'), "Hello world!\n"
  end
  
end

sha256

Every Formula has a sha256 value - this is the hash value of your compressed package files. Brew does this to ensure the tarball it downloads from url is the one specified in the Formula, and contains only the files expected by the Formula.

You can obtain the sha256 value for your package source by running brew create <url> - this will create a Formula file similar to the one above with the url and sha256 fields filled out for you.

Alternatively, you can run shasum -a 256 <tarball> to generate the sha256 value alone.

install

Brew only installs files specified under the install method. Brew will discard all package files not explicitly handled in install.

The line bin.install "hworld" tells brew to do 2 things:

  • Install the script file hworld under our package's prefix.
  • Create a symlink to our file with the same name in /usr/local/bin

It's a good practice to install your command's executable script to /usr/local/bin. It's where Homebrew generally installs package executables (for example, the script that the brew command executes lives there), and it should exist in most users' PATH.

Now, when you run the command hworld, Linux should find and execute usr/local/bin/hworld.

test

After a package is successfully installed, you can run the test method with the command brew test <package name>. This provides a good way to test your package is working properly, even if it installs without any errors. It's also a best practice - in fact, Homebrew won't even review a package for inclusion in homebrew/core if it doesn't include a few tests.

Our test simply executes our script file and tests if its output matches our expectations - e.g. it returns the string "Hello world!".

Documentation

For more information, you should refer to the documentation here.

Closing Remarks

That's it! Again, I hope this was useful.

For more information on Homebrew and Formula creation, please refer to Homebrew's website. Also, please note that for brevity and simplicity, I didn't discuss how Homebrew handles dependencies in this post. However, it's a topic worth reviewing.

Written by