Working with Builder

Habitat Builder allows you to store, automatically build, and deploy your Habitat packages. The documention below covers everything from creating an account to setting up automated build and exporting them to a variety of registries.

Table of Contents


Create a Builder Account

Setting up your Habitat account is easy! All you need is a valid GitHub account.

Sign In and Authorize Habitat

Use the button below to simultaneously sign in to Builder and create your account. Habitat Builder uses GitHub for authorization, so you'll see the prompt below authorizing the application.

Sign in to Builder

Once you have authorized Habitat and signed in, you'll arrive at the 'My Origins' view. At this point, other Habitat users can invite you to existing origins or you can proceed to create your own (see the next section).


Create an Origin on Builder

Origins are unique namespaces that can be used to denote a particular upstream of a package. For example, the "core" origin is the set of foundational packages that are managed and versioned by the core Habitat maintainers.

From the My Origins page in the Habitat Builder web app, click the Create origin button.

Note To join an existing origin, a current member of that origin will need to invite you. Pending invites will appear on the My Origins page for you to accept.

Enter an Origin Name

Pick an origin that is your company name, team name, personal name, or some other unique name that you want to associate with a given set of packages. It's important to note that once you have uploaded a package into the depot, the origin that you chose when building that package can neither be edited nor deleted.

Choose Privacy Setting

This is the default privacy setting applied to new packages. You can override this setting on individual packages when uploading or connecting a plan file.

Public packages will appear in public search results and can be used by any user, while private packages are restricted to members of the origin.

About Origin Keys

Origin keys are automatically generated when creating an origin. They are used to sign artifacts and can be downloaded from the Keys tab after creating the origin.

Note If you've already created an origin via the CLI tool, then you can upload the origin keys from the CLI to Builder and override the autogenerated set.


Upload and Promote Packages

While you can build and run Habitat packages without sharing them on Habitat Builder, uploading them there enables greater collaboration and automated package rebuilds as underlying dependencies or your connected GitHub repository are updated.

Note: Habitat Builder can only build Linux based plans (plan.sh) at this time.

Setting up Habitat Builder is easily done on the website: these steps take you through connecting your local Studio development experience with Builder.

You interact with Habitat Builder by:

  • Creating an account.
  • Creating an origin, or being invited to join an origin that already exists.
  • Setting up hab to authenticate with Builder.
  • Uploading the keys for that origin.
  • Connecting your Github Repos and opting into rebuilds.

Habitat Builder supports both public and private origins, packages, and Github repositories.

Create a Builder account

If you haven't created an account yet, see the Create a Builder Account section above.

Create or join an existing origin

You can create your own origin in Builder or be invited to join an existing one. If you already built some Habitat packages on your local computer prior to signing up for an account, you must rename your local packages' pkg_origin if the origin you want already exists.

Set up hab to authenticate to Builder

When you upload a package to Builder, you are required to supply an OAuth token as part of the hab pkg upload subcommand. Because Builder uses GitHub to authenticate, you must generate a GitHub access token for use with the hab command-line utility.

Once you have this token, you can set the HAB_AUTH_TOKEN environment variable to this value, so that any commands requiring authentication will use it.

Create origin keys

After you have done the basic account creation steps, you need to create your origin keys. The private key will be used to sign your packages and the public key will be used by supervisors to verify the integrity of your packages (.hart files).

You can either create an origin key pair by running hab setup from your host machine, or running hab origin key generate <originname> from either the host machine or from within the studio.

Your keys are located at ~/.hab/cache/keys on your host machine and /hab/cache/keys inside the studio environment.

Upload the keys for your origin

If you created a new origin and/or Builder does not have the public key that corresponds to the private key used to build your package, you must upload it. You also have the ability to upload your private key; however, if you do not upload at least the public key, Builder will reject the upload of your packages for that origin.

You can upload keys for the origin through the web interface for Builder, or by using the hab origin key upload command. You must be authenticated using the access token described earlier before you can upload keys.

Upload packages to Builder

Once Builder possesses at least the public key of the origin, you may upload one or more packages to that origin by using the hab pkg upload command. Builder will check the cryptographic integrity of the package before allowing you to upload it. Uploading packages is also a privileged operation for which you must have the access token.

Promote packages

Habitat supports continuous deployment workflows through the use of channels. A channel is a tag for a package that the Supervisors in a service group can subscribe to. Channels are useful in CI/CD scenarios where you want to gate a package before making it the default version of the package that users should consume. You can think of this split as the difference between test and production, or nightly releases versus stable releases of products.

By default, newly uploaded packages are placed in the unstable channel, however, the default package that is downloaded is the latest stable version of a package, unless overridden in commands such as hab start, hab svc load, and hab install. If you want to promote your package to the stable channel, run the hab pkg promote command as follows:

$ hab pkg promote -z <OAuth_token> origin/package/version/release stable

Note You can also promote packages to the stable channel using the promote to stable button in the web app.

For more information on how to use channels, see Continuous Deployment Using Channels.

Running packages from Builder

You can instruct the Supervisor to download and run packages from Builder by using the hab start command, for example:

$ hab start core/postgresql

If the Supervisor does not have the core/postgresql package in its local cache, it will contact Builder, retrieve the latest version and the public key for the core origin, verify the cryptographic integrity of the package, and then start it.

You may also supply a --channel argument to instruct the Supervisor to use a different channel for the purposes of continuous deployment:

$ hab start core/postgresql --channel unstable

Running packages from exported tarballs

An exported tarball package contains the Habitat client/binary as well as dependencies specified by your artifact.

After deploying the tarball to your target server, extract the contents to the root directory:

$ tar zxf core-nginx-1.11.10-20170616000025.tar.gz --directory /

You can instruct the Supervisor to run packages from an exported tarball:

$ /hab/bin/hab svc start core/nginx

Note: On a clean server, this will download additional packages to satisfy the Supervisor dependencies. You will also require a hab group and hab user on the system for most services.


Using Multiple Plans

If you have a large GitHub repository with multiple components inside, you will most likely also have individual plans for those components. When using Builder to build your packages, the default behavior is for Builder to build all packages whenever any file is updated in that repository, regardless of what component it is. This is because Builder does not know which file is associated with which package.

To provide more fine-grained control over when component packages are built, you can programatically customize how and when Builder will build your plans by specifying build behavior in a .bldr.toml file at the root of the repository that you connect to Builder.

Using this file, Builder only builds packages when source files or directories are updated in paths specified in .bldr.toml. This allows you to configure the building, publishing, and post-processing phases of a plan build in Builder.

To enable this functionality, do the following:

  1. Create a .bldr.toml in the root of your repository.

  2. Open it and add an entry for each component package that you want to build.

    The .bldr.toml file is in TOML format, so create a TOML table specifying the $pkg_name value for that plan and then add a plan_path field specifying the path to your plan.sh file (you do not need to include plan.sh explicitly in the path). If all the files related to the plan are under the plan path, then you are done. Otherwise, you will need an addition 'path' field specifying Unix-style path globs to files that are associated with the plan you specified in the 'plan_path'. File or directory changes made in these path locations determine which packages will be rebuilt. Basically, when a file is committed, Builder will check to see whether it falls underneath the plan_path hierarchy, or matches one of the globs in the path field if it was specified - if the answer is yes, then Builder will issue a build for that commit.

    For example, in the Habitat repository itself, this TOML states that the hab-launcher, hab-studio, and hab-sup packages will be rebuilt if there are any changes in any of the specified components sub-directories. Note that hab-studio does not need to specify a path because all of it's files are within the plan_path hierarchy, but that is not the case for the other projects.

    # .bldr.toml
    [hab-launcher]
    plan_path = "components/launcher/habitat"
    paths = [
    "components/launcher/*",
    "components/launcher-protocol/*",
    "support/ci/builder-base-plan.sh",
    ]
    [hab-studio]
    plan_path = "components/studio"
    [hab-sup]
    plan_path = "components/sup"
    paths = [
    "components/sup/*",
    "components/eventsrv-client/*",
    "components/launcher-client/*",
    "components/butterfly/*",
    "components/core/*",
    "components/builder-depot-client/*",
    ]

Special case where .bldr.toml does not exist

In the default case, where the .bldr.toml does not exist, there is one other condition that can impact builds. If the plan.sh file is not at the root (e.g., it is located in a habitat folder), then Builder infers that the plan files are all underneath the habitat folder, and will not kick off a build if files are committed that are outside of that folder.

In order to have automated builds kick off in this case, either move your plan.sh file to the root of the repo, or add a .bldr.toml file to specify the paths more explicitly.


Set up Automated Builds

By connecting a plan file in Habitat Builder, you can trigger automated package rebuilds any time your repo (containing the plan file) is updated.

Note Builder will watch for merges to the Master branch by default, but you can override this behanior using a .bldr.toml file as seen in the previous section.

Connect a Plan File

To connect a Plan, view one of your origins (while signed in), click on the Connect a plan file button, and complete the following steps:

  • Install the Builder GitHub App
  • Enter the GitHub Repo (that contains your plan.sh file)
  • Choose your privacy setting
  • (optional) Publish to container registries

Builder Demo

For a guided walkthrough of this process, try the Build System demo.