Habitat Builder allows you to store, automatically build, and deploy your Habitat packages. The documention below covers everything from creating an account to setting up automated builds and exporting packages to a variety of container registries.
For a guided walkthrough of this process, try the Build System demo.
Table of Contents
- Create a Builder Account
- Create an Origin on Builder
- Generate an Access Token
- Working with Origin Secrets
- Upload and Promote Packages
- Using Multiple Plans
- Set up Automated Builds
- Install and Use Builder On-Premises
Setting up your Habitat account is easy! All you need is a valid GitHub account.
Sign In and Authorize Habitat
Use the button below to simultaneously sign in to Builder and create your account. Habitat Builder uses GitHub for authorization, so you'll see the prompt below authorizing the application.
Once you have authorized Habitat and signed in, you'll arrive at the 'My Origins' view. At this point, other Habitat users can invite you to existing origins or you can proceed to create your own (see the next section).
Origins are unique namespaces that can be used to denote a particular upstream of a package. For example, the "core" origin is the set of foundational packages that are managed and versioned by the core Habitat maintainers.
From the My Origins page in the Habitat Builder web app, click the Create origin button.
Note To join an existing origin, a current member of that origin will need to invite you. Pending invites will appear on the My Origins page for you to accept.
Enter an Origin Name
Pick an origin that is your company name, team name, personal name, or some other unique name that you want to associate with a given set of packages. It's important to note that once you have uploaded a package into the depot, the origin that you chose when building that package can neither be edited nor deleted.
Choose Privacy Setting
This is the default privacy setting applied to new packages. You can override this setting on individual packages when uploading or connecting a plan file.
Public packages will appear in public search results and can be used by any user, while private packages are restricted to members of the origin.
About Origin Keys
Origin keys are automatically generated when creating an origin. They are used to sign artifacts and can be downloaded from the Keys tab after creating the origin.
Note If you've already created an origin via the CLI tool, then you can upload the origin keys from the CLI to Builder and override the autogenerated set.
You can build and run Habitat packages without authenticating to Builder; however some operations, like uploading your packages to Builder, or checking status of your build jobs from the Habitat client, will require an access token.
From the Builder site, select your user icon in the upper right-hand side, and select Profile.
At the bottom of the profile page, select Generate Token.
Copy your token, and then set it in the CLI by running:
<token> with the contents of your generated token.
With the Habitat CLI and a Builder account, you can encrypt and store secrets to expose at build time as environment variables in your Builder builds. This feature is helpful for plans requiring access to protected resources at build time, such as private source-code repositories, cloud storage providers and the like. Secrets are defined at the origin level, which makes them usable in any plan belonging to that origin.
To work with origin secrets, first obtain a Builder access token, then apply it on the command line using either the
HAB_AUTH_TOKEN environment variable or the
--auth option, along with the associated origin. For example, to list the names of all secrets defined for a given origin, use the
You can also set your origin and token as environment variables:
To create a new secret, give the secret a name and string value:
Once your secret has been uploaded, you can refer to it in your plan file as an environment variable, and Builder will supply its decrypted value during your build job.
For instance, if a plan required access to a file kept in a private bucket on Amazon S3, you might use the AWS CLI provided by the
core/awscli package to download the file using secrets named
AWS_SECRET_ACCESS_KEY, which would allow the
aws binary to read them as environment variables at build time:
You can delete a secret either with the Habitat CLI:
or in the Builder web interface, under the Settings tab for your origin:
Secrets are encrypted locally using an origin encryption key. Their values are readable only by Builder.
While you can build and run Habitat packages without sharing them on Habitat Builder, uploading them there enables greater collaboration and automated package rebuilds as underlying dependencies or your connected GitHub repository are updated.
Note: Habitat Builder can only build Linux based plans (
plan.sh) at this time.
Setting up Habitat Builder is easily done on the website: these steps take you through connecting your local Studio development experience with Builder.
You interact with Habitat Builder by:
- Creating an account.
- Creating an origin, or being invited to join an origin that already exists.
- Setting up
habto authenticate with Builder.
- Uploading the keys for that origin.
- Connecting your Github Repos and opting into rebuilds.
Habitat Builder supports both public and private origins, packages, and Github repositories.
Create a Builder account
If you haven't created an account yet, see the Create a Builder Account section above.
Create or join an existing origin
You can create your own origin in Builder or be invited to join an existing one. If you already built some Habitat packages on your local computer prior to signing up for an account, you must rename your local packages'
pkg_origin if the origin you want already exists.
Set up hab to authenticate to Builder
When you upload a package to Builder, you are required to supply an auth token as part of the
hab pkg upload subcommand. You can generate a Habitat personal access token via the Builder site Profile page for use with the
hab command-line utility.
Once you have this token, you can set the
HAB_AUTH_TOKEN environment variable to this value, so that any commands requiring authentication will use it.
Create origin keys
After you have done the basic account creation steps, you need to create your origin keys. The private key will be used to sign your packages and the public key will be used by supervisors to verify the integrity of your packages (
You can either create an origin key pair by running
hab cli setup from your host machine, or running
hab origin key generate <ORIGIN> from either the host machine or from within the studio.
Your keys are located at
~/.hab/cache/keys on your host machine and
/hab/cache/keys inside the studio environment.
Upload the keys for your origin
If you created a new origin and/or Builder does not have the public key that corresponds to the private key used to build your package, you must upload it. You also have the ability to upload your private key; however, if you do not upload at least the public key, Builder will reject the upload of your packages for that origin.
You can upload keys for the origin through the web interface for Builder, or by using the
hab origin key upload command. You must be authenticated using the access token described earlier before you can upload keys.
Upload packages to Builder
Once Builder possesses at least the public key of the origin, you may upload one or more packages to that origin by using the
hab pkg upload command. Builder will check the cryptographic integrity of the package before allowing you to upload it. Uploading packages is also a privileged operation for which you must have the access token.
Habitat supports continuous deployment workflows through the use of channels. A channel is a tag for a package that the Supervisors in a service group can subscribe to. Channels are useful in CI/CD scenarios where you want to gate a package before making it the default version of the package that users should consume. You can think of this split as the difference between test and production, or nightly releases versus stable releases of products.
By default, newly uploaded packages are placed in the
unstable channel. However, the default package that is downloaded is the latest
stable version of a package, unless overridden in commands such as
hab sup run,
hab svc load, and
hab pkg install. If you want to promote your package to the
stable channel, run the
hab pkg promote command as follows:
Note You can also promote packages to the
stablechannel using the promote to stable button in the web app.
For more information on how to use channels, see Continuous Deployment Using Channels.
Running packages from Builder
Note: When running private packages from Builder, it's necessary to add your Habitat access token to the machine where you intend to deploy the package, via
You can instruct the Supervisor to download and run packages from Builder by using the
hab sup and
hab svc commands, for example:
If the Supervisor does not have the
core/postgresql package in its local cache, it will contact Builder, retrieve the latest version and the public key for the
core origin, verify the cryptographic integrity of the package, and then start it.
You may also supply a
--channel argument to instruct the Supervisor to use a different channel for the purposes of continuous deployment:
Running packages from exported tarballs
An exported tarball package contains the Habitat client/binary as well as dependencies specified by your artifact.
After deploying the tarball to your target server, extract the contents to the root filesystem (
You can instruct the Supervisor to run packages from an exported tarball:
Note: On a clean server, this will download additional packages to satisfy the Supervisor dependencies. You will also require a
hab group and
hab user on the system for most services.
If you have a GitHub repository with multiple components inside, you will most likely also have individual plans for those components that are located inside of component subfolders. By default, Builder will only look for a package plan in either the root of the repository, or in a
habitat subfolder at the root. If it does not find a plan file in those locations, it will not automatically issue builds when it detects file changes in the repository.
In order to tell Builder about the location of the individual plan files, and in order provide more fine-grained control over when component packages are built, you can programatically customize how and when Builder will build your plans by specifying build behavior in a
.bldr.toml file at the root of the repository that you connect to Builder.
Using this file, Builder only builds packages when source files or directories are updated in paths specified in
.bldr.toml. This allows you to configure the building, publishing, and post-processing phases of a plan build in Builder.
To enable this functionality, do the following:
.bldr.tomlin the root of your repository.
Open it and add an entry for each component package that you want to build.
.bldr.tomlfile is in TOML format, so create a TOML table specifying the
$pkg_namevalue for that plan and then add a
plan_pathfield specifying the path to your
plan.shfile (you do not need to include plan.sh explicitly in the path). If all the files related to the plan are under the plan path, then you are done. Otherwise, you will need an additional 'paths' field specifying Unix-style path globs to files that are associated with the plan you specified in the 'plan_path'. File or directory changes made in these path locations determine which packages will be rebuilt. Basically, when a file is committed, Builder will check to see whether it falls underneath the
plan_pathhierarchy, or matches one of the globs in the
pathsfield if it was specified - if the answer is yes, then Builder will issue a build for that commit.
It's important to note that the entries for
pathsdo not behave the same. If you have something like
plan_path = "habitat", that behaves as if you had written
plan_path = "habitat/*"- that is, it will automatically check every file under the
habitatdirectory. However, if you have something like
paths = [ "src" ], that is not automatically expanded to
src/*. That line will only watch for changes to a file called
src. If you're wanting to watch for changes to any file inside the
srcdirectory, then you must explicitly specify the glob, like so:
paths = [ "src/*" ].
For example, in the Habitat repository itself, this TOML states that the
hab-suppackages will be rebuilt if there are any changes in any of the specified
componentssub-directories. Note that
hab-studiodoes not need to specify a
pathbecause all of it's files are within the
plan_pathhierarchy, but that is not the case for the other projects.
Notice that in order to specify that we're interested in all files inside of the directories in our
pathsentries, we had to add the
/*glob to the end manually.
It's also worth pointing out that there are multiple wildcard characters you can use when specifying path components.
?will match any single character.
*will match any (possibly empty) sequence of characters
**matches the current directory and arbitrary subdirectories. This sequence must form a single path component, so both
b**are invlid. More than two consecutive
*characters is also invalid.
[...]matches any character inside the brackets. You can also specify a range, such as
[0-9]to match any digit or
[a-z]to match any lowercase letter.
[!...]is the negation of
[...]so it will match any character not in the brackets.
Note that while the above set of rules bears a remarkable resemblence to regular expressions, we do not support full regular expression syntax. Only what's shown here is supported. Here is an example.
By connecting a plan file in Habitat Builder, you can trigger both manual (via the web UI, or via the
hab command line) as well as automated package rebuilds whenever a change is merged into the
master branch of the repository containing your Habitat plan, or when a dependent package updates (rebuilds).
Connect a Plan
To connect a plan to Builder, view one of your origins (while signed in), click the Connect a plan file button, and complete the following steps:
- Install the Builder GitHub App
- Choose the GitHub organization and repository containing your Habitat plan
- Choose a privacy setting for the package
- Specify container-registry publishing settings (optional)
- Specify auto-build option (default is off)
The auto-build option controls whether or not your package will get automatically re-built. This option is a useful capability to have - for example, if you have a demo app that doesn’t need to be kept constantly up to date when some underlying dependency updates. Auto-build encompasss both builds that are triggered by Github web hooks (on commits to master), as well as builds that are triggered by a dependency updating.
By default, new plan connections will have auto-build turned off.
In addition to our hosted service, we also support installing and running a Habitat Builder Depot on-premises, using your own network and infrastructure, which allows you to choose from a wider selection of authentication providers and to manage how Builder fits into your existing CI/CD processes.
Currently, our on-premises Builder depot only stores packages for download and upload by Supervisors and Studios. We intend to bring the full capabilities of Habitat Builder to our on premises option in the future.
For a detailed explanation of features, requirements and setup instructions, see the GitHub repository.