Habitat is a framework comprised of multiple components: Habitat Builder; the Supervisor; an HTTP interface on that supervisor to report package status; a communication model for disseminating rumors through a supervisor ring, and others.
This section will delve into each of these major components to give a shared vocabulary on what they are, what they do, and how they work together. For information about the underlying design goals that motivated these components, see Why Habitat?.
You can also check out Habitat's reference architectures.
Table of Contents
- Application Lifecycle Hooks
Almost anything you could want to do in Habitat can be managed via the
It includes subcommands for executing package builds, loading services into the process Supervisor, uploading packages to Builder, and entering the Studio. For more information on all of the functionality of
hab checkout our CLI command documentation.
The Supervisor is a process manager that has two primary responsibilities. First, it starts and monitors child services defined in the plan it is running. Second, it receives and acts upon information from other Supervisors to which it is connected. A service will be reconfigured through application lifecycle hooks if its configuration has changed.
The Supervisor Ring
Supervisors typically run in a network, which we refer to as a ring (although it is more like a peer-to-peer network rather than a circular ring). The ring can be very large; it could contain hundreds or thousands of supervisors. The membership list of this ring is maintained independently by each Supervisor and is known as the census.
The census is the core of the service discovery mechanism in Habitat. It keeps track of every Supervisor in the ring, and handles reading, writing, and serializing it with the discovery backend.
Each Supervisor in the system is a census entry that together form a census. Operations to discover or mutate the state of the census happen through algorithms that arrive at the same conclusion given the same inputs.
An example is leader election: it's handled here by having a consistent (and simple) algorithm for selecting a leader deterministically for the group. We rely on the eventual consistency of every Supervisor's census entry to elect a new leader in a reasonable amount of time.
Supervisor REST API
The Habitat Supervisor provides a HTTP API to expose cluster metadata, statistics, and general diagnostic information useful for monitoring and support in the form of a JSON document. It also provides detailed information about the Habitat package that it is supervising, including metadata such as the build and runtime dependencies and their versions.
The Supervisor control gateway is used to issue commands to a remote Supervisor. When a new Supervisor is created, a key for the
HAB_CTL_SECRET environment variable is generated for it by default, if one is not already present; this key is used to authenticate requests that are made via the control gateway. See the control gateway documentation for more details.
Habitat's Launcher is a sidecar process for the Supervisor which provides a mechanism for launching processes on behalf of the Supervisor. It is the entry point for running the Supervisor and is the Supervisor for the Supervisor. Whereas the Supervisor is able to automatically update itself, the Launcher is currently released a bit differently, by design; it should be rare that the Launcher ever needs to change.
To update your Launchers, run:
Then restart the Supervisor. This will, by necessity, require a restart of supervised services, so factor that into your planning.
The Launcher is designed to run as process 1; it is minimal by design. Its responsibilities are simply to be the parent process for the Supervisor.
The Launcher enables the Supervisor to update itself without shutting down or re-parenting the services that the Supervisor is supervising. The Launcher is versioned separately from the Supervisor and should be updated very infrequently since an update of the Launcher could require a system restart if it is running as process 1.
A plan is a set of files that describe how to build a Habitat package. At the heart of the plan is a configurable script named
plan.sh for Linux and
plan.ps1 for Windows, containing instructions on how to download, compile, and install its software.
Habitat's build phase defaults can be overidden using callbacks. Application lifecycle hooks for services can be defined so the Supervisor running your services takes specific actions in response to specific lifecycle events. Optionally included are a set of TOML variables and their defaults that can be used to generate configuration files via Handlebar.js templates.
Each plan can specify application lifecycle event handlers, or hooks, to perform certain actions during a service's runtime. Each hook is a script with a shebang defined at the top to specify the interpreter to be used.
Important: You cannot block the thread in a hook unless it is in the run hook. Never call
sleepin a hook that is not the run hook.
To see a full list of available hooks and how to use them check out our hooks documentation.
Habitat scaffoldings are standardized plans for automated building and running your application. Each scaffolding is tuned to the way your application was built, which allows it to create the appropriate application lifecycle hooks and add in the correct runtime dependencies when building the package for your application. Scaffoldings also provide some default health check hooks where appropriate to ensure your application is functioning reliably. Customized Scaffolding can be created to facilitate re-usability of common patterns in your organization for developing, building, and running your applications.
While we are targeting many platforms for automated scaffolding we currently support Ruby, Node.js and Gradle.
Scaffoldings provide certain overrideable variables for language-specific behavior. Please see the appropriate scaffolding documentation for details.
Overriding Scaffolding Callbacks
If you want to override phases of a scaffold's build in your plans, make sure to override the main
do_xxx phase, not the callback directly. ex override
do_install() instead of
A language or framework scaffolding is shipped as a Habitat package, which means that each scaffolding runtime dependency becomes a build dependency for the application being built.
To create scaffolding, a package must contain a
lib/scaffolding.sh file which gets sourced by the build program running Bash.
A optional function named
scaffolding_load() may be created in
lib/scaffolding.sh which will be called early in the build program which allows a Scaffolding author to control and augment the
pkg_build_deps arrays. At this point, no other build or run dependencies have been resolved so the code in this function can only rely on what the build program provides or software pulled in via the Scaffolding's Plan.
Default Build Phases Implementations
The remainder of the
lib/scaffolding.sh contains one or more default implementations for the build phases. These include, but are not limited to:
Try them out
The point of scaffoldings is to handle complexity for you, and make it easy to start building, deploying, and managing your application with Habitat. So try it out and let us know how it goes!
A package refers to a binary distribution for a given piece of software built with Habitat that contains a software library or application and any configuration information for that software. It's a signed tarball with a .hart file extension created from a plan definition and built with Habitat tools that can be post-processed to a specific format, such as when creating a Docker container.
Packages are identified using a four-component scheme:
origin/name/version/release, where origin, name, version, and release are replaced with corresponding values.
Origin: A name that defines a set of related packages. For example, "sample", "core", or "mycompany".
Name: The name of the application or service. For example, "postgres".
Version: The version number designation by the author(s) of the application or service. For example, "3.1.1", or "20160118".
Release: The unique Habitat id for a given version based on the timestamp pattern YYYYMMDDhhmmss. For example, "20160204220358" would be a package built at 22:03:58 on February 4th, 2016. Multiple releases of a given package version may exist using different dependencies and/or compiler options.
When referring to packages from either Builder or the Studio, you can refer to them in two ways: A package identifier and a fully qualified package identifier.
- A package identifier is typically specified using the two-component form
origin/name. For example,
- A fully-qualified package identifier includes all four components in the following format:
origin/name/version/release. For example,
If the package identifier isn't fully specified (having less than four components), then the missing components are assumed to be the most recent versions for that package. For example:
core/glibcassumes that version and release values are for the latest version of chef/glibc.
core/glibc/2.22assumes that the version of chef/glibc is 2.22 and that the release is for the latest version of chef/glibc/2.22.
core/glibc/2.22/20160310192356only refers to the specific package 20160310192356.
Package Export Formats
Habitat packages (.hart) can be exported in a number of different formats depending on what you need where. This is powerful because it means that you are using the same immutable artifact Habitat package, but then exporting it into the format that you need to do a specific job. So when you are iterating locally in a Docker container, and then promote that package to an environment running Kubernetes, and then promote it to another datacenter that's running virtual machines, the package is identical in each location -- it's simply exported to the correct format for the job you are trying to do.
You can read more about how to export packages, and what exporters are currently available, here.
Habitat has strong cryptography built into both Habitat Builder and the Supervisor. This means there are several different kinds of keys.
As described in the packages documentation, every package in Habitat belongs to an origin and is cryptographically signed with that origin's private key.
Origin key cryptography is asymmetric: it has a public key that you can distribute freely, and a private key that you should keep safe.
Supervisors, by default, will refuse to run packages for which they do not have the public key. They use this public key to verify the integrity of the Habitat package they download before running it. Supervisors can be provided the public key by pointing them at a depot that has it, or by putting the key on disk outside of Habitat.
User and Service Group Keys
User and service group keys are used to set up trust relationships between these two entities. Service groups can be set up to reject communication (e.g. applying new configuration via
hab config apply) from untrusted users.
By default, service groups will trust any communication, so for a production deployment of Habitat, setting up these relationships is essential.
User and service group keys also utilize asymmetric cryptography. To apply configuration changes to service groups when running in this mode, a user uses their own private key to encrypt configuration information for a service group, using that service group's public key. The service group then uses its private key to decrypt the configuration information, and the user's public key to verify.
Ring Encryption Key
A Supervisor ring can be optionally set up to encrypt all communication across the network. This requires the use of a symmetric pre-shared key. Any Supervisor joining the ring that does not present this key will be rejected.
The Habitat Studio is a clean, self-contained, minimal environment in which you can develop, build, and package software that is free from any upstream operating system distribution. All tools and dependencies included in the Studio are installed through Habitat packages, thus preventing any unwanted dependencies from being used by your package.
A service in Habitat is defined as a Habitat package running under a Habitat Supervisor.
A set of one or more running services with a shared configuration and topology makes up a service group. If a service is started without explicitly naming the group, it's assigned to the
default group for the name of that package. For example:
postgres.financialdb(possibly running in a cluster)
postgres.userdb(possibly running in a cluster)
Habitat allows you to define the topology of your service groups, which bakes in certain behaviors.
This is the default topology, useful for services inside a group that are completely independent from one another. Note that this still means they can share the same configuration.
Leader / Follower
This topology allows a distributed application running on at least three Habitat nodes to use a leader/follower configuration. Leaders are elected with Habitat's leader election algorithm, and followers are restarted to reflect a configuration that follows the new leader. Subsequent elections due to leader failure will update both leader and follower configuration data.
You can read more about the internals behind the elections in our advanced developer documentation.
Builder consists of a continuous build service and a searchable package repository.
Users have the option to connect their GitHub repos to Builder to enable continuous builds of their plans as either they checkin new code to GitHub, which will then initiate a new build via a Github hook, or as underlying package dependencies are updated. If you've added your Habitat plan to the root of your source code repository and a package you depend on (e.g. openssl, ruby, node) needs to be rebuilt due to a new release, your software will be rebuilt automatically and posted to your software's
unstable channel for you to review and promote per your regular procedure.
Habitat Builder's depot is a repository that stores packages for download and upload by Supervisors and Studios.