Habitat and the Modern Application

In the article, "What Is a Modern Application?" we talked about what characterizes a modern application and how such applications can address the problems of complexity and scale. In this article we'll talk about how Habitat lets any application, whether greenfield or legacy, have those characteristics.

The short version is that Habitat offers a new approach to deploying applications called application automation.

Application automation means that the automation is packaged with the application and travels with it, no matter where that application is deployed. The unit of deployment becomes the application and its associated automation. The runtime environment, whether it is a container, bare metal, or PaaS, isn't the center of focus. It does not in any way define the application.

Instead of managing the infrastructure and any supporting dependencies separately from the application, Habitat includes everything that an application needs, throughout its lifetime, within a Habitat package.

A Habitat package is isolated, immutable, and can establish relationships with peers. Habitat gives developers a straightforward way to describe how an application should behave in its eventual infrastructure environment. Habitat also has an outward-facing API that presents a standard interface to the world, so that other components, such as load balancers and monitoring solutions, can easily communicate with the application.

Wrapping any application in a Habitat package gives it the characteristics we want in a modern application.

Benefits of application automation with Habitat

Habitat speaks directly to the problem of complexity and high-velocity development. It handles all the management tasks that deflect teams from focusing on actually developing new products and features. It frees legacy applications from being tied to a particular environment, and makes them portable and simpler to manage.

Habitat also promotes collaboration and transparency. The Habitat files that describe an application's behavior are human-readable and can be stored in a version control system such as Git. They are also simple to write. The API presents a clear interface that make it easy for different team members to define the areas that concern them most, such as monitoring.

Habitat Details

Habitat is a full-featured runtime with deployment coordination and service discovery capabilities built in. Habitat is comprised of:

  • A packaging format
  • A supervisor

The format defines Habitat packages, which are isolated, immutable, and auditable. The term auditable means that the package can keep track of what happened to it, who did it, where it happened, how it happened, and when. By isolated, we mean that each package contains everything it needs to run. There are no outside dependencies. By immutable we mean that a particular version of the package never changes. If something does change—including available versions of dependencies—you create a new Habitat package, which has its own unique identifier.

The Habitat supervisor knows how to take the packages and run them. It's aware of the package's peer relationships, its upgrade strategy and security policies.

Security is important because once distributed pieces of an application begin communicating with each other you also need trust relationships. You need to have security that's built into the system, not only for policy changes but for ongoing compliance.

In addition, the Habitat ecosystem includes a build service that retrieves the source code for a package from a depot and creates the packages. There is a cloud-based, public-facing depot maintained by Chef. It is also possible to have a private depot. A Habitat depot is similar to Docker Hub and the Ubuntu apt service .

The Habitat Plan

To build a package, Habitat uses a build description, called a plan, which describes the application, its dependencies, and configuration options. The Habitat build service takes all these components and wraps them into a single, encrypted or signed artifact, which can be configured dynamically when the service starts.

Here is an example of a plan to create a Habitat package for ElasticSearch :

pkg_name=elasticsearch
pkg_origin=chefops
pkg_version=2.3.2
pkg_maintainer="Ben Rockwood "
pkg_license=('Revised BSD')
pkg_source=https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/${pkg_version}/${pkg_name}-${pkg_version}.tar.gz
pkg_filename=${pkg_name}-${pkg_version}.tar.gz
pkg_shasum=04c4d3913d496d217e038da88df939108369ae2e78eea29cb1adf1c4ab3a000a
pkg_deps=(core/glibc chefops/jdk8)
pkg_bin_dirs=(bin)
pkg_lib_dirs=(lib)
pkg_exports=(
  [http-port]=network.port
  [transport-port]=transport.port
)
pkg_exposes=(http-port transport-port)
pkg_svc_run="es/bin/elasticsearch --default.path.conf=${pkg_svc_config_path} 2>&1"
do_build() {
  return 0
}
do_install() {
  build_line "Copying files from $PWD"
  mkdir -p $pkg_prefix/es
  cp -r * $pkg_prefix/es
}

The first three values form part of the package's fully qualified name. The pkg_source value specifies the archive that contains the ElasticSearch pre-compiled binaries, configuration files, and related JAR files. The pkg_shasum value is the package's checksum. The pkg_deps values list the runtime dependencies for ElasticSearch. The pkg_bin_dirs value specifies the location for the ElasticSearch binary. The pkg_exports associative array contains a mapping of configuration keys to give public aliases to and make publicly available. The pkg_exposes values specify which exported configuration values representing service ports will be exposed if this package is exported to a Docker image. Finally, the pkg_svc_run value specifies how the ElasticSearch binary should be run when the supervisor starts the ElasticSearch package payload.

The callbacks override certain default behaviors such as compiling and building binaries from source code and installing them into the package directory. Instead, because the ElasticSearch archive does not contain any source code to compile, the do_build callback has been overridden to simply exit the function. Likewise, the do_install callback makes a directory in the package where the archive contents will be copied and copies the contents to that location. Other default callback functions such as unpacking the archive, verifying its checksum, and performing any cleanup work of the archive have not been overridden.

Habitat and the Production Environment

Applications packaged by Habitat display the following characteristics in a production environment:

  • They can be dynamically configured at runtime.
  • They can establish relationships with their peers.
  • They are security aware, which means they are encrypted at the artifact and secret level.
  • They can be replaced with new versions according to the policies of the IT organization.
  • They can express their state, be monitored, and interact with other components.

The following table summarizes how Habitat creates applications that fulfill the requirements of a production environment.

RequirementHabitat
Configurable at runtimeHabitat allows you to send in configuration information. If it's correct, Habitat changes the configuration of the service and restarts it using a defined update strategy.
Can establish peer relationshipsWhen you start a service, you can specify the peer relationship topology. A topology can be standalone (the application doesn't talk to anything else) or leader/follower (you supply the IP address of another member).
Security awareThe supervisor has a number of built-in security features. For example, it can be set up to use wire encryption when speaking with peers in a supervisor network; it can be set up to trust or distrust Habitat packages signed with a certain origin key; it can trust or distrust configuration policy changes applied by users or by other peers. Habitat uses modern cryptography implemented in the NaCl (pronounced "Salt") library.
Software upgrades and rolling deploymentsThe supervisor understands cluster topologies and has an update strategy. The supervisor triggers that strategy by watching a depot. By pointing the supervisor to the depot and specifying the service, it automatically checks, on a periodic basis, to see if there's a new package. If there is, it downloads that package and then goes into the upgrade cycle and applies the strategy. You can have rolling deployment without a central orchestrator.
StatusThe Habitat supervisor can determine the state of a service and bring it back up if it is down.
Monitoring, health checks, interactions with other componentsThe Habitat supervisor has an HTTP API that exposes information about the package it is supervising, as well as any configuration changes it has received. The HTTP API also provides a mechanism for monitoring systems to invoke health checks defined by the application author when the Habitat package was built. This provides easy integration with existing monitoring systems such as Nagios or Sensu, or even with entities external to Habitat, such as load balancers.