Ensembles

February 8, 2015

An ensemble is a complete view of an application. It is effectively synonymous to a mono-repo but micro-manages all of the components’ versions involved in the application. It is basically an attempt to support Codebase from the Twleve-Factor App, specifically, “any set of repos who share a root commit (in a decentralized revision control system like Git)”, without the complexity and drawbacks of a mono-repo.

Terms and standards used in this document

A component is any submodule of the ensemble.

ensemble:branch and component:branch are used generally to describe an ensemble or component repository at a particular branch.

The intent of an ensemble is to:

One main concept behind an ensemble are the roles involved:

release management owns the state of the application (i.e. the ensemble). Every deployed state of the application can be tracked back to a specific version of the ensemble.

development owns the state of each of the components. Every deployed version of any component involved in the application can be tracked back to a specific version within the ensemble and a specific version of its own repository.

Branching

All component submodules should ideally utilize trunk-based development.

Notes:

Managing tools

The tool-versions within the ensemble must be built with the commit from which it was built but careful consideration must be used to determine how to record the versions, especially the ensemble version. For example, a docker image might include a commit-id in its version so that other components that use the image can, and should, specify the precise version of the image. But by putting the ensemble version in the version of the image means that when the ensemble commit is updated, even for documentation changes, it will result in new docker image versions even though the image may effectively be exactly the same as prior ensemble versions. Instead, pass the ensemble version into the docker image’s build script so that it can burn it into the image for tracking purposes.

One solution to this is some tooling that can identify the changes from commit to commit, e.g. git diff --name-only ...) and rebuild only those components with source-code changes.

Ideally, if the image doesn’t exist locally, it can automatically be recreated or pulled from a known, secure repository.

Other tooling should be accessible via pathing in a similar way that components’ dependencies can reference artifacts from the ensemble.

Dependencies

All components should precisely define their dependencies by whatever method they chose. They do not typically refer to resources, artifacts, libraries, … from the ensemble but they certainly can if it can be done without relative references outside of the component’s own repository. In other words, every component within an ensemble must be buildable without the ensemble. For example, a reasonable means of utilizing resources in the ensemble is to utilize environment variables that the ensemble can override.

Referencing artifacts from the ensemble implies that the ensemble version must be included in versioning of the component. This is also reasonably easy using environment variables but it’s just generally easier for components to fully manage their dependencies within their own folder. The consequence is that it will result in a fair amount of duplication of artifacts (possibly all at different versions) which is intentional. To centralize dependencies also implies all components depending on those artifacts must iterate at the same time as the common artifacts, possibly requiring dozens of components to be updated all at once just to modify a common artifact.

In short, avoid relative references, like ../../../something, which reach outside its own repository.

Deployment

Note that “deployed” simply means an artifact is created, versioned, and available for use. One implication of this generalization means the artifact might just be for development purposes. It simply means the artifact can be tracked back to a specific version of the ensemble and to its own repository.

Versioning

TODO

Types of versions:

The application/ensemble may need to dictate some portion of these version

Use for versions:

Discussion, links, and tweets