|
|
Build frequency |
Build status and stability |
Rule to promote a Build to this server |
Usage |
|
DEV |
CONTINUOUS |
Developers have a full version of the application on their workstation, where they are free to make any change they need to implement new features and fix bugs. They must be autonomous and allowed to change code as they please. When done making changes, they unit and integration test their code locally and are ready to check it into the version control system. |
Not applicable |
Code walkthroughs, white box testing, black box testing, unit tests, integration tests |
|
DAILY |
DAILY |
The build script gets the most recent version of the application from the version control system and builds it. Once built, the application is deployed and tested on the daily build server. Tests include a full regression run of all unit tests, followed by an (automated) functional smoke test and then more complete (manual) functional tests. |
The only rule to promote the dev build is that it introduces meaningful changes (new features, bug fixes). The application must build without error and all unit tests must pass. The database administrator and the build master are notified of the changes to the code and the database. Release notes are updated. |
Regression tests (unit), integration tests |
|
QA |
WEEKLY |
QA server is where testers run extensive integration and system tests. It is essential for them to have a stable reference implementation they can refer to or perform longer running tests. QA build provides the best environment to show development progress to clients and management. |
Testers determine whether and when the daily build is stable enough to be promoted to QA. Release notes must be up-to-date. Bug and feature tracking software are up-to-date. |
Integration tests, system tests |
|
STAGING |
PER ITERATION |
Production release build candidates are deployed on the staging server to ensure production promotion goes smoothly. Tests on the staging server are done as fast as possible to ensure that no other issues are found when promoting to production. This minimum delay also ensures that issues that occur in production can be found faster. Staging hardware |
Prior to promoting the build to staging, a fresh copy of the production server software and database (schema and content) is copied to the staging server. This ensures that any issues with build promotion, especially regarding data, are caught here, not in production. |
Acceptance tests, load tests |
|
PROD |
MINOR + MAJOR VERSIONS |
This is where the end users work. It is expected that production builds are very stable and well documented. Major and minor builds are promoted from staging server to here. Sometimes, load tests are performed as it is often not cost effective to have production-grade hardware in the staging environment. |
Builds are promoted to production after having been thoroughly tested on staging server. Prior to promoting to production, the current system state is backed-up, to be reinstalled if promotion issues arise that require the system to be retrofitted with the current, “working” production build. This often requires downtime, although sometimes this downtime can be minimized with the proper use of server farms where part the farm is “switched” to the new build and tested, while the rest keeps running the old one. |
Load tests |