| #summary Release Document for Melange |
| #labels Importance-Featured,Phase-Deploy,Contents-Complete,Featured |
| |
| This document aims to describe the procedure for creating a new Melange Release. As the purpose of this document is to ease cutting a new release, it should be used as a guide on how to create a release. Paraphrasing Pirates of the Caribbean "it's more more like guidelines anyway". |
| |
| <wiki:toc max_depth="2" /> |
| |
| = Release Timing = |
| |
| In general a new Release will be cut on Monday and Thursday evening (that is, around 19-22 UTC). A good reason to cut the release at a later time could be that a certain issue needs to be resolved (but only if the issue cannot wait to be resolved until the next release). Likewise, a reason to cut a release at an earlier time could be the presence of a high priority issue that has been resolved and that needs to be deployed as soon as possible (see below for the procedure on critical fixes). |
| |
| = Release responsibility = |
| |
| In order to equally spread the burden of cutting a new release the owner that cuts the releases will be rotated with each release. The schedule is ordered alphabetically by default, and wraps around as soon as the last owner in the list has cut a release. Note that this schedule is not set in stone either, and can be modified at the discretion of the owners. If a change is made in the schedule it should of course be the goal to compensate for this change (as to not have one owner end up doing many releases in a row). |
| |
| = Critical Fixes = |
| |
| When there are critical fixes a 'patch release' should be made instead of doing a full release. The reason for this is that regular releases should always be tested extensively (see below for instructions), whereas with critical fixes there usually is no time for such exhaustive testing. The justification for this lack of testing is that critical fixes are usually small in size, and their behavior is easily verified, that is, the answer to "is the critical issue addressed by this fix?" is usually obvious. If such is not the case (e.g., the critical fix cannot be trivially verified) then it is up to the owners to decide if applying the fix without thorough testing is worth the risk of adding a new issue in doing so. |
| |
| = How to cut a release = |
| |
| There are several steps that need to be done, and one conditions that should be met, in order to cut a release. The one condition is simple, all owners should agree on a version of trunk that will be used to cut the release from. As described above, this decision is usually made on Monday and Thursday. Once a version has been chosen the chosen version should be tested to catch any obvious problems before cutting the release from that version (see below for testing instructions). |
| |
| * The name of the branch and tags are up for discussion on melange-soc-dev, once a decision has been made the document will be updated. * |
| |
| == Creating the release branch == |
| |
| Once the owner doing the release has verified that the chosen version is indeed fit to be used for the release, a new branch should be created from that version (using the release script). The name of the branch should be in the format "$major-$minor-$year$month$day", the release script will enforce this format and require you only to specify the correct major and minor version and optional patch level. For example, a valid branch name is "0-5-20090313" . Once the branch has been created, use the release script to export the branch to a separate directory, to prevent cruft in the worktree from affecting any further tests. Deploy this release to an appspot test instance with the release script and verify that everything works (see below for testing instructions). Once testing is completed use the release script to deploy this release to http://melange-demo.appspot.com and notify the testers that a new release is in need of testing through the melange-soc mailing list. |
| |
| == Creating the release tag == |
| |
| Once the testers have verified that the release works as expected, and that all issues that have been addressed with this release are indeed fixed, a tag should be created from the release branch. This testing should take between 24 and 48 hours with the preference being closer to 48. This way testers from different timezones have a chance to test (see below on test instance policies). Using the release script the owner creates a new tag from the release branch, which will have the same name as the branch. This tag should then be deployed to http://melange-demo.appspot.com with the release script. Also, send a mail to "Melange Dev List" <melange-soc-dev@googlegroups.com> announcing the new release so that Google can create a release from this tag. Tags may have an optional 'patch level' suffix, if rather then creating a new branch an existing branch is updated, the format for branches is "$major-$minor-$year$month$day$patchlevel", such as "0-5-20090313p0". |
| |
| == Changelog == |
| |
| After cutting the release a changelog (between the current and the previous release) should be made using the release script. In the changelog all new commits from trunk that are included in the release, that were not previously included, should be listed, together with a short description. |
| |
| == Dealing with critical issues in the chosen version == |
| If the chosen version contains any critical issues, these should be addressed before the release is deployed. Depending on how far the release process has been completed already, the release should be aborted, or it should be completed and the found issues should be addressed in a patch release. In the latter case the release should be deployed after these issues have been addressed and the owners have agreed on an appropriate new revision to cut the release from. |
| |
| = Rollback of a release = |
| |
| Should there be any critical issues that are noticed shortly after the release is deployed, and the release does not address an issue that is more critical than the new found issue, a rollback should be done. Due to the way AppEngine works it is possible to do such a rollback by setting the default version back to the previous release, for this reason the previous release version, and the one before that, should always be kept available. |
| |
| = Testing instructions = |
| |
| It is important that each release is tested as to ensure the quality of the new release. Since the current test suite is far from complete, manual testing is required instead. In the future this manual testing will be entirely replaced by running the test suite, which is currently only a minor quality indicator. For the purpose of stimulating the progression of the test suite a conversion guide is included (see below). |
| |
| == What to test == |
| |
| When describing what should be tested it is expected that "Create a new user" is understood to mean "Create a new user and verify that everything works as expected". For example, the new user profile should have the fields set in the way they were provided when creating the profile, etc. Also, one should test that only in the appropriate role it is possible to perform the described action (only a developer should be able to change the site settings). In order to make sure that Melange functions as expected the following things should be tested: |
| |
| * Signing in as a new user |
| * Creating your own user profile (as a new user) |
| * Editing your own user profile as an existing user |
| * Listing of a user's roles |
| * List of a user's outstanding request |
| * List of notifications |
| |
| * Creating a new document in a user's private space |
| * Editing an existing document |
| * Listing a user's documents |
| |
| * Create a new user |
| * Edit an existing user |
| * Change a user e-mail address |
| * Set a user account to invalid |
| * Make a user a developer |
| * Delete a user |
| |
| * Editing the Site Settings |
| * Adding a feed to the Site Home page |
| * Setting a Site Wide notice |
| * Putting the Site in maintenance mode |
| * List site documents |
| * Set the site home page |
| * Create a site document |
| * Feature a site document |
| |
| * Inviting a new Host |
| * Accepting a Host invitation |
| * Creating a Host profile |
| * Deleting a host profile |
| |
| * Creating a new program |
| * Editing a new program |
| * Setting a program home page |
| * Set all program agreements |
| * Edit the program timeline |
| * Create a program document |
| * Feature a program document |
| |
| * Creating a new org application |
| * List of your own org applications |
| * List of all org applications |
| * Pre-Accepting an org application |
| * Pre-Rejecting an org application |
| * Accepting an org application |
| * Rejecting an org application |
| * Ignoring an org application |
| * Deleting an org application |
| * Bulk-accepting all org applications |
| * Bulk-rejecting all org applications |
| |
| == How to test == |
| |
| In order to ease testing the /seed_db, /clear_db, and /reseed_db urls can be used. By default it creates a fair amount of the most-used entities. Refer to the seed_db module for a description of all supported arguments (among which a way to add an arbitrary amount of users to the datastore). If during testing an entity type is missing (or there are not enough entities of that type), these should be added to ease testing later on. |
| |
| == When is the testing done == |
| |
| Testing is of course never done, but a rough guideline at the moment is that is done when all functionality that is in use has been tested. When certain functionality is covered by a test suite, manual testing of that feature can be replaced by running the test suite. If the test suite fails, verify that the tested behaviour is indeed not working and take appropriate action (see below). |
| |
| == When a test fails == |
| |
| When a problem is discovered, create a http://tinyurl.com/new-issue describing the problem, describing how to reproduce the problem. If the fix is obvious and trivial it is acceptable to instead fix the issue, commit the fix, and continue testing. If however, the fix is unclear, or would take a significant amount of time, it is preferable to file an issue before fixing it, to prevent double work. |
| |
| == When a test passes == |
| |
| When an issue marked as fixed is found to work correctly on the melange-demo site its status should be changed to verified. Testers who have contributed more than a dozen useful Issues are candidates for being given issue tracker rights to enable them to modify status as well as the default of being able to create issues and add comment. |
| |
| == Converting manual testing to tests in the suite == |
| |
| To prevent manual testing, automated tests can be written that perform the required actions and validate the result. For this purpose a test harness has been created that is suited to test both the views and the logic. Model code need not be tested as AppEngine has it's own test suite, which we should not duplicate. When testing view code it is preferable to stub out Django's rendering code. This eliminates the need to test the HTML output, and allows the test to instead focus on whether the proper template is used, and the contents of the context object. Refer to the existing test code for examples. In order to ensure that the interface looks as expected Selenium can be used, this ensures that not only the views use the proper value but that they also render properly. |
| |
| = Test instance policies = |
| |
| There are several appspot instances that have been reserved for testing purposes. The two that are currently in use are http://melange-dev.appspot.com and http://melange-demo.appspot.com. |
| |
| == Policies for http://melange-dev.appspot.com == |
| The aim of melange-dev is to always have the most recent revision, regardless of whether this revision contains any issues. This allows both testers and developers to quickly identify any problems, especially those that only occur on appspot. The best way to ensure this is to have the continuous build automatically deploy the latest release to melange-dev. It is acceptable to wipe the datastore on melange-dev when the need to do so arises. |
| |
| == Policies for http://melange-demo.appspot.com == |
| |
| On melange-demo only properly tested branches (see above) should be deployed. In order to facilitate regular testing it is desirable that on melange-demo the datastore is treated with the same care as on an official instance. This means that in the case of a schema change (see below) the same measures should be taken as those that will be used on an official instance. An added benefit is that all conversions tools are tested on melange-demo first, so that issues with the conversion tool can be spotted as well. |
| |
| = Site Maintenance = |
| |
| It may be necessary to put the Site in maintenance mode (for example, when AppEngine goes into read-only mode). When the site is in maintenance mode only Developers can access it, all other users will be directed to /maintenance, which displays the current Site Wide notice. |
| |
| == Notifying users == |
| |
| When site maintenance is required this should be announced at least 48 hours in advance by means of a "Site Wide Notice" (see below). The notice should explain the reason for the maintenance, the start and end date in UTC (with a link to http://www.timeanddate.com/ for easy conversion to other timezones) and, optionally, a link to more information. |
| |
| == Enabling maintenance mode == |
| |
| Before enabling maintenance mode, a Site Wide Notice should be set explaining the reason for and duration of the maintenance (see above). This notice can be set by a Developer in the "Site Settings" menu. Once the notice has been configured, maintenance mode can be enabled by setting the start and end date (note, these should be entered in UTC). |
| |
| = Downtime = |
| |
| Due to the nature of AppEngine there should be little need for actual downtime. During a push the old version continues to serve until the new version has been set as the default. Downtime should only be necessary when major datastore modification is needed, for example in the case of a schema change (see below), when putting the site in maintenance mode is not an option. If such downtime is required, it might be desirable to deploy a version that consists of a simple app that accepts any url and redirects to a static maintenance page. |
| |
| = Schema Changes = |
| |
| If new properties are added to an entity that has "required=True", existing data will fail to load. In order to prevent the errors this should be anticipated by filling in these fields for all existing entities before pushing the a release that has the schema-breaking change. This can be achieved by adding a new url to the sitemap that, when called, updates a batch of entities that need updating. Due to the invasive nature of such changes it is desirable to avoid such changes if possible. For example, it might be possible to enforce the "required=True" in the view, rather than in the model. This way the entity cannot be saved without filling in the new property, but it will load fine. Once all entities have been updated the "required=True" can be added to the model, if desired. |
| |
| = Data backup = |
| |
| In order to prevent data loss when something goes wrong, regular backups should be made of important documents and other entities that would be cumbersome to recreate. Currently the only way to do this is through (csv) export, and there is no easy way to import these backups in the case of failure. It is desirable that an automatic solution (such as Google App Engine Backup and Restore) is implemented and used on a regular basis, but the details are beyond the scope of this document. |
