Difference between revisions of "Working with Bazaar"

Revision as of 08:05, 26 October 2015

This page contains an introduction to the Bazaar distributed version control system. It focuses on the use of Bazaar to develop Inkscape, but should be useful to all would-be users of Bazaar.

You can choose either to use the command line or to use Bazaar Explorer (get the package bzr-explorer and follow the Bazaar Explorer Tutorial.

Why version control is useful

NOTE: skip this section if you have experience with any version control system.

If you ever worked on some large document in a group without any dedicated software, you have probably encountered the problem of simultaneous modification. When two people add something to the document at the same time, then send their version to each other, no one has the correct version. One of them has to redo their work in the other's version to merge the changes. This process is tedious and time-consuming, and the time spent on merging grows significantly when more people are added to the project.

Fortunately, for text files, this task is simple enough that it can be done automatically by a special program. When merging is done by software, people have more time to focus on productive work.

Version control systems (VCS) are programs designed to eliminate the overhead of merging. They manage a set of text files and allow many people to simultaneously work on them, then periodically submit their changes to a shared location. When two people change the same file, the changes are automatically merged together. Version control systems also store the complete history of changes. When someone makes a mistake and only spots it much later, it can be easily corrected, without affecting later work by other people.

Version control basics

Most version control systems use some shared location to publish the most up-to-date version of the project. (We will from now on assume that the project contains the source code for a program.) We will call this place the trunk. The first thing to do when starting work on a project is to download its source code, which is called creating a checkout. The files in your checkout are called the working copy. A checkout contains copies of files belonging to the project, where you can make changes. When you are ready to send your changes to others, you commit them, and they are stored in the shared location for others to see. The state of the project after someone's changes is called a revision, and each revision in Bazaar is assigned a sequential number, called the revision number. Revision 0 is the empty project and revision 1 is the initial commit of the project's files. To receive the latest changes introduced by others, you update your checkout. The update command does not remove any of your uncommitted changes - they are automatically merged.

Bazaar uses the following commands for the above functionality.

$ bzr checkout project_trunk_url

This checks out the source code of a project stored at the specified URL.

$ bzr commit

This sends your changes to the shared location, so others can see them. It will display an editor window, where you should enter the summary of your changes. This description will be visible when using the bzr log command.

$ bzr update

This updates your working copy to the latest public revision. Your uncommitted changes are left in place and automatically merged with others' changes.

Useful basic commands

$ bzr revert file

This undoes all of your changes to the specified file and restores it to the state it was in after the last update.

$ bzr add file

When you create a new file it is initially unversioned, which means the version control system doesn't track changes to it. You need to tell the VCS that you want to include it in the project, or make it versioned. The add command is used to do this. The next commit command will then add this file to the shared location.

$ bzr rm file

This removes a file and its contents from version control and deletes it from disk. The next commit command will remove it from the shared location. Use the --keep option if you want to remove the shared location but keep it as an unversioned file in your working copy.

Inspecting project history

Display last 20 sets of changes (revisions):

$ bzr log -l20

Display all changes to some file since the beginning of project history:

$ bzr log file

Inspecting changes

Display all uncommitted changes:

$ bzr diff

Display all changes, both committed and uncommitted, from revision 340:

$ bzr diff -r340

Display changes introduced by revision 436:

$ bzr diff -c436

Display changes between revisions 252 and 260:

$ bzr diff -r252..260

Undoing mistakes

To remove all your uncommitted changes:

$ bzr revert

To revert a specific file to the state from revision 230:

$ bzr revert -r230 file

Let's say you committed some change, but then after some time realized it was very wrong. To back it out without affecting any later changes, you need to find the revision number of the change using the bzr log command. Let's say want to undo changes from revision 340. To do this, write:

$ bzr merge -r340..339

In the same way, you can undo any set of contiguous revisions, e.g.

$ bzr merge -r340..320

To undo sets of non-contiguous revisions, you have to use one command per set. All commands after the first need to use the --force option (by default Bazaar will refuse to do a merge if there are uncommitted changes).

$ bzr merge -r340..339
$ bzr merge -r327..326 --force
$ bzr merge -r289..286 --force

To actually make the above changes in the shared location, you need to follow them with bzr commit. The exception is bzr revert, which restores the state to the last committed revision, so there is nothing to commit after it.

Conflict resolution

Sometimes two people change the same file in a way that cannot be automatically merged, for example, they change the same line to something different. When this happens, you need to merge the problematic bits manually. You might encounter output like this:

 $ bzr update
  M  src/painting-algorithm.cpp
  M  src/ui/widget/spinner.h
  M  src/ui/widget/spinner.cpp
  M  src/ui/widget/awesome-widget.cpp
 Text conflict in src/ui/widget/spinner.cpp
 1 conflict encountered.
 Updated to revision 1337 of branch [some URL]

After that, the directory src/ui/widget will contain four related files.

spinner.cpp: this will contain an unified view of the conflicts in the file. Conflicts will be marked with lines with multiple consecutive angle brackets.
spinner.cpp.BASE: latest common ancestor version of the file. In other words, both you and other people started from this version before conflicting changes were made.
spinner.cpp.THIS: the file with your changes, with conflicting changes by others removed.
spinner.cpp.OTHER: the file with other people's changes, with your conflicting changes removed.

To display the list of conflicts again, use the command:

$ bzr conflicts

To resolve the conflict, you need to modify the contents of the spinner.cpp. You can do this either by manually editing the file, by overwriting it with spinner.cpp.BASE, spinner.cpp.THIS or spinner.cpp.OTHER, or by using special options to the resolve command. Once you are done, you need to tell Bazaar about it:

$ bzr resolve src/ui/widget/spinner.cpp

You can tell Bazaar to use your version or others' version, respectively, with the following commands:

$ bzr resolve --take-this file
$ bzr resolve --take-other file

The above command will delete the files spinner.cpp.BASE, spinner.cpp.THIS and spinner.cpp.OTHER. Manually deleting those files has the same effect as executing bzr resolve. If you resolve all the conflicts at once, you can tell Bazaar to clean up everything:

$ bzr resolve --all

More information is available here: Bazaar manual - Conflict handling.

Determining who made a change

Sometimes it is useful to see who last changed some lines in a file.

$ bzr annotate file

This will display the entire file. Each line will be prefixed with the revision when the line was last changed and the person which committed that revision. Another name for this command, which might be easier to remember, is bzr blame.

Distributed VCS concepts in Bazaar

This section assumes that you are familiar with the basic concepts of version control, like working copy, committing, updating, conflicts.

Branch: is a working copy of the project's code. Typically you use one branch per set of related changes, for example a new output extension. Once your work is finished, you merge your branch into a larger project. Initially, there is only one branch, the trunk; all other branches are its (probably indirect) descendants. To create a new branch, you have to copy an existing one. When branches A and B have a common ancestor branch but each contain changes not present in the other, they have diverged. For example, when you work on an improved rectangle tool in your own branch and at the same time somebody else applies a PDF export bugfix to the trunk, your rectangle tool branch becomes diverged from the trunk.
Checkout: is a copy of code contained in a branch that is not stored on your computer (a remote branch). Committing to a checkout will immediately apply the changes to the remote branch. Commits will not succeed if somebody else modified the branch while you were working - you need to have an up-to-date working copy - or when you're offline.
Trunk: is the main branch, which represents cutting-edge working code. You should start from it when doing any new development.
Merge: is the process of reconciling changes made between two branches since they diverged. This operation is asymmetric. When you merge A into B, all changes made to A since it was branched from B are applied to B. A is not changed. When you work on some feature, you typically work in a branch, periodically merging the trunk into your branch (to sync up with the latest changes), then you merge your work into the trunk. Merging is similar to applying a patch - it only changes your working copy. To apply a merge, you need to commit the changes it introduced. Merging un-diverges branches.
Repository: is a place where branches of a project are stored. Having a shared repository reduces the storage requirements of multiple branches of one project. Instead of O(number of branches) space, the Bazaar data takes up O(total size of changes in all branches) space.
Bind: is the process of converting a non-diverged local branch into a checkout of the remote branch.
Push: is the process of publishing an exact copy (a mirror) of your branch in some other location. The difference between pushing and checkouts is that a checked out remote branch is updated every time you commit, while a pushed remote branch is updated only when you push to it - you can commit any amount of changes between pushes. You can only push to a branch if the mirror has not diverged from your local copy. This can happen if more than 1 person can push to the same location.
Pull: is the process of creating a local exact copy (mirror) of a branch, in principle a reverse of pushing.

First steps

First you need to tell Bazaar your name. This will appear on all your commits. You should use your real e-mail, but you can obfuscate it if you are afraid of spam.

Obfuscated e-mail examples:

$ bzr whoami "John Q. Public <john dot q dot public at-sign bigmail dot com>"
$ bzr whoami "John Q. Public <removethis.john.q.public@bigmail.com>"

Unobfuscated e-mail example:

$ bzr whoami "John Q. Public <john.q.public@bigmail.com>"

If you have an account on Launchpad and want to commit changes there, you need to specify your Launchpad login. You can skip this if you do not intend to commit.

$ bzr launchpad-login johnq

Then fetch Inkscape's trunk

$ bzr checkout lp:inkscape

To carry out a later update of Inkscape's trunk

$ bzr update lp:inkscape

Using a centralized (SVN-like) workflow

In this case, every commit that you make is immediately sent to the central repository. There are two ways of achieving this:

SVN-style checkout

Get the latest Inkscape code as a checkout, as described above.

$ bzr checkout lp:inkscape

Now work as in SVN:

 <do work>
$ bzr commit
 <error, someone else has changed things>
$ bzr update
 <check all is okay>
$ bzr commit

If you add new files, add them to version control with bzr add file. You can recursively add all new files by writing simply bzr add in the top level directory.

Binding a Bazaar branch

Branch Inkscape's code:

$ bzr branch lp:inkscape

Then transform your branch into a checkout:

$ cd inkscape
$ bzr bind lp:inkscape

Now work as in SVN:

 <do work>
$ bzr commit
 <error, someone else has changed things>
$ bzr update
 <check all is okay>
$ bzr commit

Differences from SVN

Commits will fail if your checkout is not up to date, even if there would be no conflicts.
The revision number is an attribute of the working tree, not of individual files. It is not possible to have different portions of the working tree at different revisions.
bzr commit file works like svn commit file && svn update project-root. After a successful partial commit, the entire tree's revision is increased by 1.
No special server is needed to publish a branch. You can push branches to anywhere, including simple FTP shares. In our scenario, Launchpad works like a central repository, but it's not required to publish everything there.
You create branches locally. To simulate creating a SVN-style branch on Launchpad, you need to create a local branch, push it, then bind it to the pushed location. At the end of these commands you end up with a checkout of lp:~inkscape.dev/inkscape/killer-feature:

$ bzr branch trunk killer-feature
$ cd killer-feature
 <do work>
$ bzr push lp:~inkscape.dev/inkscape/killer-feature
$ bzr bind :push

Append-only branches

In the centralized workflow, you often want to ensure that revision numbering is immutable. In Bazaar, the revision log and numbering can differ depending on which way you merge (e.g. feature into trunk vs trunk into feature). To enforce immutable, monotonically increasing revision numbers on a branch, you can create it as append-only:

$ bzr init --append-revisions-only

To set the append-only flag on an already existing branch, use the following command:

$ bzr config -d url://of/branch append_revisions_only=True

You can learn the URL of a remote branch by using the command bzr info. Launchpad branches have URLs such as bzr+ssh://bazaar.launchpad.net/~inkscape.dev/inkscape/trunk/

Using a decentralized workflow

In this case you will be working locally until you are ready to publish your changes. The basic idea is to perform the work in a branch of the Inkscape codebase, since committing to a branch stores the changes locally in the branch itself. Then, to share the work with the rest of the Inkscape community, the changes in the branch are merged across to a checkout of the Inkscape repository. As the previous centralized workflow section describes, committing to a checkout sends the changes directly to the remote repository, making them part of the official Inkscape codebase.

NOTE: this workflow has been updated to avoid the "other people's commits become subrevisions of your commit" problem.

This will assume this directory layout:

inkscape
 +-trunk
 |  +-doc
 |  +-share
 |  +-src
 |  +-po
 |  +-...
 +-myproject
 +-some-other-branch

In this layout, "trunk" is the checkout used for uploading changes to the central Inkscape repository, while "myproject" and "some-other-branch" are local branches used for working on specific features.

Repository setup

To speed things up, it's good to create a local shared repository. This way revision histories will only be stored in one place, instead of in every branch, so local branching will be faster and the branches will take up less space.

Create a shared repository and enter it:

$ bzr init-repo inkscape
$ cd inkscape

Working locally

Get Inkscape's code

$ bzr branch lp:inkscape myproject

Now work:

$ cd myproject
  ... work work work ...
$ bzr commit -m "Some changes"
  ... more work work work ...
$ bzr commit -m "More changes"
  ... create new file ...
$ bzr add new-file.c
$ bzr commit -m "Added new file with kittens"

Advanced local features

To undo a commit:

$ bzr uncommit

This will leave your working tree in the state it was just before you wrote "bzr commit". This is useful for example if you forgot to add a file, and want to keep a clean revision history. This will not work on an Inkscape trunk checkout.

Let's say you started from r770, made 200 local commits, but you decided that what you did since r954 is completely wrong. You want to go back to revision 954 and do it differently. Here's one way to do this. First branch locally to preserve your work (maybe later you'll find out there's some way to save it). Then uncommit everything up to r954 and revert.

$ bzr branch myproject myproject-dead-end
$ cd myproject
$ bzr uncommit -r 954
$ bzr revert

Sometimes a big change gets committed while you are working on a feature. If you want to check whether your code still works after the big change, merge from trunk.

$ bzr merge

In this case, you can omit where you are merging from, because the default is to merge from the branch you started from (the parent).

To cherry-pick specific revisions from another branch:

$ bzr merge -r340..360 branch_location

Or to cherry-pick a single revision:

$ bzr merge -c340 branch_location

Publishing your work on Launchpad

To publish branches on Launchpad, you have to add an SSH key to your account. If you don't have one yet, generate it:

$ ssh-keygen
  ... follow instructions ...

Once you created the key, go to your Launchpad profile page, edit SSH keys and paste the contents of the key file (by default it's in ~/.ssh/id_rsa.pub or ~/.ssh/id_dsa.pub) into the window. You can now use the Launchpad integration features.

Login to Launchpad, if you didn't do it yet. You only need to do this once on each computer. You need to use your login name, not the display name.

$ bzr launchpad-login johnq

Publish your branch on Launchpad. If you push it under your username, only you will be able to modify it. If you have commit access and publish your branch under the username inkscape.dev, all Inkscape developers will be able to change it.

$ bzr push lp:~johnq/inkscape/myproject
$ bzr push lp:~inkscape.dev/inkscape/myproject

The push location will be saved. After more commits, you can simply write

$ bzr push

It's sometimes convenient to update the Launchpad copy after each commit. To save on typing, you can bind your working tree to the remote branch. This way every commit will be immediately published on the remote branch. Note that you won't be able to commit while offline.

$ bzr bind :push

Note that it is very helpful if other Inkscape developers can commit to your branch. For example, if someone is reviewing your code, it is much easier to make some quick edits than to tell you to make the edits. (for example, Johan will become very angry when he cannot commit small edits, he will stop reviewing right there, and never look at your branch again!) The easiest way to make your branch commitable to others is to upload it as:

bzr push lp:~inkscape.dev/inkscape/myproject

However, you must be a member of inkscape.dev (have commit rights) to do this. If you are not in inkscape.dev, you'll need to create a team consisting of yourself and the Inkscape Developers group, and upload as that team. (You can create a team from the front page of Launchpad.)

Putting your work in the trunk

Once your new killer feature you made in a branch is ready, you need to merge your changes into a checkout of the trunk. If you don't have one, do (in the directory containing myproject):

 $ bzr checkout lp:inkscape trunk
 $ cd trunk

If you already have one on hand, make sure it's up to date:

 $ cd trunk
 $ bzr update

Now that you have a local checkout of the current trunk:

 $ bzr merge ../myproject
 $ bzr commit -m 'added my feature'

Alternatively, if you don't have a checkout but a branch:

$ bzr merge ../myproject
$ bzr commit -m 'added my feature'
$ bzr push

Note that you need to do the above sequence on a branch that has not diverged from the trunk, otherwise it will fail, as the trunk is append-only.

The third method is to merge trunk into your local branch, then push:

$ bzr merge ../trunk
$ bzr commit -m 'added my feature'
$ bzr push

WARNING: This last method should be avoided if possible. If any work was done in trunk since you branched, all of it will be converted into subrevisions of your merge commit and revision numbers will change. This method will also fail with append-only branches, such as Inkscape's trunk on Launchpad.

Undoing a wrong-way merge

NOTE: Inkscape and Lib2geom trunks are append-only, so you will be prevented from doing the mistake discussed here, but this information is provided here in case it is useful for other projects.

Let's say you merged trunk into a feature branch, then pushed that as the new trunk, and this "erased" some revisions. In reality the revisions are still present, but they became subrevisions of the merge commit. To restore the old revision numbers with your merge as the last revision, do this. First, find the revision into which you want to merge the feature changes. This will usually be the second parent of the mangling merge revision.

$ bzr log --show-ids
$ bzr visualise   (requires bzr-gtk plugin)

Now branch from this revision:

$ bzr branch lp:myproject -r johnq@example.com-20111230042529-0iwskf9kff2xdmzg

Merge your changes the correct way and commit:

$ bzr merge lp:myproject
$ bzr commit

Replace the trunk with the correct version with unmangled history:

$ bzr push --overwrite lp:myproject

You can now convert the fixed local branch into a checkout of your project's trunk:

$ bzr bind :push

Working with patch files

If you don't have permission to commit to the trunk, you can bundle your branch's changes into a patch instead:

 $ bzr send -o mychanges.patch

To apply patches produced by the above command, just do this:

 $ bzr patch somechanges.patch

Local branching

Naturally, all this also works locally. For example, when you're in the inkscape directory, you can write bzr branch trunk export-dialog to create a new branch of the trunk called export-dialog, where you'll work only on improving the export dialog. Similarly, you can merge between local branches.

Using Git-style branches

Git stores many branches in one directory, and allows you to switch between them. This is also possible in Bazaar. One way is to use native Bazaar facilities, the other is to use the bzr-colo plugin. We'll cover the native way, since the plugin has good documentation.

First, create a no-working-trees shared repository. The repo directory can be hidden if you want.

$ bzr init-repo --no-trees .inkscape-repo

Fetch the trunk and any other branches you want into our repository. (You can also push your local branches.)

$ bzr branch lp:inkscape .inkscape-repo/trunk
$ bzr branch lp:inkscape/0.48.x .inkscape-repo/0.48.x
$ bzr push killer-feature .inkscape-repo/killer-feature

Create your working tree by making a lightweight checkout of a desired branch. The --lightweight parameter is imp[ortant, otherwise you can run into problems.

$ bzr checkout --lightweight .inkscape-repo/trunk inkscape
$ cd inkscape

If you have an existing checkout of the trunk, use the following command:

$ cd inkscape
$ bzr reconfigure --lightweight-checkout --bind-to ../.inkscape-repo/trunk

Now you can work in the inkscape directory like you would on a normal branch. If you want to switch to a different branch, use:

$ bzr switch ../.inkscape-repo/killer-feature

To create a new branch in the repository:

$ bzr switch -b ../.inkscape-repo/killer-feature-2

Best Practices for Inkscape Project

Registering Bugfixes

Launchpad will automatically link a bug report to a branch and mark it "Fix Available" once somebody commits using the flag --fixes, e.g. (if you fix those two bugs in one commit):

bzr commit --fixes lp:123456 --fixes lp:123457 -m 'patch description'

Then, bugs can be changed automatically from "Fix Available" to "Fix Released"

Test branches before committing merges

Whenever a branch is merged into Inkscape trunk, the automated tests should be run to check for a few potential errors. This is achieved by running the "make check" command after merging (but before committing any changes). See the example below.

Proper way of merging

To repeat, don't do something like:

$ bzr branch lp:inkscape myproject
... work ...
$ bzr commit
$ bzr merge               # NOOOO!!! We are doomed!
$ bzr push lp:inkscape

It would obfuscate the revision history: trunk commits that happened between the time you branched and the time you pushed would get grouped into one. Additionally, to prevent this obfuscation, Inkscape's trunk is set to append-only, so the above sequence of commands would fail. Do the merge the other way around:

$ bzr branch trunk myproject
$ cd myproject
... work ...
$ bzr commit
$ cd ../trunk
$ bzr merge ../myproject  # correct!
$ make check              # Run automated testing
$ bzr commit

You'll need a trunk checkout, but the revision history will be much less confusing, and it's useful to have a trunk checkout anyway for minor fixes.

Fixing a bug in both trunk and a release branch