Robert Cowham's Weblog 1 of 1 article Syndicate: full/short

Perforce Automatic Merging   16 Mar 06
[print permalink all comment ]

Updated: 2005-11-29 - see link to scripts.
Updated: 2006-03-16 - some clarifications and link to Miki Tebeka's scripts

A common branching pattern is to have mainline and then task branches where work is done and then "published" by merging to the mainline as shown in Figure 3 of the article Building for Success.

In perforce the "publish" and the "catchup" are both performed by using the integrate command, typically with a branch spec. For example, you might have a branch spec

Branch:	task/fred
View:
	//depot/main/... //depot/task/fred/...

The key thing is that Fred should "catchup" before doing the "publish". This is so that the more risky merging is done in his task branch and not in the mainline. When doing the merge Fred should be bringing all changes by other people in the project into his branch and with the publish he should just be able to copy his code into the mainline.

There are several ways to achieve this:

  • Education - tell Fred what to do and rely on him doing it - so what happens if he doesn't - can you "persuade" him not to?!
  • Don't give Fred write access to the mainline (e.g. via protections or a trigger), and instead have the integration team do it. The problem then being that the integration team may not know the code as well as Fred and are perhaps more likely to make a mistake.

The basic way of detecting in Perforce whether a catchup has been done is to do a preview integrate from main to task/fred and check that nothing needs to be done, so check for no results from:

p4 integrate -n -b task/fred

If the above produces no results then proceed to do the publish:

p4 integrate -r -b task/fred
p4 resolve -as

The key step is that the "resolve -as" (safe automatic merge), resolves as many files as it can. It looks to see if their are only "theirs" (source) diff chunks or "yours" (target) diff chunks and will select the theirs or yours file appropriately. (Note that "theirs" and "both" or "yours" and "both" are also processed in the same way). The key point is that if there are "theirs" and "yours" (or "conflict") diff chunks, then safe automatic resolve will not process that file.

Of course having done the automatic resolve and with the changed files sitting in our client workspace it is usually a good idea to do things like a build and smoke test - there's not a whole heap of trust otherwise...

Thus, in our script we can check for anything not safely resolved automatically (resolve -n shows what still needs to be resolved).

p4 integrate -r -b task/fred
p4 resolve -as
p4 resolve -n
if any results from above command then exit with error

build
if any problems then exit with error

run smoke tests
if any problems then exit with error

If no errors at this point we are ready to submit.

There are some extra wrinkles to this:

  • the "resolve -as" may validly not work if you have done a merge with edit during the catchup ("edit from" in the terminology from p4 integrated). You need to detect such situations and do a "resolve -at" to copy them over.
  • as soon as one person has done a publish then all other branches will require to do a catchup to pull it in - this means publishes are going to become serial
  • there is a window during which the publish is being performed when someone else might sneak in and do a publish thus meaning you need to do a catchup - consider a simple "locking" strategy to prevent this.

Note that the build and smoke test steps need to take an appropriate amount of time. If it takes hours to do them then the process is unlikely to work. Thus a few minutes or tens of minutes is likely to be the limit - this may mean cutting down on the number of tests that are executed - but that is usually OK. Apply judgement!

Although at the last point you might think it would be a brave person to do the submit automatically in a script! It's probably a good idea to do things like run diffs and give a visual once over before finally checking in, but it should be a pretty easy decision at this point.

In terms of automating the above, I can heartily recommend the various scripting languages with built-in calls to the Perforce API: Ruby, Python and Perl. Getting the results of a command is easy, and exceptions allow you to make error handling pretty easy as well.

Very brief examples of scripts designed to perform the above check in Python and Ruby are now available online. Note they are designed to be run from Custom Tools menu in p4v/p4win. Please note that Miki Tebeka has kindly published more production quality script implementations in python including a GUI front end. He also includes an example of an installer to automate installation of tools for p4win and p4v - check them out. Thanks Miki!

p.s. The above works for any codeline for which you have responsibility for accepting changes - it doesn't have to be a mainline - it can be a subsidiary integration line with third parties contributing, or team members "proposing" changes. You could automate the script and put it on an intranet page - let people try it out, and if successful then have changelists auto-checked in.

 

Copyright © 2008 Robert Cowham