Hg as a collaboration tool
02.05.08 - 06:47pm
I want to clarify the point of this post. I am not attempting to make the point that hg is a bad tool. I am sure it is very helpful to some users. My point is that, IMHO, it does not solve an existing problem with Mozilla that justifies the inherent pain of switching from cvs to hg.
Over the last few weeks the mobile team has been using Mercurial to manage the code we’re using to build xulrunner for Windows Mobile. Up to the middle of last week it worked fine as dougt was the only one checking in patches while I was off in my own personal unicode hell.Then wednesday I finally got the code to compile with UNICODE defined. At that point I all I wanted to do was update, merge and create a diff. I’m not sure where thing went wrong, but wrong they did go.
Its almost a week later, and I still don’t have my patch. I’ve given up on that eureka moment when I suddenly discover that elusive string of commands. Instead its 3am 4am and I’m hand applying my changes to a freshly checked out working directly from our new svn repository.
All joking aside, I’d like to do a bit of a post-mortem on this. From talking to a few people at work week it seems everyone that is using hg for their work at mozilla is using it for changes they’re making to one or two modules that for the most part no one else is working on. In fact, I didn’t hear of any instances of two or more people collaborating to work on the same set of files at the same time. Granted in the version control world, this is the sort of scenario that makes us all recall late nights of hand merging conflicts, but we’ve all done it. Its boring, its tedious but we know how to do it.
With hg, I don’t know how to do it. I have received a fair amount of advice and and several suggestions from various people on the subject. In fact I want to sincerely thank jorendorff for spending upwards of an hour with dougt and I trying to sort this out. But it seems once you dig down enough the conclusion becomes, “hg wasn’t designed to do that.” That being produce patches that peers can review and can eventually be checked into a central repository that everyone can pull. What it was designed for is combining the efforts of several people who are working on very distinct modules and are publishing their changes either through patch queues or their own repository (hg serve).
Please don’t take this blog post as a bleary eyed developer slinging mud. I hope this will spark some discussion around which tools are good for which jobs. From what I’ve seen, hg is not very good for collaboration on the peer reviewed central repository model that mozilla has traditionally worked on. It does seem good for experimental work like that being done by the moz2 folks. Eventually we need to make a coherent product out of those experimentations, and I don’t see how that will happen with the current set up. Maybe I’m wrong. In fact I’m sure I’m wrong on several points (I just don’t know which ones), please point them out.
There are a few things I’m sure of. First, hg works on a model completely different to that of cvs and svn. Not only does every one using it need to get their head around that model from the beginning (admittedly, my first failing). They also need to follow some new and unique procedures. In fact a whole new set of “social” norms need to be developed (or at least published) for this tool.
Finally (and probably most importantly), in its current state hg introduces a barrier to entry for new contributers that seems entirely unnecessary.
I’m sorry, but I just cannot agree with this post. I just recently switched over to hg myself, so I understand the difficulties with first using it. And I’ll note that I’m working in modules where I’m not the only one using it, and I haven’t run into an issues yet. I’m using the mq extension with hg though, which isn’t part of the default setup.
As for your last statement, this is a minor barrier to entry compared to all sorts of other barriers that exist in the mozilla world. Lots of large projects are moving to distributed VCSs, so people will have to learn one way or another. hg isn’t all that different from, git, which is what the linux kernel uses (last I knew at least).
I’ve been using Mercurial for a few months and it’s been great. Most of the time I use it just like I used to use CVS — making local changes in a sandbox and producing diffs; I haven’t had to adopt a new model. I have also maintained a local branch for a couple of months — something that’s not really possible with CVS, and it was pretty easy too.
I’m not sure what went wrong with your use of Mercurial but I don’t think there’s anything fundamentally wrong with it.
I’m not sure new contributors are severely affected by Hg. I mean what does it take to get a cvs checkout on your machine to start hacking on? Well you have to have cvs installed, set your CVSROOT, checkout client.mk, have gnu make installed, run client.mk with the appropriate options. Wait… For hg, you have to have hg installed, clone from hg.mozilla.org … oh you’re done.
I think bsmedberg did another post pointing out that merging wasn’t really going to get much if any easier in an all Mercurial world, I don’t think it is a silver bullet but at lest for me I find it vastly better to have my own hg repository that I can pull and push to from all my development machines and still keep in sync with the central repository.
I’ve been using Mercurial for a few months now, and it’s been all good. My colleague and I work with it successfully and he’s been able to get a reasonable grasp on it even though he had no former experience using a VCS (which might actually be good).
Certainly, using hg is different from using SVN or CVS, but I also find that it is indeed better than using either of those.
When developing patches for the Mercurial project itself, rather than finding it a barrier to entry, I found it was an encouragement. The social mode where I can just publish my own tree that has some additional patches which I can ask developers to review seems to require less friction than dumping diffs to email. Also, I’ve found it to be very great that even as someone without access to the crew or main repository, my name would appear on my changesets. Of course, this is just a little social honor, as otherwise my name would probably have appeared in the changelog message, but I still think it’s an extremely powerful message to send.
Merging can sometimes be a drag, but the integrity enforced by the model is also very much a Good Thing.
If you’d like more help with hg, I’ve joined #mobile on irc.mozilla.org, as djc. I’d like to help out to make Mozilla’s use of hg a success.
Hi,
is your repo accessible on the net? (and a link to the original would be nice)
Yeah, and it would be good to know what would you like to achieve?
Cheers,
teki
Hi,
I am using it (mercurial) at work in a slightly different manner. The main development happens in CVS. I have a branch in CVS where I checkin all my changes. Once I feel that my (team) branch is stable enough, I check in the same into CVS. Along the whole process of development, I keep pulling from CVS HEAD branch into my branch and checking in the same into my branch in CVS. So, merging is a continuous process with quite a few conflicting files. I have never had any problem so far. I have also used the ‘mq’ extension but had some minor hiccups with it though. The patches in the ‘mq’ series would get outdated as the files would get modified and my patch would become redundant. I am sure there are better ways of using the ‘mq’ extension to overcome the problem I experienced. Overall, I feel mercurial to be a good dVCS tool.
What is stange is that hg, bazaar, and git are all made around a paradigm that makes merges powerfull and easy (when not completely automatic). They are from the start especially tailored to a lot of people collaborating on the same module…
I know that on git, unless the very lines I modified (or the ones just next to them sometimes) also were modified by others I get no conflict at all (three-way merge FTW), and when there is a conflict, I do “git mergetool” which opens a manual three-way merge (in gvim for me). It does wonders…
I’ve been using Mercurial for almost an year on projects where I’m the only developer…. I had to adapt the workflow from CVS habits to hg but now I feel really confortable.
And for a couple of projects, I felt the need to use the MQ extension…
My main problem with hg was that… it is hg and not cvs ! You have to try some of the different possibilities to work with hg (from central “official” repository to completely distributed development) and create use policies, using both pre and post action filters and standard rules…. I did some tests and it took me some time to create the workflow that fills my needs….
Support from hg developers and “power-users” is great and responsive and I believe that if you post the description of your problem you will find a solution.
Francesco
PS: to be honest, from your blog post I couldn’t understand what your problem was…. and in what action !
If it was in update/merge, I urge you to setup a visual three-way merge tool and test that it is correctly invoked on a test repository… I lost some work because of this
could you describe the problem(s) you have/had in more detail?
from what you wrote it’s hard to find out what the actual problems were.
Hi, Brad -
Sorry you’re frustrated. Unfortunately, your posting doesn’t contain any details, and the assertions that you’re making in your state of disappointment are (as others have noted) way wide of the mark.
The best way for you to make progress is to describe what’s going wrong for you in enough detail that someone can help.
You’re more than welcome to ask for help on #mercurial, the Mercurial mailing list, or to contact me in email.
Regards, and I’m sure we can work your problem out quickly and without fuss.
This has surfaced on the Mercurial mailing list. We are all curious on what exactly went wrong. Could you provide more details on what you were attempting to do and how Mercurial was not able to help you do it?
I’ll admit that I would have thought that you would have published the changeset for review rather than a patch created from it.
Brian, Gabor and Mark:
The basic problem is that I spent a while producing a patch to make mozilla compile on windows with UNICODE defined. In that time doug checked in several changes. Every patch I created with hg using hg diff or hg export backed out all of doug’s changes.
I am not using mq, which may be the solution to all the pain. If anyone has a solution that would fix this particular issue, I’d love to hear it.
Brad –
It seems you should have pulled the changes from dougt before creating your patch/export, and created a merge changeset from there.
Brad, it seems like you’re blaming hg for something it didn’t do.
I thought that the mess with your Unicode patch finally turned out to be attributable to a Windows Vista “feature” (per-user versions of files) that had *also* screwed your team over in 2 or 3 other equally mystifying ways in the preceding weeks.
Perhaps it would be useful to write up some best practices for Mercurial wrt Mozilla, such as pull and then merge?
The workflow, btw, looks like this:
hack hack hack
decide it’s time to generate a patch
pull
merge
look at the parents of the merge commit. Find the parent that is the incoming changes, not yours, and do an hg diff -r incoming_rev:tip.
This is made easier if you do your work in a named branch, so you can do an hg diff -r default:workingbranch.
And FWIW I used Mercurial for months in a module where I was sharing changes with someone — granted they were using CVS, but I would bring those changes into an “incoming” mercurial repo first, so the workflow was basically what you wanted to do — generate patches to post to bugzilla.
Hope that helps, sorry to see you so frustrated.
I strongly disagree with the big bold letters at the bottom of your post. I find that mercurial makes it *possible* for contributors to effectively manage changes to external projects without having to manage patches and deal with out of band transports and merges and the like.
It’s still not completely clear what you were doing, though. I don’t know why “hg diff” or “hg export’ would be used at all. “hg export” at least externalizes a changeset so that it can be reapplied with hg import (the important part here is that it will apply in the right place).
An ideal workflow would be for you to do whatever it is you’re doing, commit as you go, and enjoy life. dougt would do the same. You two would be working in different directions with a common ancestor somewhere up the tree. At the point(s) where you want to bring the work together, you merge the two trees. hg will do most of the three-way merging automatically (hopefully), and the conflicts will be presented to you in the three-way merge tool of your choice.
jorendorff:
yea, vista has screwed us over and over. However, I believe I have adjusted for its many features in this instance. As we discussed in mountain view, I made a diff of the changes I had made while in mountain view (command line diff between my working directory and the tar ball I was working off of) and applied that to the working directory on my vista machine at home. I then did:
hg pull
hg merge
hg diff 10000:10002 > unicode.diff
Unfortunately the resulting patch still backed out dougt’s changes.
You say “Unfortunately the resulting patch still backed out dougt’s changes.” actually means you have dougt changes with a “-” in front of it ?
Looks like you have a non linear commit history, probably you merge from another head or something similar that hg experts may check.
Can you please post me the result of “hg log –rev 10000:10002″ ?
(adding the -p switch it will also print the patch related to that revision)
Double check the “parent:” line to check for non-linear commits, or merges done on other heads… adding glog extension would give you a graphical representation (or using tortoise hg)
Francesco
# hg pull
# hg merge
# hg diff 10000:10002 > unicode.diff
A commit is missing, but I think you just forget it.
> Unfortunately the resulting patch still backed out dougt’s changes.
Which means that you are generating the wrong diff.
Still the best would be to publish your repository to have a look.
teki
If the merge went haywire, maybe this is more of an issue with the merge tool or the way the merge tool is started up. Could you possibly have pointed directly at kdiff3 as the merge tool? That could be the problem because the argument order is not right. More details would be helpful.
I have been using Mercurial for over a year and find it great, but I’ve also been using a distributed SCM (not a central repository system) for 15 years. I’ve heard from many CVS/Subversion users complaining about how complicated using Mercurial is, but the bottom line is that it’s the distributed nature of SCM’s like git or Mercurial that is the basic issue here, and getting used to a new tool.
It is a learning experience, I suspect you need to give it more time. It is not something you can jump into green and get comfortable in a few minutes or even hours. Brian’s book at http://hgbook.red-bean.com/ is required reading.
-kto
Brad,
Just tried a little test case here in XP system. Seems be able to reproduce your trouble. In mercurial windows binary install, there is no default hgmerge.cmd batch file. So hg merge will fail when you call “hg merge”. The file will be marked as changed, but it is actually the same as your change without dougt’s change. Then after commit, your diff will show dought’s change gone. Something like below. But in my test case, hg will complain merge failed.
Could you post your hgmerge.cmd batch file? You can find out this problem by running “hg debuginstall”.
regards,
PS: if you want, you can ask for help from mercurial@selenic.com mailing list.
@ your change without dought’s change, bad merge
|\
| |
| |
| |
| |
| |
| |
| o dought’s change
| |
| |
| |
| |
| |
o | your change
|/
|
|
|
o base
Guo,
Thanks, that’s great work. I saw the warnings about hgmerge.exe not existing and switched to using cygwin’s hg. Still, because the files were marked as changed the damage was already done.
Any thoughts on forcing the merge despite them being marked as changed?
this is the output form hg debuginstall:
Checking encoding (cp1252)…
Checking extensions…
Checking templates…
Checking patch…
Checking merge helper…
No merge helper set and can’t find default hgmerge script in PATH
(specify a merge helper in your .hgrc file)
Checking commit editor…
Checking username…
No username found, using ‘Administrator@brad-wsq225woz7.localdomain’ instead
(specify a username in your .hgrc file)
No problems detected
there is no hgmerge.cmd
If, like it seems to be, your problem was due to a misconfigured mercurial setup (as I suggested you in my yesterday message) can you please amend your blog post ?
I’d like also to suggest you to use tortoise hg (but I don’t know if it works on vista) that includes three-way merge GUI tools and other extensions already setup.
Ciao,
Francesco
Francesco,
What would you have me change? In essence, I completely agree with your first comment. Hg is not cvs, and is very different.
I question the need for Mozilla to switch over. Especially considering that you can use hg with a cvs based project, but not the other way around (as indicated by dhruva and Collin’s comments).
Brad,
Here is the instruction to setup merge for mercurial.
http://www.selenic.com/mercurial/wiki/index.cgi/MergeProgram
You can check whether your favorite tool is there.
I use winmerge myself. My hgmerge.cmd is this one. But I have a little trouble figuring out how to let winmerge return success even if it really succeed.
So most of the time, my work flow is this:
Let say your base is rev 0.
Your change create rev 1.
dought’s change create rev 2, and a new head.
I will just do
” diff -r 0 -r 2 | patch -u -p1″
and only use winmerge when patch failed.
There gonna be a better way. But I just use mercurial to track my own work, I don’t have a big merging challenge.
@echo off
setlocal
:: Look in the registry for WinMerge location
for /f “skip=2 tokens=2*” %%A in (
‘”reg query “HKEY_CURRENT_USER\Software\Thingamahoochie\WinMerge” /v Executable 2> nul”‘ ) do set WMergePath=%%B
if “%WMergePath%”==”" (goto :notfound) else (goto :merge)
:merge
“%WMergePath%” /e /ub /wl /dl “Repository” %3 %1
if %errorlevel% neq 0 (exit 1) else (exit 0)
:notfound
echo hgmerge: cannot find WinMerge location in the registry.
exit 1
Hi Brad,
> What would you have me change?
don’t know, this is your blog…
Probably someone reading your article AND carefully reading all the comments can realize that it’s actually not an hg fault, strictly speaking, but a configuration error… and that hg actually alerted you about the problem !
But since there are a lot of posts, it would be nice if you can put some note in the text, as an addendum or a post scriptum, describing what the problem was. And perhaps remove the bold in the last paragraph…. to me, hg is not the greater barrier to participate to Mozilla development !
And yes, hg is not cvs++, it’s a completely different idea of vcs ! hg needs to be configured, and development repository strategies should be tested and deployed. I again suggest you to have a look at tortoisehg, that comes preconfigured with same external tools.
Please have a look at mercurial web site for some repository samples.
Francesco
Brad,
Here is some help from Lee. I would suggest you try his binary installer. There is one issue in 0.9.5 regarding “\r\n” converted to “\r\r\n” in hg diff output. That is fixed in latest build.
Lee Cantey to me, Zbynek, mercurial
show details 10:34 am (3½ hours ago)
The latest installer (at http://mercurial.berkwood.com) supports a
number of merge tools out of the box now due to the recent changes in
the core merge logic. The configuration is located in the
Mercurial.ini that the installer writes for you so backup your copy if
needed and let it write a new configuration or take at look at
mergetools.hgrc in contrib directory of the sources (http://selenic.com/repo/hg/file/b7f44f01a632/contrib/mergetools.hgrc
is current tip).
Regards,
Lee