I love using Mercurial’s MQ extension for managing patch queues, even though I have a strong suspicion that it’s fundamentally the wrong idea. I’m only going to discuss one part of that wrongness now, though: it forgets things. Lots of things.
Much of the point of using a revision control system is to not forget anything. I should be able to freely try various lines of development, and get back any of my earlier work. Normally, that would just mean being able to revert to earlier revisions of my source tree, although even there I should really be able to revert portions of changesets. But when using additional tools like mq that manage how I got to a particular source tree, I should be able to back up to any previous state with the tool’s assistance. Fundamentally, it’s not about moving back and forth through a history of artifact versions. It’s that I should never lose any work, even if I do something that in retrospect turns out to be dumb. Or especially when I do something dumb, I should say — that’s why I’m using a revision control system instead of a dumb backup system. It’s supposed to understand source code and what perverse things developers do when writing and modifying it.
Here’s a concrete example:
- Developer edits code
- hg qnew my-amazing-patch
- Developer edits code some more
- hg qrefresh
- Developer says “oh f#@@#!!!!”
The problem is that when the developer refreshed the patch, Mercurial forgot the original patch. It also forgot the source tree that existed when the original patch was applied. So if those further edits turned out to be a Bad Idea, well, oops!
Yes, there is a way out: mq patch queues can themselves be revision controlled. Then as long as the developer remembers to hg commit --mq after every change to the patch queue, everything is golden.
You could even argue that this is the Right Way to work. After all, you don’t expect — or even want — your revision control system to remember every character you type. You’d never be able to identify the right point in time to back up to amid the mass of older revisions. Leaving the decision to the developer as to when a state is important enough to remember just makes sense.
Except it doesn’t. The developer already decided that the state was important by running qnew or qrefresh. Why burden the poor sap with yet another decision? Especially when making that decision requires typing in another command, which means that the mental threshold for interestingness is higher, which means it’ll pretty much never happen.
See https://bitbucket.org/sfink/mqext/ for the obvious solution. That’s actually a grab bag of mq extensions, all of which should really be submitted upstream. But I haven’t bothered.
The part that’s relevant to what I’m about here is that I added -Q options to all of the patch queue-modifying functions I could think of. Specifying -Q will commit the change to the patch queue repository, with a commit message describing the basic change (or you can set the message with -M).
Or you can go a step further, as I did, and use the [defaults] section in your ~/.hgrc to set the -Q flag automatically for whichever commands. See the help message (or the README) for details on installation and usage. Update: and now it’s easier, because you can set qcommit = auto in your [mqext] section and it’ll add the -Q option to the relevant commands. Which is good, since there are more of them than you think.
If you install this, you may want to try out the ‘qshow’ command, too. It’s my favorite of the other things implemented in that extension (I alias it to just ‘show’ because my left pinky is slow.) I use it constantly to review the various patches in the queue. hg show <n> is the way I usually use it; it prints out patch #n in your queue (the numbers come from hg qseries -v, though you really ought to just put -v in your [defaults] section too. Or alias series=qseries -v as I did.)
Feel free to use it, fork it, complain about it, or whatever. I’m still trying to figure out whether I really like it or not. It slows down qref operations, which kinda sucks. But I guess if I really cared I would turn off the default -Q for that one command, and just specify it manually. And I haven’t done that yet.
Oh right. One crucial thing I should mention: actually using any of this saved state is a dangerous affair. Why? Well, because you probably have a couple of patches in your queue applied at the time you decide to back up to an older state, and modifying applied patches is not very healthy. Especially if you reordered your series file. In fact, I would probably recommend doing these steps before (or just after, it doesn’t matter) reverting to an older revision of your patch queue:
hg update -r qparent -Crm $(hg root --mq)/stateThat will “unapply” all patches, forcefully. You can then qpush (or better, qgoto) the place you want in your queue. Note that shell $(…) is the modern version of backticks, in case you’re unfamiliar.Finally, here’s a sampler of the sorts of log messages the extension extension produces:UPDATE: multipage-test js/jsd/jsd_xpc.cpp | 1 + js/jsd/test/Makefile.in | 3 +- js/jsd/test/browser_multipage.js | 428 +++++++++++++++++++++++++++++++++++++++ js/src/jsapi.cpp | 4 + js/src/jscntxt.cpp | 4 +- js/src/jscompartment.cpp | 1 + js/src/jswrapper.cpp | 9 + 7 files changed, 447 insertions(+), 3 deletions(-) NEW: rename-multipage RENAME: bug615277-JM-execHook-3 -> bug615277-JM-execHook DELETE: bug-612717.diff UPDATE: better-note-dump
Or as the output of
hg log --mq(which only shows the 1st line of each commit message):changeset: 92:e2ed45b4a8bf user: Steve Fink date: Tue Dec 07 14:54:46 2010 -0800 summary: UPDATE: bug615277-JM-execHook changeset: 91:6e36813b7291 user: Steve Fink date: Tue Dec 07 14:51:45 2010 -0800 summary: NEW: rename-multipage changeset: 90:b66861e98c29 user: Steve Fink date: Tue Dec 07 14:38:15 2010 -0800 summary: RENAME: bug615277-JM-execHook-3 -> bug615277-JM-execHook changeset: 89:c02111e0d18d user: Steve Fink date: Tue Dec 07 14:37:27 2010 -0800 summary: NEW: bug615277-JM-execHook-3
There’s something ironic about hosting your Mercurial extension on GitHub. 🙂
That being said, this sounds like a cool idea, and I’m definitely going to give it a try. I screw up and qref changes into the wrong patch all the time. I have learned that you can often qrefresh/qselect your way out of it, depending on how badly you screwed your patch up.
Have you considered using PBranch instead of mq to address these concerns?
You still are missing some information about the history of your changes (specifically the parent of each patch in the queue; this missing info is what eventually causes failed patches). I’ve done some work to figure out a solution to that but really, everything just gets messy when you try to fix it (this flaw in mq runs very deep; I’ve only been able to fix it for the case where you do not use guards or reorder patches; see the code in the attic extension about using the merge functionality to reparent patches).
I believe that once you get past the initial learning curve of PBranch, you will find that it solves all of your concerns and some that you haven’t considered yet.
1. try pbranch
2. you could use versioned mq’s
I haven’t gotten around to using pbranch myself.
But for things where I know i’m going to do a lot of rewriting of history (contributions to Mercurial, Symbian, VirtualBox, and a CVS->Hg migration for Bonsai), I have used versioned mq’s, which does allow me to track my changes. — And I have taken advantage of this, it helps a lot.
@Bill Barry: Thanks for the pointer to PBranch. It definitely looks promising, but does the learning curve really need to be that steep? I haven’t tried it out yet, because the intro docs were pretty confusing and I’m worried about switching away from what everyone else around here is comfortable with. Ok, so it’s mostly inertia. My fundamental problem with mq is that your patches are grafted on extraneous bits. It’s great for juggling a collection of patches, but awful for integrating with other revisions — or in other words, it gains functionality at the expense of revision control functionality. Which is not a good thing for a revision control system to do. I’d much rather have commands that keep track of a “local/tentative” portion of the revision history and display it much as mq does. This seems to be what pbranch is about, so I really need to look closer, but can’t they simplify the 90% case somehow by hiding some of that complexity? Also, I want to be able to *explicitly* export changes as patches, edit, and re-import them. And preferably have an autoupdated read-only view of the patches. They just shouldn’t be the master artifact unless your whole VCS works that way. (Another annoyance with mq is that the patches that I want it to work off of are different from the patches I want to submit for inclusion in the tree — mainly, that means I want less context in the internal patch, more in the external.)
@timeless: Using a versioned queue is exactly what this post was about, so I assume you meant multiple queues? I played with that once, but ended up hopelessly confusing myself so I gave it up. Specifically, you pretty much have to always qpop -a before moving between queues. The tool could have intelligence to help you out with switching queues with patches applied, but instead it just mangles everything beyond recognition if you do the wrong thing. It’s the same problem with a single versioned queue, actually. (Switching queues or changing to a different version of a queue really ought to pop everything or as much as needed off for you, then push back to get to as close a state as it could. Preferably without updating timestamps unless necessary.)