Bitkeeper was neat, and my overall take on it mirrors Larry McVoy's: I wish he had open sourced it, made his nut running something just like github but for Bitkeeper, and that it had survived.
I only had one interaction with him. In the early '00s, I had contributed a minor amount of code to TortoiseCVS. (Stuff like improving the installer and adding a way to call a tool that could provide a reasonable display for diffs of `.doc` and `.rtf` files.) I had a new, very niche, piece of hardware that I was excited about and wanted to add support for in the Linux kernel. Having read the terms of his license agreement for Bitkeeper, and intending to maintain my patches for TortoiseCVS, I sent him an email asking if it was OK for me to use Bitkeeper anyway. He told me that it did not look like I was in the business of version control software (I wasn't!) and said to go ahead, but let him know if that changed.
I use git all the time now, because thankfully, it's good enough that I shouldn't spend any of my "innovation tokens" in this domain. But I'd still rather have bitkeeper or mercurial or fossil. I just can't justify the hit that being different would impose on collaboration.
I hope that the CLA goes away one day.
(*Update:* Though IANAL, you should read the child comment and the CLA itself and make up your own mind. https://cla.developers.google.com/about/google-individual. The rest of my comment is mostly independent of the previous paragraph).
OTOH, IANAL, but AFAIK anyone can fork `jj` and sell a proprietary product based on jj (and distribute it under pretty much whatever license they like, with very few restrictions) because it is currently Apache licensed, but that is unrelated to the Google CLA.
Let me conjecture even more wildly about things I don't know. The following is a guess on my part.
One way to interpret this is that Google tends to publish their projects under Apache, and there is no need to demand that people transfer copyright to Google. By releasing your work under Apache, you are already giving Google (or anyone else) all the rights it needs.
AFAIK, the main purpose of the Google Individual CLA is to have you sign a statement claiming that you own the rights to your own work and didn't give up those rights to your employer.
That is a substantially more permissive license than Apache-2.0 (let alone other licenses, which apply to works to which Google also applies that CLA). That term means that Google can ignore the terms of Apache-2.0 and instead use the work under this much more permissive license, while everyone else is bound by Apache-2.0. In other words, they can do whatever they want with the code. Others could ship it in a proprietary product, sure, but they can't ignore the terms of the license while doing so.
"Permissive license" doesn't mean "do whatever you want". Apache-2.0, among other things, requires maintaining license notices,
(Note that the "and to recipients of" clause doesn't imply others can ignore the license terms, because they'd still be subject to the terms of whatever license Google puts on the software they distribute, whether that's Apache-2.0 or some proprietary license.)
So I maintain that "there's a single company with the right to change the licensing arbitrarily" is a largely accurate summary/gloss. Or, if you prefer, "there's a single company with the right to ignore the license terms".
I'm still not sure whether it really matters in light of the Apache license, but I don't feel qualified to argue about that.
I guess the straw-man I was arguing against was that some people think you transfer your copyright to Google (you don't), but that's different from what you claimed.
It matters less for something like the Apache license than it does for a copyleft license, but there are still reasons people use Apache rather than MIT or public domain, and it does include several protections people care about.
Re your edit:
> AFAIK, the main purpose of the Google Individual CLA is to have you sign a statement claiming that you own the rights to your own work and didn't give up those rights to your employer.
The Developer Certificate of Origin (DCO, what you're signing if you use a "Signed-off-by: Your Name <email@example.org>" line) serves this same purpose, isn't a CLA, and doesn't cause any of the same problems. Legal departments generally don't have concerns with developers signing a DCO, while many will rightfully prevent or restrict signing a CLA (even when they were otherwise fine with a developer contributing to Open Source in general).
https://github.com/martinvonz/jj
Seems an interesting take indeed :)
To me, Git is almost exactly like a ground-up cleaner rewrite of BitKeeper. Gitk and git-gui are essentially clones of the BitKeeper GUI.
I don't understand why you'd want to keep using BitKeeper.
I was exposed to BitKeeper when I was managing my team's CVS server. On my next team, we moved to svn, which always felt like cvs with better porcelain from a developer perspective, but when administering that server fell onto my plate, I liked it a lot better than CVS. And I thought BitKeeper would be nicer from a developer perspective.
Then on my next team, we used mercurial. I really, really, really liked mercurial, both as a developer and as a dev infrastructure administrator. It also sucked a lot less on Windows than git or BitKeeper.
The last time I had to decide for a new team, mercurial and git were the obvious options. I went with git because that was clearly what the world liked best, and because bringing new team members up to speed would require less from me that way.
All that goes to say... my direct comparison of git and bitkeeper came from when bitkeeper was mature and git decidedly was not. Then I lumped it in with mercurial (which I really would still prefer, right now) and fossil (ditto). You're probably exactly right about BK.
FWIW, mercurial seems to have an advantage over git, and that support for BIG repositories which seems to be provided by facebook of all people, so until facebook moves to git, mercurial lives on.
https://www.fossil-scm.org/home/doc/trunk/www/caps/login-gro...
It's the most complete history of git that I know now. Exceptional!
I'd love to read more historical articles like this one, of pieces of software that have helped shape our world.
I wasn't going to read the story until I read your comment. I knew the summary of BitKeeper and the fallout, but wow this was so detailed. Thanks!
(Although it's especially silly in this case, since accessing that copy[1] in the Wayback Machine reveals that the GitHub website included screenshots of itself that look nothing like the screenshot in this article.)
1. <https://web.archive.org/web/20080514210148/http://github.com...>
Why is this crappy? What would be better?
Edit: @luckydude Thank you for generously responding to the nudge, especially nearly instantly, wow :)
- No rename support, it guesses
- no weave. Without going into a lot of detail, suppose someone adds N bytes on a branch and then that branch is merged. The N bytes are copied into the merge node (yeah, I know, git looks for that and dedups it but that is a slow bandaid on the problem).
- annotations are wrong, if I added the N bytes on the branch and you merged it, it will (unless this is somehow fixed now) show you as the author of the N bytes in the merge node.
- only one graph for the whole repository. This causes multiple problems: A) the GCA is the repository GCA, it can be miles away from the file GCA if there was a graph per file like BitKeeper has. B) Debugging is upside down, you start at the changeset and drill down. In BitKeeper, because there is a graph per file, let's say I had an assert() pop. You run bk revtool on that file, find the assert and look around to see what has changed before that assert. Hover over a line, it will show you the commit comments to the file and then the changeset. You find the likely line, double click on it, now you are looking at the changeset. We were a tiny company, we never hit the claimed 25 people, and we supported tons of users. This form of debugging was a huge, HUGE, part of why we could support so many people. C) commit comments are per changeset, not per file. We had a graphic check in tool that walked you through the list of files, showed you the diffs for that file and asked you to comment. When you got the the ChangeSet file, now it is asking you for what Git asks for comments but the diffs are all the file names followed by what you just wrote. It made people sort of uplevel their commit comments. We had big customers that insisted the engineers use that tool rather a command line that checked in everything with the same comment.
- submodules turned Git into CVS. Maybe that's been redone but the last time I looked at it, you couldn't do sideways pulls if you had submodules. BK got this MUCH closer to correct, the repository produced identical results to a mono repository if all the modules were present (and identical less whatever isn't populated in the sparse case). All with exactly the same semantics, same functionality mono or many repos.
- Performance. Git gets really slow in large repositories, we put a ton of work into that in BitKeeper and we were orders of magnitude faster for things like annotate.
In summary, Git isn't really a version control system and Linus has admitted it to me years ago. A version control system needs to faithfully record everything that happened, no more or less. Git doesn't record renames, it passes content across branches by value, not by reference. To me, it feels like a giant step backwards.
Here's another thing. We made a bk fast-export and a bk fast-import that are compatible with Git. You can have a tree in BK, have it updated constantly, and no matter where in the history you run bk fast-export, you will get the same repository. Our fast-export is idempotent. Git can't do that, it doesn't send the rename info because it doesn't record that. That means we have to make it up when doing a bk fast-import which means Git -> BK is not idempotent.
I don't expect to convince anyone of anything at this point, someone nudged, I tried. I don't read hackernews any more so don't expect me to defend what I said, I really don't care at this point. I'm happier away from tech, I just go fish on the ocean and don't think about this stuff.
Git doesn't track changes yes, it tracks states. It has tools to compare those states but doesn't mean that it needs to track additional data to help those tools.
I'm unconvinced that tracking renames is really helpful as that is only the simplest case of of many possible state modifications. What if you split a file A into files B and C? You'd need to be able to track that too. Same for merging one file into another. And many many many more possible modifications. It makes sense to instead focus on the states and then improve the tools to compare them.
Tracking all kinds of changes also requires all development tools to be aware of your version control. You can no longer use standard tools to do mass renames and instead somehow build them on top of your vcs so it can track the operations. That's a huge tradeoff that tracking repository states doesn't have.
> submodules
I agree, neither submodules nor subtrees are ideal solutions.
I suppose Bitkeeper can meaningfully deal with that since their data model drills down into the file contents.
I still have fond memories of the bk revool. I haven't found anything since that's been as intuitive and useful.
0. https://arstechnica.com/information-technology/2017/05/90-of...
TeamWare - somewhat easy branching (by copying whole workspace from the parent and the bringover/putback of the changes, good merge tool), the history is local, partial commits.
BitKeeper added distributed mode, changesets.
Git added very easy branching, stash, etc.
Any other currently available source control usually is missing at least one of those features. Very illustrative is the case of Mercurial which emerged at about the same time responding to the same need for the modern source control at the time, yet was missing partial commits for example and had much cumbersome branching (like no local history or something like this - i looked at it last more than a decade ago) - that really allowed it to be used only in very strict/stuffy settings, for everybody else it was a non starter.
it is a feature which allows, for example, to work simultaneously on several releases, patches, hot fixes, etc. Once better alternative emerges we'll jump the git ship as we did before when we jumped onto the git ship.
>the repository separate from the data
that was a feature of a bunch of source controls and a reason among others why they lost to git.
>it forces you to
that is another reason why source controls lose to git as git isn't forcing some narrow way of doing things upon you.
I don't deny of course that for some people/teams/projects other source controls work better as you comment illustrates. I'm just saying why git won and keeps the majority of situations.
It's not that easy at this point in time. git carries a lot of momentum, especially in combination with GitHub.
Anybody learning about software development learns about git and GitHub.
Software is expected to be in GitHub.
At the time git became successful there were arguably better systems like mercurial and now we got fossil, but git's shortcomings are too little of a pain point compared to universal knowledge about it and integration into every tool (any editor, any CI system, any package manager, ...) and process.
CVS back then was like this too, including public repos, etc.
>At the time git became successful there were arguably better systems like mercurial
I specifically mentioned Mercurial above because they both emerged pretty simultaneously responding to the same challenges, and Mercurial happened to be just inferior due to its design choices. Companies were jumping onto it too, for example our management back then chose it, and it was a predictable huge pain in the neck, and some years down the road it was replaced with git.
Not really.
CVS had too many flaws (no atomicity, no proper branching, no good offline work, etc.) Subversion as "natural successor" fixed some things and was eating some parts of CVS.
At the same time sourceforge, the GitHub of that time, started to alienate their users.
And then enterprises used different tools to way larger degree (VSS, sccs, Bk, perforce, whatever) while that market basically doesn't exist anymore these days and git is ubiquitous.
And many people went way longer without any version control than today. Today kids learn git fundamentals very early, even on Windows and make it a habit. Where's in the early 2000s I saw many "professional" developers where the only versioning was the ".bak" or ".old" file or copies of the source directory.
Perforce still has a solid following in the gamedev space - even with LFS, git's handling of binaries is only mildly less than atrocious.
What are you talking about here? I'm not talking about eliminating branching, but the fact that merging a branch is usually just a fake single commit that hides away the complexity and decisions of the branch. see [0] into how you can leverage branches and the log for a sane commit history.
> that was a feature of a bunch of source controls and a reason among others why they lost to git.
Given the article, git won because it was foss, torvalds and speed, if you have proof of a good amount of people saying "I hate the division of data and repository!" then its a believable claim, or maybe you're confusing the data/repo division with cvs? git also didn't have to fight much, the only contender was hg
> “Here’s a BitKeeper address, bk://thunk.org:5000. Let’s try connecting with telnet.”
Famously, Tridge gave a talk about this, and got the audience of the talk to recreate the "reverse engineering". See https://lwn.net/Articles/133016/ for a source.
> I attended Tridge's talk today. The best part of the demonstration was that he asked the audience for each command he should type in. And the audience instantly called out each command in unison, ("telnet", "help", "echo clone | nc").
Tridge neglected to tell people that he was snooping the network while Linus was running BK commands when Linus was visiting in his house. THAT is how he did the clone.
The fact that you all believe Tridge is disappointing, you should be better than that.
The fact that Tridge lied is disappointing but I've learned that open source people are willing to ignore morals if it gets them what they want. I love open source, don't love the ethics. It's not just Tridge.
The network protocol, according to multiple sources and the presented talk at LCA, was "send text to the port that's visible in the URL, get text back". The data received was SCCS, which was an understood format with existing tools. And the tool Tridge wrote, sourcepuller, didn't clone all of BitKeeper, it cloned enough to fetch sources, which meant "connect, send command, get back SCCS".
Anything more than that is hearsay that's entirely inconsistent with the demonstrated evidence. Do you have any references supporting either that the protocol was more complicated than he demonstrated on stage at LCA, or that Tridge committed the network surveillance you're claiming?
And to be clear, beyond that, there's absolutely nothing immoral with more extensively reverse-engineering a proprietary tool to write a compatible Open Source equivalent. (If, as you claim, he also logged a friend's network traffic without their express knowledge and consent, that is problematic, but again, the necessity of doing that seems completely inconsistent with the evidence from many sources. If that did happen, I would be mildly disappointed in that alone, but would still appreciate the net resulting contribution to the world.)
I appreciate that you were incensed by Tridge's work at the time, and may well still be now, but that doesn't make it wrong. Those of us who don't use proprietary software appreciate the net increase in available capabilities, just like we appreciate the ability to interoperate with SMB using Samba no matter how inconvenient that was for Microsoft.
the one you're replying to, @luckydude, is Larry McVoy, who created BitKeeper.
> The data received was SCCS, which was an understood format with existing tools.
You'd be surprised. SCCS is not broadly understood. And BK is not exactly SCCS.
I read the SourcePuller code when it was published (sp-01). It's pretty easy reading. I give Tridge credit for that. I wrote a little test, got it to checkout the wrong data with no errors reported. Issue was still there in sp-02 .
Then he proceeded to fix my code over and over again. I had a basic understanding of SCCS but Rick understood the details.
Rick knows more about SCM than any guy I know.
And he is right, SCCS is not well understood and BK even less so.
BitKeeper itself is open source now and (an old version) of the protocol is documented at https://github.com/bitkeeper-scm/bitkeeper/blob/master/doc/p....
Tridge reverse engineering bk and kicking off a series of events that led to git is probably one of the most positively impactful things anyone has done for the software industry, ever. He does not deserve the flack he got for it, either then or today. I'm grateful to him, as we all should be. I know that it stings for you, but I hope that with all of this hindsight you're someday able to integrate the experience and move on with a positive view of this history -- because even though it didn't play out the way you would have liked, your own impact on this story is ultimately very positive and meaningful and you should take pride in it without demeaning others.
There is absolutely zero chance he figured out the pull protocol via telnet. I will happily pay $10,000 to anyone could do that with zero access to BK. Can't be done. If I'm wrong, I'll pay up. But I'll have a lot of questions that can't be answered.
So he cheated, he got Linus to run BK commands at his house and he snooped the network. He had no legal access to those bytes. Without those snoops, no chance he reverse engineered it.
As I have seen over and over, when the open source people want something, they will twist themselves in knots to justify getting it, legality be damned.
How about you be better than this and admit that open source is not without its skeletons?
Snooping the network is a common and entirely legal means of reverse engineering.
>There is absolutely zero chance he figured out the pull protocol via telnet. I will happily pay $10,000 to anyone could do that with zero access to BK. Can't be done. If I'm wrong, I'll pay up. But I'll have a lot of questions that can't be answered.
I just tried this myself. Here's the telnet session:
https://paste.sr.ht/~sircmpwn/0b3f1f1d77896a96b0777471785cdc...
I confess that I had to look up the name of the BK_REMOTE_PROTOCOL environment variable after a few false starts to put the pieces together, but it would be relatively easy to guess.
I also looked over Tridge's original sourcepuller code and didn't really see anything that you couldn't infer from this telnet session about how bk works.
So, do I just send you my bank account number or?
Edit:
There was another thread from him linked as well https://news.ycombinator.com/item?id=26205688
> Never in history has a version control system dominated the market like Git. What will be the next to replace Git? Many say it might be related to AI, but no one can say for sure.
I doubt it's getting replaced. It's not just that it's got so much of the market, but also that the market is so much larger than back in the days of CVS.
It's hard to imagine everyone switching from Git. Switching from GitHub, feasible. From Git? That's much harder.
Anyway, those are the four things you'd have to hit in order to replace git, as I see them.
If you had such a system, getting people off git wouldn't be the issue - offer git compatibility and if they don't want to use the advanced features, they can just keep using their existing workflow with git. The problem with that though, is that then why use your new system.
Which gets to the point of, how do you make this exist as a global worldwide product? FAANG-sized companies have their own internal tools team to manage source code. Anywhere smaller doesn't have the budget to create such a thing from scratch but
You can't go off and make this product and then sell it to someone because how many companies are gonna go with an unproven new workflow tool that their engineers want? What's the TAM of companies for whom "git's not good enough", and have large enough pocketbooks?
But, there are industries that need it, like game industry. So they should use tool that allow that. I heard that Plastic-SCM is pretty decent at it. Never used it so cant tell personally.
Replacing GIT is such a stupid idea. There is no ONE tool to handle all cases. Just use right one for your workflows. I, for example, have a need to version binary files. I know GIT handles them badly, but I really like the tool. Solution? I wrote my own simple DVFS tool for that usecase: dot.exe (138KB)
Its very simple DVFS for personal use, peer to peer syncing (local, TCP, SSH). Data and Metadata are SHA-1 checksumed. Its pretty speedy for my needs :) After weeks of use use I liked it so much, I added pack storage to handle text files and moved all my notes from SVN to DOT :)
Maybe you're confusing it with SCM which are source control managers, that's the only ones that handle strict source only, but scm can mean other things.
Not handling binary files is not a downside for me because GIT should not be a tool to handle binary files versioning and we should use something else for that.
And what is wrong with a single blob? This seems work great for deduplication, if you were using git to manage a photo library and happen to add something twice.
As for deduplication, I do not think so. If you have single blob, lets say 1GB and you change just 1B, whole blob changes and no more dedup. If you use basic method of static block size, lets say 512KB, this will work much better. Futher, there are more advanced techniques to handle dedups like roling checksums to carve out even smaller sub-block.
I pointed out that GIT have some rough cases and its good to be aware of them. I said it already and will say it again: choose right tool for the task.
If your workflow with GIT works, use it. No need to discourage other solutions. Yes, I use ancient platforms and I want DVFS on them too.
> If you had such a system, getting people off git wouldn't be the issue - offer git compatibility and [...]
Git is already doing exactly that.
How do I check out, eg https://github.com/neovim/neovim/tree/master/scripts into a directory and work with it as if it was a repo unto itself?
git clone --filter=blob:none --sparse https://github.com/neovim/neovim
cd neovim
git sparse-checkout add scripts
TLDR: the problem is more complex and pretending like “you can checkout a subtree” solves the problem is missing the proverbial forest for the (sub)tree
A replacement would be niche, only for the huge orgs, which is usually made by them anyway. For everyone else, git is good enough.
I’m sure I’m not the first to point out that Junio (the appointed git “shepherd”) works at Google where mercurial is the “recommend local vcs” internally instead of git.
Also, it is a good thing if Junio can do his job independently of Google's immediate needs.
So thank to all of you that made this possible by creating Git, Gerrit and all the life saving tools this industry was missing! The passing of Shawn Pearce was really sad, but he won't be forgotten!
Is this true? I thought GitHub had no official affiliation with the git project
The quotes on "official" imply non-official to me. i.e. official seeming to people who don't know any better.
But I think they mean here what the official git project ‘site’ is with docs and so on. And that is now https://git-scm.com/ and indeed as the article describes that was initially set up by GitHub people, to promote git
They meant exactly what they wrote: GitHub took over hosting duties for the official Git site (because they did).
I'm going to borrow this phrase from now on to everything slow.
The licensing of bitkeeper was a real thing. Although I don't follow the kernel mailing list at all nowadays, I remember Alan Cox calling it out as buttkeeper. Good Times.
"In software development terminology, comparing low-level infrastructure to plumbing is hard to trace, but the use of “porcelain” to describe high-level packaging originated in the Git mailing list. To this day, Git uses the terms “plumbing” and “porcelain” to refer to low-level and high-level commands, respectively. "
Also, unrelated, the "Ruby people, strange people" video gave me a good chuckle.
https://git-scm.com/docs/git-am
https://git-scm.com/docs/git-send-email
Git was built around supporting the linux kernel email lists. And while there are a number of other options out there that sprang up around the same time, many of them didn't fill the core need for git at that time - to reduce the stress / workload on Linus.
Open source wins
It's alive today! Sr.ht has categories of work you can't host too. Still marinating.
> You couldn’t use BitKeeper for version control if you were working on version control software.
> You had to get BitMover’s permission if you wanted to run BitKeeper alongside other similar software.
That just strains credulity.
Even outcome was the same :) flurry of more reasonable competitors with fully open source GreaseWeazle offering the whole nine yards.
That sounds a bit backwards: actually Git works similar to BitKeeper (can't say to what extent, as I'm not familiar with bk), not the other way around.
It's the "X Window System" or just "X".