git Status

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

git Status

dag-7
Did the project ever come to a decision about making a transition to
git?  I'm trying to do some longer-term planning and it would be helpful
to know what the roadmap is.

Thanks!

                                 -Dave
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

james woodyatt-3
On Aug 18, 2011, at 10:57 AM, David Greene wrote:
>
> Did the project ever come to a decision about making a transition to
> git?  I'm trying to do some longer-term planning and it would be helpful
> to know what the roadmap is.

Me too.  I've been catching up on the thread from a couple weeks ago, and I didn't see any clear conclusion.  I have some comments about the Mercurial aspect of the discussion.

On Jul 22, 2011, at 2:02 PM, Andrew Trick wrote:
>
> Comments on that point [Mercurial and local revision numbers] are welcome.

Oh, I really doubt that's true.

I'm a big Mercurial proponent, c.f. ...

  <http://jhw.dreamwidth.org/1868.html>
  <http://jhw.dreamwidth.org/2049.html>

...but even I know when to shut up and just go along to get along.  If folks want LLVM to use a DVCS instead of Subversion, then I will be cheering from the sidelines if LLVM switches to Git.

For my purposes, I'll just use the HgGit extension <http://mercurial.selenic.com/wiki/HgGit> as my view on the Git repository.  I use that routinely with other projects that use Git as their authoritative DVCS repository, and it works reasonably well.  (There are performance improvements coming in both Mercurial and the HgGit extension that should make life even easier for Mercurial users in the not too distant future.)

On Jul 28, 2011, at 3:56 PM, FlyLanguage wrote:

>>> Have you considered mercurial?
>>
>> Please lets not go there.
>>
>> I have to use mercurial form time to time in another project and it is
>> really painful. I use hg<->git bridge as often as possible, but it
>> doesn't work as well as the git<->svn one.
>
> Ahh, the joy of anectodal evidence.
>
> I'll add one too: I use bot Git and Hg in numerous projects, and they
> are both stable and fast DVCS's and have great communities.
>
> Any one of those two blows svn out of the water in so many ways, and
> either choice will work great (bridged or not.)

+1 to this rebuttal.

I've a sneaking suspicion from reading Chris Lattner's messages about reviewing patches in a queue that Mercurial's MQ facility would come in handy for that.

  <http://hgbook.red-bean.com/read/managing-change-with-mercurial-queues.html>

The good news is that with recent Mercurial and Dulwich versions installed, the Hg-git extension is really quite reasonable, and it allows you to use MQ among your distributed Mercurial clones, and still push to a remote Git repository when you've finished editing a patch.  I do this all the time.

One point worth noting about the Hg-git extension is that it *really* isn't very good at letting Git users interoperate with a Mercurial repository, mainly because there is metadata captured in the Mercurial format that Git doesn't know how to represent, but it's very nice for Mercurial users who need to interoperate with a Git repository, where a round-trip from Mercurial through Git and back is completely transparent.

Ignore the kvetching about Hg-git from the Git users.  It's really not made for them, and most of them don't know what they're missing by not using Mercurial natively.

Seriously, I would be pleased to see LLVM switch the authoritative repository from Subversion to Git.  I think it's a good idea for two reasons: A) to improve the integrity of the source code history, by distributing it widely, and B) to improve the performance of source code control operations.  (Using a SVN repository is like sucking cold malt extract through coffee stirring straw.  The speed improvement alone, actually, is reason to do it.)  An important secondary reason for switching to Git is for its hugely improved representation of merge operations over how Subversion operates.

People like me, who hate Git and much prefer to use Mercurial, will generally be able to work with a Git repository using Hg-git, and with much less grousing than we do today using the Subversion repository and Hg-Subversion.  Maybe we won't be as happy as if you picked Mercurial over Git, but we're going to be *greatly* outnumbered by the Git users who would scream bloody murder if you picked Mercurial instead.

In closing, just go with Git and tell people who don't like it to use Mercurial and Hg-git.  We're adaptable.

p.s. The Mercurial subrepositories feature is loads better than git submodules and it's built into the tool.  But never mind that.  Just go with Git and don't look back.  Nobody ever got fired for buying from the market leader.


--
j h woodyatt <[hidden email]>

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

FlyLanguage
> On Aug 18, 2011, at 10:57 AM, David Greene wrote:
>>
>> Did the project ever come to a decision about making a transition to
>> git?  I'm trying to do some longer-term planning and it would be helpful
>> to know what the roadmap is.

It's stuck on:

1) A misunderstanding that global revision numbers are necessary and
that 'git describe' along with frequent tagging (commit hooks) isn't
good enough.

2) Nobody writing up how git should be used with the current llvm
workflow (which is not going to adapt to an SCM, but the other way
around, which is understandable.)
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Matthieu Moy-4
In reply to this post by james woodyatt-3
james woodyatt <[hidden email]> writes:

> p.s. The Mercurial subrepositories feature is loads better than git
> submodules and it's built into the tool. But never mind that. Just go
> with Git and don't look back. Nobody ever got fired for buying from
> the market leader.

I don't use submodules enough to be a good juge, but my understanding is
that Git's submodules are conceptually nice, but that most people agree
that the UI is not good. There's a summer of code on that subject, so
there's still hope ;-).

--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Dave Zarzycki
In reply to this post by FlyLanguage

On Aug 19, 2011, at 2:57 AM, FlyLanguage wrote:

>> On Aug 18, 2011, at 10:57 AM, David Greene wrote:
>>>
>>> Did the project ever come to a decision about making a transition to
>>> git?  I'm trying to do some longer-term planning and it would be helpful
>>> to know what the roadmap is.
>
> It's stuck on:
>
> 1) A misunderstanding that global revision numbers are necessary and
> that 'git describe' along with frequent tagging (commit hooks) isn't
> good enough.
>
> 2) Nobody writing up how git should be used with the current llvm
> workflow (which is not going to adapt to an SCM, but the other way
> around, which is understandable.)

>From chats with some llvm/clang developers at Apple, we're also stuck on:

3) Somebody doing all of the thankless infrastructure work to ensure that we don't regress on basic things like the web interface, post-commit email hooks, etc.

4) Having a few stakeholders audit the conversion process to ensure that "we do it right".

5) There are organizations that use the current git-svn bridge extensively to maintain long term topic branches before proposing/integrating them into the public repository. If the conversion changes the SHA1 of commits (and it likely will due to goal #4), then these organizations will need time to prepare for the conversion (to avoid unscheduled downtime and/or any loss of history).

davez
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

greened
In reply to this post by FlyLanguage
FlyLanguage <[hidden email]> writes:

> 1) A misunderstanding that global revision numbers are necessary and
> that 'git describe' along with frequent tagging (commit hooks) isn't
> good enough.

I'm not sure what's needed here.  I don't care about global revision
numbers at all so I can't really comment on what should be used to
replace them with git.  Hopefully someone else who cares can chime in.

> 2) Nobody writing up how git should be used with the current llvm
> workflow (which is not going to adapt to an SCM, but the other way
> around, which is understandable.)

I'm willing to take a crack at this as I have to do something like this
already anyway.  I won't be able to get to it until next week, though.

                                -Dave
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

greened
In reply to this post by Dave Zarzycki
Dave Zarzycki <[hidden email]> writes:

>>From chats with some llvm/clang developers at Apple, we're also stuck on:
>
> 3) Somebody doing all of the thankless infrastructure work to ensure
> that we don't regress on basic things like the web interface,
> post-commit email hooks, etc.

It's certainly a concern.  Is Apple the only entity that has access to
this stuff or can others help?

> 4) Having a few stakeholders audit the conversion process to ensure
> that "we do it right".

Has anyone been identified to play this role?  I haven't seen any
requests go out on the list for volunteers.

> 5) There are organizations that use the current git-svn bridge
> extensively to maintain long term topic branches before
> proposing/integrating them into the public repository. If the
> conversion changes the SHA1 of commits (and it likely will due to goal
> #4), then these organizations will need time to prepare for the
> conversion (to avoid unscheduled downtime and/or any loss of history).

I am probably not one of the organizations you're thinking of, but I
currently do a lot of this and am willing to throw away everything I
have to move to git proper.  It's that much better.

                             -Dave
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

greened
In reply to this post by Matthieu Moy-4
Matthieu Moy <[hidden email]> writes:

> james woodyatt <[hidden email]> writes:
>
>> p.s. The Mercurial subrepositories feature is loads better than git
>> submodules and it's built into the tool. But never mind that. Just go
>> with Git and don't look back. Nobody ever got fired for buying from
>> the market leader.
>
> I don't use submodules enough to be a good juge, but my understanding is
> that Git's submodules are conceptually nice, but that most people agree
> that the UI is not good. There's a summer of code on that subject, so
> there's still hope ;-).

git-subtree is a very nice way to handle the submodule project.  Even
though it's clunky through the git-svn bridge, I still use it on this
end.  With native git it's a breeze.

git-submodule is indeed a putrid pile of donkey dung.  :)

                              -Dave
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Talin-3
In reply to this post by dag-7
The Python language recently migrated from SVN to Mercurial as their version control system. As part of this effort, a detailed migration plan was written:


Now, I'm not proposing that we favor using Mercurial over Git*. But I would suggest that perhaps you could use the Python migration plan as a template for LLVM's migration.

(*I use both Mercurial and Git on a regular basis. Although I think Git's branching and history model makes more sense, as a command-line tool I find Mercurial significantly easier to use for day-to-day work.)

-- Talin

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Chris Lattner-2

On Aug 19, 2011, at 2:36 PM, Talin wrote:

The Python language recently migrated from SVN to Mercurial as their version control system. As part of this effort, a detailed migration plan was written:


Hi guys,

Just to be absolutely clear here, Mercurial is not being considered.

-Chris


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Chris Lattner-2
In reply to this post by greened

On Aug 19, 2011, at 1:41 PM, David A. Greene wrote:

> Dave Zarzycki <[hidden email]> writes:
>
>>> From chats with some llvm/clang developers at Apple, we're also stuck on:
>>
>> 3) Somebody doing all of the thankless infrastructure work to ensure
>> that we don't regress on basic things like the web interface,
>> post-commit email hooks, etc.
>
> It's certainly a concern.  Is Apple the only entity that has access to
> this stuff or can others help?

This is an llvm.org thing, not an Apple thing.  Several non-apple people have llvm.org access, e.g. Anton.

-Chris
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Talin-3
In reply to this post by Chris Lattner-2


On Fri, Aug 19, 2011 at 2:53 PM, Chris Lattner <[hidden email]> wrote:

On Aug 19, 2011, at 2:36 PM, Talin wrote:

The Python language recently migrated from SVN to Mercurial as their version control system. As part of this effort, a detailed migration plan was written:


Hi guys,

Just to be absolutely clear here, Mercurial is not being considered.

If you look again at what I wrote, I wasn't proposing Mercurial. I was proposing a migration process.

-Chris




--
-- Talin

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

greened
In reply to this post by FlyLanguage
FlyLanguage <[hidden email]> writes:

> 2) Nobody writing up how git should be used with the current llvm
> workflow (which is not going to adapt to an SCM, but the other way
> around, which is understandable.)

Here is a first cut at that.  Other git users, please chime in with
suggestions, edits, etc.  Non-git users, please ask for clarification
where needed.  This is based on my notes on working with LLVM via
git-svn, modified to assume native git.  There are hundreds of ways to
design a workflow that works with the current review process.  This is
but one.

We should add this to the web page once it's polished if we make the
transition to git.

                         -Dave


Working with the LLVM git Repository
====================================

This guide provides information about how to access and interact with
LLVM's git repository.  Specifically, it covers how to check out the
LLVM git repository, create patches for review, revise those patches
as necessary and commit them to the LLVM upstream repository.

Initially, this guide is a sketch of how a transition to git may be
done without disrupting the current LLVM review process.  It may be
adapted later into a full-fledged user guide.

This guide does not specifically cover interacting with Clang and
other LLVM projects.  However, the information here is likely to be
adaptable to those projects.

This guide assumes the developer only wants a single clone of LLVM.
It is possible to set up complex work environments with a local shared
clone of LLVM and various clones of the shared local repository, but
this guide does not attempt to describe such scenarios.  Such setups
are highly useful for tracking independent third-paty development but
they are beyond the scope of this document.

Getting LLVM
------------

To check out the current LLVM sources, use git clone:

git clone http://llvm.org/git/llvm.git llvm

This will get you a repository that looks like this:

 ...-A[HEAD]

where "..." is the history of the project leading up to the most
recent commit A.  HEAD is a reference to this most recent commit.

Naming Upstream
---------------

The intial clone from upstream results in a git remote reference with
the rather unhelpful name of "origin."  As more remote sources get
added, it is easy to forget what "origin" is.  Therefore, add a remote
with a more descriptive name.

git remote add llvm-upstream http://llvm.org/git/llvm.git master

Updating LLVM - no local changes
--------------------------------

To update your clone to the latest LLVM sources, use git pull:

git pull llvm-upstream

This may result in a dag like this

...-A-B-C[HEAD]

showing two new commits made since the last pull.

git pull should ONLY be used where there is no local history waiting
to be sent upstream.  See below for managing updates in the presence
of local history.

Working with LLVM
-----------------

Actual code development in the workarea proceeeds much like any
project, except that with git you may commit changes locally without
disturbing upstream.  Let us say that the curent dag looks like this:

...-A[HEAD]

We want to make some changes to if conversion.  First we create a
local branch to hold the changes:

git checkout -b ifconvert

Now we edit the files and commit them.

git add lib/CodeGen/IfConversion.cpp

git commit

<edit commit message>

git has some conventions about commit message format.  The most important one is that if there is a single line separated from the rest, like this:

  This is the subject

  This patch does some really funky stuff.  I hope it will work!

Then "This is the subject" will become the line printed when doing
--oneline log displays and will also be the subject of e-mails sent to
the mailing list.

Our dag now looks like this:

...-A{master}
     \
      B{ifconvert}[HEAD]

Let's make some more changes:

...-A{master}
     \
      B-C-D{ifconvert}[HEAD]

Preparing a Commit
----------------------

Let's say we want to commit the changes to if conversion, contained in
revisions B and C.  We don't yet want to send D for review.  There are
several strategies available, but the fundamental choice is whether to
create a separate branch to do the merge and commit.  Usually this is
not necessary but for very complex situations it can help to organize
things a bit more explicitly.

Let's say we choose the simple strategy of merging from master.  The
first step is to get the identifiers of B and C:

git log --oneline:

8fe2a Finish if conversion work
afe3d Middle of if conversion work, something interesting to commit
44ef3 Start if conversion work

(we hope subjects are more descriptive than this!)

We go back to master to start the commit process:

git checkout master

...-A{master}[HEAD]
     \
      B-C-D{ifconvert}

Now we choose the commits we want to send upstream:

git cherry-pick 44ef3
git cherry-pick afe3d

The dag looks like this:

...-A---E-F{master}[HEAD]
     \ / /
      B-C-D{ifconvert}

E and F are the cherry-picked B and C.

Updating LLVM - with local changes
----------------------------------

Now we have the commits we want but we should apply them against the
latest upstream master.  git rebase is the tool to do this:

git fetch llvm-upstream master

This pulls in the changes from upstream but does not yet apply them to
the workarea.  A local branch gets created to track this content.

Now we want to take those upstream changes and apply our local changed
on top of them.  This maintains a linear history, making reviews much
easier.

git rebase llvm-upstream/master

There may be conflicts from this operation.  If so, resolve them in
the usual way.  git has some tools to help but they are beyond the
scope of this guide.

Now our history looks like this:

...-A-----G-H-I-E'-F'{master}[HEAD]
     \   /
      B-C-D{ifconvert}

Where E' and F' are the new commits of E and F on top of the latest
master.

Now we are ready to begin the review process.

Sending Patches for Review
--------------------------

git includes a whole set of tools for managing the patch review
process.  We kick things off with git format-patch:

git format-patch -o $HOME/patches/ifconvert --thread --src-prefix=old/ \
                 --dst-prefix=new/ --cover-letter HEAD~1..HEAD

This places three text files in $HOME/patches/ifconvert, one for each
commit, plus a cover letter to send before each patch.  These will get
sent to the e-mail list with the subject "[PATCH n/2] <commit
subject>" where "n" is the patch number (0 for the cover letter) and
<commit subject> is the first line of the commit message.

Edit these files to add any commentary you desire.

It is helpful for format-patch and send-email to have various bits of
information pre-selected for e-mail interaction.  For example, I put
this in my .git/config file in the local repository:

[format]
        numbered = auto
        to = [hidden email]
        inline = "---------"
        thread = shallow
[sendemail]
        smtpencryption = tls
        smtpserver = <my mail server>
        smtpuser = <my username>
        smtpserverport = 25
        thread = false
        chainreplyto = false
        to = [hidden email]
        signedoffbycc = false
        from = David Greene <[hidden email]>
        suppressfrom = true

"git help format-patch" and "git help send-email" gives all te
details.

Now use git send-email to actually send the review request messages:

git send-email --annotate $HOME/patches/ifconvery/* 2>&1 | tee email.out

Your patches have been sent to llvm-commits for review.  Interact over
e-mail and respond to feedback.

Updating Patches
----------------

Your patches will probably require some editing.  git rebase -i and
git add -i are your friends.

For the typical case of editing your patches a bit, use git rebase -i:

git rebase -i HEAD~2

This brings up an editor with a document that looks something like
this:

  pick ef723 Start if conversion work
  pick 443de Middle of if conversion work, something interesting to commit

This is a control file you edit to state how git-rebase should work.
There are essentially three commands pick, edit and squash.

"pick" tells rebase to apply that change as-is.  "edit" tells rebase
to suspend working immediately after applying that commit.  This
allows you to edit the patch as necessary.  "squash" tells rebase to
combine that commit with the previous one and apply them as a single
change.

One can also reorder commits within the file to change the order they
are applied.  If you delete a line, that commit disappears from the
rewritten history.

In this case, let's say that the second commit needs some work.  We
edit the control file to do that:

  pick ef723 Start if conversion work
  edit 443de Middle of if conversion work, something interesting to commit

After saving and quitting, git-rebase does its work and we are left in
this state, assuming no intervening upstream changes:

...-A-----G-H-I-E'-F'{master}[HEAD]
     \   /
      B-C-D{ifconvert}

Note that F' is still applied.  To actually change it, we need to back
up one commit:

git reset --soft HEAD^

This puts all the files from F' into the "changes to be committed"
state.  That is, they are in the index, but not in the history.  You
can now edit files to your heart's content.  When everything is ready,
you can re-commit with the original log message:

git commit -a -c ORIG_HEAD

ORIG_HEAD is a temporary branch created from the origin HEAD before
the git reset.  It is a convenient handle to refer to information from
that commit.

Now tell rebase to continue and finish up:

git rebase --continue

Splitting Patches
-----------------

Sometimes reviewers request that a patch be broken up into smaller
components.  This is easy to do with git add -i.  Let's say the first
patch is too large.

git rebase -i HEAD~2

  pick ef723 Start if conversion work
  pick 443de Middle of if conversion work, something interesting to commit

Edit the first patch:

  edit ef723 Start if conversion work
  pick 443de Middle of if conversion work, something interesting to commit

After rebase suspends, back out the commit:

git reset --mixed HEAD^

Your files are left in the "untracked files" state and will have to be
added again.  This is what you want because you need to add just a few
files or parts of files.  So decide what to add:

git add -i

This brings up a little menu which lets you pick which files or hunks
of files to add.  Going through all the options is beyond the scope of
this guide.  One good guide is here:
http://book.git-scm.com/4_interactive_adding.html

After finishing the add process, do a commit.

git commit

Do NOT do commit -a as that will re-add everything.

Go on to do some more add -i and commits to stage everything as
desired.  Then tell rebase to finish up:

git rebase --continue

Sending to Upstream
-------------------

Finally, we are ready to send the patches.  First make sure we are as
up-to-date as possible:

git fetch llvm-upstream master
git rebase llvm-upstream/master
<fix any conflicts>

Now commit:

git push llvm-upstream master

Resources
---------

Here are some helpful git resources.

Pro Git book:
  http://progit.org/

  This is a great book to grok git.  It explains the tools in a
  straightforward way.  Get it in paperback for a great reference!
  All of the other resources will make much more sense after having
  read this book.

git fetch/merge vs. pull:
  http://longair.net/blog/2009/04/16/git-fetch-and-merge/

Git rebase worflow:
  http://mettadore.com/analysis/a-simple-git-rebase-workflow-explained/

The git manpages are pretty good once you are reasonably familiar with
the tools.  They contain a lot of good examples for various scenarios.

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Mark Lacey-3
David,

A few comments.
 
Naming Upstream
---------------

The intial clone from upstream results in a git remote reference with
the rather unhelpful name of "origin."  As more remote sources get
added, it is easy to forget what "origin" is.  Therefore, add a remote
with a more descriptive name.

git remote add llvm-upstream http://llvm.org/git/llvm.git master


If the intent is to rename origin, this can be done directly:
  git remote rename origin llvm-upstream

 
Updating LLVM - no local changes
--------------------------------


Splitting this into "no local changes" / "with local changes" seems unnecessarily complicated. Why not just recommend doing 'git pull --rebase' all of the time?

I have to ask as well - is a linear history really desired? That seems to be the intent of your instructions.

I ask because for something like a reasonably sized feature that might have multiple commits, having the merge history in place can be useful, if only to separate the set of related changes from sets of unrelated changes. It also aids in reverting an entire feature composed of multiple commits.

One other comment - the way I use git (and from what I've read I am not alone), I end up committing multiple times per hour / many times per day, often into different branches. I then go back and use rebase to reorder and squash commits into logically related bite-sized chunks (e.g. combine some commits related to feature A into one or more commits, some related to feature B into one or more commits, and then there might be three bug fix commits for bugs C, D, and E.). It might be helpful to add some guidelines related to this that are in line with current LLVM review process - somewhat the reverse scenario of someone asking for commits to be split. In this case, they probably don't want to see 15 different commits that in total add 75 lines of code to feature A.

Mark


_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Tobias Grosser-5
In reply to this post by greened
On 08/22/2011 09:37 PM, David A. Greene wrote:

> FlyLanguage<[hidden email]>  writes:
>
>> >  2) Nobody writing up how git should be used with the current llvm
>> >  workflow (which is not going to adapt to an SCM, but the other way
>> >  around, which is understandable.)
> Here is a first cut at that.  Other git users, please chime in with
> suggestions, edits, etc.  Non-git users, please ask for clarification
> where needed.  This is based on my notes on working with LLVM via
> git-svn, modified to assume native git.  There are hundreds of ways to
> design a workflow that works with the current review process.  This is
> but one.
>
> We should add this to the web page once it's polished if we make the
> transition to git.
>
>                           -Dave

Hi Dave,

thanks a lot. This already reads very nice. Two smaller comments:
> Sending Patches for Review
> --------------------------
>
> git includes a whole set of tools for managing the patch review
> process.  We kick things off with git format-patch:
>
> git format-patch -o $HOME/patches/ifconvert --thread --src-prefix=old/ \
>                   --dst-prefix=new/ --cover-letter HEAD~1..HEAD

I personally dislike typing all the time such a long command line. Maybe
you can also point out, how to configure this in .git/config.

> This places three text files in $HOME/patches/ifconvert, one for each
> commit, plus a cover letter to send before each patch.  These will get
> sent to the e-mail list with the subject "[PATCH n/2]<commit
> subject>" where "n" is the patch number (0 for the cover letter) and
> <commit subject>  is the first line of the commit message.
>
> Edit these files to add any commentary you desire.
>
> It is helpful for format-patch and send-email to have various bits of
> information pre-selected for e-mail interaction.  For example, I put
> this in my .git/config file in the local repository:
>
> [format]
>          numbered = auto
>          to =[hidden email]
>          inline = "---------"
I believe the current policy is not to inline patches, but to attach
them. I believe we should keep following this policy to reduce the
changes of this transition.

> Updating Patches
> ----------------
>
> Your patches will probably require some editing.  git rebase -i and
> git add -i are your friends.
>
> For the typical case of editing your patches a bit, use git rebase -i:
>
> git rebase -i HEAD~2
>
> This brings up an editor with a document that looks something like
> this:
>
>    pick ef723 Start if conversion work
>    pick 443de Middle of if conversion work, something interesting to commit
>
> This is a control file you edit to state how git-rebase should work.
> There are essentially three commands pick, edit and squash.

You missed the one I use most: 'fixup'. I use a different approach to
edit patches. Here the text, feel free to add parts of it or to ignore
it for the sake of simplicity.

-------------------
For the typical case of editing your patches a bit, use git rebase -i:

Let's assume you have the following two patches in your queue:

$ git log --oneline
ef723 Start if conversion work
443de Middle of if conversion work, something interesting to commit

To fix some problems with commit 'Start if conversion work', add your
fixes directly to the working copy and commit them as new changeset
'Fixes for: Start if conversion work'.

Now we call

$ git rebase -i HEAD~2

This gives us the following text file:

pick ef723 Start if conversion work
pick 443de Middle of if conversion work, something interesting to commit
pick 33ed3 Fixes for: Start if conversion work

Here, we move the fixup right after the commit to fix and replace the
default action 'pick' by the comment 'fixup'.

pick ef723 Start if conversion work
fixup 33ed3 Fixes for: Start if conversion work
pick 443de Middle of if conversion work, something interesting to commit

By saving and closing this file, the patches will be merged and
there will be a new history.

sdd24 Start if conversion work
eba32 Middle of if conversion work, something interesting to commit

Commit sdd24 now contains the changes of both ef723 and 443de. In case
you want to edit the commit message of the commit to fix, you can also
use 'squash' instead of 'fixup'.
---------------------------

Cheers
Tobi
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Matthieu Moy-4
In reply to this post by Mark Lacey-3
Mark Lacey <[hidden email]> writes:

> David,
>
> A few comments.
>  
>
>     Naming Upstream
>     ---------------
>    
>     The intial clone from upstream results in a git remote reference with
>     the rather unhelpful name of "origin."  As more remote sources get
>     added, it is easy to forget what "origin" is.  Therefore, add a remote
>     with a more descriptive name.
>    
>     git remote add llvm-upstream http://llvm.org/git/llvm.git master
>
> If the intent is to rename origin, this can be done directly:
>   git remote rename origin llvm-upstream

or simpler, do it at clone time:

  git clone --origin llvm-upstream

Note that git clone does not only set a remote, it also sets it as
"upstream" for the master branch, i.e. "git pull" without argument will
fetch from origin. Adding a new remote will not do that (but one can use
the --set-upstream option of various commands to fix that later).

That said, I don't think it's a good idea to ask users to rename their
upstream in a guideline document. Naming upstream is personnal
preference, and keeping the default seems sane for most users. If you
don't remember what "origin" is, then .git/config can remind you.

>     Updating LLVM - no local changes
>     --------------------------------
>
> Splitting this into "no local changes" / "with local changes" seems
> unnecessarily complicated. Why not just recommend doing 'git pull
> --rebase' all of the time?

I'd also recommand "git pull --rebase" if the goal is to keep history
linear. Note that this has to go with a big, fat, warning, telling the
user that rebasing published history is bad. Rebase is a very good tool
to work with private history, but as soon as you've pushed it to some
place visible by other people, you should stop using it.

> I ask because for something like a reasonably sized feature that might
> have multiple commits, having the merge history in place can be
> useful, if only to separate the set of related changes from sets of
> unrelated changes.

Rebase is exactly meant to do this separation, and avoid history looking
like

- start working on feature
- merge from upstream
- continue working on feature
- merge again from upstream
...

and remove the useless "merge from upstream" commits, that would just
distract reviewers. Keeping merge history when merging several clean,
published branches is good though.

If you look at how Git itself is developped, people use rebase a lot to
send clean patch series, and the maintainer uses merge a lot to merge
multiple patch series together.

--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

FlyLanguage
> I'd also recommand "git pull --rebase" if the goal is to keep history
> linear. Note that this has to go with a big, fat, warning, telling the
> user that rebasing published history is bad. Rebase is a very good tool
> to work with private history, but as soon as you've pushed it to some
> place visible by other people, you should stop using it.

This is enormously important - once submodule maintainers starts
rebasing, we're screwed.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

Matthieu Moy-4
In reply to this post by greened
[hidden email] (David A. Greene) writes:

> Updating LLVM - no local changes
> --------------------------------
>
> To update your clone to the latest LLVM sources, use git pull:
>
> git pull llvm-upstream

That should probably be either

  git pull

or

  git pull llvm-upstream master

since the first is shorter and works if llvm-upstream is the upstream
for the current branch, and "git pull llvm-upstream" will ask for a
branch name if it isn't.

> git cherry-pick 44ef3
> git cherry-pick afe3d
>
> The dag looks like this:
>
> ...-A---E-F{master}[HEAD]
>      \ / /
>       B-C-D{ifconvert}

Err, no. cherry-pick won't record B and C as parents.

A common convention would be to call E and F as B' and C' to reflect the
fact that they are different, but similar to B and C.

Also, if you are to draw DAGs in your explanations (which is good), you
should mention gitk and/or git log --oneline --graph so that users can
experiment by themselves.

> git rebase llvm-upstream/master
>
> There may be conflicts from this operation.  If so, resolve them in
> the usual way.

... and use "git rebase --continue" as needed.

Perhaps point to git-rebase(1) for more details.

> Now our history looks like this:
>
> ...-A-----G-H-I-E'-F'{master}[HEAD]
>      \   /
>       B-C-D{ifconvert}

I don't get it. B and C were local commits, and you didn't explain the
user how to push them, so they're still local. As I Understand It, G and
H are the remote commits you've just fetched, so G cannot have C as
parent.

You probably wanted to show this history after fetch

      G-H{llvm-uptream}
     /
...-A--E-F{master}[HEAD]

and this one after rebase

...-A-G-H-E'-F'{master}[HEAD]
   
> git format-patch -o $HOME/patches/ifconvert --thread --src-prefix=old/ \
>                  --dst-prefix=new/ --cover-letter HEAD~1..HEAD

I usually use git send-email right away, it can call format-patch in
modern Gits.

> Edit these files to add any commentary you desire.

... comments that are not meant to appear in the commit message should
be added after the --- and before the diffstat in the patches.

?

> [format]
>         numbered = auto
>         thread = shallow

these are the default, so better not annoy new users with them.

> git rebase -i HEAD~2

I usually do

git rebase -i origin/master

or in this case

git rebase -i llvm-upstream/master

which has the advantage of offering me to edit local commits, and only
them. I cannot rewrite upstream history by mistake (which would lead to
weird things afterwards).

> In this case, let's say that the second commit needs some work.  We
> edit the control file to do that:
>
>   pick ef723 Start if conversion work
>   edit 443de Middle of if conversion work, something interesting to commit

It's not the best example: rebase is not needed at all to edit the last
commit. You should edit the first to make the example more relevant.

> git reset --soft HEAD^

> git commit -a -c ORIG_HEAD

That seems to be a rather complex and potentially dangerous way of
saying

git commit --amend

> git reset --mixed HEAD^
>
> Your files are left in the "untracked files"

... or "changed, but not staged for commit"

> state

> Now commit:
>
> git push llvm-upstream master

"Now commit" is confusing. If you talk to former SVN users, you can say
"now, do the equivalent of svn commit", and otherwise, "Now, send your
changes to the upstream repository".

My 2 (perhaps 3?) cents,

--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

FlyLanguage
In reply to this post by greened
>> 2) Nobody writing up how git should be used with the current llvm
>> workflow (which is not going to adapt to an SCM, but the other way
>> around, which is understandable.)
>
> Here is a first cut at that.  Other git users, please chime in with
> suggestions, edits, etc.  Non-git users, please ask for clarification
> where needed.  This is based on my notes on working with LLVM via
> git-svn, modified to assume native git.  There are hundreds of ways to
> design a workflow that works with the current review process.  This is
> but one.
>
> We should add this to the web page once it's polished if we make the
> transition to git.

Great that you work on this, but I don't quite get the approach, nor
some of your examples (which seems buggy).

I feel that:

a) We don't need another git tutorial - there's a lot of really good
ones out there, even ones explaining git from the perspective of a
current SVN user.

b) We need a document explaining how patches are reviewed and referred
to, what commit rules (and hooks) are in place and how this would work
in a git world. It would be short and sweet.
_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reply | Threaded
Open this post in threaded view
|

Re: git Status

greened
In reply to this post by Mark Lacey-3
Mark Lacey <[hidden email]> writes:

>     git remote add llvm-upstream http://llvm.org/git/llvm.git master
>
> If the intent is to rename origin, this can be done directly:
>   git remote rename origin llvm-upstream

Much better.  Thanks.

>     Updating LLVM - no local changes
>     --------------------------------
>
> Splitting this into "no local changes" / "with local changes" seems unnecessarily complicated. Why not just recommend doing 'git pull --rebase'
> all of the time?

Actually git pull can sometimes get you into trouble.  Probably git
fetch / git rebase is the better combination for LLVM.  I agree my
distinction is artificial but for the users who simply want the most
up-to-date LLVM, git pull is simpler.  I certainly debated back a forth.
I have no problem reworking this.

> I have to ask as well - is a linear history really desired? That seems
> to be the intent of your instructions.

It is a stated requirement of the project.

> I ask because for something like a reasonably sized feature that might
> have multiple commits, having the merge history in place can be
> useful, if only to separate the set of related changes from sets of
> unrelated changes. It also aids in reverting an entire feature
> composed of multiple commits.

Reverting is just as easy with linear history, I think.  Chris has
stated her wants reviews to be a simple as possible and linear history
gets us that.

There are some more interesting points made here:

http://randyfay.com/node/91

Here's an interesting article with a proposal on how to get the best of
both:

http://softwareswirl.blogspot.com/2009/04/truce-in-merge-vs-rebase-war.html

The follow-up linked at the bottom explains how it mgiht be implement.
It requires changes to git, however.  Personally, I kind of like the
ideas presented but I doubt it will happen any time soon.

> One other comment - the way I use git (and from what I've read I am
> not alone), I end up committing multiple times per hour / many times
> per day, often into different branches. I then go back and use rebase
> to reorder and squash commits into logically related bite-sized chunks
> (e.g.  combine some commits related to feature A into one or more
> commits, some related to feature B into one or more commits, and then
> there might be three bug fix commits for bugs C, D, and E.). It might
> be helpful to add some guidelines related to this that are in line
> with current LLVM review process - somewhat the reverse scenario of
> someone asking for commits to be split. In this case, they probably
> don't want to see 15 different commits that in total add 75 lines of
> code to feature A.

That makes sense.  I briefly mentioned squashing in the rebase -i
explanation but I can expand on that.

Thanks for your feedback!

                         -Dave

_______________________________________________
LLVM Developers mailing list
[hidden email]         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
123