[llvm-dev] [PITCH] Improvements to LLVM Decision Making

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

[llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
Hi Everyone,

Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Here is the outline of the draft proposal.  Caveats: this is itself just a draft proposal to inspire discussion not a mandate - I welcome feedback, thoughts and concerns, ideas of how to make this better, or better ideas/approaches overall on how we can level up the community.

Please let me know what you think, thanks!

-Chris





_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
On Tue, Jan 14, 2020, 22:58 Chris Lattner via llvm-dev <[hidden email]> wrote:
Hi Everyone,

Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Here is the outline of the draft proposal.  Caveats: this is itself just a draft proposal to inspire discussion not a mandate - I welcome feedback, thoughts and concerns, ideas of how to make this better, or better ideas/approaches overall on how we can level up the community.

Please let me know what you think, thanks!

Looks like a good idea to me! I particularly like that it is similar to Rust's RFC model, which has seemed to work out really well for Rust.

Jacob

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
Overall, this sounds like a good proposal to me. I think I have some mild concerns regarding the decision making body, as I can't help but suspect that such a body will inevitably focus their decisions based on the opinions of the louder or more well known members of the community. While to a certain extent that may be appropriate (the more well-known members will typically have greater experience), I think it's important to make sure that the less well-known etc are not ignored completely. I don't have any concrete suggestions though, and I do agree that a decision making body would help clarify things if nothing else.

One other thought: any formal review period needs to be long enough for people to contribute to if they have any annual leave from work or whatever. For example, if the review period were to be set to two weeks, I'd have missed proposals made at the start of roughly 2-3 different 2 week periods last year. It would have been worse for 1 week. On the other hand, a 3 week period would have meant I'd be able to read and respond to every review. Note this is just an example - I'm not concretely suggesting 3 weeks; perhaps it should be longer for bigger changes etc?

On Wed, 15 Jan 2020 at 06:59, Chris Lattner via llvm-dev <[hidden email]> wrote:
Hi Everyone,

Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Here is the outline of the draft proposal.  Caveats: this is itself just a draft proposal to inspire discussion not a mandate - I welcome feedback, thoughts and concerns, ideas of how to make this better, or better ideas/approaches overall on how we can level up the community.

Please let me know what you think, thanks!

-Chris




_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
Hi Chris,

I experience a fair share of problems when it comes to code reviews and
change requests so I appreciate any improvement we can make here.


On 01/14, Chris Lattner via llvm-dev wrote:
> Numerous people have been bringing up challenges with consensus driven
> decision making in the LLVM community.  After considering this and
> seeing the frustrations it is causing many people, I think we should
> make a formal process change to help improve decision making going
> forward.

Improving is always a good idea.


> Here is the outline of the draft proposal
> <https://gist.github.com/lattner/e3679998a7609c99b1243f09d30f0132>.
> Caveats: this is itself just a draft proposal to inspire discussion
> not a mandate - I welcome feedback, thoughts and concerns, ideas of
> how to make this better, or better ideas/approaches overall on how we
> can level up the community.
>
> Please let me know what you think, thanks!

First thought: This should not be a gist on github. Maybe it should be
on phabricator or at least in an email so it is easier to respond to it.


I'll try to inline the gist below so I can respond to the points.

---

> "It isn't clear how to propose some changes in the first place, and it
> is often unclear who the decision makers are."

I feel that patches and RFCs are well established *and documented* [1,2]
ways to propose changes. In addition, the *-dev lists, IRC, etc. do
provide ways to clear uncertainties. Adding more documentation on this
can obviously be helpful. Also, we are already improving the existing
documentation [0].

That said, we need to differentiate in more detail what the problems
actually are. Do people not find the documentation? Is the documentation
unclear? Are people waiting for "a decision maker" to step in or afraid
to ask questions? Are questions not answered? ...


> "It isn't clear what mailing lists to send such proposals to: llvm-dev
> gets a lot of traffic and many people don’t read it."

I argue that people that do participate in the decision making (wording
took from the previous point) already do, or at least should, follow
llvm-dev.
I am not quite sure why it is unclear what mailing list is the right one
given that we have one *-dev mailing list per subproject and on llvm-dev,
often the first points-of-contact, people are referred to the right one.


> "It is hard to know whether something is a serious proposal that you
> must take seriously, and so it is easy to miss important changes
> because you don't have time to review everything. Even though you
> technically had a chance to participate, you can end up surprised when
> some change goes into effect."

I'm unsure what kinds of proposal are supposed to be "not serious" and
who is supposed to be making that decision.


> "Sometimes people chime in late with dissent after a decision has been
> apparently made: this can be frustrating to people who need a decision
> made, because they aren't sure how to proceed."

It is unclear to me how the proposal helps on this front. Could you
elaborate?


> "Sometimes people express a loud voice on discussion threads even if
> they aren't active contributors, and they can derail discussions.
> There is no "moderator" for these discussions."

With the caveat of finding the moderator (as mentioned below), this
makes sense. We probably/hopefully don't need a moderator (for this
reason) on many discussions but it might certainly help if people step
up if a discussion is derailed (for any reason and by anyone).


> "The initial discussion phase of a proposal can have lots of back and
> forth discussions to shape a idea, and the eventual proposal can have
> significant changes from that initial review. It can be difficult to
> discern what feedback from the initial review is addressed or ignored
> in the final rounds of discussions."

Yes. I am not sure how the proposed solution remedies the problem
though. Could you elaborate?


> "Complex changes (e.g. the relicensing project) sometimes take many
> rounds of iteration, and it can be easy to lose track of where the
> proposal is and what the history was."

This is certainly true. The proposed solution (with moderators and
rounds) is probably implementable and reasonable for "complex changes".


> "Occasionally, Code Owners may be faced with a technical decision and
> not be sure what to do, particularly for highly impactful design
> decisions - e.g. for core LLVM IR changes. It could be useful to have
> a formal escalation process for these decisions."

TBH, I always thought "code owner" is not a "decision making" but rather
an "administrative" title. The person can be part of the decision
making, but the role of "code owner" does not imply special privileges,
only tasks, e.g., making sure reviews are done by the right people.


> I recommend that we add a process similar to (but adapted and changed
> where it makes sense) the Swift Evolution process. This process was
> designed to help guide decision making for high impact language and
> standard library changes to the Swift language. It was inspired by
> similar processes in other language communities (e.g. the Rust
> Language RFC process, the Python PEP process, etc) and has worked well
> in that context - it stands to reason that a variant should work well
> for LLVM as well.

I tried to determine how Rust RFCs (and Python PEPs) influenced the
proposed Swift Evolution process but from the history of the linked
pages (in the gist) that was not clear. Could you elaborate on that?


> The solution entails several parts. First, the process should be
> written down!

Agreed!


> This written process should include things like:
>
>     An informal "pitch" phase to help collect requirements and shape
>     the ultimate form of an idea, but which can be ignored by people
>     who aren't interested in following all of the details of a
>     discussion.

How is this different form the discussion that happens after an initial
RFC is send?

People already ignore it if they are not interested in the details.  If
people chime in late, as mentioned in the problems above, this will not
help, right? I mean, if the pitch phase is done and then people start to
chime it starts again.  This can have any reason, they are late, they
want to see if it is really "a serious proposal", or they just want to
wait until the first round of discussion changed the proposal to start
the second round.


>     A new mailing list (or Discourse channel) dedicated to formal
>     proposal review. This would be relatively low volume, allowing
>     effectively everyone to follow it. This could be named something
>     like "llvm-proposals".

1) We have already 30+* Discourse channels. Having so many, and one
   more, makes it harder to actually monitor them. Additionally, people
   that do not have Discourse are already excluded from the discussion.
   (I feel this had/has the opposite effect it was supposed to have.)
2) Arguably you could filter *-dev lists for the tag RFC instead of
   having a new mailing list^. People sending RFCs send without the tag
   can be asked to send it again with the tag.
3) This would only be low-volume if you do not count the
   responses/discussion.

* I haven't counted them but I am probably close with my estimate.
^ We have a lot already which can be, as implicit mentioned above,
  confusing.


>     A written proposal template, which provides guidelines for how to
>     write a proposal. This proposal is written in an example template
>     we can use, and the template can evolve over time.

I'm in favor.


>     A new directory on Github that serves as the archive for official proposals.

I don't see how that helps so I'd like to ask you to elaborate why this
is not yet another place one has to look for information.


>     A review manager who helps shepherd the discussion in official
>     rounds of review (which are time bound, e.g. to a week or two).

Could you elaborate on how these review managers are determined?


>     A decision making body that evaluates the results, asking for
>     additional discussions (with advice for how to change the proposal
>     in the next round), and that ultimately approves or denies a
>     proposal. This body should aim to get the community to consensus
>     whenever possible, but can help split decisions in the case of
>     extreme ambiguity when overwise all hope is lost. Denied proposals
>     can of course be re-run if a pertinent situation changes or when
>     revised

Could you elaborate on how these "decision making bodies" are
determined?


Thanks for initiating this,
  Johannes


[0] https://reviews.llvm.org/D71916
[1] http://llvm.org/docs/DeveloperPolicy.html#code-reviews
[2] https://www.llvm.org/docs/Contributing.html

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
On 01/15, James Henderson via llvm-dev wrote:
> One other thought: any formal review period needs to be long enough for
> people to contribute to if they have any annual leave from work or
> whatever. For example, if the review period were to be set to two weeks,
> I'd have missed proposals made at the start of roughly 2-3 different 2 week
> periods last year. It would have been worse for 1 week. On the other hand,
> a 3 week period would have meant I'd be able to read and respond to every
> review. Note this is just an example - I'm not concretely suggesting 3
> weeks; perhaps it should be longer for bigger changes etc?

There are various opinions on this (see for example the discussion here
[0]).

My take is that there is no fixed reasonable time to review and respond.
There is a minimal one, due to weekends and time zones, but as soon as
we take vacation/trips into account the problem is unbounded. Instead, I
argue that post-reviews and potential revers are acceptable. If a
consensus was reached and a reasonable* amount of time has passed
changes should make it into the repository to guarantee timely progress
for contributors. If problems are encountered later, either because the
change was not on someones radar or because no one anticipated some
problematic interaction, we should be flexible. A post-review discussion
is appropriate if improvements are needed, a potential revert and
follow-up review are appropriate if it was an actual breaking change.


* Both "consensus" and "reasonable amount of time" are arguably
  vague here. Appropriate metrics depend on the impact of the proposed
  change and written guidelines would be helpful [1].


[0] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136808.html
[1] https://reviews.llvm.org/D71916

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

signature.asc (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
On Wed, 15 Jan 2020 at 06:58, Chris Lattner via llvm-dev
<[hidden email]> wrote:
> Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Hi Chris,

Having proposed this many years ago, I think it's a good move forward. :)

I also mostly agree with your points, here are just some additional
material to think about.

My main concerns about the current model:
 * You get hit by a bus (hopefully not that drastic, but other life
changing moments can occur).
 * You get a lot of pressure to take decisions that are guaranteed to
divide the community (bad for mental health)
 * You defer to "the community" to avoid mental health breakdown, then
chaos ensue (we have good examples)
 * At the end of chaos, we get to a certain solution, again, with the
community divided

This is the main problem with the "benevolent dictator" model, and I
think it's worse for you than it is for most of us.

I have had a few bad experiences (a no good ones) in building a
decision making body that represents a diverse community, and it boils
down to representativity.

If an unelected body appoints people, no matter how trusted and
truthful they are, the community will already be divided with the
inherent bias (intentional or not).

If we want to go for meritocracy, which I think most people would
agree, then it would have to be people that already represent the
community, internally or externally.

Active code owners is an easy solution, but also fraught with problems
(like what constitutes an active code owner?). It also leaves external
active users from the equation (all the languages and side projects).

Perhaps we could refine the definition of a code owner as one that
actively participates in or around an area and refine the list to
include external users as well, for example Swift or Julia owners.

Code owners are already somewhat elected by either creating the code
base (back-end, passes) or stepping up and being accepted by the
community. Existing code owners can also step down and propose others,
which are again accepted or not by the community. This is a reasonably
democractic process, even if not encoded, and I think it's the closes
to representation we have.

But we also need representation of the users and wider groups that do
not relate to code, and I tihnk that's where the foundation comes in.
We should also look for other opportunities (ethnical groupos?
minorities?) to provide their own point of view.

Hoewever, I'd strongly advise against a simple voting system.

After all, technical decisions should not be taken by the opinion of
the majority, but by strong and corroborated facts. True, those also
tend to fall into categories (see GihtHub PR vs Phab, both sides have
strong points). But voting should only be a last resort, when there is
no clear majority on any side.

Voting systems also create biase in themselves, by over/under
representing certain groups in the number of members allowed to vote.
This will create yet another meta-argument and it'll drag us forever.

If we stick to facts first, and ask the representatives to bring any
concerns their sub-community may have, and we collate all of those,
and there is a clear majority, then the process was fair. Not everyone
was happy, but that was never possible to begin with, so it's ok.

We should also differentiate between code, process and community
changes. Examples are new passes, GitHub and policy discussions. They
need very different time frames, from weeks, to months, to years, not
necessarily respectively.

We also need to be scalable. If we have a decision that only affects a
small part of the code, we can handle in the sub-community (like we
currently do RFCs), but if some points may affect a wider range (like
stepvector in SVE), then we need to widen the scope, involve more
people, and we need those people to be pro active and responsive.

Finally, I think we need to be flexible.

If new information comes to light, or if someone was on extended
holidays and didn't catch up, etc. We need to make sure we don't just
alienate a sub-community because of a rigid process.

After all, the end goal is to make better decisions without burdening
the same few individuals every time or fragmenting the community.

cheers,
--renato
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
+1 from us on this.

The only caveat I'll add is:
  • Getting the right mix of representatives is key.
  • You need a chair that'll do the book-keeping but be impartial to the decisions.
  • Having a nuclear option of 'the representatives cannot agree' is important too.
Drawing from some life lessons when I spent 5 years designing Vulkan as part of Khronos, the general philosophy was that we'd discuss until we came to a consensus on an issue. This meant we definitely spent more time than was required to solve the technical issues, but I think in each instance the time sunk was of benefit to the individual design decisions we took. But you always need the nuclear option in that sometimes pushing through any decision is better than the paralysis of none at all. In Khronos it was done with a one company / one vote -> majority passes. This was clean-ish there, but failed to take into account that huge companies like Google might have two very different teams working on the technology internally that might have differing views, which couldn't be represented cleanly. You also don't want to overbalance the decision making process by having all representatives from one/a-few companies (this would be way worse than the BDFL we have currently!).

Cheers,
-Neil.

On Wed, Jan 15, 2020 at 10:42 AM Renato Golin via llvm-dev <[hidden email]> wrote:
On Wed, 15 Jan 2020 at 06:58, Chris Lattner via llvm-dev
<[hidden email]> wrote:
> Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Hi Chris,

Having proposed this many years ago, I think it's a good move forward. :)

I also mostly agree with your points, here are just some additional
material to think about.

My main concerns about the current model:
 * You get hit by a bus (hopefully not that drastic, but other life
changing moments can occur).
 * You get a lot of pressure to take decisions that are guaranteed to
divide the community (bad for mental health)
 * You defer to "the community" to avoid mental health breakdown, then
chaos ensue (we have good examples)
 * At the end of chaos, we get to a certain solution, again, with the
community divided

This is the main problem with the "benevolent dictator" model, and I
think it's worse for you than it is for most of us.

I have had a few bad experiences (a no good ones) in building a
decision making body that represents a diverse community, and it boils
down to representativity.

If an unelected body appoints people, no matter how trusted and
truthful they are, the community will already be divided with the
inherent bias (intentional or not).

If we want to go for meritocracy, which I think most people would
agree, then it would have to be people that already represent the
community, internally or externally.

Active code owners is an easy solution, but also fraught with problems
(like what constitutes an active code owner?). It also leaves external
active users from the equation (all the languages and side projects).

Perhaps we could refine the definition of a code owner as one that
actively participates in or around an area and refine the list to
include external users as well, for example Swift or Julia owners.

Code owners are already somewhat elected by either creating the code
base (back-end, passes) or stepping up and being accepted by the
community. Existing code owners can also step down and propose others,
which are again accepted or not by the community. This is a reasonably
democractic process, even if not encoded, and I think it's the closes
to representation we have.

But we also need representation of the users and wider groups that do
not relate to code, and I tihnk that's where the foundation comes in.
We should also look for other opportunities (ethnical groupos?
minorities?) to provide their own point of view.

Hoewever, I'd strongly advise against a simple voting system.

After all, technical decisions should not be taken by the opinion of
the majority, but by strong and corroborated facts. True, those also
tend to fall into categories (see GihtHub PR vs Phab, both sides have
strong points). But voting should only be a last resort, when there is
no clear majority on any side.

Voting systems also create biase in themselves, by over/under
representing certain groups in the number of members allowed to vote.
This will create yet another meta-argument and it'll drag us forever.

If we stick to facts first, and ask the representatives to bring any
concerns their sub-community may have, and we collate all of those,
and there is a clear majority, then the process was fair. Not everyone
was happy, but that was never possible to begin with, so it's ok.

We should also differentiate between code, process and community
changes. Examples are new passes, GitHub and policy discussions. They
need very different time frames, from weeks, to months, to years, not
necessarily respectively.

We also need to be scalable. If we have a decision that only affects a
small part of the code, we can handle in the sub-community (like we
currently do RFCs), but if some points may affect a wider range (like
stepvector in SVE), then we need to widen the scope, involve more
people, and we need those people to be pro active and responsive.

Finally, I think we need to be flexible.

If new information comes to light, or if someone was on extended
holidays and didn't catch up, etc. We need to make sure we don't just
alienate a sub-community because of a rigid process.

After all, the end goal is to make better decisions without burdening
the same few individuals every time or fragmenting the community.

cheers,
--renato
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


--
Neil Henning
Senior Software Engineer Compiler

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
On 15/01/2020 10:04, Doerfert, Johannes via llvm-dev wrote:
>> "It isn't clear how to propose some changes in the first place, and it
>> is often unclear who the decision makers are."
> I feel that patches and RFCs are well established*and documented*  [1,2]
> ways to propose changes. In addition, the *-dev lists, IRC, etc. do
> provide ways to clear uncertainties. Adding more documentation on this
> can obviously be helpful. Also, we are already improving the existing
> documentation [0].

This came up at the WICT workshop (which still doesn't have a writeup on
the LLVM web site) and has previously come up in Cambridge and Paris
LLVM socials that I've attended and at EuroLLVM:

There is no formal definition of 'the LLVM community' and therefore
anything that requires consensus of the community is difficult to
define.  I have spoken to several people who have believed based on
mailing list discussions that consensus on an issue is one thing, but
then the decision has been taken to do the other option, often with the
justification that this was the consensus of the people at a BoF at the
San Jose dev meeting.

There is no formal leadership of the LLVM community.  The LLVM
Foundation leadership is self-selected and not necessarily
representative, so cannot fill the 'buck stops here' role of someone who
has to make the call and justify their decisions when they come up for
reelection.

David
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
To be clear, my comment was purely intended to refer to non-technical decisions (i.e. not things like code reviews etc that stall because there's disagreement about what to do, but rather things like switching to the monorepo/github PRs etc). Indeed, my understanding from Chris's pitch is that his proposal isn't intended to address that either:

" One note: While there are challenges with patch review and code owners, this proposal focuses on non-technical decisions that do not have a clear "code owner" escalation path today. This can include things like the introduction of new subprojects, introduction of a new social policies, change to core infrastructure like bug review tools or patch review processes, changes to the LLVM Developer Policy, etc."

These decisions are significantly more impacting than the more technical ones as they usually impact pretty much every single developer. They are also often irreversible after a certain point, or at least would cause serious issues if we tried to reverse. Finally, once a decision has been made and started to be implemented, I always feel like there's a greater level required for objections, so people who weren't able to be involved are less likely to voice their opinions after the fact in a way that will actually generate any further discussion. Don't get me wrong, I agree that we can't keep a review open forever, since you can't accommodate everyone (e.g. months-long parental leave/long-term sicknesses/sabbaticals etc), but surely 1-2 weeks for such decisions isn't enough.

On Wed, 15 Jan 2020 at 10:18, Doerfert, Johannes <[hidden email]> wrote:
On 01/15, James Henderson via llvm-dev wrote:
> One other thought: any formal review period needs to be long enough for
> people to contribute to if they have any annual leave from work or
> whatever. For example, if the review period were to be set to two weeks,
> I'd have missed proposals made at the start of roughly 2-3 different 2 week
> periods last year. It would have been worse for 1 week. On the other hand,
> a 3 week period would have meant I'd be able to read and respond to every
> review. Note this is just an example - I'm not concretely suggesting 3
> weeks; perhaps it should be longer for bigger changes etc?

There are various opinions on this (see for example the discussion here
[0]).

My take is that there is no fixed reasonable time to review and respond.
There is a minimal one, due to weekends and time zones, but as soon as
we take vacation/trips into account the problem is unbounded. Instead, I
argue that post-reviews and potential revers are acceptable. If a
consensus was reached and a reasonable* amount of time has passed
changes should make it into the repository to guarantee timely progress
for contributors. If problems are encountered later, either because the
change was not on someones radar or because no one anticipated some
problematic interaction, we should be flexible. A post-review discussion
is appropriate if improvements are needed, a potential revert and
follow-up review are appropriate if it was an actual breaking change.


* Both "consensus" and "reasonable amount of time" are arguably
  vague here. Appropriate metrics depend on the impact of the proposed
  change and written guidelines would be helpful [1].


[0] http://lists.llvm.org/pipermail/llvm-dev/2019-November/136808.html
[1] https://reviews.llvm.org/D71916

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
On Wed, 15 Jan 2020 at 11:58, James Henderson via llvm-dev
<[hidden email]> wrote:
> To be clear, my comment was purely intended to refer to non-technical decisions (i.e. not things like code reviews etc that stall because there's disagreement about what to do, but rather things like switching to the monorepo/github PRs etc). Indeed, my understanding from Chris's pitch is that his proposal isn't intended to address that either:

High level technical also fall into that category, for example,
changes in one area impacting others (like changing the IR semantics,
pass manager structure, etc) as well as accepting new projects with
varying degree of compatibility (like new front or back ends).

> These decisions are significantly more impacting than the more technical ones as they usually impact pretty much every single developer. They are also often irreversible after a certain point, or at least would cause serious issues if we tried to reverse. Finally, once a decision has been made and started to be implemented, I always feel like there's a greater level required for objections, so people who weren't able to be involved are less likely to voice their opinions after the fact in a way that will actually generate any further discussion. Don't get me wrong, I agree that we can't keep a review open forever, since you can't accommodate everyone (e.g. months-long parental leave/long-term sicknesses/sabbaticals etc), but surely 1-2 weeks for such decisions isn't enough.

I agree, that's why we need to be flexible on all counts. From
understanding that people sometimes miss the time (and waiting for or
pinging people), to accepting the responsibility of looking at those
issues with high priority.

First, we need to separate the noise, because the llvm-dev list is not
suitable. I oftten only pick up when Alex's weekly comes up, sometimes
it's too late. We need a clear and separate channel for decisions
which can interrupt us with high priority.

But also people that want/need to be involved in the decision process
must be responsible for their own domain. For example asking someone
else to look for important changes while you're away.

cheers,
--renato
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
On Tue, Jan 14, 2020 at 10:58 PM Chris Lattner via llvm-dev <[hidden email]> wrote:
Hi Everyone,

Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Here is the outline of the draft proposal.  Caveats: this is itself just a draft proposal to inspire discussion not a mandate - I welcome feedback, thoughts and concerns, ideas of how to make this better, or better ideas/approaches overall on how we can level up the community.

Please let me know what you think, thanks!

-Chris

I think this is a great idea and is sorely needed. I agree that selecting the body will be difficult, but I feel that any reasonable body is better than what we do today.

- Michael Spencer

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev


On Jan 15, 2020, at 1:15 AM, James Henderson <[hidden email]> wrote:

Overall, this sounds like a good proposal to me. I think I have some mild concerns regarding the decision making body, as I can't help but suspect that such a body will inevitably focus their decisions based on the opinions of the louder or more well known members of the community. While to a certain extent that may be appropriate (the more well-known members will typically have greater experience), I think it's important to make sure that the less well-known etc are not ignored completely. I don't have any concrete suggestions though, and I do agree that a decision making body would help clarify things if nothing else.

Agreed, I think that picking the decision making body is likely to be the hardest part.


One other thought: any formal review period needs to be long enough for people to contribute to if they have any annual leave from work or whatever. For example, if the review period were to be set to two weeks, I'd have missed proposals made at the start of roughly 2-3 different 2 week periods last year. It would have been worse for 1 week. On the other hand, a 3 week period would have meant I'd be able to read and respond to every review. Note this is just an example - I'm not concretely suggesting 3 weeks; perhaps it should be longer for bigger changes etc?

I think it is probably dependent on the decision in question.  If it is something like relicensing, you’d want a month or something really long.  If it is something simpler, then perhaps a week is ok.  I think we can figure this out on a case-by-case basis when the process goes into effect, it isn’t something that we need to prearrange as part of the process.

-Chris


On Wed, 15 Jan 2020 at 06:59, Chris Lattner via llvm-dev <[hidden email]> wrote:
Hi Everyone,

Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Here is the outline of the draft proposal.  Caveats: this is itself just a draft proposal to inspire discussion not a mandate - I welcome feedback, thoughts and concerns, ideas of how to make this better, or better ideas/approaches overall on how we can level up the community.

Please let me know what you think, thanks!

-Chris




_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev


> On Jan 15, 2020, at 3:40 AM, David Chisnall via llvm-dev <[hidden email]> wrote:
>
> On 15/01/2020 10:04, Doerfert, Johannes via llvm-dev wrote:
>>> "It isn't clear how to propose some changes in the first place, and it
>>> is often unclear who the decision makers are."
>> I feel that patches and RFCs are well established*and documented*  [1,2]
>> ways to propose changes. In addition, the *-dev lists, IRC, etc. do
>> provide ways to clear uncertainties. Adding more documentation on this
>> can obviously be helpful. Also, we are already improving the existing
>> documentation [0].
>
> This came up at the WICT workshop (which still doesn't have a writeup on the LLVM web site) and has previously come up in Cambridge and Paris LLVM socials that I've attended and at EuroLLVM:
>
> There is no formal definition of 'the LLVM community' and therefore anything that requires consensus of the community is difficult to define.  I have spoken to several people who have believed based on mailing list discussions that consensus on an issue is one thing, but then the decision has been taken to do the other option, often with the justification that this was the consensus of the people at a BoF at the San Jose dev meeting.

Yes, I agree.  The model I have seen work in the Swift community is that there is no formal voting or other pre-structured way a decision is made.  Instead, the community provides input to the core team, and the core team (in practice) always goes with the community if there is consensus.  If there isn’t consensus or if the core team believes that an important point has come up but hasn’t gotten enough discussion, then it kicks the discussion back to another round with guidance to focus on specific topics etc.  It is sort of a managed discussion process in practice.

> There is no formal leadership of the LLVM community.  The LLVM Foundation leadership is self-selected and not necessarily representative, so cannot fill the 'buck stops here' role of someone who has to make the call and justify their decisions when they come up for reelection.

I agree that this is the hardest part of getting a structure in place, as well as your concerns with the foundation board.  What do you suggest?  Do you have other ideas, recommendations, or a preferred approach?

-Chris  
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
Hi Chris,

tl;dr Sounds good, when can we start?

Thank you for picking up on this topic, I believe this is really important! The way we reach decisions - or don't - has a major impact on the culture of our  project. And the culture has a major impact on the happiness and health of the community. As a newcomer to LLVM I find it hard to understand how I can get an agreement on a particular topic affecting many people (pre-merge testing in my case).

For me it's important to have a way of getting a timely decision on a topic. How we do that in detail is less important to me. I'm also a big fan of continuous improvement and the Swift approach sounds reasonable. So let's start with that and then iterate/adapt if we see the need to do so.

Best,
Christian

On Wed, Jan 15, 2020 at 7:58 AM Chris Lattner via llvm-dev <[hidden email]> wrote:
Hi Everyone,

Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Here is the outline of the draft proposal.  Caveats: this is itself just a draft proposal to inspire discussion not a mandate - I welcome feedback, thoughts and concerns, ideas of how to make this better, or better ideas/approaches overall on how we can level up the community.

Please let me know what you think, thanks!

-Chris




_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


--
Best,
Christian

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
Renato Golin via llvm-dev <[hidden email]> writes:

> If we want to go for meritocracy, which I think most people would
> agree, then it would have to be people that already represent the
> community, internally or externally.

Given your close attention to representation I know that you have all
the best intentions here, so please take this in the spirit of
productive conversation that is intended.

"Meritocracy" always makes me feel uncomfortable.  Many many people
interpret that to mean those with the most experience/social capital are
best equipped to make decisions.  Not only is that not always true, it
leads to an organization with structural barriers to newcomers, those
with different/less common backgrounds and dissenting views.  I
definitely do not want to go for "meritocracy."

> Hoewever, I'd strongly advise against a simple voting system.

Agreed!

> We also need to be scalable. If we have a decision that only affects a
> small part of the code, we can handle in the sub-community (like we
> currently do RFCs), but if some points may affect a wider range (like
> stepvector in SVE), then we need to widen the scope, involve more
> people, and we need those people to be pro active and responsive.

My first reaction upon reading Chris' proposal was, "Great!  I'm glad
we're finally tackling this."  My second reaction was, "There probably
can't be a single decision-making group."

It's really hard to find a group of people with the passion, domain
knowledge, communication skills and time to make decisions on every
aspect of the project.  Note that by "domain knowledge" I'm including
expertise outside whatever specific technical aspect is under
discussion.  It's meant to be a broad term, not a narrow one that
excludes people.  I think we may want to consider specialty
decision-making groups composed of members who understand lots of
different aspects of particular areas.

For some decisions we might want to form ad-hoc committees (the
git/monorepo decision comes to mind).  It also may be worth having a
handful of standing committees with representatives most interested in
specific areas.  Here's a list of possible standing committees,
generated by me thinking back on various proposals and threads:

- Welcoming community (inclusivity/code of conduct)

- Developer tools and processes

- IR evolution

- Sponsorship/Summer of Code

We might also consider committees that aren't decision-making bodies per
se but carry on various project-wide activities:

- Social (maintain database of meetups, produce resources for social
  activities, etc.)

- Communication (maintain web site, run a Twitter account, etc.)

- Conference organization (would form ad-hoc review committees for
  individual conferences among other tasks)

Not knowing exactly what the LLVM Foundation board does in its
day-to-day work I don't know if one or more of these might fall under
its purview.  In my imagination, the LLVM Foundation board primarily
handles legal aspects, funding and perhaps final arbitration.  I'm not
sure whether it should take on any other formal decision-making roles.

I have some ideas on how to determine membership but they are just
ideas.  We'd want to keep these relatively small (10-20 people in my
mind).  I'm assuming some kind of rotating membership, maybe a couple of
people leave and are replaced each year so that committees maintain
institutional knowledge.  I imagine asking for volunteers and if more
people volunteer than available spots, there could be a membership
backlog where previous volunteers have the right of first refusal when
membership spots open up.  I'm trying to avoid elections and all of the
social problems that come about with them.

Committees would be expected to operate in the open, via public mailing
lists or some other mechanism.  Committees would be expected to solicit
input from the broader community by whatever means they determine best
(a designated mailbox, for example, possibly allowing for anonymous
input depending on the topic).

Learning from other projects will be important.  I am not very familiar
with how governance in other specific projects works.

Maybe formal committees is too heavyweight, I'm not sure, but I've tried
to construct a model to provide a solid framework that improves
inclusiveness and transparency in decision-making.

                      -David
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
Hi David,

I absolutely agree with the spirit is most of the contents of your response.

Some replies inline.

On Thu, 16 Jan 2020 at 16:21, David Greene <[hidden email]> wrote:
> "Meritocracy" always makes me feel uncomfortable.  Many many people
> interpret that to mean those with the most experience/social capital are
> best equipped to make decisions.  Not only is that not always true, it
> leads to an organization with structural barriers to newcomers, those
> with different/less common backgrounds and dissenting views.  I
> definitely do not want to go for "meritocracy."

You nailed it. I suggested meritocracy as "one of the" methods of
choosing representatives, not the only one. I also mention other ways
to look for non-obvious candidates, so that we can represent the
multiple dimensions of the community, as I believe is the spirit of
your comment.


> Not knowing exactly what the LLVM Foundation board does in its
> day-to-day work I don't know if one or more of these might fall under
> its purview.  In my imagination, the LLVM Foundation board primarily
> handles legal aspects, funding and perhaps final arbitration.  I'm not
> sure whether it should take on any other formal decision-making roles.

IIUC, making technical decisions, even final arbitration, is a clear
non-goal of the Foundation.

So far, every big decision we had was done via list threads, BoFs,
socials and in the few cases where it was required, the final
arbitration has been done by Chris.


> I have some ideas on how to determine membership but they are just
> ideas.  We'd want to keep these relatively small (10-20 people in my
> mind).  I'm assuming some kind of rotating membership, maybe a couple of
> people leave and are replaced each year so that committees maintain
> institutional knowledge.  I imagine asking for volunteers and if more
> people volunteer than available spots, there could be a membership
> backlog where previous volunteers have the right of first refusal when
> membership spots open up.  I'm trying to avoid elections and all of the
> social problems that come about with them.

I feel that creating a complex multi-dimensional rotational system
will do more harm than good.

When I said "scalable" I meant we should do what needs done when it
needs done, and not more.

So, we can have a large pool of representatives (by whatever fair
method we find), and whenever a topic needs discussion, those
representatives "subscribe" to the discussion. Topics appear in a
public forum, but not the llvm-dev list, so that it's easy to see
what's being discussed without pulloting day-to-day work.

All those subscribed are expressing that they want/need to be
consulted before a final decision (modulo timing issues, multiple
warnings, strong consensus, etc). People that don't subscribe are
stating whatever decision that is taken is ok for them (and their
sub-communities).

A sub-community can raise interest, in which case their
representative(s) are being strongly encouraged to participate, even
if they themselves are not pesonally interested. Such is the price of
representation (like we ask code owners to be the last line of review
and technical decisions).

Bigger discussions will have more people involved, smaller, less. To
deal with the big ones, an equally organic nature should be taken.
Discussions, consensus and decisions have time frames to be completed,
mostly set by the needs of the problems at hand, so that nothing drags
forever.

As a last resort, some kind of fair voting [1] system can reduce the
pain of representative bias and take some action that is endorsed by a
majority. If we assume a small majority is still better than no
decision, this should work fine, though I'm not claiming it is.

> Maybe formal committees is too heavyweight, I'm not sure, but I've tried
> to construct a model to provide a solid framework that improves
> inclusiveness and transparency in decision-making.

I think we need to accept that we won't get it right the first time,
and be ready to change the process, and potentially re-discuss closed
topics, as we go.

Starting from a very simple, flexible and organic system would
probably mean we'll make more mistakes up front, but I hope that it
will result in faster convergence to a system that trully represents
the community.

cheers,
--renato

Suggestion, not endorsement:
[1] https://en.wikipedia.org/wiki/Condorcet_method
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev

Trying to define a process without defining who implements the process seems a bit cart-before-horse, and I’m happy to see people debating how to compose the Core Team or equivalent who will implement the process.

 

LLVM to date has really had only one distinguished role, which is Code Owner.  Other projects have a more defined set of roles, and the LLVM community might benefit from that.  (I had one dev meeting proposal rejected due to lack of participation by a “core contributor” which left me at something of a loss, as LLVM has no such defined role.)

 

So, defining a Core Team or equivalent seems like a worthwhile preliminary step.  Whether they choose to adopt Chris’s decision-making process proposal or something else, should be up to them IMO.

--paulr

 

From: llvm-dev <[hidden email]> On Behalf Of Chris Lattner via llvm-dev
Sent: Wednesday, January 15, 2020 1:59 AM
To: llvm-dev <[hidden email]>
Subject: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

 

Hi Everyone,

Numerous people have been bringing up challenges with consensus driven decision making in the LLVM community.  After considering this and seeing the frustrations it is causing many people, I think we should make a formal process change to help improve decision making going forward.

Here is the outline of the draft proposal.  Caveats: this is itself just a draft proposal to inspire discussion not a mandate - I welcome feedback, thoughts and concerns, ideas of how to make this better, or better ideas/approaches overall on how we can level up the community.

Please let me know what you think, thanks!

-Chris




_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev


On Wed, Jan 15, 2020 at 2:04 AM Doerfert, Johannes via llvm-dev <[hidden email]> wrote:
Hi Chris,

I experience a fair share of problems when it comes to code reviews and
change requests so I appreciate any improvement we can make here.


On 01/14, Chris Lattner via llvm-dev wrote:
> Numerous people have been bringing up challenges with consensus driven
> decision making in the LLVM community.  After considering this and
> seeing the frustrations it is causing many people, I think we should
> make a formal process change to help improve decision making going
> forward.

Improving is always a good idea.


> Here is the outline of the draft proposal
> <https://gist.github.com/lattner/e3679998a7609c99b1243f09d30f0132>.
> Caveats: this is itself just a draft proposal to inspire discussion
> not a mandate - I welcome feedback, thoughts and concerns, ideas of
> how to make this better, or better ideas/approaches overall on how we
> can level up the community.
>
> Please let me know what you think, thanks!

First thought: This should not be a gist on github. Maybe it should be
on phabricator or at least in an email so it is easier to respond to it.


I'll try to inline the gist below so I can respond to the points.

---

> "It isn't clear how to propose some changes in the first place, and it
> is often unclear who the decision makers are."

I feel that patches and RFCs are well established *and documented* [1,2]
ways to propose changes. In addition, the *-dev lists, IRC, etc. do
provide ways to clear uncertainties. Adding more documentation on this
can obviously be helpful. Also, we are already improving the existing
documentation [0].

That said, we need to differentiate in more detail what the problems
actually are. Do people not find the documentation? Is the documentation
unclear? Are people waiting for "a decision maker" to step in or afraid
to ask questions? Are questions not answered? ...


> "It isn't clear what mailing lists to send such proposals to: llvm-dev
> gets a lot of traffic and many people don’t read it."

I argue that people that do participate in the decision making (wording
took from the previous point) already do, or at least should, follow
llvm-dev.
I am not quite sure why it is unclear what mailing list is the right one
given that we have one *-dev mailing list per subproject and on llvm-dev,
often the first points-of-contact, people are referred to the right one.


> "It is hard to know whether something is a serious proposal that you
> must take seriously, and so it is easy to miss important changes
> because you don't have time to review everything. Even though you
> technically had a chance to participate, you can end up surprised when
> some change goes into effect."

I'm unsure what kinds of proposal are supposed to be "not serious" and
who is supposed to be making that decision.


> "Sometimes people chime in late with dissent after a decision has been
> apparently made: this can be frustrating to people who need a decision
> made, because they aren't sure how to proceed."

It is unclear to me how the proposal helps on this front. Could you
elaborate?


> "Sometimes people express a loud voice on discussion threads even if
> they aren't active contributors, and they can derail discussions.
> There is no "moderator" for these discussions."

With the caveat of finding the moderator (as mentioned below), this
makes sense. We probably/hopefully don't need a moderator (for this
reason) on many discussions but it might certainly help if people step
up if a discussion is derailed (for any reason and by anyone).


> "The initial discussion phase of a proposal can have lots of back and
> forth discussions to shape a idea, and the eventual proposal can have
> significant changes from that initial review. It can be difficult to
> discern what feedback from the initial review is addressed or ignored
> in the final rounds of discussions."

Yes. I am not sure how the proposed solution remedies the problem
though. Could you elaborate?


> "Complex changes (e.g. the relicensing project) sometimes take many
> rounds of iteration, and it can be easy to lose track of where the
> proposal is and what the history was."

This is certainly true. The proposed solution (with moderators and
rounds) is probably implementable and reasonable for "complex changes".


> "Occasionally, Code Owners may be faced with a technical decision and
> not be sure what to do, particularly for highly impactful design
> decisions - e.g. for core LLVM IR changes. It could be useful to have
> a formal escalation process for these decisions."

TBH, I always thought "code owner" is not a "decision making" but rather
an "administrative" title. The person can be part of the decision
making, but the role of "code owner" does not imply special privileges,
only tasks, e.g., making sure reviews are done by the right people.


> I recommend that we add a process similar to (but adapted and changed
> where it makes sense) the Swift Evolution process. This process was
> designed to help guide decision making for high impact language and
> standard library changes to the Swift language. It was inspired by
> similar processes in other language communities (e.g. the Rust
> Language RFC process, the Python PEP process, etc) and has worked well
> in that context - it stands to reason that a variant should work well
> for LLVM as well.

I tried to determine how Rust RFCs (and Python PEPs) influenced the
proposed Swift Evolution process but from the history of the linked
pages (in the gist) that was not clear. Could you elaborate on that?


> The solution entails several parts. First, the process should be
> written down!

Agreed!


> This written process should include things like:
>
>     An informal "pitch" phase to help collect requirements and shape
>     the ultimate form of an idea, but which can be ignored by people
>     who aren't interested in following all of the details of a
>     discussion.

How is this different form the discussion that happens after an initial
RFC is send?

People already ignore it if they are not interested in the details.  If
people chime in late, as mentioned in the problems above, this will not
help, right? I mean, if the pitch phase is done and then people start to
chime it starts again.  This can have any reason, they are late, they
want to see if it is really "a serious proposal", or they just want to
wait until the first round of discussion changed the proposal to start
the second round.


>     A new mailing list (or Discourse channel) dedicated to formal
>     proposal review. This would be relatively low volume, allowing
>     effectively everyone to follow it. This could be named something
>     like "llvm-proposals".

1) We have already 30+* Discourse channels. Having so many, and one
   more, makes it harder to actually monitor them. Additionally, people
   that do not have Discourse are already excluded from the discussion.
   (I feel this had/has the opposite effect it was supposed to have.)

Are you sure that you are not confusing Discourse with Discord here?

Discord is an IRC replacement, Discourse is more of a candidate to replace the mailing-list (there is a mailing list mode where every post end up in your mailbox: it should not be a regression over the mailing-list).

-- 
Mehdi


 
2) Arguably you could filter *-dev lists for the tag RFC instead of
   having a new mailing list^. People sending RFCs send without the tag
   can be asked to send it again with the tag.
3) This would only be low-volume if you do not count the
   responses/discussion.

* I haven't counted them but I am probably close with my estimate.
^ We have a lot already which can be, as implicit mentioned above,
  confusing.


>     A written proposal template, which provides guidelines for how to
>     write a proposal. This proposal is written in an example template
>     we can use, and the template can evolve over time.

I'm in favor.


>     A new directory on Github that serves as the archive for official proposals.

I don't see how that helps so I'd like to ask you to elaborate why this
is not yet another place one has to look for information.


>     A review manager who helps shepherd the discussion in official
>     rounds of review (which are time bound, e.g. to a week or two).

Could you elaborate on how these review managers are determined?


>     A decision making body that evaluates the results, asking for
>     additional discussions (with advice for how to change the proposal
>     in the next round), and that ultimately approves or denies a
>     proposal. This body should aim to get the community to consensus
>     whenever possible, but can help split decisions in the case of
>     extreme ambiguity when overwise all hope is lost. Denied proposals
>     can of course be re-run if a pertinent situation changes or when
>     revised

Could you elaborate on how these "decision making bodies" are
determined?


Thanks for initiating this,
  Johannes


[0] https://reviews.llvm.org/D71916
[1] http://llvm.org/docs/DeveloperPolicy.html#code-reviews
[2] https://www.llvm.org/docs/Contributing.html
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
On 16/01/2020 06:44, Chris Lattner wrote:
> Yes, I agree.  The model I have seen work in the Swift community is that there is no formal voting or other pre-structured way a decision is made.  Instead, the community provides input to the core team, and the core team (in practice) always goes with the community if there is consensus.  If there isn’t consensus or if the core team believes that an important point has come up but hasn’t gotten enough discussion, then it kicks the discussion back to another round with guidance to focus on specific topics etc.  It is sort of a managed discussion process in practice.

How is the Swift core team selected?

>> There is no formal leadership of the LLVM community.  The LLVM Foundation leadership is self-selected and not necessarily representative, so cannot fill the 'buck stops here' role of someone who has to make the call and justify their decisions when they come up for reelection.
> I agree that this is the hardest part of getting a structure in place, as well as your concerns with the foundation board.  What do you suggest?  Do you have other ideas, recommendations, or a preferred approach?

I've never seen a model that works well in all situations.  I've been in
a few projects where there's a clear maintainer and as long as most
decisions are reached by consensus that the maintainer stamps with their
approval, things work well.  This model breaks down when there are
decisions that do not have clear consensus, for example moving a project
to GitHub when a significant fraction of the community have moral
objections to infrastructure that does not meet the FSF's approval.  In
this case, there is nothing that the maintainer can do without annoying
half of the community.

The FreeBSD model works reasonably well.  First, there is an explicit
notion of a committer as an official project member.  Committers have
voting rights for the Core Team.  To retain these rights, you must have
committed something in the 12 months before a Core election.  To become
a committer, you must be proposed, voted for by the Core Team, and must
then be mentored by an existing committer, who must approve all of your
commits for a little while.

We discussed a mentorship system similar to this for LLVM at the WiCT
workshop and I'd be in favour of something along those lines,
irrespective of the rest of this process.

There are a few problems with the FreeBSD model:

Being a member of the community is tied to committing code (or docs).
There are a number of incredibly valuable members of the community whose
contributions are in other forms.  The WiCT keynote (which is still not
online as far as I could see!) highlighted the value of these people.
Decisions made by the Core Team can often affect these people, but they
have no say in the elections.

FreeBSD also has a large downstream community that is not necessarily
represented by this system.  In LLVM's case, this community is an even
larger part of the whole ecosystem.  There are a lot of out-of-tree
front ends, for example, yet most of the developers of those are not
represented by our current processes.  It is a mistake to ignore this
community because it should be one of our primary recruiting pools:
writing a tool that uses LLVM is the best introduction to starting to
work on LLVM itself.  The current decision-making process can often
leave these people feeling ignored.  The monorepo decision was a good
example of this: it made life easier for active contributors to a set of
core LLVM projects and harder for a number of categories of downstream
consumers.  Making these people feel as if they don't have a voice does
not encourage them into our community.

Electing a core team can work very well if the electorate represents the
interests of the community.  The LLVM community has a very sharp
in-group / out-group divide currently and I struggle to see any way of
defining an electorate for our wider community that would work well.  I
would love to see more effort made to address this split independently,
because I think it would give us a more healthy community and a better
path to a democratic decision making process.

David
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Reply | Threaded
Open this post in threaded view
|

Re: [llvm-dev] [PITCH] Improvements to LLVM Decision Making

Alex Brachet-Mialot via llvm-dev
In reply to this post by Alex Brachet-Mialot via llvm-dev
Renato Golin <[hidden email]> writes:

> Starting from a very simple, flexible and organic system would
> probably mean we'll make more mistakes up front, but I hope that it
> will result in faster convergence to a system that trully represents
> the community.

This makes a lot of sense to me.  +1.

                 -David
_______________________________________________
LLVM Developers mailing list
[hidden email]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev