Lizka's Quick takes

Thanks for writing this Lizka! I agree with many of the points in this [I was also a visiting fellow on the longtermist team this summer]. I'll throw my two cents in about my own reflections (I broadly share Lizka's experience, so here I just highlight the upsides/downsides things that especially resonated with me, or things unique to my own situation):

Vague background:

• Finished BSc in PPE this June
• No EA research experience and very little academic research experience
• Introduced to EA in 2019

Upsides:

• Work in areas that are intellectually stimulating and feel meaningful (e.g. Democracy, AI Governance).
• Become a better researcher. In particular, understanding reasoning transparency, reaching out to experts, the neglected virtue of scholarship, giving and receiving feedback, and being generally more productive. Of course, there is a difference between 1. Understanding these skills, and 2. internalizing & applying them, but I think RP helped substantially with the first and set me on the path to doing the second.
• Working with super cool people. Everyone was super friendly, and clearly supportive of our development as researchers. I also had not written an EA forum post before RP, but was super supported to break this barrier.

Downsides:

• Working remotely was super challenging for me. I underestimated how significant a factor this would be to begin with, and so I would not dismiss this lightly. However, I think there are ways that one can remedy this if they are sufficiently proactive/agent-y (e.g. setting up in-person co-working, moving cities to be near staff, using Focusmate, etc). Also, +1 to getting a fast computer (and see Peter's comment on this).
• Imposter syndrome. One downside of working with super cool, brilliant, hard working people was (for me) a feeling that I was way out of my depth, especially to begin with. This is of course different for everyone, but one thing I struggled to fully overcome. However, RP staff are very willing to help out where they can, should this become a problem.
• Ugh fields. There were definitely times when I felt somewhat overwhelmed by work, with sometimes negative spirals. This wasn't helped by personal circumstances, but my manager (Michael) was super accommodating and understanding of this, which helped alleviate guilt.

If it's helpful, I might write-up a shortform on some of these points in more depth, especially the things I learnt about being a better researcher, if that's helpful for others.

Overall, I also really enjoyed my time at RP, and would highly recommend :)

(I did not speak to anyone at RP before writing this).

Protip: if you find yourself with a slow computer, fix that situation asap.

Note to onlookers that we at Rethink Priorities will pay up to $2000 for people to upgrade their computers and that we view this as very important! And if you work with us for more than a year, you can keep your new computer forever.

I realize that this policy may not be a great fit for interns / fellows though, so perhaps I will think about how we can approach that.

This is awesome! Is there a page somewhere that collates the results of a bunch of internal forecasting by the end of the grant period? I'd be interested

I’d be excited to see more celebration or appreciation for people’s work.

I would be excited about this and have wondered for a while if we should have EA awards. This Washington post article brought the idea to my mind again:

Civil servants who screwed up were dragged before Congress and into the news. Civil servants who did something great, no one said a word about. There was thus little incentive to do something great, and a lot of incentive to hide. The awards were meant to correct that problem. “There’s no culture of recognition in government,” said Max Stier, whom Heyman hired to run the Partnership. “We wanted to create a culture of recognition.”

“Slow-rolling mistakes” are usually much more important to identify than “point-in-time blunders”

After reading your post, I wasn't sure you were right about this. But after thinking about it for a few minutes, I can't come up with any serious mistakes I've made that were "point-in-time blunders".

The closest thing I can think of is when I accidentally donated $20,000 to the GiveWell Community Foundation instead of The Clear Fund (aka GiveWell), but fortunately they returned the money so it all worked out.

For anyone interested in watching a dramatic reconstruction of this incident, go to timestamp 43:30–47:05 of The Man Who Saved The World. (I recommend watching at 1.5x speed.)

I’m a researcher on voting theory, with a focus on voting over how to divide a budget between uses. Sorry I found this post late, so probably things are already decided but I thought I’d add my thoughts. I’m going to assume approval voting as input format.

There is an important high-level decision to make first regarding the objective: do we want to pick charities with the highest support (majoritarian) or do we want to give everyone equal influence on the outcome if possible (proportionality)?

If the answer is “majoritarian”, then the simplest method makes the most sense: give all the money to the charity with the highest approval score. (This maximizes the sum of voter utilities, if you define voter utility to be the amount of money that goes to the charities a voter approves.)

If the answer is “proportionality”, my top recommendation would be to drop the idea of having only 3 winners and not impose a limit, and instead use the Nash Product rule to decide how the money is split [paper, wikipedia]. This rule has a nice interpretation where let’s say there are 100 voters, then every voter is assigned 1/100th of the budget and gets a guarantee that this part is only spent on charities that the voter has approved. The exact proportions of how the voter share is used is decided based on the overall popularity of the charities. This rule has various nice properties, including Pareto efficiency and strong proportionality properties (guaranteeing things like “if 30% of voters vote for animal charities, then 30% of the budget will be spent on animal charities”).

If you want to stick with the 3 winner constraint, there is no academic research about this exact type of voting situation. But if proportionality is desired, I would select the 3 winners as not the 3 charities with the highest vote score, but instead use Proportional Approval Voting [wikipedia] to make the selection. This would avoid the issue that @Tetraspace identified in another comment, where there is a risk that all 3 top charities are similar and belong to the largest subgroup of voters. Once the selection of 3 charities is done, I would not split the money in proportion to approval scores but either (a) split it equally, or (b) normalize the scores so that a voter who approved 2 of the 3 winners contributes 0.5 points to each of them, instead of 1 point to each. Otherwise those who approved 2 out of 3 get higher voting weight.

I'm happy to discuss further.

I think since there can be multiple winners, letting people vote on the ideal distribution then averaging those distributions would be better than direct voting, since it most directly represents "how voters think the funds should be split on average" or similar, which seems like what you want to capture? And also is still very understandable I hope.

E.g. if I think 75% of the pool should go to LTFF and 20% to GiveWell, and 5% to the EA AWF, 0% to all the rest, I vote 75%/20%/5%/0%/0%/0% etc. Then, you take the average of those distributions across all voters. I guess it gets tricky if you are only paying out to the top three, but maybe you can just scale their percentage splits? IDK.

If not that or if it is annoying to implement, IMO approval voting or quadratic are probably best, but am not really sure. Ranked choice feels like it is so explicitly designed for single winner elections that it is harder to apply here. 

some thoughts on different mechanisms:

Quadratic voting:

I think this could be fun. An advantage here is that voters have to think about the relative value of different charities, rather than just deciding which are better or worse. This could also be an important aspect when we want people to discuss how they plan to vote/how others should vote. If you want to be explicit about this, you could also consider designing the user interface so that users enter these relative differences of charities directly (e.g. "I vote charity A to be 3 times as good as charity B" rather than "I assign 90 vote credits to charity A and 10 vote credits to charity B"). Note however, that due to the top-3 cutoff, putting in the true relative differences between charities might not be the optimal policy.

A technical remark: If you want only to do payouts for the top three candidates, instead of just relying on the final vote, I think it would be better to rescale the voting credits of each voter after kicking out the charity with the least votes and then repeating the process until there are only 3 charities left. This would reduce tactical voting and would respect voters more who pick unusual charities as their top choices. This process has some similarities with ranked-choice voting. Additionally, users should have the ability to enter large relative differences (or very tiny votes like 1 in a billion), so their votes are still meaningful even after many eliminations.

Approval voting:

I think voting either "approve" or "disapprove" does not match how EAs think about charities. I generally approve a lot of charities within EA space, but would not vote "approve" for these charities.

I worry that a lot of tactical voting can take place here, especially if people can see the current votes or the pre-votes. For example, a person who both approves of the 3rd-placed charity and the 4th-placed charity (by overall popularity), might want to switch their vote to "disapprove" for the (according to them) worse charity. For example, voters are incentivized to give different votes to the 3rd-placed and 4th-placed charity, because there the difference will have the biggest impact on money paid out. Or a person who disapproves of all the top charities might switch a vote from "disapprove" to "approve" so that their vote matters at all.

Ranked-choice voting:

I am assuming here that the elimination process in ranked-choice stops once you reach the top 3 and that votes are then distributed proportionally. I think this would be a good implementation choice (mostly because proportional voting itself would be a decent choice by itself, so doing it for the top 3 seems reasonable). Ranking charities could be more satisfying for voters than having to figure out where to draw the line between "approve" and "disapprove", or putting in lots of numeric values.

Generally, ranked-choice voting seems like an ok choice.

how well will these allocate funds?:

I am quite unsure here, and finding a best charity based on expressed preferences of lots of people with lots of opinions will be difficult in any case. My best guess here is that ranked-choice voting > quadratic voting > approval voting. A disadvantage of quadratic voting here is that it can happen that some fraction of the money will be paid out to sub-optimal charities (even if everyone agrees that charity C is worse than A and B, then it will likely still be rational for voters to assign non-zero weight to charity C, corresponding to non-zero payout).

understandability:

I think approval voting is easier to understand than ranked-choice voting, which is easier to understand than quadratic voting. This is both for the user interface and for understanding the whole system. Also, the mental effort for making a voting decision is less under ranked-choice and approval voting. I think the precise effects of the voters choices will be difficult to estimate in any system, so keeping

general remarks:

Different voting mechanisms can be useful for different purposes, and paying 3 charities different amounts of money is a different use case than selecting a single president, so not all considerations and analyses of different voting mechanisms will carry over to our particular case. The top-3 rule will incentivize tactical voting in all these systems (whereas in a purely proportional system there would be no tactical voting). Maybe this number should be increased a bit (especially if we use quadratic voting). If there are lots of charities to choose from, it will be quite an effort to evaluate all these charities. Potentially, you could give each voter a small number of charities to compare with each other, and then aggregate the result somehow (although that would be complicated and would change the character of the election). Or there can be two phases of voting, where the first phase narrows it down to 3-5 charities and then the second phase determines the proportions.

My personal preferences:

Obviously, we should have a meta-vote to select the three top voting methods among user-suggested voting methods and then hold three elections with the respective voting methods, each determining how a fraction of the fund (proportional to the vote that the voting method received in the meta-vote) gets distributed. And as for the voting method for this meta-vote, we should use... ok, this meta-voting stuff was not meant entirely seriously.

In my current personal judgement, I prefer quadratic voting over ranked-choice and ranked-choice over approval voting. I might be biased here towards more complex systems. I think an important factor is also that I might like more data about my preferences as a voter: With quadratic voting, I can express my relative preferences between charities quantitatively. With ranked-choice voting, I can rank charities, but cannot say by how much I prefer one charity over another. With approval voting, I can put charities in only two categories.

One issue that comes up with multi-winner approval voting is: suppose there are 15 longtermists and 10 global poverty people. All the longtermists approve the LTFF, MIRI, and Redwood; all the global poverty people approve the Against Malaria Foundation, GiveWell, and LEEP.

The top three vote winners are picked: they're the LTFF, with 15 votes, MIRI, with 15 votes, and Redwood, with 15 votes. 

It is maybe undesirable that 40% of the people in this toy example think those charities are useless, yet 0% of money is going to charities that aren't those. (Or maybe it's not! If a coin lands heads 60% of the time; then you bet on heads 100% of the time.)

I'm going to stick my neck out and say that approval voting is the best option here. Why?

1. It avoids almost all of the problems with plurality voting. In non-pathological arrangments of voter preferences and candidates, it will produce the 'intuitively' correct option - see here for some fun visualisations.
2. It has EA cred, see Aaron Hamlin's interview on 80k here
3. And most importantly, it's understandable and legible - you don't need people to trust an underlying apportionment algorithm or send the flyers explaining the D'Hondt method to voters or whatever. Just vote for the options you approve of on the ballot. One person, one ballot. Most approvals wins. Simple.

I fear that EAs who are really into this sort-of thing are going to nerd-snipe the whole thing into a discussion/natural experiment about optimal voting systems instead of what would be most practical for this Donation Election. A lot of potential voters and donors may not be interested in using a super fancy optimal but technically involved voting method, and be the kind of small inconvenience that might turn people off the whole enterprise.

Now, before all you Seeing Like a State fans come at me saying how legibility is the devil's work I think I'm just going to disagree with you pre-emptively.^[1] Sometimes there is a tradeoff between fidelity and legibility, and too much weighting on illegible technocracy can engender a lack of trust and have severe negative consequences.

1. ^{^}

   Actually it's interesting that Glen references Scott as on his side, I think there's actually some tension between their positions. But that's probably a topic for another post/discussion

I'd prefer a voting mechanism that factored in as much of the vote as possible. I suspect that cause area will be a major determinant of individuals' votes, and would prefer that the voting structure promote engagement and participation for people with varying cause prioritizations. 

Suppose we have 40% for cause A orgs, 25% for cause B orgs, 20% for cause C orgs, and 15% for various smaller causes. I would not prefer a method likely to select three organizations from cause A -- I don't think that outcome would be actually representative of the polis, and voting rules that would lead to such an outcome will discourage engagement and participation from people who sense that their preferred causes are not the leading one.

I'm not sure how to effectuate that preference in a voting system, although maybe people who have thought about voting systems more deeply than I could figure it out. I do think approval voting would be problematic; some voters might strategically disapprove all candidates except in their preferred cause area, which could turn the election into a cause-area election rather than an organization-specific one. Otherwise, it might be appropriate to assign each organization to a cause area, and provide that (e.g.) no more than half of all funds will go to organizations in the same cause area. If that rule were invoked, it would likely require selecting additional organizations than the initial three.

Proportional voting with some number of votes. between 1 and 10.

If it were me, the thing I'd experiment on is being able to donate votes to someone else. That feels like something I'd like to see more of on a larger scale. I give a vote to Jenifer and Alan, she researches longterm stuff, he looks into animal welfare.

edit: I should have read the post more carefully

Do you intend to have one final winner or would it be ok to pay out the fund to various charities in different proportions (maybe with a minimum payout to avoid logistical hassle)? In the latter case, a consideration could also be proportional voting. But it is not clear how approval voting and ranked choice would work exactly in those cases.

Also, am I understanding correctly that donating more to that fund does not get you additional votes?

I tried to do something similar a while ago looking at under-5 mortality.

Great!

A similar story exists here: https://archiveofourown.org/works/30351690

I really liked this. It was simply, but a smooth read and quite enjoyable. I'd  be happy to see more of this type of content.

I attended and thoroughly enjoyed your workshop! Thanks for posting these notes

I don’t see mention of quantifying the uncertainty in each component and aggregating this (usually via simulation). Is this not fundamental to Fermi? (Is it only a special version of Fermi, the “Monte Carlo” version?)

Moderation update: We have indefinitely banned 8 accounts^[1] that were used by the same user (JamesS) to downvote some posts and comments from Nonlinear and upvote critical content about Nonlinear. Please remember that voting with multiple accounts on the same post or comment is very much against Forum norms.

(Please note that this is separate from the incident described here)

1. ^{^}

   my_bf_is_hot, inverted_maslow, aht_me, emerson_fartz, daddy_of_upvoting, ernst-stueckelberg, gpt-n, jamess

Moderation update:

We have strong reason to believe that Torres (philosophytorres) used a second account to violate their earlier ban. We feel that this means that we cannot trust Torres to follow this forum’s norms, and are banning them for the next 20 years (until 1 October 2042). 

LukeDing (and their associated alt account) has been banned for six months, due to voting & multiple-account-use violations. We believe that they voted on the same comment/post with two accounts more than two hundred times. This includes several instances of using an alt account to vote on their own comments.

This is against our Forum norms on voting and using multiple accounts. We will remove the duplicate votes. 

As a reminder, bans affect the user, not the account(s). 

If anyone has questions or concerns, please feel free to reach out, and if you think we made a mistake here, you can appeal the decision.

Moderation update: A new user, Bernd Clemens Huber, recently posted a first post ("All or Nothing: Ethics on Cosmic Scale, Outer Space Treaty, Directed Panspermia, Forwards-Contamination, Technology Assessment, Planetary Protection, (and Fermi's Paradox)") that was a bit hard to make sense of. We hadn't approved the post over the weekend and hadn't processed it yet, when the Forum team got an angry and aggressive email today from the user in question calling the team "dipshits" (and providing a definition of the word) for waiting throughout the weekend.

If the user disagrees with our characterization of the email, they can email us to give permission for us to share the whole thing.

We have decided that this is not a promising start to the user's interactions on the Forum, and have banned them indefinitely. Please let us know if you have concerns, and as a reminder, here are the Forum's norms.

Moderation update: 

We have strong reason to believe that Charles He used multiple new accounts to violate his earlier 6-month-long ban. We feel that this means that we cannot trust Charles He to follow this forum’s norms, and are banning him from the Forum for the next 10 years (until December 20, 2032).

We have already issued temporary suspensions to several suspected duplicate accounts, including one which violated norms about rudeness and was flagged to us by multiple users. We will be extending the bans for each of these accounts to mirror Charles’s 10-year ban, but are giving the users an opportunity to message us if we have made any of those temporary suspensions in error (and have already reached out to them). While we aren’t >99% certain about any single account, we’re around 99% that at least one of these is Charles He.

You can find more on our rules for pseudonymity and multiple accounts here. If you have any questions or concerns about this, please also feel free to reach out to us at forum-moderation@effectivealtruism.org.

Moderation update: We have banned "Richard TK" for 6 months for using a duplicate account to double-vote on the same posts and comments. We’re also banning another account (Anin, now deactivated), which seems to have been used by that same user or by others to amplify those same votes. Please remember that voting with multiple accounts on the same post or comment is very much against Forum norms.

(Please note that this is separate from the incident described here)

We’re issuing [Edit: identifying information redacted] a two-month ban for using multiple accounts to vote on the same posts and comments, and in one instance for commenting in a thread pretending to be two different users. [Edit: the user had a total of 13 double-votes, most far apart and are likely accidental, two upvotes close together on others' posts (which they claim are accidental as well), but two cases of deliberate self upvote from alternative accounts]

This is against the Forum norms around using multiple accounts. Votes are really important for the Forum: they provide feedback to authors and signal to readers what other users found most valuable, so we need to be particularly strict in discouraging this kind of vote manipulation.

A note on timing: the comment mentioned above is 7 months old but went unnoticed at the time, a report for it came in last week and triggered this investigation.

If [Edit: redacted] thinks that this is not right, he can appeal. As a reminder, bans affect the user, not the account.

[Edit: We have retroactively decided to redact the user's name from this early message, and are currently rethinking our policies on the matter]

It's kind of jarring to read that someone has been banned for "violating a norm" - that word to me implies that they're informal agreements between the community. Why not call them "rules"?

pinkfrog (and their associated account) has been banned for 1 month, because they voted multiple times on the same content (with two accounts), including upvoting pinkfrog's comments with their other account. To be a bit more specific, this happened on one day, and there were 12 cases of double-voting in total (which we’ll remove). This is against our Forum norms on voting and using multiple accounts.

As a reminder, bans affect the user, not the account(s).

If anyone has questions or concerns, please feel free to reach out, and if you think we made a mistake here, you can appeal the decision.

Multiple people on the moderation team have conflicts of interest with pinkfrog, so I wanted to clarify our process for resolving this incident. We uncovered the norm violation after an investigation into suspicious voting patterns, and only revealed the user’s identity to part of the team. The moderators who made decisions about how to proceed aren't aware of pinkfrog's real identity (they only saw anonymized information).

Reply to this comment from @John G. Halstead 

(Written in a personal capacity, I did not check this with other moderators)

Thank you for the feedback! I didn't want to go too off-topic, as this is unrelated to this post, so I'm replying here, but I want to quickly share some factual information for other readers.

The post itself obviously violates forum norms and the moderators are defending the post

You're writing this in multiple comments. I want to make it clear that moderators did not endorse or "defend" (or symmetrically "attack") the post as moderators. But of course, we do comment as users on parts we agree or disagree with (like any other user). Let us know if it's not clear whether we're commenting as users or as moderators.

As for your other warnings, I want to make sure other readers know that your last warning was not for discussing a specific topic, but for being uncivil and not constructive to the discussion. I agree that the situation in the first warning is less relevant to this case, apologies for bringing it up.

Just a quick note to say that we’ve removed a post sharing a Fermi estimate of the chances that the author finds a partner who matches their preferred characteristics and links to a date-me doc.

The Forum is for discussions about improving the world, and a key norm we highlight is “Stay on topic.” This is not the right space for coordinating dating. (Consider exploring LessWrong, ACX threads/classifieds, or EA-adjacent Facebook/Reddit/Discord groups for discussions that are primarily social.)

We’re not taking any other action about the author, although I've asked them to stay on topic in the future.

Moderation update: 

Around a month ago, a post about the authorship of Democratising Risk got published. This post got taken down by its author. Before this happened, the moderation team had been deciding what to do with some aspects of the post (and the resulting discussion) that had violated Forum norms. We were pretty confident that we’d end up banning two users for at least a month, so we banned them temporarily while we sorted some things out. 

One of these users was Throwaway151. We banned them for posting something a bit misleading (the post seemed to overstate its conclusions based on the little evidence it had, and wasn’t updated very quickly based on clear counter-evidence), and being uncivil in the comments. Their ban has passed, now. As a reminder, bans affect the user, not the account, so any other accounts Throwaway151 operated were also affected. The other user was philosophytorres — see the relevant update.

Quick update: we’ve banned Defacto, who we have strong reason to believe is another sockpuppet account for Charles He. We are extending Charles’s ban to be indefinite (he and others can appeal if they want to).

You can find more on our rules for pseudonymity and multiple accounts here. If you have any questions or concerns about this, please also feel free to reach out to us at forum-moderation@effectivealtruism.org.

We’ve banned Vee from the Forum for 1 year. Their content seems to be primarily or significantly AI-generated,^[1] and it’s not clear that they’re using it to share thoughts they endorse and have carefully engaged with. (This had come up before on one of their posts.) Our current policy on AI-generated content makes it clear that we’ll be stricter when moderating AI-generated content. Vee’s content doesn’t meet the standards of the Forum.

If Vee thinks that this is not right, they can appeal. If they come back, we’ll be checking to make sure that their content follows Forum norms. As a reminder, bans affect the user, not the account.

1. ^{^}

   Different detectors for AI content are giving this content different scores, but we think that this is sufficiently likely true to act on.

   It’s hard to be certain that something is AI-generated, and I’m not very satisfied with our processes or policies on this front. At the same time, the increase in the number of bots has made dealing with spam or off-topic/troll contributions harder, and I think that waiting for something closer to certainty will have costs that are too high.

Moderation update:
I'm indefinitely banning JasMaguire for an extremely racist comment that has since been deleted. We'll likely revisit and update our forum norms to explicitly discourage this sort of behavior.

Please feel free to get in touch with forum-moderation@effectivealtruism.org if you have any concerns.

3-4 minutes, mostly on playing through various elimination-order scenarios in my head and trying to ensure that my assigned values would still reflect my preferences in at least more likely scenarios.

took me ~5min

It took me just under 5 minutes.

The percentages I inputted were best guesses based on my qualitative impressions. If I'd been more quantitative about it, then I expect my allocations would have been better—i.e., closer to what I'd endorse on reflection. But I didn't want to spend long on this, and figured that adding imperfect info to the commons would be better than adding no info.

IIRC it took me about a minute or two. But I already had high context and knew how I wanted to vote, so after getting oriented I didn't have to spend time learning more or thinking through tradeoffs.

It took me ~1 minute. I already had a favourite candidate so I put all my points towards that. I was half planning to come back and edit to add backup choices but I've seen the interim results now so I'm not going to do that.

Probably about 30 minutes of unfocused thought on the actual voting. Mainly it was spent negotiating between what I thought was sort of best and some guilt and status based obligation stuff. 

On top of that I perhaps read 2-4 articles and chatted to 1-2 people involved in orgs. I guess that was 1- 3 hours.

I think around 5-10 mins? I tried to compare everything I cared at all about, so I only used multipliers between 0 and 2 (otherwise I would have lost track and ended up with intransitive preferences). The comparison stage took the most time. I edited things in the end a little bit, downgrading some charities to 0.

And it's fixed! 💜