Giving and receiving feedback

By Max_Daniel @ 2020-09-07T07:24 (+94)

[Based on a couple of years work and volunteering experience. Sometimes people tell me they found my feedback especially useful, and some people told me they found this advice useful. But I haven't gone out of my way to find dissenting views.

I’d appreciate it if people chimed in with their own experiences as well as additional or conflicting advice they may have.]

Suppose you want to invest additional time and care to make feedback you receive or give more useful. Here's some advice on how to do this.

These aren’t iron-clad rules. Suboptimal feedback is often better than no feedback, and it can be infeasible (e.g. cost too much time) to maximally stick to all of the following advice.

Also, feedback varies a lot by type of relationship (e.g. manager-report or among friends) and cultural norms. While I think the below is good advice for many situations, it will be decidedly inappropriate for some.

Receiving feedback

Now and in the future, you’ll tend to receive feedback that’s more useful to you if you …

• Explicitly invite feedback, and make it as easy and cognitively non-demanding to give as possible.
• This often includes saying until when feedback would be useful and which communication channel you prefer.
• For example, for these notes, I appreciate feedback any time, but it’d be particularly useful before the end of SRF. Commenting here or email are both fine.
• Give context
• E.g. for a talk or paper, the target audience is often crucial.
• It can also be helpful to say how much time you’ve spent on something, what your currently planned next steps are, and what you hope to ultimately achieve.
• For example, here's what I said when I first shared this advice with people on FHI's Summer Research Fellowship:
• I plan to share them with future Summer Research Fellows at FHI as well as the organizers of similar programmes elsewhere (e.g. this year at CLR or Stanford), who may adapt and share them with their programme participants.
• I’ll also likely share them within the Research Scholars Programme, and I could imagine using them for onboarding new Research Scholars and maybe other FHI staff (if other FHI stakeholders think that’d be good).
• Depending on how useful people tell me they find this, I might post it on the EA Forum.
• Ask specific questions
• E.g. for these notes in addition to any general reactions I’m particularly interested in:
• When you think of particularly helpful feedback you received, can you say what made it so helpful? Does this suggest additions to this list?
• To what extent are these notes consistent with other advice on feedback you’ve seen, and where are contradictions?
• Which bits of these notes (if any) are novel to you?
• After reading them, do you plan to change anything about how you give or receive feedback? If so, what?
• Don’t argue, defend, or justify yourself. Instead, cultivate a reaction of gratitude in response to feedback, and express it to the source of feedback.
• Bad: “You said my explanation of neural nets was hard to follow. But I in fact copied it from a popular online course, and several of my non-technical friends said they liked the explanation!”
• It’s fine to ask clarifying questions. E.g. “Just to make sure I understand you correctly – when saying my argument for X was off, were you just saying that you felt I gave insufficient reasons for X or do you actually think X is wrong?”
• Note I think this is good advice for situations that are purely about collecting evidence on people's perceptions. However, in practice, feedback can also happen in situations that have aspects of collaborative truth-seeking or decision making, and in these situations I think it's often valuable to push back if you think feedback is based on misconceptions. [1]
• Give feedback on feedback
• I.e. say which parts were particularly useful, and how future feedback could be even more useful.
• If feedback caused specific changes or actions, it can be nice to let the source of feedback know what they’ve accomplished.

Giving feedback

Feedback you give will tend to be more useful to the receiver if it is …

• Immediate
• Bad: “I think you’re not the best person to give this presentation because your last talk 4 months ago didn’t meet my expectations.” [If this is the first time the old talk is being mentioned.]
• Better: “Would you like some feedback on the talk you just gave? … ”
• Desired by the receiver
• Bad: <Cornering a speaker who is about to leave:> “So, about your talk: … ”
• Better: “I have some feedback on the talk you gave earlier today. When would be a good time to talk about this?”
• For this reason, usually when I send unsolicited written feedback, I try to make it easy for the receiver to choose the time they look at it. E.g. in an email leaving some white space before the feedback, or putting it in a separate document; and on Slack sending a message saying I have feedback and then posting the actual feedback in a thread to that message.
• Specific
• Not specific: “I liked your talk.”
• Even worse: “You’re bad at giving talks.” [Unless grounded in a review of feedback received for many instances of talks.]
• Better: “I enjoyed your talk, in particular I found your explanation of the backpropagation algorithm really engaging and easy to follow, despite not having a technical background myself.”
• Precise
• Not precise: “That’s a really good literature review.”
• Better: “This is among the 3 best literature reviews I know of, similar to X and Y and much better than e.g. Z.”
• [Note this is different from being specific: the above could also be more specific by saying things like: “I’d guess it’s less accessible to non-specialists than X, but has the advantage of being more comprehensive.” And the more specific example from above could be more precise, e.g. by saying “this was one of the more enjoyable talks during this conference for me, though probably not in the top 5 – though it had the best explanation of the backprop algorithm I’ve seen”.]
• Transparently grounded in observations of specific behaviors and personal reactions
• Bad: “Your talk was boring!”
• Better: “Throughout the talk, except the last two slides, I noticed that I was already familiar with what you were saying. I therefore found it quite hard to focus, and my thoughts were often trailing off.”
• This is similar to the NVC advice to distinguish between (a) observations, (b) feelings, (c) needs, and (d) requests.
• One difference is that for the purpose of professional feedback I do think it’s often useful to include factual claims that go beyond observations of specific behaviors or personal reactions. However, ideally I think factual claims are grounded in observations and personal reactions or transparent reasoning. E.g. the above could go on with “Most audience members are colleagues I know quite well, and I’d guess their experience was similar”, or in a different context one might say “I’ve seen many people submit a paper to this journal, and based on this I expect the chance of your manuscript being accepted in its current form is below 10%”.
• Explicitly connected to the receiver’s goals
• Bad: “The summary really needs to be shorter!”
• Better: “You said that you hope to bring your main conclusion to the attention of policymakers. I think they often have very little time, therefore I expect they’re more likely to read your paper if its summary is at most one page long.”
• Honest
• Bad: Having major criticisms, but saying something evasive like “I thought your talk was OK.”
• Better: “<Consider saying some positive things first – there almost always are some.> I’d like to point out a few major issues. Most importantly, I noticed you overran the time allocated for your talk. I think this is really important to avoid because I know many audience members have a tight schedule, and they might miss meetings or incur a considerable admin overhang by having to reschedule their day. Some ways to avoid this in the future could be to make sure you know the allotted time, to calibrate the duration of the talk via practice runs; and if despite all efforts you do run up against the time limit, it’s important to notice and stop – you could even set an alarm. Another thing on my mind is that in the second section you used several technical terms like ‘regularization’ or ‘softmax’ without having introduced them. I believe many members of the audience won’t have heard of these terms before, and as a consequence might have had trouble following that part of the talk. [...] Overall, for these reasons, this talk fell short of meeting the performance expectations we’ve discussed, and I worry that most audience members feel that listening to that talk wasn’t a good use of their time. I’ll therefore work with you to improve before assigning you another talk.”
• (Of course, the last part is only appropriate if this is a manager giving feedback to a report, or similar.)
• Balanced, i.e. conveying an accurate impression of your overall view rather than focusing too much on positives or negatives.
• Bad: <A large number of comments in a Google doc, each pointing out small issues you think should be fixed, but not mentioning that you overall really like the text.>
• Better: A quick preamble like “I’ll focus on room for improvement in my comments, but overall think this is really great work!”
• Even better: Giving detail on both negative and positive points by being specific, precise, etc.

Proposing solutions – a double-edged sword?

• The above list notably doesn’t include the advice to offer suggestions for how the feedback receiver can improve.
• Often I think this can be really valuable, perhaps even the most valuable part of feedback.
• However, I think its value varies more depending on the situation, and can sometimes be negative.
• It’s still useful to give feedback even if you can’t think of any suggestions! So I wouldn’t want to discourage anyone from sharing their feedback without any. Offering suggestions often requires one to go beyond one’s personal reactions or expertise. It may simply be too hard, or require more effort than one can expend.
• Relatedly, in some domains suggestions tend to be systematically bad. For example, I believe it’s common wisdom in UI design that users are good at identifying which parts of an interface they don’t like but bad at anticipating what they’d like instead.
• More generally, humans aren’t great at affective forecasting, i.e. predicting how they’ll feel about some hypothetical future event.
• Suggestions can prematurely narrow down a conversation to assessing the pros and cons of potential solutions. But sometimes it’d be more fruitful to first spend more time understanding the problem.
• My weak impression is that offering suggestions has a higher risk of coming across as patronizing or belittling, in particular if the suggestions are obvious to the speaker or unsolicited.
• Silly example: probably better to just say “there’s a typo on slide 2” than “I noticed you misspelled ‘the’ as ‘teh’ on slide 2 – consider spelling it ‘the’ next time”.
• Somewhat related to this and the “sometimes solutions will be systematically bad” point: Beware of Other-Optimizing

On the difference between feedback and performance reviews [2]

Ultimately we don't just want to know how well we did in particular instances. It's much more useful if we can evaluate our skills and abilities, and thus predict our future performance at a wide range of tasks, identify which job we might be a good personal fit for, etc.

E.g. wouldn’t it be much more useful to know, say, “I’m bad at giving talks” than just that “Bob didn’t understand what I said in the middle of slide 3 last week”?

And conversely, doesn't this mean that some of the above advice on how to give feedback is misguided?

Yes and no. I suggest to view evaluating generalizable skills and abilities as a two-step process:

1. ‘Feedback’: Record many instances of feedback on specific behaviors as described above.
2. ‘Performance review’: For each skill or ability, periodically review all relevant instances of specific feedback (and any other evidence you might have, e.g. number of views your talk has received on YouTube). Based on this make an aggregate judgment about how strong you are at this trait, by how much you have improved, etc.
1. Crucially, this second step can only be done by yourself (or people you share all your data with, or people who've seen a lot of your work). In the above example, if Bob has seen just this one talk, it'd still be better for him to just say "I didn't follow you in the middle of slide 3" because he simply cannot appropriately make generalizations such as "you're bad at giving talks!".

This can happen at different time scales, and in particular what I’ve called ‘performance review’ need not be a separate conversation with that label. E.g. after having started a new job, you might ask a colleague for feedback after your first week. As that colleague, you may want to convey both feedback on specific events as well as tentative assessments of skills and abilities, all in the same conversation.

But even then I suggest to clearly distinguish between these two types of evaluation, and where possible to ground the assessment of skills and traits in instances of specific feedback. E.g. “I recall that every day this week you suggested a new project idea in the team meeting. I liked all of these ideas, especially the idea of an interactive website you raised on Tuesday. I don’t remember anyone apart from Alice who contributed that many ideas that early. It’s of course too early to say with any confidence, but based on this I’m tentatively quite optimistic about your ability to find creative solutions to problems”.

Acknowledgements

For helpful comments I'd like to thank Kwan Yee Ng, Nora Ammann, Brad Saad, Eliana Lorch, Will Hunt, and Aaron Gertler.

Endnotes

[1] Thanks to Kwan Yee Ng for reminding me of this important point.

[2] Will Hunt tells me that a distinction similar to my 'feedback vs. performance review' is made in the book The Effective Manager.