Metascience/Progress: Michael Nielsen interview/podcast

By Ben Yeoh @ 2022-11-16T23:37 (+10)

Link to podcast/transcipt here.

Summary: for the progress studies/ metascience wing of EA interview with meta scientist thinker Michael Nielsen

For the EAs interested in the progress studies, or science of science / meta-science area. I held a conversation with Michael Nielsen on his latest work on metascience.

If you put some wieght on metascience being a possibly underrated field that systemically might enable a lot of future long term value then this conversation could be useful for you.

Also of interest in thinking about funding.

Less interesting to the other wings of EA (eg animals, x-risk; more seconf order for health/science/blobal development funding)

Summary of podcast and overview of  questions

Michael Nielsen is a scientist at the Astera Institute. He helped pioneer quantum computing and the modern open science movement. He is a leading thinker on the topic of meta science and how to improve science, in particular, the social processes of science. His latest co-authored work is  ‘A Vision of metascience: An engine of improvement for the social processes of Science’ co-authored with Kanjun Qiu (open source book link). His website notebook is here, with further links to his books including on quantum, memory systems, deep learning, open science and the future of matter. 

I ask: What is the most important question in science or meta science we should be seeking to understand at the moment ?

We discuss his vision for what a metascience ecosystem could be; what progress could be and ideas for improving the the culture of science and social processes.

We imagine what an alien might think about our social processes and discuss failure audits, high variance funding and whether organisations really fund ‘high risk’ projects if not that many fail, and how we might measure this.

We discuss how these ideas might not work and be wrong; the difficulty of (the lack of) language for new forming fields; how an interdisciplinary institute might work. 

The possible importance of serendipity and agglomeration effects; what to do about attracting outsiders, and funding unusual ideas. 

We touch on the stories of Einstein, Katalin Kariko (mRNA) and Doug Prasher (molecular biologist turned van driver) and what they might tell us.

We discuss how metascience can be treated as a research field and also as an entrepreneurial discipline.

“...."How good a use of the money actually is? Would it be better to repurpose that money into more conventional types of thing or not?" It's difficult to know exactly how to do that kind of evaluation, but hopefully, meta-scientists in the future will in fact think very hard and very carefully about how to do those kinds of evaluation. So that's the meta-scientist research discipline.

As an entrepreneurial discipline, somebody actually needs to go and build these things. For working scientists it's often remarkably difficult to do that because it doesn't look like a conventional activity. This isn't sort of science as normally construed. Something that I found really shocking-- you may be familiar with and hopefully many listeners maybe familiar with, the replication crisis in social psychology. So this was, I guess most famously in 2015, there was a paper published in which 100 well-known experiments in social psychology were replicated. I think it was 36% of the significant findings were found to replicate and typically the effect size was about roughly halved.

So this was not a great look for social psychology as a discipline and raised a lot of questions about what was going on. That story I just told is quite well-known. What is much less well-known is that in fact going back many decades, people had been making essentially the same set of sort of methodological criticisms. Talking about the file drawer effect, talking about p-hacking, talking about all these kinds of things which can lead to exactly this kind of failure. And there are some very good papers written in-- I think the earliest I know is from the early sixties. Certainly in the 1970s and 1980s you see these kinds of papers. They point out the problems, they point out the solutions. “Why did nothing happen?” "Well, because there's no entrepreneurial discipline which actually allows you to build out the institutions which need to be built out if anything is actually to change."

We discuss how decentralisation may help. How new institutions may help. The challenges funders face in wanting to wait until ideas become clearer.

We discuss the opportunity that developing nations such as Indonesia might have.

We chat about rationality and critical rationality.

Michael gives some insights into how AI art might be used and how we might never master certain languages, like the languages of early computing.

We end on some thoughts Michael might give his younger self:

The one thing I wish I'd understood much earlier is the extent to which there's kind of an asymmetry in what you see, which is you're always tempted not to make a jump because you see very clearly what you're giving up and you don't see very clearly what it is you're going to gain. So almost all of the interesting opportunities on the other side of that are opaque to you now. You have a very limited kind of a vision into them. You can get around it a little bit by chatting with people who maybe are doing something similar, but it's so much more limited. And yet I know when reasoning about it, I want to treat them like my views of the two are somehow parallel but they're just not.

Link to podcast/transcipt here.