Infosphere — June 2021 🧵

I recently joined the Centre for Collective Intelligence Design at Nesta, the UK’s Innovation Foundation. My team studies, designs and builds systems that combine human and machine intelligence to tackle social challenges. Collective intelligence is a highly rich set of disciplines and methods covering nano, micro, meta and global macro perspectives and includes practices like citizen engagement, collective decision-making, crowdsourcing, human computation, open-source intelligence, prediction markets, and super forecasting among others.

This is the first blog in my infosphere series that will document various resources that shape my thinking every month and hopefully increase my Luck Surface Area. Some themes of interest this month include:

Superforecasting & risk

In a previous post, I mentioned my interest in understanding how novel tools and practices can help institutions develop more agile and long term thinking, especially in a volatile era of long term emergencies. I’m currently working with Kathy and our COO to set up Nesta’s first internal crowd forecasting pilot. The aim is to forecast the likelihood of identified operational, financial, and strategic risks using the method of ‘superforecasting.

Superforecasting has been popularised by the political scientist Philip Tetlock and successfully used in industry, government, academia and third sector. The theory behind this method is that certain people can make more accurate forecasts than specialists in an area. Much of their success is due to their independence from events and the structured nature of their thought processes. We hope that this approach can help surface risks and emerging issues from a broader range of Nesta staff and can be the basis of a more dynamic and adaptive internal risk management system that allows us to see risk outside traditional audit and risk reporting.

I also took a course on how to be a superforecaster by Good Judgement. The course offers advice on assessing one’s knowledge, weighting new information, distinguishing signal from noise, and calibrating confidence in beliefs. These are not new practices, especially if you are trained in Statistics & Probabilities, and you’ve been following the rationalist community. But I find the design considerations of developing and deploying superforecasting pilots particularly interesting e.g. how do you incentivize participants, how do you create /phrase forecastable questions, how do you time the questions, what mechanisms can keep people engaged overtime etc.

On this topic, I also enjoyed reading the Future Proof report by Toby Ord at Oxford University’s Future of Humanity Institute and the Centre of Long term resilience. The report sets out a roadmap for the UK Government to transform its resilience to extreme risks. It focuses on biosecurity and AI risks and calls for better risk management processes and more RnD funding on extreme risks.

Open infrastructure

I attended two fantastic events: a talk by Kaitlin Thaney from Invest in Open Infrastructure, which was curated by the Tools Practices and Systems programme at the Alan Turing Institute (I’m very lucky to be part of their extended network 🙏) and a seminar on the Maintenance & Labor of Open Infrastructure. Invest in Open is a new initiative with two founding premises: that open, community-owned infrastructure is required for research to thrive, and the way we fund and resource open projects is insufficient and doesn’t contribute to a healthy, collaborative environment. Hence they are committed to improving funding and resourcing for open technologies and systems for research and scholarship by highlighting challenges, undertaking research, and collaborating with decision-makers to enact change.

These ideas are relevant to a new project I’m working on with a large French financial institution on collective intelligence commons. The project will imagine a new technical and social infrastructure that allows development actors, communities, and governments to pool and use information, knowledge and ideas to accelerate progress on the Sustainable Development Goals (more in the coming months).

Through these conversations, I also learned about:

Collective intelligence

I’m working on parts of a bigger project in my team which is about Collective Crisis Intelligence with the Red Cross and Red Crescent Societies. Collective Crisis Intelligence can be defined as the capabilities and systems which bring together AI with human decision making and allow for early warning, early action, enhanced response and better recovery in disaster risk and response efforts.

In particular, Aleks and I are working on Participatory Machine Learning, which in its broadest sense refers to the involvement of a wider range of stakeholders than just technology developers in creating an AI system, model, tool or application. We are documenting and synthesising resources on the co-design and development of models by communities; how communities can give validation and feedback to analytical tools; new participatory analysis and interpretation methods; and how users and focus communities can benefit directly from advances in AI. We will publish many more resources and analyses on this in the coming weeks. Related to this topic, I liked Google’s recent major update on their resource, the People + AI Guidebook, which documents a set of methods, best practices, and examples on designing with AI.

I also attended Kathy’s presentation at the launch of my team’s work with the UNDP Accelerator Labs, which was 🔥. My team has published comprehensive research of 277 case studies presenting how diverse collective intelligence approaches are being used to speed up progress on all 17 Sustainable Development Goals.

Other threads:

AI & Science

A few highlights I’m keeping an eye on:

Books I’m reading

I’m currently reading Julia Galef’s The Scout Mindset, The first 90 days which was suggested by my friend Hushpreet (and I find quite helpful unlike other business books), and I also just started Emergent Strategy.

on this quiet Sunday, I’ll leave you with some David Lynch vibes…