Retroactive Public Goods Funding (RetroPGF) is one of the leading economic experiments to help decentralize and grow crypto ecosystems. While still in its very early days, the Optimism Collective is excited to lead the charge on RetroPGF experimentation, and we aim to inspire more builders to contribute to the Optimism ecosystem with the hope – and expectation – of getting funded retroactively for their contributions.

Why does RetroPGF exist? 

Without public goods funding, core tools and infrastructure required to keep blockchains running could eventually run out of cash to operate.

Without public goods funding, projects that can’t accept payment from users in order to stay neutral and nonpartisan could run out of cash and cease to operate.

Without public goods funding, the future for a decentralized web is murky at best.

For that reason, we’re truly stoked to share the results and learnings from RetroPGF Round 2.  RetroPGF Round 2 is the Collective’s second experiment in allocating retroactive public goods funding. In this round, 69 out of the 71 selected badgeholders – our early “citizens” – voted to allocate 10M OP to projects who have supported the development and usage of the OP Stack.

RetroPGF Round 2 stretched over multiple months and involved participation from hundreds of projects and community members across the Optimism Collective. 

The process:

1. Badgeholder selection - badgeholders have the power to distribute OP tokens to projects. They’re instrumental in running an effective RetroPGF round. For RetroPGF Round 2, badgeholders were selected across four different criteria.

   1. 14 badgeholders were selected based on their participation as badgeholders in round one of RetroPGF

   2. 21 badgeholders were selected by the Optimism Foundation

   3. 10 badgeholders were elected by Optimism’s Token House

   4. 29 badge holders were nominated by badgeholders from the three categories above

2. Nominations (Jan 17 - Jan 31st) - anyone could nominate a project in the forum by providing a name, impact description and link to Github/Twitter

3. Project profile creation (Feb 7th - Feb 21st) - Projects had to create a profile where they were asked for general information, as well as a description of their project and its impact. Information provided by Projects can be viewed on the RetroPGF discovery page.

Voting (Mar 7th - Mar 21st) - Badgeholders were provided with a badgeholder manual and asked to evaluate and vote on nominated projects via a voting form. (Mar 7th - Mar 21st)

Analyzing the Results

Results for round 2 are in! Similar to the results of round 1, the variance among projects funded in RetroPGF 2 was relatively low, with little difference in payouts between extremely high impact projects and those with a more moderate impact. Different to Round 1 there was a large spread in token allocation: in Round 1, 58 out of the 76 nominated projects received votes, while in Round 2 all nominated projects (195) received votes. This was likely a result of the small number of badgeholders compared to the tokens allocated, as even a small allocation by a single badgeholder could result in a significant token allocation to a project.

The diversity of projects funded during this round improved compared to Round 1. The majority of the funded projects in RetroPGF 2 were not specific to Optimism but were part of the broader Ethereum ecosystem. Funded projects spanned a wide range of areas, from infrastructure to education, and operated in different languages and regions.

Community reaction to the funding round served as an interesting signal on the health of the RetroPGF flywheel. Many projects expressed excitement, gratitude, and surprise about the OP tokens allocated to them for their past impact. While spreading strong positive vibes, this indicates that the RetroPGF flywheel – where projects and investors make upfront investments in public goods, anticipating that their impact will be retroactively funded by the collective – is still in its early stages. The overall surprise at the rewards demonstrated that this flywheel has not taken shape yet.

How Badgeholders Assessed Impact

Retroactive Public Goods Funding aims to reward past impact based on the idea that it is easier to agree on what was useful than what will be useful. One of the most important experiment parameters in RetroPGF is determining how and what information about projects is presented to voters.

In order for a project to be eligible for RetroPGF, they had to be nominated in the forum and subsequently sign-up via the Project intake form. In addition to functioning as a lightweight filter for qualifying projects, the two steps aimed to collect useful information from projects that badgeholders could use to evaluate the projects’ impact.

The Project Nominations were designed as an open process in which the community could signal what projects generated impact for the Optimism Collective. In total 262 unique nominations were submitted with a significant number of projects nominating themselves.

The nominations played out as a mini-dilemma of the commons, in which few felt responsible or incentivized to nominate relevant projects. Even some of the top recipients of the round had to self-nominate, such as Protocol Guild, Lodestar, Goerli Testnet, OpenZeppelin and Snapshot.

As a result, the information provided during the nomination process ultimately wasn’t used as a valuable community signal on impact and was not actively considered by badgeholders in the project evaluation process. On the other hand, it seems that the fact of being nominated was a positive signal, as every one of the 195 projects nominated received a nonzero funding amount.

Once a project was nominated, the project was asked to sign-up using an intake form. The Project intake form asked for information that was intended to help badgeholders evaluate projects’ impact and potential sources of funding outside retroPGF.

In addition to questions about team size and history, the intake form asked projects the following questions:

• “How do you support development and usage of the OP Stack? What public good do you provide to the Collective?”

• “How do you sustain yourself? Please list sources of funding and revenue.”

This was a lightweight attempt at giving projects an opportunity to contextualize their impact (the first question) and their profit (the second question) to set up badgeholders to evaluate along the general heuristic that impact should equal profit, consistent with the Optimistic Vision.

However, the information provided by projects was often too vague, making it difficult for badgeholders to accurately assess impact.

*     “Many projects did not provide enough information on the elements they were evaluated on: what is their impact to optimism, and what is their funding situation like,” Anonymous*

*     “The application form for next season should be more aligned with review assessment process,” Krzysztof Urbanski*

*     “…ask projects to more clearly submit information that will be relevant to the evaluation criteria (impact and access to funding being the main one,” Cassidy*

Project descriptions and impact descriptions were usually narrative driven, while funding sources were listed without actual numbers. This was likely the result of a minimal prompt provided to projects in the profile intake form, and few examples to model off. Without proper guidance, projects tended to default to their standard messaging, which is often concerned with aspirations about future impact rather than descriptions of the impact they’ve had to date.

Overall the nominations process and project profiles failed to provide high quality context or information to help badgeholders evaluate past impact.

Going forward: How can the Collective surface more high quality data that serves as a “proof of impact” for the evaluation and voting by badgeholders?  What structured data could help badgeholders make less impressionistic evaluations?

Scaling Impact Evaluation

Badgeholders faced challenges evaluating projects’ impact — not only because of the lack of high quality data, but also because of the scale of projects to review.

Badgeholders were asked to broadly evaluate as many projects as they felt comfortable with instead of solely focusing on their area of expertise.

The most consistent feedback from badgeholders during the evaluation process was around the overwhelming quantity of projects to review.

*     “Smooth experience, but way too many projects.”*

*     “...this is really about it being unmanageable for badgeholders.”*

In this round, 195 projects were eligible for voting. In comparison, Round 1 of RetroPGF had 76 eligible projects.

Although some badgeholders went the extra mile, reviewing a majority of the projects, most tended to distribute their votes among 20-40 projects, with the median badgeholder allocating their votes among 30 projects.

If RetroPGF is to scale to support hundreds or thousands of projects and people across the Collective, evaluation using the current model will not scale.

This problem is amplified when evaluating the impact of small individual contributions. To tackle this issue, the Optimism Foundation experimented with nominating “Collections.” Each collection was a list of contributors along with a weight for funding distribution across that list.

5 Collections were nominated, including Monorepo Dependencies, EIP-4844 Contributors and Optimism Support NERDS, Ambassadors and Translators. This experiment did help badgeholders allocate funding to these broader groups of contributors, something that might have been more challenging were the collection participants up for funding as individuals.

While most collections were uncontroversial, the EIP4844 collection received significant pushback by Ethereum contributors. Some criticism revolved around creating unnecessarily strong incentives for work that could be retro funded, and the risk that this incentive could push teams to alter their prioritization. Other concerns centered on the challenges of assessing individual contributions to generate the “weights” in each collection, including possible biases towards rewarding work with high visibility.

*     “It also may have impact among core dev teams themselves: should I be careful in the future deciding in which subprojects do I work within Prysm to expect a higher payout if I work on projects that are of interest to Optimism?” potuz*

*     “I think doing RPGF in this granular of a form incentivizes doing more visible and “popular” work in a degenerate way.”, djrtwo*

*     “Effectively surfacing the people who’ve done the work requires intimate domain knowledge.”*

*     “Even though OP Labs was deeply involved with 4844, the final list still missed some people who should have been included.” trent*

Feedback showed that the collection was far from perfect in capturing contributions on a granular level. Some of the sentiment expressed that individual contributions to Ethereum core development should not be rewarded separately, but rewards should rely on a self-curated approach like Protocol Guild, which currently doesn’t take the quality of individual contributions into account.

While Protocol Guild is an interesting solution that sidesteps the problem of evaluating individual contributions, and has been recognized as the top recipient of round 2, it doesn’t allow us to realize the vision of retroactive public goods funding in which impact and profit are aligned on the contributor level and there is an outsized incentive to do high impact work.

This pushback amplified that tracking, indexing, and evaluating contributions is hard and we’re at a very early stage of uncovering solutions to this.

RetroPGF 2 also experimented with using three categories to organize projects. The goal of using categories was to help define and expand the scope of Round 2 and give community members some signals around what type of work might be eligible. At a high level, this approach was moderately successful: Round 3 funded more Education projects than Round 1, in part perhaps because Education was identified as a standalone category worthy of funding.

Categories also had some downside: during project intake, the applicant had to specify which category they fell into, which was challenging for some multidisciplinary builders. And it’s hard to know how many people in the Collective were turned off from applying for RetroPGF because they didn’t see their work represented in one of the three buckets.

*     “BuidlGuidl is all of these things, which one should I pick for @optimismFND RetroPGF?!?” tweet by Austin Griffith*

Going forward, categories may also be used as a form of high-leverage voting, where a voter who doesn’t have expertise in a particular area could allocate funding to the overall category, which is then distributed pro-rata with other badgeholders’ votes. Overall, this is a dimension worthy of further exploration.

How can impact evaluation scale to represent not just the individual experience and impression of badgeholders, but the evaluation of all observed impact within the collective?

Badgeholder Collaboration

In Round 2, 71 badgeholders distributed 10M OP – a substantial amount of responsibility for each participant. To make this evaluation process easier, the Optimism Foundation tried to facilitate a high context environment with extensive guidelines in the badgeholder manual and an onboarding call recapping the most important concepts.

Guidelines were often loose, giving badgeholders frameworks on how to evaluate, but only few explicit criteria or rules. This put the burden on badgeholders themselves to leverage their own judgment in applying these philosophical concepts to the real world.

To make sense of this collaboration between badgeholders was strongly encouraged, a-sync via Discord & Telegram, as well as via collaborative calls hosted by Other Internet.

“These collaborative calls seemed to have the intended effect of understanding how fellow badgeholders were thinking about voting, and circulating that information. However, they also provided a much needed opportunity for real-time reflection on the process, and a collaborative “working out” of what tools were needed to succeed in future iterations.” Toby from OtherInternet

The first call aimed to provide co-working time and leverage badgeholders’ collective intelligence. Attendees spent some time looking through projects with their breakout groups and discussed questions they ran into, and insights about their decision-making processes. The second call focused on feedback, reflection, and aggregating ideas for the next round. Many badgeholders reflected positively on these sessions, and they should likely remain a part of future RetroPGF rounds.

*     “It was really helpful to have sessions like this one with fellow badgeholders to talk about the process, different methods people were using, and the different ways others were approaching voting”*

*     “[I appreciate the] diversity of badgeholders and the work done by badgeholders to collaborate.”*

Defining Impact = Profit

One of the Optimism Collective's values is 'impact=profit', the idea that individuals should receive profit equal to the impact they provide to the collective. But applying this framework can be challenging, especially without a quantitative framework for evaluating either "impact" or "profit.".

*     “Are we taking impact vs profit literally?”*

*     “On the philosophy of impact = profit, we wish there was a clearer definition of what impact looks like.”*

This is one of the most complex pieces of RetroPGF: how does a badgeholder evaluate what types of public goods actually deserve funding? Badgeholders across the board expressed the need for more clarity on impact evaluation, and philosophical alignment on the type of project RetroPGF should support.

*     “Introduce better categories and stronger evaluation heuristics for each category”*

*     “more discussion and structure of what the assessment criteria are - what are the goals and criteria that everyone can agree on”*

*     “different people were assessing different ways and had their own criteria for assessing. People were not in sync on the criteria, and how we each determined that based on the applications was unclear.”*

Some badgeholders leaned into the ambiguity and created their own evaluation framework.

A common theme in badgeholders’ evaluation frameworks was the use of categorizations of criteria and their combination to simplify impact assessment. This spanned from binary criteria like “Is the Project Optimism specific [Y/N]” to simplified impact sizing “Contribution Type [Large, Medium, Small]”.

With the popularity of pro-active grant models the consideration for future impact during impact evaluation is hard to unlearn. Badgeholders had multiple discussions on the topic of expected future impact and if it should be considered.

Tim Beiko ran a Twitter poll on “How much should retroactive public goods funding (RPGF) weight “future value delivered” when allocating funds?” with a majority of the votes in favor of considering “future value delivered” in the weighting. While Twitter polls are far from representative, and this poll did not reflect badgeholders beliefs, it shows that there’s still a long way to go on making the concept of retroactive funding well understood.

How do we provide better mental models and definitions for impact evaluation? How can we support badgeholders to more effectively collaborate? How do we communicate the core mechanics of retro funding in a way that preserves badgeholder agency?

Voting Tooling

The voting experience in round 2 was far from optimal. In the interest of learning and iterating quickly, the Optimism Foundation de-scoped the implementation of an integrated voting interface and instead implemented an MVP solution.

To submit votes, Badgeholders used a DeForm form with wallet verification. To support the vote allocation, badgeholders were provided with a voting scratchpad, to first allocate their votes, making sure they add up 100%, and then copy them over into the form. The scratchpad also emerged as a convenient way for badgeholders to share their vote allocations with other community members.

Voting UX was not ideal, but badgeholders were quite resourceful in identifying creative solutions to the problem. Ludens from Lattice created a script that exports votes from the voting scratchpad and allows you to import them into the form. And during the last days of voting, Vitalik provided a script one could paste into the browser console to quickly tally votes allocated across the ballot.

Future rounds need voting tooling that doesn’t create unnecessary operational issues for badgeholders. But beyond the friction, this emergent behavior provides early validation for a key design hypothesis: that retroPGF should move towards a permissionless protocol that allows community members to build voting, evaluation, and aggregation experiences on top. This deserves exploration in future rounds.

How can the Collective provide a better voting experience to Badgeholders? How can the  Optimism community create tooling that improves the RetroPGF system for all types of participants?

Conclusion

RetroPGF Round 2 supported nearly 200 creators of public goods across the Optimism Collective. It also brought us valuable insights into the design challenges of the retroactive results oracle, bringing us one step closer to summoning Ether’s Phoenix.

In Round 3 - announcement coming soon™ - the Collective will iterate on the core challenges outlined in this piece:

• How can the Collective gather more high quality data that serves as a “proof of impact” for the evaluation and voting by badgeholders? What structured data could help badgeholders make less impressionistic evaluations?

• How can impact evaluation scale to represent not just the individual experience and impression of badgeholders, but the evaluation of all observed impact within the collective?

• How do we provide better mental models and definitions for impact evaluation? How can we support badgeholders to more effectively collaborate? How do we communicate the core mechanics of retro funding in a way that preserves badgeholder agency?

• How can the Collective provide a better voting experience to Badgeholders? How can the Optimism community create tooling that improves the RetroPGF system for all types of participants?

We will tackle these challenges as a Collective, inviting contributors to build different pieces of the puzzle. Stay tuned for requests for proposals on how to contribute to the RetroPGF protocol.

RetroPGF is a pillar of the Collective, a never-ending cycle, an infinite game. If done right, RetroPGF will scale beyond Optimism to demonstrate a new type of economy for the world that rewards impact. Between then and now, there’s plenty to experiment with.

The scope and shape of RetroPGF Round 3 will be announced in the coming weeks. Sign up for the RetroPGF Newsletter to be the first to know!