# How to score accelerator applications so a second panel would pick the same startups

By Alperen Zeki Gokmen. Published 24 October 2026.
Canonical: https://brutallyhonestecosystem.com/library/score-accelerator-applications

## How should an accelerator score startup applications?

Score evidence before people. Have every co-founder answer the same structured questions on their own, ask each judge to score each criterion alone against written anchors, and only then discuss, starting with the startups where judges disagree most. Then score the cohort again at the end, so the same instrument shows your funder what changed.

Below is the scorecard, with the anchors written out, and the six steps for running a panel with it. The scorecard leans on what a programme can collect before a panel meets: each co-founder's own written answers.

## Why two panels pick two different cohorts

We think selection is half of what a programme can achieve. Select well and you are halfway to results. Select badly and, in our experience, no curriculum or mentor bench produces them, either for your portfolio or for the ecosystem you report to.

The usual method looks sensible. You invite mentors and investors who have done this for years, give them a spreadsheet, ask them to rank the applicants, and take the top ten. They all read the same sheet. But each of them reads it through their own lens: the sector they know, or the founder who reminds them of someone. In our experience that is where the bias comes in, along with shortcuts and misreadings, and it cuts both ways. A false positive takes a seat from a team that would have used it. A false negative sends home the founder you should have backed.

Research on accelerators shows how much weight the panel carries. Berna Beyhan, Semih Akçomak and Dilek Cetindamar studied ten accelerators in Ankara and Istanbul, with interviews done between March 2016 and January 2017, and published the work in the Entrepreneurship Research Journal in 2021. The authors describe a staged process in which, at least in the second stage, a jury or committee decided; their own table shows one of the ten choosing in a single stage. One of them used a jury of 40 people, including professionals from corporations, investors, entrepreneurs and academics. The authors found that the programmes tended to select teams that were coachable, willing to collaborate with the accelerator, mentors and others, and passionate enough to cope with the pressure of building fast.

Those are the qualities a ranking spreadsheet is worst at seeing. A judge can feel that a founder is coachable in a ten-minute pitch. But a programme only knows it by watching what the founder does with advice.

Daniel Kahneman, Dan Lovallo and Olivier Sibony named the wider problem in MIT Sloan Management Review in 2019. One of their opening examples of an evaluative judgement was this: "A venture capital investment committee chooses among an array of startups to fund." They separated errors that come from known biases from random errors, sometimes called noise, and drew on research into hiring and job interviews for a structured approach they called the Mediating Assessments Protocol. Choosing a cohort is the same kind of judgement, so it should respond to the same kind of structure.

## The scorecard

Five criteria. The first four are the four dimensions of the assessment we built for programmes, so the evidence for them can come from founders' own written answers. The fifth is the one panels guess at.

| Criterion | Evidence you score | 1 | 3 | 5 |
|---|---|---|---|---|
| Current state | What exists today: product, users, customers, revenue | An idea and a deck | A working product with first users or pilots | Paying customers who come back |
| Road to revenue | Who pays, how much, how often, and how the team knows | No price tested | A price tested with real buyers, no repeat yet | Repeat revenue and a known cost to win a customer |
| Scalability and fund raising | Whether growth needs costs to grow at the same rate; whether the funding plan fits the stage | Every new customer needs the founders' time | A plan to grow without matching cost, not yet tested | Growth already outpacing cost, with a funding plan that fits |
| Competitive landscape | What customers use today instead, and why they would switch | "No competitors" | Named alternatives, a stated reason to switch | Customers who switched, and can say why |
| Team alignment and coachability | Whether co-founders give the same answers on direction and runway; what they did with feedback between stages | Co-founders contradict each other; feedback ignored or copied wholesale | Aligned on direction, apart on details; feedback used in part | Aligned on direction; feedback weighed and used with judgement |

Weights are the programme's call, and they'll follow the programme's job. A pre-accelerator that exists to create first revenue reads the second row differently from an investment-led programme that has to show follow-on funding. Write the weights down before applications open, while nobody has met the founders yet.

## How to run the panel with it

1. Write one sentence on what the programme is selecting for, before the call opens. Every judge gets it with the scorecard.
2. Collect the evidence before anyone pitches. Each co-founder answers the same questions separately; in the assessment we built that takes about 90 minutes per founder, and they can save and come back. Differences between co-founders are part of the evidence, so keep them visible.
3. Judges score alone, criterion by criterion, against the anchors, before any discussion. No one sees anyone else's scores until all are in.
4. For each startup, look at the spread: the highest judge score minus the lowest. Where judges agree, the discussion can be short. Spend the panel's time on the startups with the widest spread, and ask each judge which evidence moved their score.
5. Between the first and second stage, give each shortlisted team one small piece of homework built from their weakest row. Score what comes back. That turns coachability from a feeling into evidence.
6. Keep every score. At the end of the programme, run the same instrument again.

Step 6 is what you show your funder. The same scorecard at intake and at the end shows what changed in each team, and over two or three cycles it shows whether your selection picked the teams that grew.

## Before you change anything

Test the scoring you run today. If a different panel judged the same applicants next month, would they pick the same cohort? Our [cohort scoring check](https://brutallyhonestecosystem.com/cohort-check) answers that in six questions and two minutes, and it is free. For EU-funded and publicly funded programmes, the [selection accountability check](https://brutallyhonestecosystem.com/selection-check) looks at your process the way a sponsor would.

> Part of: Running a programme funders renew. Next in this cluster: twelve intake questions that find the expectation gap before good founders drop out.

## Where we fit

The programme edition of the [startup assessment](https://brutallyhonestecosystem.com/assessment) we built runs steps 2 and 6 for every startup: each co-founder answers on their own, and your team gets readiness scores on four dimensions, team alignment and flagged answers, with up to five evaluations compared side by side. It starts from 3,000 EUR for the first year for up to 25 startup teams, VAT included. Our mentors and [cohort workshops](https://brutallyhonestecosystem.com/mentorshipandtraining) can then work on the weakest rows inside the programme you already run.

## Questions programme managers ask

### What criteria do accelerators use to select startups?

A study of ten accelerators in Ankara and Istanbul (interviews in 2016 and 2017, published in 2021) found that they selected on two things: the opportunity, and how well the team suits it. The programmes also tended to pick teams that were coachable, willing to collaborate and passionate enough to cope with a fast pace.

### How do you reduce bias in startup selection?

Separate evidence from judgement. Collect structured answers from each founder before anyone sees a pitch, ask each judge to score each criterion alone against written anchors, and discuss last, starting with the startups where the judges' scores are furthest apart.

### Should the same scorecard be used at the end of the programme?

Yes. Scoring the cohort again with the same instrument shows a funder what changed in each team, and over two or three cycles it tells you whether your selection picked the teams that grew.

## Sources

- Berna Beyhan, Semih Akçomak and Dilek Cetindamar, [The Startup Selection Process in Accelerators: Qualitative Evidence from Turkey](https://opus.lib.uts.edu.au/bitstream/10453/150686/2/Manuscript_selection_ERJ2021-0122.pdf), Entrepreneurship Research Journal, published online 1 October 2021 (author manuscript; interviews March 2016 to January 2017).
- Daniel Kahneman, Dan Lovallo and Olivier Sibony, [A Structured Approach to Strategic Decisions](https://sloanreview.mit.edu/article/a-structured-approach-to-strategic-decisions/), MIT Sloan Management Review, 4 March 2019.
- The Brutally Honest Ecosystem, [prices](https://brutallyhonestecosystem.com/pricing.md), last checked 14 October 2026.