Building a scoring rubric you will actually use

Lesson 10 of 50Module 2 · Judging link quality7 min read

By Bill Hartzer · Hartzer Consulting · 25+ years in search, domains and expert witness work

Why a rubric, and why most of them are abandoned

The point of a rubric is not accuracy. No scoring system predicts what a link will do for your rankings, and anyone selling one that claims to is selling a model with no validation behind it. The point is consistency and auditability. If you score every prospect the same way and write down the score, then in six months you can look at which links you built, what they scored, and what happened — and adjust. Without a record, you have opinions about your own past decisions, and opinions about your own past decisions are unreliable.

Most rubrics get abandoned for one of three reasons, and all three are avoidable. They have too many criteria, so scoring takes forty minutes per prospect and nobody does it. They use scales nobody can apply consistently, so two people score the same site three points apart. Or they produce a number that nobody acts on, because no threshold was ever set.

The rubric below has four dimensions, a 0 to 5 scale with defined anchors, and explicit decision thresholds. It takes about ten minutes per prospect once you are practiced. Ten minutes is affordable. Forty is not, and a rubric you do not run is worth nothing.

The four dimensions and a worked set of weights

Here is a rubric that works as a starting point for most sites. The weights are deliberate and I will defend each one, but they are yours to change — that is the next section.

Relevance takes 40% because it is the dimension that determines whether the link does anything for the page you are trying to rank, and because it is the one every competing framework underweights. Authority takes 25% — significant, but deliberately less than relevance, because it is an estimate of one property and the most manipulable input on the list. Page liveness takes 20% because it is close to binary in effect: an unindexed orphaned page is worth approximately nothing regardless of the other three. Editorial signals take 15% because they are the best available proxy for whether the link will still be there in three years.

Scoring each dimension 0 to 5

Vague scales are what kill rubrics. Define the anchors, write them down, and score against the definition rather than the feeling.

Relevance

• 5 — the site is about your exact subject and the specific page covers the thing your page covers.
• 3 — the site is in an adjacent field with a plausible shared audience, and the page is at least loosely on topic.
• 1 — the site covers your subject as one of many unrelated categories.
• 0 — no meaningful topical connection, or the site has no identifiable topic at all.

Authority

• 5 — strong metrics from two providers that agree, a healthy TF/CF relationship, topical trust in your category.
• 3 — moderate metrics, providers roughly agreeing, nothing alarming in the ratio.
• 1 — weak metrics, or providers disagreeing sharply, or Citation Flow far above Trust Flow.
• 0 — metrics that look manufactured: high volume, no trust, no topical category that fits the content.

Page liveness

• 5 — indexed, ranking for something, internally linked from several places, evidence of readers.
• 3 — indexed and internally linked, little or no measurable traffic.
• 1 — indexed but structurally isolated, or in the sitemap but nowhere else.
• 0 — not indexed, orphaned, noindexed, or canonicalized to another URL.

Editorial signals

• 5 — named authors who exist elsewhere, a substantive About page, a real organization, irregular human publication cadence, sensible outbound links.
• 3 — a small genuine site: one real author, thin but honest About page, irregular posting.
• 1 — no named authors or unverifiable ones, template About page, mechanical cadence.
• 0 — footprints of a network, or a visible "write for us / sponsored post" pricing page as the site's main purpose.

Multiply each score by its weight, sum, and express out of 5. A prospect scoring 4.2 is a strong yes; 3.0 to 4.2 is worth pursuing if the effort is proportionate; below 2.5 is a no.

Veto rules that override the total

Weighted averages have a known weakness: a catastrophic failure on one dimension gets diluted by strong scores on the others. A DR 80 site with excellent editorial signals and a completely irrelevant topic can still total above your threshold. Fix this with veto rules, applied before the arithmetic.

• Relevance 0 is an automatic reject. Whatever else is true.
• Page liveness 0 is an automatic reject. An unindexed orphaned page passes nothing; the other dimensions are describing a page that does not functionally exist.
• Editorial 0 is an automatic reject when the reason is network footprints. Network membership is not a quality deficit to be traded off — it is a different category of risk.
• Any two dimensions at 1 or below is a reject regardless of total.

Vetoes are what make the rubric a decision tool rather than a ranking exercise. They also stop the most common failure I see in agency link approval: a prospect list that clears the average and looks nothing like a link profile a real business would have.

Calibrating the rubric against links you already have

Untested weights are guesses. Calibration turns them into something you can defend, and it uses data you already own.

1. Pull thirty to fifty existing referring domains — ideally links you or a predecessor built deliberately, so there is a decision behind each one.
2. Score every one against the rubric, blind to outcome. Do the scoring first. Knowing which links you liked will contaminate the scores.
3. Then attach the outcomes. Is the link still live? Did it send referral traffic? Did the target page's performance move in the period after acquisition? Was the link ever indexed?
4. Compare. Do high-scoring links show better survival and more referral traffic than low-scoring ones? If your rubric cannot separate the links that worked from the links that did not, the weights are wrong.
5. Adjust one weight at a time and re-run. Changing three at once tells you nothing about which change helped.

Two things to expect. First, link survival is the outcome your rubric will predict best, because it is the outcome most directly tied to whether the link had a reason to exist. Survival is also strongly tied to authority: in my own profile analysis, referring domains at Trust Flow 0 had a median link lifespan of 859 days, while those at Trust Flow 61 or above lasted a median of 3,353 days — roughly four times as long. Second, ranking impact is the outcome your rubric will predict worst, because ranking has too many other inputs. Do not throw away a rubric because it fails to predict rankings from a sample of forty links. Judge it on whether it separates durable, relevant, live links from disposable ones.

Running it without it becoming bureaucracy

A rubric survives contact with real work only if it is cheap to run and visibly useful.

• Keep it to one spreadsheet row per prospect: URL, four scores, total, veto flag, one line of notes, date, and who scored it.
• Score in batches. Twenty prospects in one sitting are scored more consistently than twenty scored a day apart, because your internal standard drifts.
• Have a second person score a sample. If two people differ by more than a point on the same site, your anchors are not specific enough. Rewrite them.
• Record rejections and why. The rejected list is more useful than the accepted list — it stops you re-evaluating the same bad prospect next quarter, and it is the evidence you show a client who asks why the volume is lower than a competing proposal promised.
• Re-score annually. Sites change hands. A site that scored 4.5 two years ago may now have a sponsored post price list and a new owner.
• Do not add a fifth dimension without removing one. Bloat is how rubrics die.

One last point about what a rubric is for. It is a tool for saying no. Most people building links have no shortage of prospects and no defensible basis for choosing between them, so they choose by metric and volume and end up with a profile that looks bought. A written rubric with veto rules gives you a reason to decline that you can explain to a client, a manager, or yourself.