Skip to content
Plumb Line

How Plumb Line works, rule by rule

Five sets of rules: what a review passes through before it appears, how its stars become a score, where reviews come from, how listings are kept, and how to flag a review.

The path to publication

Every review written on Plumb Line waits in a queue, marked pending, until a moderator has read it against the review guidelines. The moderator then does one of two things: publishes it or rejects it. Until it is published, a review shows on no company page and adds nothing to any score.

What the moderator weighs:

  • Did the reviewer deal with this company? The review must rest on their own order, booking, visit or quote, not on hearsay or a different business.
  • Is anything in it out of bounds? Links and phone numbers come out, and so do personal details about staff, threats and hate speech, wherever they appear in the text.
  • Who is behind it? We watch for repeat posts from a single person, and for reviews planted by the company’s own staff or by a rival.
  • How soon after the last one? One reviewer may review the same company once every 30 days.

Praise and complaint pass through the same check on the same terms. A company may answer any review in public, but it cannot edit, hide or remove one, and no payment changes how a review is handled.

From stars to a score

A company’s score is arithmetic and nothing more: add up the stars in its published reviews, divide by the number of those reviews, and round to one decimal place. No part of it is estimated, adjusted by hand or influenced by payment.

A handful of reviews makes an unsteady average, so color waits for numbers. Under 5 published reviews, the score is printed in grey and labeled “Tentative”. Once a company reaches 5, its score takes the color of its band:

BandRangeShown as
Admirable4.5 – 5.04.7
Favorable4.0 – 4.44.2
Adequate3.0 – 3.93.4
Subpar2.0 – 2.92.5
Failingunder 2.01.6

Buying and service, weighed separately

A shop that sells well can still be poor at repairs, and the reverse happens too. So each reviewer says what kind of dealing it was: a purchase, a request for a quote, or a service. Once enough have answered, the company’s page shows a buying score and a service score next to each other.

Four details and two price questions

Reviewers can also score four details: how forthright and responsive the company was, how reasonable its charges were, its pace and handling, and how closely the result matched the promise. They can answer two yes-or-no questions as well: did the final bill equal the quote, and were they charged anything beyond what was stated? Each of these figures stays off a company’s page until 5 customers have answered it.

Ranking order

Where a list is sorted by rating, plain averages alone would let a company with three five-star reviews jump ahead of one with forty reviews averaging 4.8. To prevent that, each company enters the ranking carrying 5 imaginary reviews of 3.5 stars. Its real reviews outweigh them more and more as they accumulate, until its rank score sits close to its own average:

rank score = (5 × 3.5 + stars received) ÷ (5 + reviews received)

Rank scores set the order of lists; they never replace the plain average, which is the only score a company’s page shows.

Where reviews originate

Every review on a company page reached us in one of three ways:

SourceWhat it means
Filled out on Plumb Line's own review formThe customer wrote it here, on this site.
Filled out on this company's own feedback formGathered by the company itself, from its own customers, and handed to us as it came in. Anything the company picked out, altered or paid for is refused.
Reviews from other platformsThe customer first posted it on another review site; it appears here as it was published there.

How listings are kept

Where does a listing come from? From business information we collect and check, down to the name, the street address, the phone number and the website. What happens when a company closes? Its listing is hidden, never deleted, and the reviews stay with it. The same applies when we can no longer confirm its details.

A company can claim its listing at no charge, then correct its details and reply to customers. Its score is the same the day after a claim as the day before.

When a review seems off

Under every review is a Report link. Use it for any of three reasons: the review breaks the guidelines, it is about a different company, or you have reason to think it is not genuine. A moderator reads every report and settles it one of three ways: the review comes down, it stays, or the reviewer is asked for more detail. While that happens, the review stays visible.