Our editorial policy sets out the principles. This is the working detail: what goes into a score, what stays out, and where the method runs out of road.
What a score is and is not
The figure on each vendor review is an editorial assessment. We apply a published weighting across six criteria to the evidence we have gathered, and publish the result next to the analysis that produced it. That is the whole mechanism.
What the number is not is worth stating bluntly, because review sites are often vague here:
- It is not a laboratory measurement. Nothing here is benchmarked under controlled conditions. Two analysts working from the same evidence could land a few tenths apart.
- It is not a survey result. We report no sample size because we do not run a statistically constructed panel. Any site quoting a satisfaction percentage without saying how respondents were recruited is asking you to take it on faith.
- It is not an aggregate of third-party review scores. We read public reviews as evidence; we do not average other people's ratings and present the mean as our judgement.
Used well, a score is a shortlisting instrument: it tells you which four or five platforms deserve a demo and a reference call. Used badly, it becomes a substitute for your own evaluation, and no score — ours included — can carry a six-figure decision on its own. Read the analysis, not just the digit, and treat gaps of under half a point as noise rather than a ranking.
The six criteria
Every review applies the same weighting. It does not change by vendor, by segment, or by whether the vendor advertises with us — how we make money explains why that separation is structural rather than a promise.
| Criterion | Weight | What we look at | What would lower the score |
|---|---|---|---|
| Feature Completeness | 25% | Depth across receiving, put-away, slotting, picking strategies, replenishment, cycle counting, wave and waveless fulfilment, labour management, yard and dock, and reporting. Assessed from documentation and hands-on evaluation where available. | Core functions available only through custom development; capabilities on a roadmap rather than shipped; modules sold separately but described as included. |
| Ease of Use | 20% | Task flow for the people who use it daily: pickers on a handheld, a supervisor rebalancing waves, an administrator changing a rule. Screen counts for routine tasks, terminology consistency, and how much training the vendor's materials assume. | Configuration that requires scripting for ordinary changes; interfaces that differ sharply between modules; consistent user reports of slow or unreliable handheld clients. |
| Integration Capabilities | 15% | Whether published API documentation is complete enough to build against, availability of pre-built ERP and carrier connectors, and how automation interfaces are handled. | APIs documented only under NDA; integration available solely through the vendor's professional services arm; batch-file exchange presented as a modern interface. |
| Implementation Experience | 15% | Typical deployment shape and duration, how much work falls on the customer, quality of migration and testing tooling, and input from partners who have delivered the product. | Repeated accounts of timelines overrunning materially; thin or outdated implementation documentation; a partner network too small to give buyers a choice. |
| Customer Support | 15% | Published support tiers and response commitments, coverage across time zones, escalation paths, the state of the knowledge base, and publicly available user feedback on support quality. | Meaningful support locked behind a premium tier; unresolved issues described consistently by unconnected users; no route to a human when the floor stops. |
| Value for Investment | 10% | Capability delivered relative to total cost of ownership — licence, implementation, integration and ongoing administration — judged against comparable products in the same segment. | Pricing that makes cost unpredictable as volume grows; essential functionality priced as an add-on; a cost profile out of line with comparable peers. |
The weights are deliberately uneven. Feature completeness carries the most because a missing capability is the hardest problem to fix after signature; value carries the least because a good platform bought at a poor price is still recoverable.
Where our evidence comes from, and its limits
Four sources feed a review. Each has a specific weakness we correct for rather than ignore.
Vendor documentation
Documentation and release notes are the fullest description of what a system is meant to do. They describe intent, not behaviour. A documented feature may be mature, or awkward, or available only on a version most customers have not upgraded to. Where we cannot tell which, we say so.
Hands-on evaluation
Where we have access to a system, we use it. Access is uneven: entry-level products often offer trials, while enterprise platforms are demonstrated rather than handed over. A demo shows a happy path chosen by the vendor — useful, but not the same as running the product.
Publicly available user feedback
Public reviews skew toward extremes. People write them when a system has rescued their operation or ruined their quarter, and the large middle goes unrepresented. Some is solicited by vendors at moments of maximum goodwill. We look for patterns repeated by unconnected reviewers rather than treating any single account as decisive.
Implementation partner input
The consultancies that deliver these systems see more deployments than any single customer ever will. They also have commercial ties to the vendors they implement, so their view is informed and not disinterested.
Two structural limits sit above all four. Enterprise WMS pricing is rarely published, so any range we give is inferred from what buyers and partners describe rather than quoted from a rate card — where we are inferring, we label it. And these platforms are configured so heavily that two deployments of the same product behave like different systems: the same licence can be a fast, tidy operation in one building and a slow one next door. Treat any single score as one input among several, and give a reference call with an operation shaped like yours more weight than a number on a page.
What we do not score
Much of what determines whether a WMS project succeeds cannot be assessed generically, so we do not pretend to score it. These factors decide outcomes more often than the choice of software does.
- Fit with your specific process. A platform built for a cold-chain distributor with heavy compliance requirements may suit a direct-to-consumer parcel operation badly. Our comparisons and industry guides narrow the field; only your own requirements document closes it.
- The implementation partner you hire. The same product delivered by two teams produces two results. Reference the partner as rigorously as the vendor.
- Your data quality. Item masters, dimensions, weights and location data that are wrong going in come out wrong, faster. Data remediation is routinely the most underestimated line in these projects.
- Change management. Adoption on the floor, supervisor buy-in and the willingness to change a process rather than customise software around it are decisive, and no vendor can supply them for you.
When we update a review
Every review carries the date of its most recent assessment. A rating reflects the product as we found it on that date and nothing after. We revisit a review when:
- A major release changes capability in a way that touches one of the six criteria
- An acquisition or change of ownership alters the roadmap or support structure
- The pricing or licensing model changes, including a shift between perpetual, subscription and consumption-based terms
- User reports accumulate that contradict our assessment, particularly on support or implementation
- A vendor or reader supplies evidence that something we published is wrong
Where a re-review moves a score, the page carries the new assessment and the reasoning. Material corrections are noted rather than quietly patched.
Challenge a rating
If you think a score is wrong — whether you work for the vendor, sell against it, or run it in your own warehouse — email [email protected] with the page URL, the criterion you are disputing, and what you believe the correct position is. We reply within two business days.
Because scores decompose into six named criteria, a challenge only has to address the criterion at issue. Evidence that moves an assessment: documentation showing a capability we recorded as absent, release notes dated after our assessment, published pricing or support terms contradicting what we described, and first-hand implementation accounts detailed enough to be checked. What does not move one: disagreement with the weighting itself, comparisons to competitor scores, and aggregate ratings from other sites.
Two things are worth stating without hedging. Vendors get no advance sight of reviews: no pre-publication approval, no embargo arrangement, no right of reply before a piece goes live. And no rating can be bought — advertising is sold on placement only, and if an advertiser asks for a change to a score the answer is no. Both are set out in full on how we make money.
Related: who writes this · RFP template · contact