The problem this service answers
Three suppliers have presented three convincing demonstrations. Their documentation is not comparable, their claims cannot be verified from the outside, and a choice has to be justified in front of a committee within weeks.
A supplier choice on this subject is rarely reversible at low cost. Data is transferred, processes are rebuilt around a particular interface, teams are trained, and integration work accumulates. Eighteen months later, changing provider means redoing a large part of that work, which is exactly the position a serious comparison is meant to avoid.
The comparison also has an internal function. Whoever signs has to explain why one option was retained and the others were not, sometimes to a board, sometimes to an auditor. A documented benchmark, with stated criteria and traceable sources, turns a defensible judgement into something that can be reread a year later by someone who was not in the room.
What the assignment covers
The benchmark starts with your criteria, not with the market. We work with you to establish what actually matters: the capability required, the deployment model imposed by your data, the languages and document types involved, the volume expected in production, the integration constraints and the contractual conditions your legal team will insist on. Those criteria are weighted and written down before any solution is examined.
Each candidate is then assessed against that grid, from documentation, published evaluations, licence texts, service commitments and, where relevant, supervised trials on representative material. We state explicitly how each element was established, because a claim taken from a commercial page and a result obtained under controlled conditions do not have the same value in a decision.
The report presents the comparison, the criteria that separate the candidates and the ones on which they are equivalent. It also documents what the comparison does not establish, including performance on data we could not test. Vendor dependency is treated as a criterion in itself: what a change of provider would cost you later is part of what you are choosing today.
How we work on it
- We define the criteria and their weighting with the technical, legal and business stakeholders concerned.
- We establish the list of candidates, including options your teams had not considered.
- We gather documentation, licence texts, service commitments and independent evaluations for each one.
- We assess every candidate against the same grid, noting the origin and reliability of each element.
- We organise supervised trials on representative material when the criteria require them.
- We produce the comparative report, including the limits of the exercise.
- We take part in the internal presentation of the results when that is useful.
Where the information comes from
Sources are selected with you at the start of the assignment and reviewed as the subject evolves.
- Technical documentation, service commitments and licence texts of each candidate
- Independent evaluations, published test protocols and reproduction reports
- Public procurement records and published references where they exist
- Regulatory and standards documents applicable to the intended use
- Specialised professional press and user community feedback, weighted for reliability
Possible deliverables
The format is chosen with you. A single assignment can combine several of them.
- A comparative report with the grid, the assessments and their sources
- A criteria matrix that your teams can reuse for a later comparison
- A short synthesis written for a decision-making committee
- A note on contractual conditions, reversibility and vendor dependency
- A presentation session with your stakeholders to discuss the findings
Who this service is designed for
What this service does not promise
We do not certify a solution and we do not guarantee the result of a deployment. A benchmark documents what can be established at a given date, under stated conditions, on the criteria you retained. It does not replace a trial on your own data.
How to start
Describe the use you need to cover, your deployment constraints and your timeline. We propose a criteria grid and a candidate list for validation before the comparison starts.