How does Klemens.AI work?
LLM proposes, code verifies
Most AI document tools work similarly: a language model analyzes a document fragment and writes a response.
The problem is that LLM optimizes sentence flow, not source fidelity. This can be detected with a single question. Not so with a few parameters and dozens of documents.
Document analysis in 5 steps
1. Source repository and versioning
Documents are placed in the repository assigned to the task. Each file has a version, so it's clear what material was used to create a specific result. It can be recreated in six months.
2. Question/prompt
You determine exactly what should be extracted from the documents: which products or offers, which fields, and which attributes. Here, you also decide on the output format: a table, a list of differences, a set of negotiation questions, or a descriptive summary. This is the actual content of the command and where you have the most control.
3. Content extraction
Each document is read in its original form, not as a previously transcribed text. This isn't an aesthetic choice, but rather a measurement: working with a flattened transcript loses a significant amount of value because it disrupts tables and page layouts, and in insurance documents, most of the specific information is contained in tables.
The system also understands that a single document may describe several insurance products, and that a single product may be scattered across multiple documents. It distinguishes between the product document and the general terms and conditions.
4. Verification of the quote against the source
Before a response reaches you, the associated citation is checked against the document's content. An unverified citation is rejected, and the field is marked as missing data or an analysis error, depending on the cause.
Every response that comes out of the system is checked for consistency with the source document by the code, not LLM.
5. Reply with link
You receive the answer in an agreed-upon format. If it's a comparison table, each filled-in cell has the file name, page number in the document being analyzed, and a verbatim citation, while each empty cell has a reason. In the list of differences or in the descriptive summary, a reference to the source appears next to each statement.
Document based on responses
On the Klemens.AI platform, you can also create a finished document: a response to a client, a note, or meeting materials. The content is based on your analysis or comparison. You edit the document on the platform, view previous versions, and export it to DOCX or PDF. You remain responsible for the content.
What does the result of document comparison look like?
Below is a diagram of the table format, as this is the most common form of the result.
The answers are addressed in the document, the quote is verbatim, and any missing data remains a missing data, not a guess.
Atrybut | Ubezpieczyciel 1 | Ubezpieczyciel 2 |
|---|---|---|
Exemption: employee error | [value]
OWU_A.pdf, p. 22
"[literal quote from the document]" | [value]
OWU_B.pdf, p. 9
"[literal quote from the document]" |
Limit on legal defense costs | [value]
OWU_A.pdf, p. 14
"[literal quote from the document]" | —
no record in the document |
Why can't a general AI assistant and semantic search handle it?
Answer accuracy . With hundreds of cells, widely available AI assistants begin to truncate, generalize, and round numbers. In comparing insurance documents, this can mean the difference between a correct and incorrect decision.
Completeness of the response . There is no guarantee that every field for every product has been covered. No one will notice a missing cell because the table appears complete.
Honest "—" . For a field that is not present in the document, the system will not return the closest match. This will not create a value that looks plausible and is false.
Verifiability . Without a source address, checking a single value means rereading the document.
Klemens.AI Limitations
-
The requirement for comprehensive lists and verbatim detail results in cells in the comparison table being long and verbose. We choose precision over brevity.
-
The quality of the result depends on the quality of the sources. The completeness and quality of the documents influence the quality of the analysis and responses.
-
The result must always be verified by a human. We don't design the process without your input.