Checklist: 12 questions for every provider of AI document solutions

If you are choosing an AI solution for company documents, it is best to compare not brochures but answers to specific questions. The following checklist brings together twelve questions you should ask every provider – from data storage and sources to leaving the contract. For each question, you will find why it matters and how to recognise a good answer.

In brief

• A structured checklist makes provider selection comparable, because all candidates have to answer the same twelve questions.
• The most critical topics are data storage in the EU, sources for every AI answer and the contractual exclusion of your documents from AI training.
• Good providers answer specifically and in writing – evasive answers are an answer in themselves.
• Questions about maintenance effort, access rights and data export separate solutions that work in everyday use from mere demonstration systems.
• A test run with your own documents says more about suitability than any product presentation.

Why do you need a checklist for choosing a provider?

The market for document management and AI solutions is confusing: classic DMS systems, cloud storage with AI add-ons, chatbots, specialised knowledge bases. Which type of solution fits your needs in the first place is explained in the big comparison of document management solutions. But even within one type of solution, providers differ considerably – in data protection, traceability and ongoing effort.

A checklist forces comparability. If every candidate answers the same questions in writing, you quickly see who gets specific and who dodges. The twelve questions in the next section cover the points on which projects fail or succeed in practice.

The 12 questions for every provider

Question 1: Where do the AI’s answers come from – exclusively from our documents?

Why it matters: Freely trained AI models mix general knowledge from the internet into their answers. For binding information in your company, this is risky, because plausible-sounding statements do not necessarily come from your documents.

A good answer: The system answers exclusively from the company documents that have been loaded and says openly when it finds no basis there – instead of inventing an answer.

Question 2: Does every answer name the specific source document?

Why it matters: AI can be wrong. Without a source, you can neither check an answer nor take responsibility for it towards customers or authorities.

A good answer: Every answer refers to the document it comes from, ideally with the source passage – and the original can be opened with one click.

Question 3: Where are our data stored and processed?

Why it matters: For the GDPR, it is not only where files are stored that counts, but also where the AI processes them. A storage location in the EU is of little use if the analysis takes place elsewhere.

A good answer: Storage and AI processing demonstrably take place on servers in the EU – guaranteed in writing, not just promised verbally.

Question 4: Are our documents used to train AI models?

Why it matters: Your manuals, costing bases and customer correspondence are trade secrets. They must not flow into third-party AI models from which they could, in theory, resurface.

A good answer: A clear, contractually recorded no – customer documents are not used to train AI models.

Question 5: Is there a data processing agreement (DPA), and which subcontractors are involved?

Why it matters: The data processing agreement (DPA) is the mandatory data protection agreement as soon as personal data is processed. The list of subcontractors shows who actually has access to your data.

A good answer: The agreement is available as a template, the subcontractors are listed by name, and changes are announced.

Question 6: How do our existing documents get into the system – including scans and paper?

Why it matters: The most valuable knowledge of many companies is in legacy documents: folders, scans, old service reports. A solution that only understands new digital files leaves exactly this knowledge untouched.

A good answer: There is a structured process for the initial load, and integrated text recognition (OCR) makes scans searchable too.

Question 7: How are access rights mapped?

Why it matters: Not everyone in the company is allowed to see everything – personnel records, contracts and costings need boundaries. An AI that reveals everything to everyone is a security problem.

A good answer: Rights can be controlled per area and per user, and the AI only answers questions from documents that the person asking is allowed to see.

Question 8: What happens if the AI finds no answer in the documents?

Why it matters: The biggest risk of AI systems is invented but convincingly worded answers. What matters is how the system behaves in borderline cases.

A good answer: The system reports honestly that nothing reliable can be found in the documents – and does not simply guess.

Question 9: How much maintenance effort is required in day-to-day operation?

Why it matters: Many knowledge systems fail not because of the technology but because of maintenance: if every document has to be filed and tagged by hand, the system falls asleep.

A good answer: New documents are processed and tagged automatically; nobody has to rewrite content into a second system.

Question 10: How do we get our data back if we cancel?

Why it matters: No company should allow itself to be locked into a solution. What happens to documents and data when you leave is best clarified before you start.

A good answer: Your documents are retained in common formats anyway, an export is possible at any time, and deletion after the end of the contract is confirmed.

Question 11: Who supports us during implementation and day-to-day operation?

Why it matters: Between signing the contract and everyday use lies the implementation – and that is where it is decided whether the team adopts the solution. An anonymous ticket system in another time zone is of little help to an SME.

A good answer: A named, German-speaking contact person accompanies the implementation, and there are defined response channels for faults.

Question 12: Can we test with our own real documents?

Why it matters: Demonstrations with prepared sample data show the ideal case. Whether the solution can cope with your grown, imperfect documents can only be shown by a test with exactly those documents.

A good answer: A test run with your documents and your typical questions is possible before you commit for the longer term.

How do you use the checklist in practice?

Send the twelve questions to two or three candidates and insist on written answers. Then assess not only the content but also the type of answer: specific information with reference to the contract carries more weight than fine-sounding declarations of intent. The product overview shows how a document-based AI knowledge base answers these twelve points.

Ideally, the final step is the practical test from question 12 – with real documents and the knowledge questions that actually come up in your company. How such a start looks step by step, from the initial inventory to day-to-day operation, is described in the implementation process.