Checklist: 12 questions for every provider of AI document solutions

If you are choosing an AI solution for company documents, compare answers to specific questions, not brochures. The following checklist brings together twelve questions you should ask every provider – from data storage and sources to getting out of the contract. For each question, you will find why it matters and how to recognise a good answer.

In brief

• A structured checklist makes provider selection comparable, because all candidates have to answer the same twelve questions.
• The most critical topics are data storage in the EU, sources for every AI answer and the contractual exclusion of your documents from AI training.
• Good providers answer specifically and in writing – evasive answers are an answer in themselves.
• Questions about maintenance effort, access rights and data export separate solutions that work in everyday use from systems that only shine in a demo.
• A test run with your own documents says more about suitability than any product presentation.

Why do you need a checklist for choosing a provider?

The market for document management and AI solutions is confusing: classic DMS systems, cloud storage with AI add-ons, chatbots, specialised knowledge bases. The big comparison of document management solutions explains which type of solution fits your needs in the first place. But even within one type, providers differ considerably – in data protection, traceability and ongoing effort.

A checklist makes providers comparable. If every candidate answers the same questions in writing, you quickly see who gets specific and who dodges. The twelve questions in the next section cover the points that make or break projects in practice.

The 12 questions for every provider

Question 1: Where do the AI’s answers come from – exclusively from our documents?

Why it matters: General-purpose AI models mix general knowledge from the internet into their answers. Where your company needs information it can rely on, this is risky, because plausible-sounding statements do not necessarily come from your documents.

A good answer: The system answers exclusively from the company documents that have been uploaded and tells you so when it finds no basis there – instead of inventing an answer.

Question 2: Does every answer name the specific source document?

Why it matters: AI can be wrong. Without a source, you can neither check an answer nor stand behind it in front of customers or authorities.

A good answer: Every answer refers to the document it comes from, ideally with the source passage – and the original opens in one click.

Question 3: Where is our data stored and processed?

Why it matters: Under the GDPR, what counts is not only where files are stored but also where the AI processes them. A storage location in the EU is of little use if the analysis happens elsewhere.

A good answer: Storage and AI processing verifiably happen on servers in the EU – guaranteed in writing, not just promised verbally.

Question 4: Are our documents used to train AI models?

Why it matters: Your manuals, costing data and customer correspondence are trade secrets. They must not end up in third-party AI models from which they could, in theory, resurface.

A good answer: A clear no, written into the contract – customer documents are not used to train AI models.

Question 5: Is there a data processing agreement (DPA), and which subcontractors are involved?

Why it matters: The data processing agreement (DPA) is the mandatory data protection agreement as soon as personal data is processed. The list of subcontractors shows who actually has access to your data.

A good answer: The agreement is available as a template, the subcontractors are listed by name, and changes are announced.

Question 6: How do our existing documents get into the system – including scans and paper?

Why it matters: In many companies, the most valuable knowledge sits in older documents: folders, scans, old service reports. A solution that only understands new digital files leaves precisely this knowledge untouched.

A good answer: There is a structured process for the initial import, and integrated text recognition (OCR) makes scans searchable too.

Question 7: How are access rights handled?

Why it matters: Not everyone in the company is allowed to see everything – personnel records, contracts and costings need boundaries. An AI that reveals everything to everyone is a security problem.

A good answer: Rights can be set per area and per user, and the AI only answers questions from documents that the person asking is allowed to see.

Question 8: What happens if the AI finds no answer in the documents?

Why it matters: The biggest risk with AI systems is answers that are made up but sound convincing. What matters is how the system behaves in borderline cases.

A good answer: The system reports honestly that nothing reliable can be found in the documents – and does not simply guess.

Question 9: How much maintenance does it need day to day?

Why it matters: Many knowledge systems fail not because of the technology but because of maintenance: if every document has to be filed and tagged by hand, the system soon falls into disuse.

A good answer: New documents are processed and tagged automatically; nobody has to rewrite content into a second system.

Question 10: How do we get our data back if we cancel?

Why it matters: No company should let itself be locked into a solution. Clarify what happens to your documents and data when you leave before you start.

A good answer: Your documents are retained in common formats anyway, an export is possible at any time, and deletion after the end of the contract is confirmed.

Question 11: Who supports us during implementation and day-to-day use?

Why it matters: Between signing the contract and everyday use comes the implementation – and that is where the team either takes to the solution or doesn’t. An anonymous ticket system in another time zone is of little help to an SME.

A good answer: A named, German-speaking contact person supports you through the implementation, and there are clear channels for reporting faults.

Question 12: Can we test with our own real documents?

Why it matters: Demonstrations with prepared sample data show the ideal case. Only a test with your own real-world, imperfect documents will show whether the solution can cope with them.

A good answer: A test run with your documents and your typical questions is possible before you commit for the longer term.

How do you use the checklist in practice?

Send the twelve questions to two or three candidates and insist on written answers. Then look not only at what they say but at how they say it: specific information backed by the contract carries more weight than fine-sounding statements of intent. The product overview shows how a document-based AI knowledge base answers these twelve points.

Ideally, the final step is the practical test from question 12 – with real documents and the knowledge questions that actually come up in your company. The implementation process describes getting started step by step, from the initial inventory to day-to-day use.