
What should an institution check when choosing an AI assistant? Six questions before procurement
How should an institution choose an AI assistant? Six vendor-neutral questions on roles, data handling, auditability and quality. Daskalos Consulting, 2025–2026
More and more universities are looking for an AI assistant for student services, and it quickly becomes clear that the choice isn't as simple as picking software. The same name, "AI assistant", covers very different systems. What suits one institution well can be a serious risk at another.
This article, written independently of any vendor, shows why the choice isn't obvious and which six questions to ask before making a decision.
Why isn't the choice obvious?
At a university there are at least three kinds of assistant:
- Public chatbot. Answers prospects and students from information available on the website. Relatively low risk, since it only works with public content.
- Internal assistant. Helps staff with policies and internal documents. Here it matters who gets access to what information.
- Acting assistant. It doesn't just answer: it opens tickets, drafts emails, books appointments and connects to other systems. A mistake has bigger consequences here.
A single procurement often involves all three types, but each comes with different security and governance expectations. That's why it's better to start with questions rather than a feature list.
1. Who can see and do what?
Different roles work at a university: students, student services staff, administrators, IT. An AI assistant shouldn't treat them all the same.
Ask: is there role-based access control (RBAC)? Does everyone see only what their job requires? Is there multi-step authentication (for example with an authenticator app) on more sensitive interfaces? Can personal data be masked where it isn't needed?
2. Who decides when the assistant is about to act?
There is a big difference between an informational answer and an executed action. If the assistant opens a case, changes data or sends an email, the university needs a person to approve it.
This principle is known as "human in the loop": a human stays in the loop. Ask: for which actions does the assistant ask for approval, and can the institution configure that itself?
3. Can you trace what happened?
When a student files a complaint, or an investigation needs to establish what was said and who accessed what, you need a trail. Auditability means the system logs activity and the AI's operation can be traced end to end.
Ask: is there a full activity log? Can the path of an individual answer be traced back? Who can access the logs?
4. Where does the data go, and does the model learn from it?
This is the question most university leaders want answered most precisely. It helps to split it up:
- Where is the data stored? Does the system run on EU-based servers?
- Who is the data controller and who is the processor? Properly, the university is always the controller and the provider acts as a processor under a data processing agreement (DPA).
- Does the model learn from the data? The university's data shouldn't be used to train models, and it shouldn't flow into another institution's system.
- Does the university keep control? Can it archive, export and delete its data whenever it wants?
If a vendor points to security certifications (for example ISO 27001 or SOC 2), ask whose certification it is. It is common for the certificate to belong to the cloud infrastructure provider, not the software vendor. Both are valuable, but they are not the same.
5. Where do the answers come from, and who checks quality?
An AI assistant is only as good as the knowledge it is built on. Ask: are answers produced from the university's own documents, with the source shown? Are public and internal content kept separate, so an internal policy can't reach an outside prospect?
Quality also can't be promised in general terms: it depends on how well organised the university's own website and documents are. A fair vendor shows this in advance rather than explaining it afterwards. It is worth asking for a trial using the university's own real student questions, and comparing the results with reference answers.
6. Who is responsible for operations?
Running an AI assistant takes ongoing work after launch. Models change, and integrations (for example with the student information system) need maintenance. Ask: how are answer quality and safety tested on a regular basis? Can you see in one place which models and integrations are running? Is there a public status page showing operating status? And if the university wants to switch providers, how does it get its data back?
How did we prepare for these questions?
We built Emily so that each of the six questions above has a concrete answer. This is how we work:
- Roles. Everyone sees and does only what their job requires. Sensitive interfaces use multi-step authentication with an authenticator app, and we mask personal data on request.
- Approval. When the assistant is about to take an action, a staff member approves it.
- Traceability. We log every activity, and the AI's operation can be traced end to end.
- Data. The system runs on EU-based servers, and Emily works as a data processor under a data processing agreement. We do not train models on the institution's data, and the data never reaches another institution's system. The institution can archive, export or delete its data at any time. The cloud infrastructure runs on certified providers (Azure, Vercel, DigitalOcean), and the ISO 27001 and SOC 2 certifications cover their services.
- Answers and quality. Answers come from the institution's own documents, with sources cited. The public and the internal knowledge bases run separately. We measure quality in advance using the institution's own questions and reference answers.
- Operations. We test answer quality and safety on a regular basis. We manage the models and integrations in one place, and the operating status is visible on a status page.
Emily has four layers. Students meet the public chatbot and get answers with sources cited. Student services get analytics, an overview and an internal assistant. Administrators and IT staff set up the assistants and the chatbot and build their own agents. The deepest layer is platform oversight, where the AI's operation is governed and verified.
If an institution wants to see how this works in practice, the AI Quality Report is free and carries no commitment. We use the institution's own real questions and reference answers, and show the strengths, weaknesses and recommendations in a PDF report.