Anyone considering introducing AI software should be able to ask the right questions. This blog post helps companies to ask the right questions when selecting such systems.
amberSearch is a provider of in-house software that enables employees to chat with internal company content through a combination of intelligent search and generative AI and thus build their own “in-house” ChatGPT.
All about the application
- Is it a standard product and how quickly is it installed and ready to use?
- To what extent is the software dependent on open source/third-party providers and does it guarantee sufficient investment security?
- Is the AI solution fit for the future? Does the provider offer modern technology concepts (intelligent search, RAG, multihop Q&A, ….)?
- Who is behind the AI models developed? Do they rely on external expertise or do they have their own know-how?
- Does the product take into account generative artificial intelligence, e.g. for text summaries or chatbots based on internal data? If so, how is this implemented and what are the differences to other methods?
- Can I work directly with the provider or only via partners?
- Does the provider have experience in my industry?
User-friendliness
- Is the solution intuitive for users and can it be used without training?
- How does the user access the software? Via web app or are there other integration options (mobile app, iframe, team app, desktop app, etc.)?
- How modern/simple is the user interface? Does it meet the requirements of users who are quite spoilt by B2C?
- To what extent are you limited to text only or is multimedia content such as images etc. also searchable?
- To what extent can personal items (e.g. OneDrive folders, emails, teams etc.) be integrated?
- If so, are these functions checked for user rights?
- Are support functions such as autocomplete and autocorrect offered?
- Are there filter options within the answers? If so, which ones?
- Is there a document preview for PowerPoint slides, individual pages in PDF documents, etc.?
- Are the hits highlighted and immediately visible?
- Can users continue to work with the search results, e.g. share them with colleagues or make direct enquiries about a document?
- Are there personalisation options? If so, which ones?
- Can results be saved?
- Can the user interface be customised (e.g. company CI)?
- How well can the software support the desired use case?
Functionalities
- How good is the quality of the hits? How does the solution ensure relevant hits?
- Does the product take classic full-text search AND modern semantic (intelligent) vector search into account when generating answers?
- Is the software based on a retrieval augmented generation or multi-hop question answering system?
- Is it possible to use chat exclusively and neglect the search?
- Can you chat with documents?
- Can you create your own groups of information and only chat with them? (internal company GPT’s)
- Is it possible to search for metadata?
- Can drives be connected?
- Do documents have to be prepared manually before the software can handle them?
- Do documents have to be exported somewhere or does the software retrieve the latest information automatically?
- Does the software recognise the language(s) of my content? Even in mixed-language texts?
- Does it recognise organisation-specific terms? Or can this be made available?
- Are interfaces/APIs for creating your own connectors or for connecting your own frontends, for example, accessible and well documented at no additional cost?
- Can new AI models be added flexibly?
- Is the system a black box? Is there a risk of vendor lock-in?
Data sources
- Which data sources can I connect? Are standard connectors available? If so, which ones?
- Can I have a connector created for my specific application?
- Can the software search unstructured data and/or structured data?
- Does the software cover all file formats relevant to me?
- How many documents can the solution capture? Is the database limited? Is the solution scalable?
- How up-to-date is the indexed data?
Data protection and data security
- If I prefer a cloud/SaaS solution, where will my data be stored? (EU, USA, etc.)
- How does the provider guarantee data security?
- Are the users’ access rights to the documents automatically respected?
Operations
- Where is the solution operated? In the cloud, on-premise, private cloud? Is a hybrid approach also conceivable?
- How much effort is required to integrate synonyms and the like?
- Is the software easy to install, including by the customer?
- Could generative AI also be hosted on premise?
- Does the customer receive project support from the provider during implementation?
- Is the solution multi-client capable? Can several locations, branches, etc. be connected?
- Is the solution stable if many users access the software at the same time?
- How can I test the software? Is a proof of concept possible? If so, is there a charge for this?
- How time-consuming is the administration of the solution?
- Is user training necessary?
Service and consulting
- What support models are available on site?
- Is support provided by internal representatives of the provider or by external partners?
- Does the provider offer additional functions in the application if required?
- How often are software updates carried out?
- How long will I receive support for my software?
Commercial criteria
- Is the pricing model easy to understand? Is the pricing model based on usage, documents or users?
- Are there additional costs for development, testing or staging instances of the solution?
- What ROI calculations need to be taken into account?
Provider information
- How long has the provider been on the market?
- In which country is the software developed?
- Are there any certifications required for this type of software?
- Does it have its own development department?
- How do the AI models used/developed compare to the competition?
- Are there any customer references?