...
Solution | Integration Complexity | Fully Managed | Comment |
|---|---|---|---|
Open AIOpenAI | Very High | No | Provide access to separate APIs that would allow us to build a RAG system, but it’s not a managed solution and would require a consistent amount of effort and maintenance to build. Custom GPTs would allow to build an application that integrates external knowledge for example from a search API. This might be worth investigating for “chatting with a project/folder”. |
Azure AI Search | High | No | See https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview. Similar to the Open AI solution, Azure provides all the tools needed (plus the search index management) to build a RAG solution but it would nevertheless be a big effort. |
AWS Bedrock | Medium | Yes | Provides a fully managed solution that is integrated with AWS, provide access to several open source models and does most of the heavy lifting. The knowledge base component is what enables the RAG system managing the storage and retrieval of external data. The default vector database is an AWS OpenSearch index. |
AWS Kendra | High | No | It’s a managed semantic search index (starts from $2k/month for 100K documents) that can potentially be used as part of a RAG system. Kendra can potentially be integrated into Bedrock as the backing document index. |
Vertex AI Search | High | Yes | Provides a (presumably) fully managed solution to deploy a RAG system using their own models. |
...
We are already on AWS, we know how to use it and we do not have to learn another cloud platform/api
There is no easy way to test this out today quickly and compare across solutions. Ideally with a first prototype we could build a static knowledge base that can be reused, including the indexed content and the questions/answers plus feedback.
Bedrock seems to be the most comprehensive and easier to integrate solution out there, despite that it might not have the best models available, with several tools that include model evaluation (https://docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation.html ), prompt management (https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management.html ), content filtering (See https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html ) etc.
Bedrock provide access to several different foundation models that can be tried (See https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html )
Bedrock has native support for metadata filtering (https://aws.amazon.com/about-aws/whats-new/2024/03/knowledge-bases-amazon-bedrock-metadata-filtering/) that could be used for access control in a similar way we do with CloudSearch today.
We could potentially reuse OpenSearch for both normal search and the backing vector database
Pricing is a bit of a complex topic as in all the above solutions it follows a pay-for-what-you-use model that depends on the user input, number of tokens, type of model used and amount of data. Given my recommendation I can estimate the cost of bedrock to be around $1k-$2k/month depending on usage (the default vector database is AWS OpenSearch, which should cost between $400 and $800/month).
...