Amazon flicks switch on AI-powered enterprise search service Kendra – but where are all the connectors? The concept behind Kendra, the enterprise search service that Amazon Web Services made generally available today, is pretty simple. Hook up connectors to all the data sources relevant to your organisation. Add a FAQ for good measure. Then leave it to the Kendra engine to do the indexing and add some machine learning, AI and …
>The Enterprise edition supports 500,000 documents
That is _astonishingly_ small. It has been common for Solr and Elasticsearch indices to store, index and retrieve tens of billions of documents for many, many years now. At a maximum storage of 150GB this thing is basically a toy. Enterprise search it most definitely is not.
... I would proffer that if you have 500,000 documents under management control, you have too many. You need to be more prescriptive with the term "document". By avoiding the noise, and being able to find what you are looking for from a small subset, quality and access are improved and benefits realised. Indexing everything, using something like the private Google Search Engine pointed at a fileshare, is just adding to noise. As usual, it is the humans that will tame the technology. I am keen to see how good Kendra is at getting me what I am looking for, rather than 5,950,000 hits for the term "storage latest WDC". I probably only want 3 out of those, sniff.
Even if we constrain the term "document" to mean something human-centric like an email, PDF or doc file that is still a really small number. In legal and regulatory contexts in particular that is a tiny number. Look at something like the Enron scandal. Just the emails from that scandal - one that happened nearly two decades ago - clock in at 600,000. It's common for even modest legal cases to rack up millions of physical documents.
The reality is in the world of search "document" is equivalent to "row" in RDBMS world - or in simple terms "anything that I want to search". That might be a line of chat text, log record, a financial trade or a sensor report. Billions of documents are common in enterprise search. That's what enterprise search is for.
I have to agree - we have more than 500,000 documents stored locally, let alone those now stored in various clouds. OK a fair few may be duplicate (90% certainty there - its Education we are talking) but even so 500k seems like a really small limit.
The other thing that occurs to me is if the search is anything like as atrocious as the search box on the Amazon web site... would you want to pay for that?
Elastic search with Ultrawarm https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/ultrawarm.html
I doubt it will be like Alexa for you business documents because of the variety, quality and ad-hoc nature, even if they have an AI semantic quantum powered engine as this is an NLP hard problem.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
