Search across all sandstorm documents

[judytuna]

This is a wishlist item that came out of a conversation with @jadeqwang, @paulproteus and @jenmei !

whose docs	In One App	Across Multiple Apps
Just My Docs	Searching across all of my own documents in an app. Probably done inside of the app itself.	Searching across all documents across all of my own apps. Requires Sandstorm.io auth, and is limited to my own Sandstorm account.
Everybody on Sandstorm's Docs	Searching across all documents for a specific app (e.g., Etherpad, Mediagoblin) that I can see on Sandstorm (either because they're mine or they're public). Would this be publicly accessible, or require you to be logged in as a Sandstorm user?	Searching across all documents on all Sandstorm apps on all of Sandstorm. Search the whole Sandstorm garden (desert oasis?)

Some ideas for implementation:

• Turn all docs into some JSON format that's searchable across different apps. A new standard is born? Maybe this already exists.
• Web search...stuff? Use other engines? Unsure.

[kentonv]

Yep, search is on the long-term roadmap. I think we'll probably need to maintain a separate search index for each user. There will need to be some API by which a grain reports its searchable content, and Sandstorm will call it to index grains. A user's index should include content from all grains that would appear in their grain list -- so, their own grains as well as grains shared with them. It will of course be possible to narrow the search as desired by specifying query terms.

This will be a big project but it's definitely a critical feature.

[jenmei]

Would be great to make those documents available to any application w/ access. E.g. if someone wanted to use their own search engine instead of (or in addition to) the one that comes w/ sandstorm.

Would also be great for other metadata APIs. E.g., which users have access, size, date of last update, how frequently used, etc.

[judytuna]

And a Sandstorm app that searches Sandstorm apps

Github style score like number of commits

On Dec 6, 2015, at 21:15, Jen-Mei Wu [email protected] wrote:

Would be great to make those documents available to any application w/ access. E.g. if someone wanted to use their own search engine instead of (or in addition to) the one that comes w/ sandstorm.

Would also be great for other metadata APIs. E.g., which users have access, size, date of last update, how frequently used, etc.

—
Reply to this email directly or view it on GitHub.

[kentonv]

I agree. In fact, I think it makes a lot of sense for us to architect the search feature as an app from the start. We're at the point where it is entirely feasible for us to do that.

We can even give it full access to your data without any security risk, even if the search app is evil: If the grain can only receive the data to index, and can only output search results to you, without being able to leak anything to a third party, then the worst thing it can do is give you incorrect search results. :)

[asprionj]

Has there been any work / progress on this feature? IMHO it is an absolutely critical feature for sandstorm. I am evaluating sandstorm as some sort of "knowledge management" platform for my company. But without a search functionality over all documents (grains), finding any related content is as cumbersome as if every single collaboration tool (app) was used independenty. (BTW, this also applies to links between cards, pages, documents, ... between different apps.)

[ocdtrekkie]

This is a pretty major scale feature, and Sandstorm is more or less in maintenance mode at current, so I would not count on any significant progress here in the short-term future.

[ocdtrekkie]

I feel we should probably document here that #2868 is the beginnings of this API.