Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Really? Have you ever tried to search for specific content in structured documents on a filesystem?


I have, and found what I was looking for. What am I doing wrong?


How much data were you grepping for? How much time did it take? How did you transmit those results over the network?

Or does your data fit in one csv file?


You have too little data and/or too little metadata, and are not taking into account time, flexibility, tooling, etc etc.

Essentially you say something akin to "Death metal is just pulsating air-waves, like jazz, so what's the big difference?".


You are referring to using an index, correct? Because grep is absolutely, madly efficient for a doing a full search.

The index portion of a file system are called files and directories.

Several file names can refer to the same data. Those are called hard links.

So with hard links, I can refer to a Foo by their related Bar.

/foo/foo1 /foo/foo2 /foo/by_bar/bar1 /foo/by_bar/bar2 /foo/by_bar/bar3 /bar/bar1 /bar/bar2 /bar/bar3 /bar/by_foo/foo1 /bar/by_foo/foo2

If I am not mistaken, this accurately describes the limits of MongoDB in terms of mapping relations. I'm not a Mongo expert because no one could convince me otherwise to date, somebody correct me?


>You are referring to using an index, correct? Because grep is absolutely, madly efficient for a doing a full search.

I'm not sure why you imply that a full search is incompatible with an index.

Perhaps you meant "full scan", that is reading everything while searching, instead of "full search" (searching everything). The first is not a prerequisite for the second.

In any case, grep is a very inefficient way of doing a full search. An index is so much faster it's not even funny.

>The index portion of a file system are called files and directories.

Those are just indexes for the names of the files and folders, and a few other select metadata. Nothing like a full-text search index, or even actual indexes on metadata.

(Some filesystems allow those too, e.g. in BeOS, but nowhere as comprehensive and flexible as using a dedicated tool for this, be it MongoDB or something else).

>Several file names can refer to the same data. Those are called hard links. So with hard links, I can refer to a Foo by their related Bar.

Sounds like a convoluted and inefficient way of building something somewhat like a "document database" with 1/10 the features (if that).

>I'm not a Mongo expert because no one could convince me otherwise to date, somebody correct me?

I'm far from a fan of Mongo, but you seem like you have already made up your mind, and nothing will change it.

Plus, if a filesystem is enough of a document database for you (with no cheating, e.g piling up tons of hacks and add-ons like external full-text scanning tools), then be all means, us one.


Grep is essentially the slowest way to search content. It has to read every byte. You can do much better with term/field indexing.

Why would you be calling grep from an online application anyway?


Grep doesn't read every byte.


I used the word "read" and not the word "compare" for a reason.

You are missing the big picture here of linear search versus indexed search. Optimizations in grep don't magically make it better than O(n).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: