How ELK stores data

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How ELK stores data

vikas gopal
Hi Experts,

I am totally new to this tool, so I have couple of basic queries 

1) How ELK stores indexed data. Like traditional analytic tools stores data in flat files or in their own database . 
2) How we can perform historical search
3) How license is provided , I mean is it based on data indexed per day ?
4) If I want to start do I need to download 3 tools (ElasticSearch,Logstash, Kibana)

Please assist

Thanks
VG

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/26f99ae1-7b99-467f-94d3-71a01b3b6ce7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How ELK stores data

Magnus Bäck
On Monday, March 09, 2015 at 16:34 CET,
     vikas gopal <[hidden email]> wrote:

> I am totally new to this tool, so I have couple of basic queries
> 1) How ELK stores indexed data. Like traditional analytic tools
> stores data in flat files or in their own database .

Elasticsearch is based on Lucene and the data is stored in
whatever format Lucene uses. This isn't something you have
to care about.

> 2) How we can perform historical search

Using the regular query APIs. Sorry for such a general answer
but your question is very general.

> 3) How license is provided , I mean is it based on data
> indexed per day ?

It's free Apache-licensed software so you don't have to pay
anything. If you feel you need a support contract that's
being offered at a couple of different levels. I'm sure there
are third parties offering similar services.

http://www.elasticsearch.com/support/

> 4) If I want to start do I need to download 3 tools
> (ElasticSearch,Logstash, Kibana)

If you want the whole stack from log collection to storage
to visualization then yes, you need all three. But apart
from a dependency from Kibana to Elasticsearch the tools
are independent.

I suggest you download them and try them out. That's the
quickest way to figure out whether the tool stack (or a subset
thereof) fits your needs. There are also a number of videos
available.

--
Magnus Bäck                | Software Engineer, Development Tools
[hidden email] | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20150309161010.GA18116%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How ELK stores data

Austin Harmon
Hello,

to add on to the searching historical data question, I know Elasticsearch using JSON to index documents but how do you get it to index the body of the document without copy and pasting the body into JSON. I assume there is a way to do this. I have used analyzers in my mapping but it didn't get the body of the document.

thanks,
Austin

On Monday, March 9, 2015 at 11:10:40 AM UTC-5, Magnus Bäck wrote:
On Monday, March 09, 2015 at 16:34 CET,
     vikas gopal <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="xJIzUGPm1FkJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">vikas...@...> wrote:

> I am totally new to this tool, so I have couple of basic queries
> 1) How ELK stores indexed data. Like traditional analytic tools
> stores data in flat files or in their own database .

Elasticsearch is based on Lucene and the data is stored in
whatever format Lucene uses. This isn't something you have
to care about.

> 2) How we can perform historical search

Using the regular query APIs. Sorry for such a general answer
but your question is very general.

> 3) How license is provided , I mean is it based on data
> indexed per day ?

It's free Apache-licensed software so you don't have to pay
anything. If you feel you need a support contract that's
being offered at a couple of different levels. I'm sure there
are third parties offering similar services.

<a href="http://www.elasticsearch.com/support/" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.elasticsearch.com%2Fsupport%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEYZwrT2N97YIfQhDOyPVadFmZ4pg';return true;" onclick="this.href='http://www.google.com/url?q\75http%3A%2F%2Fwww.elasticsearch.com%2Fsupport%2F\46sa\75D\46sntz\0751\46usg\75AFQjCNEYZwrT2N97YIfQhDOyPVadFmZ4pg';return true;">http://www.elasticsearch.com/support/

> 4) If I want to start do I need to download 3 tools
> (ElasticSearch,Logstash, Kibana)

If you want the whole stack from log collection to storage
to visualization then yes, you need all three. But apart
from a dependency from Kibana to Elasticsearch the tools
are independent.

I suggest you download them and try them out. That's the
quickest way to figure out whether the tool stack (or a subset
thereof) fits your needs. There are also a number of videos
available.

--
Magnus Bäck                | Software Engineer, Development Tools
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="xJIzUGPm1FkJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">magnu...@... | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c1f07b87-b8d3-4401-8dae-431264352809%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.