Documents vanishing

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Documents vanishing

Alexis Okuwa
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

phill
I think the obvious question is: what is your mapping for sysMetrics types?
Might you try to add another field (or use an existing field)
particularly a non-analyzed field, retrieve the document by that, and
see what the document retrieved by this other has in it, particularly
for its ID.
Alternatively for debugging, retrieve everything, dump it and see what
you see in your index.  Has the _id been converted in some way?  How
about the other fields.

-Paul

On 5/4/2013 6:42 PM, Wojons Tech wrote:

> I having a werid issue where I feel like i am loosing documents, a get
> request fails to get the document but if i do a search for the
> document it works just fine. So this is the output of a document write
>
> Array
> (
>     [ok] => 1
>     [_index] => latest
>     [_type] => sysMetrics
>     [_id] => t2QjFSAfQjSJRPv-J8neeA
>     [_version] => 1
> )
>
> if I curl for the document
>
> curl http://localhost:9200/latest/sysMetrics/t2QjFSAfQjSJRPv-J8neeA
> {"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}
> --
> You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [hidden email].
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.


Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

InquiringMind
In reply to this post by Alexis Okuwa
When I issue a get-by-id request for an existing document in my "test" index, I get:

$ curl http://localhost:9200/test/subscriber/7214560007
{"_index":"test","_type":"subscriber","_id":"7214560007","_version":1,"exists":true, "_source" : { "telno" : "7214560007", "name" : "FXE", "o" : "FLL Executive Airport", "city" : "Ft. Lauderdale", "state" : "FL" }}

And when I issue another get-by-id request for an _id that I know to not exist:

$ curl http://localhost:9200/test/subscriber/9999999999
ERROR: HTTP response status: 404

I don't recall any documentation of the meaning of "exists":false (a lot has been going on and it's likely I've missed it), but empiracally it doesn't seem to apply to the document itself.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Alexander Reelsen-2
In reply to this post by Alexis Okuwa
Hey,

how are you indexing your data? Maybe using a TTL? Are you using routing maybe? These are wild guesses, because I do not know your setup.
Without more information it is hard to help out. Can you recreate your issue by using curl calls, so we can reproduce it?


--Alex


On Sun, May 5, 2013 at 3:42 AM, Wojons Tech <[hidden email]> wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Alexis Okuwa


On Sat, May 11, 2013 at 4:57 AM, Alexander Reelsen <[hidden email]> wrote:
Hey,

how are you indexing your data? Maybe using a TTL? Are you using routing maybe? These are wild guesses, because I do not know your setup.
Without more information it is hard to help out. Can you recreate your issue by using curl calls, so we can reproduce it?


--Alex


On Sun, May 5, 2013 at 3:42 AM, Wojons Tech <[hidden email]> wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

InquiringMind
Your previous post showed the TTL enabled mapping, but your gist's mapping (link below) does not.

Also, you need to check your documents to see where _ttl is specified. Here is the subset of my initial testing. The elf type has TTL enabled but there is no default (documents without a TTL are immortal). Only the third document below is mortal (finite TTL). Note that the TTL is in this bulk input stream's action_and_meta_data and not in the source:

{"create":{"_index":"mortal","_type":"elf","_id":"1"}}
{ "cn" : "Celeborn" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"2"}}
{ "cn" : "Galadriel" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"3","_ttl":"3m"}}
{ "cn" : "Haldir of Lórien" , "text" : [ "Dies in the battle of Helm's Deep", "Gone in 3m" ] }

One other behavior: If your mapping does not enable TTL, then any _ttl you add to a document seems to go into the bit bucket and does not come back in a response. The _ttl only comes back as a field in this case when it's added to the _source; but in that case it isn't a _ttl that ElasticSearch knows about.

Brian

On Saturday, May 11, 2013 8:01:18 AM UTC-4, Wojons Tech wrote:


--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Alexis Okuwa
i am very sure that i am do not have any ttl on. 


On Sat, May 11, 2013 at 6:58 AM, InquiringMind <[hidden email]> wrote:
Your previous post showed the TTL enabled mapping, but your gist's mapping (link below) does not.

Also, you need to check your documents to see where _ttl is specified. Here is the subset of my initial testing. The elf type has TTL enabled but there is no default (documents without a TTL are immortal). Only the third document below is mortal (finite TTL). Note that the TTL is in this bulk input stream's action_and_meta_data and not in the source:

{"create":{"_index":"mortal","_type":"elf","_id":"1"}}
{ "cn" : "Celeborn" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"2"}}
{ "cn" : "Galadriel" , "text" : [ "Lives forever" ] }

{"create":{"_index":"mortal","_type":"elf","_id":"3","_ttl":"3m"}}
{ "cn" : "Haldir of Lórien" , "text" : [ "Dies in the battle of Helm's Deep", "Gone in 3m" ] }

One other behavior: If your mapping does not enable TTL, then any _ttl you add to a document seems to go into the bit bucket and does not come back in a response. The _ttl only comes back as a field in this case when it's added to the _source; but in that case it isn't a _ttl that ElasticSearch knows about.

Brian

On Saturday, May 11, 2013 8:01:18 AM UTC-4, Wojons Tech wrote:


--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

InquiringMind
In reply to this post by Alexis Okuwa
I looked at the Javadoc for the GetResponse.isExists method and the related getSourceAsBytes method. It seems to tell me that your response that contains "exists":false means that the source of the document doesn't exist. But if there is an index.type.id that is found then what is that? ElasticSearch's terminology is confusing me here; the source might not exist but the index, type, and _id exist or else you'd get an HTTP 404 status. So what is the index, type, and _id if it's not a document? Or is the term "document" loosely equated to "document source". Here is the relevant Javadoc:

isExists

public boolean isExists()
Does the document exists.

getSourceAsBytes

public byte[] getSourceAsBytes()
The source of the document if exists.

isSourceEmpty

public boolean isSourceEmpty()
Is the source empty (not available) or not.

One downside to documentation after the code is written is that the code becomes difficult to explain. I've never ever created a source-less document nor have I seen the issue you are seeing, so I don't know all of the corner cases that I would normally use to recreate my own documentation of how things work.

Perhaps somewhere in your load you are creating or updating the document with the meta data but not providing a source. That's the only thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

InquiringMind
Or, you are providing a source but somehow not storing it. Then your get request returns with the source not existing, but your normal queries return fields because the fields are stored and asked for, but the source is not stored.

Again, just some thoughts. I don't store fields: I always store the full source and parse it using the supplied Jackson stream parser (works well, and helps with updates too).

Brian

On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document with the meta data but not providing a source. That's the only thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Alexis Okuwa
Is there anything else i could provide that would help find what the exact issue is?


On Sat, May 11, 2013 at 7:41 PM, InquiringMind <[hidden email]> wrote:
Or, you are providing a source but somehow not storing it. Then your get request returns with the source not existing, but your normal queries return fields because the fields are stored and asked for, but the source is not stored.

Again, just some thoughts. I don't store fields: I always store the full source and parse it using the supplied Jackson stream parser (works well, and helps with updates too).

Brian


On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document with the meta data but not providing a source. That's the only thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Derry O' Sullivan
Hey,

Not sure if this is related but we had a similar problem before with parent child documents:
http://elasticsearch-users.115913.n3.nabble.com/Having-problem-retrieving-child-documents-with-their-IDs-td4031448.html

Pretty much get only looked at 1 shard (out of default 5) but search worked every time...

D

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.


Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Alexander Reelsen-2
In reply to this post by Alexis Okuwa
Yes, a possibility for us to recreate your problem locally (using curl) would be great in order to find out if it is a bug or not.


--Alex


On Sun, May 12, 2013 at 4:53 AM, wojonstech <[hidden email]> wrote:
Is there anything else i could provide that would help find what the exact issue is?


On Sat, May 11, 2013 at 7:41 PM, InquiringMind <[hidden email]> wrote:
Or, you are providing a source but somehow not storing it. Then your get request returns with the source not existing, but your normal queries return fields because the fields are stored and asked for, but the source is not stored.

Again, just some thoughts. I don't store fields: I always store the full source and parse it using the supplied Jackson stream parser (works well, and helps with updates too).

Brian


On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document with the meta data but not providing a source. That's the only thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to [hidden email].

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
<a href="tel:424.835.1223" value="+14248351223" target="_blank">424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Alexis Okuwa
just wanted to let everyone know that i found the answer.


i was writin the documents with routing and not when i did the pulling. for now i  have disabled routing. how does everyone feel about routing.


On Sun, May 12, 2013 at 12:56 AM, Alexander Reelsen <[hidden email]> wrote:
Yes, a possibility for us to recreate your problem locally (using curl) would be great in order to find out if it is a bug or not.


--Alex


On Sun, May 12, 2013 at 4:53 AM, wojonstech <[hidden email]> wrote:
Is there anything else i could provide that would help find what the exact issue is?


On Sat, May 11, 2013 at 7:41 PM, InquiringMind <[hidden email]> wrote:
Or, you are providing a source but somehow not storing it. Then your get request returns with the source not existing, but your normal queries return fields because the fields are stored and asked for, but the source is not stored.

Again, just some thoughts. I don't store fields: I always store the full source and parse it using the supplied Jackson stream parser (works well, and helps with updates too).

Brian


On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document with the meta data but not providing a source. That's the only thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to [hidden email].

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
<a href="tel:424.835.1223" value="+14248351223" target="_blank">424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

simonw-2
phew! thanks for bringing closure to this!

simon

On Monday, May 20, 2013 10:21:49 AM UTC+2, Wojons Tech wrote:
just wanted to let everyone know that i found the answer.


i was writin the documents with routing and not when i did the pulling. for now i  have disabled routing. how does everyone feel about routing.


On Sun, May 12, 2013 at 12:56 AM, Alexander Reelsen <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="AIRXUNwBmDkJ">a...@...> wrote:
Yes, a possibility for us to recreate your problem locally (using curl) would be great in order to find out if it is a bug or not.


--Alex


On Sun, May 12, 2013 at 4:53 AM, wojonstech <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="AIRXUNwBmDkJ">wojon...@...> wrote:
Is there anything else i could provide that would help find what the exact issue is?


On Sat, May 11, 2013 at 7:41 PM, InquiringMind <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="AIRXUNwBmDkJ">brian....@...> wrote:
Or, you are providing a source but somehow not storing it. Then your get request returns with the source not existing, but your normal queries return fields because the fields are stored and asked for, but the source is not stored.

Again, just some thoughts. I don't store fields: I always store the full source and parse it using the supplied Jackson stream parser (works well, and helps with updates too).

Brian


On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document with the meta data but not providing a source. That's the only thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="AIRXUNwBmDkJ">elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="AIRXUNwBmDkJ">elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="AIRXUNwBmDkJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Documents vanishing

Alexis Okuwa
for all the known elasticsearch masters that may ever find this. what i am guessing is the issue is the _id for this document would normally route to a selective shard. based on some hash distrobution. but since i had custom routing i would see the documents when the document would have ended up on the shard it got routed to based on the _id anyway. 

Searches were not effected because they were not using routing and calling all nodes.


On Mon, May 20, 2013 at 4:22 AM, simonw <[hidden email]> wrote:
phew! thanks for bringing closure to this!

simon


On Monday, May 20, 2013 10:21:49 AM UTC+2, Wojons Tech wrote:
just wanted to let everyone know that i found the answer.


i was writin the documents with routing and not when i did the pulling. for now i  have disabled routing. how does everyone feel about routing.


On Sun, May 12, 2013 at 12:56 AM, Alexander Reelsen <[hidden email]> wrote:
Yes, a possibility for us to recreate your problem locally (using curl) would be great in order to find out if it is a bug or not.


--Alex


On Sun, May 12, 2013 at 4:53 AM, wojonstech <[hidden email]> wrote:
Is there anything else i could provide that would help find what the exact issue is?


On Sat, May 11, 2013 at 7:41 PM, InquiringMind <[hidden email]> wrote:
Or, you are providing a source but somehow not storing it. Then your get request returns with the source not existing, but your normal queries return fields because the fields are stored and asked for, but the source is not stored.

Again, just some thoughts. I don't store fields: I always store the full source and parse it using the supplied Jackson stream parser (works well, and helps with updates too).

Brian


On Saturday, May 11, 2013 10:30:41 PM UTC-4, InquiringMind wrote:

Perhaps somewhere in your load you are creating or updating the document with the meta data but not providing a source. That's the only thing I can think of at this time.

Brian

On Saturday, May 4, 2013 9:42:57 PM UTC-4, Wojons Tech wrote:
I having a werid issue where I feel like i am loosing documents, a get request fails to get the document but if i do a search for the document it works just fine. So this is the output of a document write

Array
(
    [ok] => 1
    [_index] => latest
    [_type] => sysMetrics
    [_id] => t2QjFSAfQjSJRPv-J8neeA
    [_version] => 1
)

if I curl for the document

{"_index":"latest","_type":"sysMetrics","_id":"t2QjFSAfQjSJRPv-J8neeA","exists":false}

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
<a href="tel:424.835.1223" value="+14248351223" target="_blank">424.835.1223

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/8diEvl-OoRc/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.