"TTL value must be > 0. Illegal value provided" when updating document with _ttl set

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

"TTL value must be > 0. Illegal value provided" when updating document with _ttl set

Brad Cavanagh
I'm using ES 0.90.1. I've run into a little race condition-ish problem concerning sending an update request for a document after its TTL has expired but before ES has run the TTL cleanup.

Here are the replication steps:


% curl -XPUT localhost:9200/test-ttl -d '{
   "settings" : {
     "index.number_of_shards" : 1,
     "index.number_of_replicas" : 0
   },
   "mappings" : {
     "test-doc" : {
       "_ttl" : {
         "enabled" : true,
         "default" : "1m"
       }
     }
   }
 }'
% curl -XPUT 'localhost:9200/test-ttl/test-doc/1' -d '{ "title" : "test title" }'
% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

In another window, I have this running:

% while true ; do date ; curl -XGET 'http://localhost:9200/test-ttl/test-doc/1?pretty=true&fields=title,_ttl' ; sleep 1 ; done

While that's running, I can watch the _ttl field decrease. When it becomes negative, the document is in a state such that its TTL has expired, but ES hasn't yet run the cleanup step. I can still retrieve the document, even though technically speaking it's "expired".

When the document is in this state, if I do:

% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

... I get the following error message:

{"error":"ElasticSearchIllegalArgumentException[TTL value must be > 0. Illegal value provided [-209]]","status":400}

This is causing me a little bit of grief because the program I have written to update documents in ES doesn't handle this properly, so it just bombs out entirely (which I really need to fix regardless). Is there another way I should be updating documents that could be in this "expired but still available" state?

Thanks,
 Brad.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: "TTL value must be > 0. Illegal value provided" when updating document with _ttl set

InquiringMind
Brad,

From my "outside looking in" perspective, I've never encountered this specific case.... so thanks for describing it! But my suggestion would be to handle this specific exception as if the document wasn't there any more.

"Expired but still available" really is "expired, or as good as expired". I've noticed that the TTL value goes negative and the document stuck around a little longer in 0.90 than it did in 0.20.4, but I'm guessing it's related to some default configuration change. However, the document still expired and I only noticed it because my test cases created TTL values in minutes. Whereas a real application would typically set TTL values of hours or days or perhaps weeks and not worry too much about the length of time it sepnds in this "expired but still there with a negative TTL" state.

Exceptions are much cleaner to handle in Java than via the HTTP interface, I realize.

Brian

On Wednesday, June 26, 2013 5:59:49 PM UTC-4, Brad Cavanagh wrote:
I'm using ES 0.90.1. I've run into a little race condition-ish problem concerning sending an update request for a document after its TTL has expired but before ES has run the TTL cleanup.

Here are the replication steps:


% curl -XPUT localhost:9200/test-ttl -d '{
   "settings" : {
     "index.number_of_shards" : 1,
     "index.number_of_replicas" : 0
   },
   "mappings" : {
     "test-doc" : {
       "_ttl" : {
         "enabled" : true,
         "default" : "1m"
       }
     }
   }
 }'
% curl -XPUT 'localhost:9200/test-ttl/test-doc/1' -d '{ "title" : "test title" }'
% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

In another window, I have this running:

% while true ; do date ; curl -XGET 'http://localhost:9200/test-ttl/test-doc/1?pretty=true&fields=title,_ttl' ; sleep 1 ; done

While that's running, I can watch the _ttl field decrease. When it becomes negative, the document is in a state such that its TTL has expired, but ES hasn't yet run the cleanup step. I can still retrieve the document, even though technically speaking it's "expired".

When the document is in this state, if I do:

% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

... I get the following error message:

{"error":"ElasticSearchIllegalArgumentException[TTL value must be > 0. Illegal value provided [-209]]","status":400}

This is causing me a little bit of grief because the program I have written to update documents in ES doesn't handle this properly, so it just bombs out entirely (which I really need to fix regardless). Is there another way I should be updating documents that could be in this "expired but still available" state?

Thanks,
 Brad.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: "TTL value must be > 0. Illegal value provided" when updating document with _ttl set

Benjamin Devèze
In reply to this post by Brad Cavanagh
Brad,

I think you are right it should probably behave as if the document has been deleted when it is in expired state but not yet expunged, i.e. in your case it should return a DocumentMissingException I guess.

Thanks for reporting, would you mind opening an issue for it?

On Wednesday, June 26, 2013 11:59:49 PM UTC+2, Brad Cavanagh wrote:
I'm using ES 0.90.1. I've run into a little race condition-ish problem concerning sending an update request for a document after its TTL has expired but before ES has run the TTL cleanup.

Here are the replication steps:


% curl -XPUT localhost:9200/test-ttl -d '{
   "settings" : {
     "index.number_of_shards" : 1,
     "index.number_of_replicas" : 0
   },
   "mappings" : {
     "test-doc" : {
       "_ttl" : {
         "enabled" : true,
         "default" : "1m"
       }
     }
   }
 }'
% curl -XPUT 'localhost:9200/test-ttl/test-doc/1' -d '{ "title" : "test title" }'
% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

In another window, I have this running:

% while true ; do date ; curl -XGET 'http://localhost:9200/test-ttl/test-doc/1?pretty=true&fields=title,_ttl' ; sleep 1 ; done

While that's running, I can watch the _ttl field decrease. When it becomes negative, the document is in a state such that its TTL has expired, but ES hasn't yet run the cleanup step. I can still retrieve the document, even though technically speaking it's "expired".

When the document is in this state, if I do:

% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

... I get the following error message:

{"error":"ElasticSearchIllegalArgumentException[TTL value must be > 0. Illegal value provided [-209]]","status":400}

This is causing me a little bit of grief because the program I have written to update documents in ES doesn't handle this properly, so it just bombs out entirely (which I really need to fix regardless). Is there another way I should be updating documents that could be in this "expired but still available" state?

Thanks,
 Brad.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: "TTL value must be > 0. Illegal value provided" when updating document with _ttl set

Brad Cavanagh


On Wed, Jun 26, 2013 at 11:24 PM, Benjamin Devèze <[hidden email]> wrote:
Brad,

I think you are right it should probably behave as if the document has been deleted when it is in expired state but not yet expunged, i.e. in your case it should return a DocumentMissingException I guess.

Thanks for reporting, would you mind opening an issue for it?


On Wednesday, June 26, 2013 11:59:49 PM UTC+2, Brad Cavanagh wrote:
I'm using ES 0.90.1. I've run into a little race condition-ish problem concerning sending an update request for a document after its TTL has expired but before ES has run the TTL cleanup.

Here are the replication steps:


% curl -XPUT localhost:9200/test-ttl -d '{
   "settings" : {
     "index.number_of_shards" : 1,
     "index.number_of_replicas" : 0
   },
   "mappings" : {
     "test-doc" : {
       "_ttl" : {
         "enabled" : true,
         "default" : "1m"
       }
     }
   }
 }'
% curl -XPUT 'localhost:9200/test-ttl/test-doc/1' -d '{ "title" : "test title" }'
% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

In another window, I have this running:

% while true ; do date ; curl -XGET 'http://localhost:9200/test-ttl/test-doc/1?pretty=true&fields=title,_ttl' ; sleep 1 ; done

While that's running, I can watch the _ttl field decrease. When it becomes negative, the document is in a state such that its TTL has expired, but ES hasn't yet run the cleanup step. I can still retrieve the document, even though technically speaking it's "expired".

When the document is in this state, if I do:

% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

... I get the following error message:

{"error":"ElasticSearchIllegalArgumentException[TTL value must be > 0. Illegal value provided [-209]]","status":400}

This is causing me a little bit of grief because the program I have written to update documents in ES doesn't handle this properly, so it just bombs out entirely (which I really need to fix regardless). Is there another way I should be updating documents that could be in this "expired but still available" state?

Thanks,
 Brad.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/ifvWZJjQuvU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: "TTL value must be > 0. Illegal value provided" when updating document with _ttl set

InquiringMind
In reply to this post by Benjamin Devèze
Benjamin,

In this case, should the document also be omitted from a match-all query or other type of search instead of being returned with a negative TTL? And not just a get-by-id request?

Brian

On Thursday, June 27, 2013 2:24:17 AM UTC-4, Benjamin Devèze wrote:
Brad,

I think you are right it should probably behave as if the document has been deleted when it is in expired state but not yet expunged, i.e. in your case it should return a DocumentMissingException I guess.

Thanks for reporting, would you mind opening an issue for it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: "TTL value must be > 0. Illegal value provided" when updating document with _ttl set

Benjamin Devèze
In reply to this post by Brad Cavanagh
Thanks Brad let's continue the technical discussion about this on your issue and wait for some points of view about it.

On Thursday, June 27, 2013 11:00:39 PM UTC+2, Brad Cavanagh wrote:


On Wed, Jun 26, 2013 at 11:24 PM, Benjamin Devèze <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="XMLYxDK1NvQJ">benjami...@...> wrote:
Brad,

I think you are right it should probably behave as if the document has been deleted when it is in expired state but not yet expunged, i.e. in your case it should return a DocumentMissingException I guess.

Thanks for reporting, would you mind opening an issue for it?


On Wednesday, June 26, 2013 11:59:49 PM UTC+2, Brad Cavanagh wrote:
I'm using ES 0.90.1. I've run into a little race condition-ish problem concerning sending an update request for a document after its TTL has expired but before ES has run the TTL cleanup.

Here are the replication steps:


% curl -XPUT localhost:9200/test-ttl -d '{
   "settings" : {
     "index.number_of_shards" : 1,
     "index.number_of_replicas" : 0
   },
   "mappings" : {
     "test-doc" : {
       "_ttl" : {
         "enabled" : true,
         "default" : "1m"
       }
     }
   }
 }'
% curl -XPUT 'localhost:9200/test-ttl/test-doc/1' -d '{ "title" : "test title" }'
% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

In another window, I have this running:

% while true ; do date ; curl -XGET 'http://localhost:9200/test-ttl/test-doc/1?pretty=true&fields=title,_ttl' ; sleep 1 ; done

While that's running, I can watch the _ttl field decrease. When it becomes negative, the document is in a state such that its TTL has expired, but ES hasn't yet run the cleanup step. I can still retrieve the document, even though technically speaking it's "expired".

When the document is in this state, if I do:

% curl -XPOST 'localhost:9200/test-ttl/test-doc/1/_update' -d '{ "doc" : { "title" : "test update title" } }'

... I get the following error message:

{"error":"ElasticSearchIllegalArgumentException[TTL value must be > 0. Illegal value provided [-209]]","status":400}

This is causing me a little bit of grief because the program I have written to update documents in ES doesn't handle this properly, so it just bombs out entirely (which I really need to fix regardless). Is there another way I should be updating documents that could be in this "expired but still available" state?

Thanks,
 Brad.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/ifvWZJjQuvU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="XMLYxDK1NvQJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: "TTL value must be > 0. Illegal value provided" when updating document with _ttl set

Benjamin Devèze
In reply to this post by InquiringMind
InquiringMind yeah you are certainly right and this is more or less what I have begun to talk about in the issue.

I think the negative TTL value error message is not very good in any case.

On Thursday, June 27, 2013 11:08:24 PM UTC+2, InquiringMind wrote:
Benjamin,

In this case, should the document also be omitted from a match-all query or other type of search instead of being returned with a negative TTL? And not just a get-by-id request?

Brian

On Thursday, June 27, 2013 2:24:17 AM UTC-4, Benjamin Devèze wrote:
Brad,

I think you are right it should probably behave as if the document has been deleted when it is in expired state but not yet expunged, i.e. in your case it should return a DocumentMissingException I guess.

Thanks for reporting, would you mind opening an issue for it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.