transform script when indexing data

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

transform script when indexing data

Nick Wood
Hello,

I'm trying to implement a transform script when my data is being indexed, but it doesn't seem to have any affect.  I basically want to multiply a date_created field by 1000.

I've tried adding this to my mapping

--------------------
        "transform" : {
            "script" : "ctx._source['date_created'] = ctx._source['date_created'] * 1000;",
            "lang": "groovy"
        },
        "properties" : { ...
--------------------

When I index documents, I don't get any errors, but I also can't tell that the script actually ran or changed the data.  How would I troubleshoot whether this script is even firing, and if so, why it's not updating the data?

Best,

  Nick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5cf1ca40-3ae4-4555-ae75-733a1d8d4571%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: transform script when indexing data

Nikolas Everett

I'd add a new field and check for it. Or do a search that won't find anything unless it took effect. The document is stored untransformed so just fetching the document won't show you anything.

On Dec 22, 2014 12:35 PM, "Nick Wood" <[hidden email]> wrote:
Hello,

I'm trying to implement a transform script when my data is being indexed, but it doesn't seem to have any affect.  I basically want to multiply a date_created field by 1000.

I've tried adding this to my mapping

--------------------
        "transform" : {
            "script" : "ctx._source['date_created'] = ctx._source['date_created'] * 1000;",
            "lang": "groovy"
        },
        "properties" : { ...
--------------------

When I index documents, I don't get any errors, but I also can't tell that the script actually ran or changed the data.  How would I troubleshoot whether this script is even firing, and if so, why it's not updating the data?

Best,

  Nick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5cf1ca40-3ae4-4555-ae75-733a1d8d4571%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2WPT_3zSGLFzq12h2a8Zdzv-C8EgKnHT2g1hs3X6YneA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: transform script when indexing data

vineeth mohan-2
In reply to this post by Nick Wood
Hi Nick , 

This wont change the _source , so no point in looking there.
Try script field to read the data of this field from store ( if enabled ) or reverse index.

Thanks
          Vineeth

On Mon, Dec 22, 2014 at 11:05 PM, Nick Wood <[hidden email]> wrote:
Hello,

I'm trying to implement a transform script when my data is being indexed, but it doesn't seem to have any affect.  I basically want to multiply a date_created field by 1000.

I've tried adding this to my mapping

--------------------
        "transform" : {
            "script" : "ctx._source['date_created'] = ctx._source['date_created'] * 1000;",
            "lang": "groovy"
        },
        "properties" : { ...
--------------------

When I index documents, I don't get any errors, but I also can't tell that the script actually ran or changed the data.  How would I troubleshoot whether this script is even firing, and if so, why it's not updating the data?

Best,

  Nick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5cf1ca40-3ae4-4555-ae75-733a1d8d4571%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGdPd5nOVjnbQgLBCqs4SfY93x41b3x1S3omMH0fwxowsH836w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: transform script when indexing data

Nick Wood
In reply to this post by Nick Wood
Thanks for the tips.

In case anyone else is looking for a similar solution, my problem had to do with types.  The mapping that ended up working is:

{
    "test" : {
"transform" : {
            "script" : "ctx._source['date_created'] = ctx._source['date_created'].toLong() * 1000",
            "lang": "groovy"
        },
        "properties" : {
            "date_created" : {"type" : "date"},
        }
    }
}

Note the "toLong()".

Also, once a script is modified, it appears as though the type has to be deleted and the mapping re-applied for the changes to take effect.

  Nick

On Monday, December 22, 2014 10:35:08 AM UTC-7, Nick Wood wrote:
Hello,

I'm trying to implement a transform script when my data is being indexed, but it doesn't seem to have any affect.  I basically want to multiply a date_created field by 1000.

I've tried adding this to my mapping

--------------------
        "transform" : {
            "script" : "ctx._source['date_created'] = ctx._source['date_created'] * 1000;",
            "lang": "groovy"
        },
        "properties" : { ...
--------------------

When I index documents, I don't get any errors, but I also can't tell that the script actually ran or changed the data.  How would I troubleshoot whether this script is even firing, and if so, why it's not updating the data?

Best,

  Nick

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7b4d0870-aba4-4405-8100-5ec127de72f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.