Create parent field mapping in ES with MongoDB-river

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Create parent field mapping in ES with MongoDB-river

Margaret-2
I have some data stored in MongoDB. These data are separated into two collections: one is 'bare_authors' which contains basic information of authors like name; the other one is 'books' which contains information of books written by the author. For example, in MongoDB, we have: 
db.bare_authors.find()
{ "_id" : ObjectId("512b01b955922c6270d96d8e"), "name" : "Multi G.Enre" }
{ "_id" : ObjectId("512b01c455922c6270d96d8f"), "name" : "Alastair Reynolds" }

db.books.find()
{ "_id" : ObjectId("512af8d455922c6270d96d8c"), "name" : "Revelation Space", "genre" : "scifi" }
{ "_id" : ObjectId("512af99155922c6270d96d8d"), "name" : "Dead in the night", "genre" : "thriller" }

I want to use the 'parent field mapping' in ES. So I created a mapping in elasticsearch:
$ curl -XPOST localhost:9200/authors/book/_mapping -d '{
  "book":{
    "_parent":{"type":"bare_author"}
  }
}'

Then I created the river:
$ curl -XPUT 'http://localhost:9200/_river/authors/_meta' -d '
{
  type:"mongodb",
  mongodb:{
    db:"authors",collection:"bare_authors",
    host:"localhost",
    port:"27017"
  },
  index:{
    name:"authors",
    type:"bare_author",
    bulk_size:"1000",
    bulk_timeout:"30"
  }
}'
{"ok":true,"_index":"_river","_type":"authors","_id":"_meta","_version":1}
and
$ curl -XPUT 'http://localhost:9200/_river/books/_meta' -d '
{
  type:"mongodb",
  mongodb:{
    db:"authors",collection:"books",
    host:"localhost",
    port:"27017"
  },
  index:{
    name:"authors",
    type:"book",
    bulk_size:"1000",
    bulk_timeout:"30"
  }
}'
{"ok":true,"_index":"_river","_type":"books","_id":"_meta","_version":1}
 
But I got error:  
org.elasticsearch.action.RoutingMissingException: routing is required for [authors]/[book]/[512b01dd55922c6270d96d91]

Can anyone help? Thanks. 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Create parent field mapping in ES with MongoDB-river

Martijn v Groningen
I have no experience with the mongo river, but in general when indexing child documents the parent id needs to be provided during the index operation. That doesn't seem to happen here. A quick look at the plugin tells me that specifying the parent id during indexing isn't implemented. Maybe writing a small script that syncs your data with ES is sufficient in your case?

Martijn

On 25 February 2013 08:09, Maggie <[hidden email]> wrote:
I have some data stored in MongoDB. These data are separated into two collections: one is 'bare_authors' which contains basic information of authors like name; the other one is 'books' which contains information of books written by the author. For example, in MongoDB, we have: 
db.bare_authors.find()
{ "_id" : ObjectId("512b01b955922c6270d96d8e"), "name" : "Multi G.Enre" }
{ "_id" : ObjectId("512b01c455922c6270d96d8f"), "name" : "Alastair Reynolds" }

db.books.find()
{ "_id" : ObjectId("512af8d455922c6270d96d8c"), "name" : "Revelation Space", "genre" : "scifi" }
{ "_id" : ObjectId("512af99155922c6270d96d8d"), "name" : "Dead in the night", "genre" : "thriller" }

I want to use the 'parent field mapping' in ES. So I created a mapping in elasticsearch:
$ curl -XPOST localhost:9200/authors/book/_mapping -d '{
  "book":{
    "_parent":{"type":"bare_author"}
  }
}'

Then I created the river:
{
  type:"mongodb",
  mongodb:{
    db:"authors",collection:"bare_authors",
    host:"localhost",
    port:"27017"
  },
  index:{
    name:"authors",
    type:"bare_author",
    bulk_size:"1000",
    bulk_timeout:"30"
  }
}'
{"ok":true,"_index":"_river","_type":"authors","_id":"_meta","_version":1}
and
{
  type:"mongodb",
  mongodb:{
    db:"authors",collection:"books",
    host:"localhost",
    port:"27017"
  },
  index:{
    name:"authors",
    type:"book",
    bulk_size:"1000",
    bulk_timeout:"30"
  }
}'
{"ok":true,"_index":"_river","_type":"books","_id":"_meta","_version":1}
 
But I got error:  
org.elasticsearch.action.RoutingMissingException: routing is required for [authors]/[book]/[512b01dd55922c6270d96d91]

Can anyone help? Thanks. 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.