Big unstructure Text Data Query Problem

classic Classic list List threaded Threaded
5 messages Options
jj
Reply | Threaded
Open this post in threaded view
|

Big unstructure Text Data Query Problem

jj
Hello,I have a file system.Many unstructure text store in it .For example ,
ISSUED BY: CHINA EASTERN AIRLINES    ORG/DST: SHA/XNN                 ARL-D     
E/R:                                                     
TOUR CODE:                                                                      
PASSENGER: NONAME                                                                 
EXCH:                               CONJ TKT: 881-88888888/14                 
O FM:1XIY MU    OPEN  Q OPEN          SUPQ21                   20K OPEN FOR USE 
     T3-- RL:                                                                   
  TO: XNN                                                                       
FC: 06MAR13SHA MU KMG480.00//XIY MU XNN200.00CNY680.00END                       
FARE:           CNY  680.00|FOP:CC                                              
TAX:             CNY100.00CN|OI:                                                 
TAX:             CNY200.00YQ|                                                    
TOTAL:          CNY  980.00|TKTN: 881-888888888         


Now I want to transfer this unstructure text to structure object  fot query by field.For exmple,
select * from TKT where  TKT.ORG=SHA

TKT database may have TB data.
I use oracle to do this . But performance is poor. Also I try some object DB like Versant.But performane is sitll poor.
Can elasticsearch hold this query with good performance(Query can return result in 2-3 second)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Big unstructure Text Data Query Problem

dadoonet
IMHO you should think of building a JSon doc from your text files using an ETL tool or your own code.
Then you will be able to query it easily (and build facets on top of it BTW) and very fast.

My 2 cents.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 9 mai 2013 à 02:57, jj <[hidden email]> a écrit :

Hello,I have a file system.Many unstructure text store in it .For example ,
ISSUED BY: CHINA EASTERN AIRLINES    ORG/DST: SHA/XNN                 ARL-D     
E/R:                                                     
TOUR CODE:                                                                      
PASSENGER: NONAME                                                                 
EXCH:                               CONJ TKT: 881-88888888/14                 
O FM:1XIY MU    OPEN  Q OPEN          SUPQ21                   20K OPEN FOR USE 
     T3-- RL:                                                                   
  TO: XNN                                                                       
FC: 06MAR13SHA MU KMG480.00//XIY MU XNN200.00CNY680.00END                       
FARE:           CNY  680.00|FOP:CC                                              
TAX:             CNY100.00CN|OI:                                                 
TAX:             CNY200.00YQ|                                                    
TOTAL:          CNY  980.00|TKTN: 881-888888888         


Now I want to transfer this unstructure text to structure object  fot query by field.For exmple,
select * from TKT where  TKT.ORG=SHA

TKT database may have TB data.
I use oracle to do this . But performance is poor. Also I try some object DB like Versant.But performane is sitll poor.
Can elasticsearch hold this query with good performance(Query can return result in 2-3 second)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
jj
Reply | Threaded
Open this post in threaded view
|

Re: Big unstructure Text Data Query Problem

jj
Thank you~ Where is the ES China company?How to contact ?

在 2013年5月9日星期四UTC+8上午10时31分31秒,David Pilato写道:
IMHO you should think of building a JSon doc from your text files using an ETL tool or your own code.
Then you will be able to query it easily (and build facets on top of it BTW) and very fast.

My 2 cents.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 9 mai 2013 à 02:57, jj <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="2noQwq7kH-cJ">r7rau...@...> a écrit :

Hello,I have a file system.Many unstructure text store in it .For example ,
ISSUED BY: CHINA EASTERN AIRLINES    ORG/DST: SHA/XNN                 ARL-D     
E/R:                                                     
TOUR CODE:                                                                      
PASSENGER: NONAME                                                                 
EXCH:                               CONJ TKT: 881-88888888/14                 
O FM:1XIY MU    OPEN  Q OPEN          SUPQ21                   20K OPEN FOR USE 
     T3-- RL:                                                                   
  TO: XNN                                                                       
FC: 06MAR13SHA MU KMG480.00//XIY MU XNN200.00CNY680.00END                       
FARE:           CNY  680.00|FOP:CC                                              
TAX:             CNY100.00CN|OI:                                                 
TAX:             CNY200.00YQ|                                                    
TOTAL:          CNY  980.00|TKTN: 881-888888888         


Now I want to transfer this unstructure text to structure object  fot query by field.For exmple,
select * from TKT where  TKT.ORG=SHA

TKT database may have TB data.
I use oracle to do this . But performance is poor. Also I try some object DB like Versant.But performane is sitll poor.
Can elasticsearch hold this query with good performance(Query can return result in 2-3 second)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="2noQwq7kH-cJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
jj
Reply | Threaded
Open this post in threaded view
|

Re: Big unstructure Text Data Query Problem

jj
In reply to this post by jj
Hello,I want to use ES to build index for query. Use Versant to store object. How to write river plugin ?

在 2013年5月9日星期四UTC+8上午8时57分19秒,jj写道:
Hello,I have a file system.Many unstructure text store in it .For example ,
ISSUED BY: CHINA EASTERN AIRLINES    ORG/DST: SHA/XNN                 ARL-D     
E/R:                                                     
TOUR CODE:                                                                      
PASSENGER: NONAME                                                                 
EXCH:                               CONJ TKT: 881-88888888/14                 
O FM:1XIY MU    OPEN  Q OPEN          SUPQ21                   20K OPEN FOR USE 
     T3-- RL:                                                                   
  TO: XNN                                                                       
FC: 06MAR13SHA MU KMG480.00//XIY MU XNN200.00CNY680.00END                       
FARE:           CNY  680.00|FOP:CC                                              
TAX:             CNY100.00CN|OI:                                                 
TAX:             CNY200.00YQ|                                                    
TOTAL:          CNY  980.00|TKTN: 881-888888888         


Now I want to transfer this unstructure text to structure object  fot query by field.For exmple,
select * from TKT where  TKT.ORG=SHA

TKT database may have TB data.
I use oracle to do this . But performance is poor. Also I try some object DB like Versant.But performane is sitll poor.
Can elasticsearch hold this query with good performance(Query can return result in 2-3 second)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Big unstructure Text Data Query Problem

dadoonet
In reply to this post by jj

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 9 mai 2013 à 05:06, jj <[hidden email]> a écrit :

Thank you~ Where is the ES China company?How to contact ?

在 2013年5月9日星期四UTC+8上午10时31分31秒,David Pilato写道:
IMHO you should think of building a JSon doc from your text files using an ETL tool or your own code.
Then you will be able to query it easily (and build facets on top of it BTW) and very fast.

My 2 cents.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 9 mai 2013 à 02:57, jj <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="2noQwq7kH-cJ">r7rau...@...> a écrit :

Hello,I have a file system.Many unstructure text store in it .For example ,
ISSUED BY: CHINA EASTERN AIRLINES    ORG/DST: SHA/XNN                 ARL-D     
E/R:                                                     
TOUR CODE:                                                                      
PASSENGER: NONAME                                                                 
EXCH:                               CONJ TKT: 881-88888888/14                 
O FM:1XIY MU    OPEN  Q OPEN          SUPQ21                   20K OPEN FOR USE 
     T3-- RL:                                                                   
  TO: XNN                                                                       
FC: 06MAR13SHA MU KMG480.00//XIY MU XNN200.00CNY680.00END                       
FARE:           CNY  680.00|FOP:CC                                              
TAX:             CNY100.00CN|OI:                                                 
TAX:             CNY200.00YQ|                                                    
TOTAL:          CNY  980.00|TKTN: 881-888888888         


Now I want to transfer this unstructure text to structure object  fot query by field.For exmple,
select * from TKT where  TKT.ORG=SHA

TKT database may have TB data.
I use oracle to do this . But performance is poor. Also I try some object DB like Versant.But performane is sitll poor.
Can elasticsearch hold this query with good performance(Query can return result in 2-3 second)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="2noQwq7kH-cJ">elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.