Documents not deleted when using DeleteRequest within BulkProcessor

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Documents not deleted when using DeleteRequest within BulkProcessor

Diana Tuck
Trying to index/delete documents within one BulkProcessor object in the Java API.  Indexing documents works great!  Deleting, however, does not.

bulkProcessor.add(new DeleteRequest(index.getIndexingAlias(), index.getType(), entityId));

Nothing happens.  Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Documents not deleted when using DeleteRequest within BulkProcessor

dadoonet
Do you try to delete a doc using an alias?
Any failure or error reported by the bulk processor?

Hard to tell more without seeing the code / data.

David

Le 30 avr. 2015 à 02:03, Diana Tuck <[hidden email]> a écrit :

Trying to index/delete documents within one BulkProcessor object in the Java API.  Indexing documents works great!  Deleting, however, does not.

bulkProcessor.add(new DeleteRequest(index.getIndexingAlias(), index.getType(), entityId));

Nothing happens.  Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/512A5411-A00C-4F02-B11E-A0B24A27E7E3%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Documents not deleted when using DeleteRequest within BulkProcessor

Diana Tuck
Thank you for the reply, David.

We are using an alias to delete.  Is that a problem?  Indexing with the alias through the bulk processor works fine.

There are no errors reported, it just seems to disappear into the oblivion.  Here's our code for the BulkProcessor:

public static BulkProcessor getBulkProcessor(Client client, int esConcurrencyLevel, int esBulkSize, int esFlushInterval) {
return BulkProcessor.builder(client, new BulkProcessor.Listener() {

@Override
public void beforeBulk(long executionId, BulkRequest bulkRequest) {
LOG.debug("Processing {} requests in bulk process {}", bulkRequest.numberOfActions(), executionId);
}

@Override
public void afterBulk(long executionId, BulkRequest bulkRequest, BulkResponse response) {

if (response.hasFailures()) {
for (BulkItemResponse item : response.getItems()) {
LOG.error("Processing to index \"{}\" failed for entity id {} with message {}", item.getIndex(),
item.getId(), item.getFailureMessage());
}
}
}

@Override
public void afterBulk(long executionId, BulkRequest bulkRequest, Throwable throwable) {
LOG.error("Failed to process {} requests in bulk request {}: {}", bulkRequest.numberOfActions(),
executionId, throwable.getMessage());
throwable.printStackTrace();
}
})
.setBulkActions(esBulkSize)
.setFlushInterval(TimeValue.timeValueSeconds(esFlushInterval))
.setConcurrentRequests(esConcurrencyLevel)
.build();
}

Code for the delete request:

bulkProcessor.add(new DeleteRequest(index.getIndexingAlias(), index.getType(), entityId));

where index.getIndexingAlias() is an alias (same alias used for indexing which is working), type is the document type "company" and entityId is the document ID.

What data would be helpful?  An example document, the index metadata, something else?

On Wednesday, April 29, 2015 at 9:53:41 PM UTC-7, David Pilato wrote:
Do you try to delete a doc using an alias?
Any failure or error reported by the bulk processor?

Hard to tell more without seeing the code / data.

David

Le 30 avr. 2015 à 02:03, Diana Tuck <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="j3fxrvQ9TYEJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">dtu...@...> a écrit :

Trying to index/delete documents within one BulkProcessor object in the Java API.  Indexing documents works great!  Deleting, however, does not.

bulkProcessor.add(new DeleteRequest(index.getIndexingAlias(), index.getType(), entityId));

Nothing happens.  Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="j3fxrvQ9TYEJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">elasticsearc...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;">https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/01b6ab18-78a8-44d0-b574-c649501ec21a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Documents not deleted when using DeleteRequest within BulkProcessor

dadoonet
No as soon as you have only one index for this alias, indexing and deleting should work.

I don’t see anything suspicious here.

Any chance you could share on github your full code?


When you say that "nothing happens", do you mean that you never get the debug LOG « Processing {} … » ?
Or do you mean that the document has not been removed ?

How do you test all that?

-- 
David Pilato - Developer | Evangelist 





Le 30 avr. 2015 à 19:10, Diana Tuck <[hidden email]> a écrit :

Thank you for the reply, David.

We are using an alias to delete.  Is that a problem?  Indexing with the alias through the bulk processor works fine.

There are no errors reported, it just seems to disappear into the oblivion.  Here's our code for the BulkProcessor:

public static BulkProcessor getBulkProcessor(Client client, int esConcurrencyLevel, int esBulkSize, int esFlushInterval) {
return BulkProcessor.builder(client, new BulkProcessor.Listener() {

@Override
public void beforeBulk(long executionId, BulkRequest bulkRequest) {
LOG.debug("Processing {} requests in bulk process {}", bulkRequest.numberOfActions(), executionId);
}

@Override
public void afterBulk(long executionId, BulkRequest bulkRequest, BulkResponse response) {

if (response.hasFailures()) {
for (BulkItemResponse item : response.getItems()) {
LOG.error("Processing to index \"{}\" failed for entity id {} with message {}", item.getIndex(),
item.getId(), item.getFailureMessage());
}
}
}

@Override
public void afterBulk(long executionId, BulkRequest bulkRequest, Throwable throwable) {
LOG.error("Failed to process {} requests in bulk request {}: {}", bulkRequest.numberOfActions(),
executionId, throwable.getMessage());
throwable.printStackTrace();
}
})
.setBulkActions(esBulkSize)
.setFlushInterval(TimeValue.timeValueSeconds(esFlushInterval))
.setConcurrentRequests(esConcurrencyLevel)
.build();
}

Code for the delete request:

bulkProcessor.add(new DeleteRequest(index.getIndexingAlias(), index.getType(), entityId));

where index.getIndexingAlias() is an alias (same alias used for indexing which is working), type is the document type "company" and entityId is the document ID.

What data would be helpful?  An example document, the index metadata, something else?

On Wednesday, April 29, 2015 at 9:53:41 PM UTC-7, David Pilato wrote:
Do you try to delete a doc using an alias?
Any failure or error reported by the bulk processor?

Hard to tell more without seeing the code / data.

David

Le 30 avr. 2015 à 02:03, Diana Tuck <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="j3fxrvQ9TYEJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" class="">dtu...@...> a écrit :

Trying to index/delete documents within one BulkProcessor object in the Java API.  Indexing documents works great!  Deleting, however, does not.

bulkProcessor.add(new DeleteRequest(index.getIndexingAlias(), index.getType(), entityId));

Nothing happens.  Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="j3fxrvQ9TYEJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;" class="">elasticsearc...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" onclick="this.href='https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;" class="">https://groups.google.com/d/msgid/elasticsearch/e2774458-8542-4634-bd8d-1ccfd9837409%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;" class="">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/01b6ab18-78a8-44d0-b574-c649501ec21a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/C7602498-6322-4F06-86ED-01B5147FA1FE%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.