Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-25234

Analytics must define explicit mapping for ecm:path@level index

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 10.2-SNAPSHOT
    • Fix Version/s: 10.2
    • Component/s: Elasticsearch

      Description

      nuxeo analytics add extra index on path by level.
      An explicit mapping must be set (keyword type) to prevent elastic to assign a wrong type of field depending on the first term indexed.
      For instance if the first ecm:path@5 is taken for a timestamp, this will create errors for the following docs:

      2018-06-26 12:56:08,553 ERROR [Nuxeo-Work-elasticSearchIndexing-1:1024302662188230.778594692] [org.nuxeo.elasticsearch.core.ElasticSearchIndexingImpl] failure in bulk execution:
[0]: index [nuxeo], type [doc], id [c06991b0-baea-423a-8634-0a05309242ab], message [MapperParsingException[failed to parse [ecm:path@level5]]; nested: IllegalArgumentException[Invalid format: "localConfigNote"];]
[1]: index [nuxeo], type [doc], id [99c55218-7cc3-4366-b687-ce6eb01d0382], message [MapperParsingException[failed to parse [ecm:path@level5]]; nested: IllegalArgumentException[Invalid format: "localConfigNote"];]
2018-06-26 12:56:08,563 WARN  [http-nio-0.0.0.0-8080-exec-1] [org.nuxeo.ecm.platform.ui.web.restAPI.SystemLogRestlet] ----- WebDriver: Starting test 'org.nuxeo.ftest.cap.ITContentViewLocalConfigTest#testContentViewLocalConfig'
2018-06-26 12:56:14,024 ERROR [http-nio-0.0.0.0-8080-exec-8] [org.nuxeo.ecm.core.work.AbstractWork] Exception during work: IndexingWorker(c06991b0-baea-423a-8634-0a05309242ab, , Progress(?%, ?/0), null)
MapperParsingException[failed to parse [ecm:path@level5]]; nested: IllegalArgumentException[Invalid format: "localConfigNote._1530017774011_...."];
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:302)
	at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:481)
	at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:603)
	at org.elasticsearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:403)
	at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:380)
	at org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:95)
	at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:69)
	at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:261)
	at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:700)
	at org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:677)
	at org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:658)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.lambda$executeIndexRequestOnPrimary$2(TransportShardBulkAction.java:553)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeOnPrimaryWhileHandlingMappingUpdates(TransportShardBulkAction.java:572)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:551)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequest(TransportShardBulkAction.java:142)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:248)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.performOnPrimary(TransportShardBulkAction.java:125)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:112)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:74)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:1018)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:996)
	at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:103)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:357)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:297)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:959)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:956)
	at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:270)
	at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
	at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2213)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:968)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.access$500(TransportReplicationAction.java:98)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:318)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:293)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:280)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66)
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:656)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:724)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Invalid format: "localConfigNote._1530017774011_...."
	at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187)
	at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:780)
	at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parse(DateFieldMapper.java:248)
	at org.elasticsearch.index.mapper.DateFieldMapper.parseCreateField(DateFieldMapper.java:456)
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:297)
	... 41 more
2018-06-26 12:56:14,049 ERROR [http-nio-0.0.0.0-8080-exec-8] [org.nuxeo.ecm.core.work.AbstractWork] Exception during work: IndexingWorker(c06991b0-baea-423a-8634-0a05309242ab, , Progress(?%, ?/0), null)
MapperParsingException[failed to parse [ecm:path@level5]]; nested: IllegalArgumentException[Invalid format: "localConfigNote._1530017774011_...."];
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:302)
	at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:481)
	at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:603)
	at org.elasticsearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:403)
	at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:380)
	at org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:95)
	at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:69)
	at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:261)
	at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:700)
	at org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:677)
	at org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:658)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.lambda$executeIndexRequestOnPrimary$2(TransportShardBulkAction.java:553)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeOnPrimaryWhileHandlingMappingUpdates(TransportShardBulkAction.java:572)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:551)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequest(TransportShardBulkAction.java:142)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:248)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.performOnPrimary(TransportShardBulkAction.java:125)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:112)
	at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:74)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:1018)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:996)
	at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:103)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:357)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:297)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:959)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:956)
	at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:270)
	at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:237)
	at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2213)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:968)
	at org.elasticsearch.action.support.replication.TransportReplicationAction.access$500(TransportReplicationAction.java:98)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:318)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:293)
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:280)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66)
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:656)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:724)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Invalid format: "localConfigNote._1530017774011_...."
	at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187)
	at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:780)
	at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parse(DateFieldMapper.java:248)
	at org.elasticsearch.index.mapper.DateFieldMapper.parseCreateField(DateFieldMapper.java:456)
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:297)
	... 41 more

      Seen on CI: https://qa.nuxeo.org/jenkins/job/Deploy/job/IT-nuxeo-master-tests-cap-tomcat-mariadb/716/

      So far only seen since elasticsearch 6.3.0 upgrade so limit fix to 10.2

        Attachments

          Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. NXP-24102 Upgrade Elasticsearch from 5.6.3 to 6.3.0

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. NXP-27445 Backport NXP-25234 to 9.10 fix es mapping for ecm:path

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 4 hours
                  4h