-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 10.10-HF67, 2023.0, 2021.27
-
Component/s: Elasticsearch
-
Release Notes Summary:Elasticsearch indexing is more robust by skipping the corrupted documents.
-
Tags:
-
Team:PLATFORM
-
Sprint:nxplatform #71
-
Story Points:3
The JsonESDocumentWriter which is building the elastic json representation of a document is going to raise an exception when trying to read a corrupted property, resulting in indexing error.
When this is happening in a BucketWorker the entire bucket of documents is in error and therefore not indexed.
Corrupted properties exist because of a wrong schema update (changing type of a field),
when this is happening in production it is very hard to fix, it requires a manual (db level) intervention.
We should trace an error and skip the property (or the document if not possible), so other docs in the bucket are indexed.
Example:
org.nuxeo.ecm.core.api.model.PropertyConversionException: Unable to read property: videoasset:transcodedAssets for document: 83c9e57d-5cad-47d7-8077-36bf99599529
at org.nuxeo.ecm.core.storage.BaseDocument.readComplexProperty(BaseDocument.java:766) ~[nuxeo-core-storage-10.10-HF55.jar:?]
at org.nuxeo.ecm.core.storage.BaseDocument.readComplexProperty(BaseDocument.java:710) ~[nuxeo-core-storage-10.10-HF55.jar:?]
at org.nuxeo.ecm.core.storage.dbs.DBSDocument.readDocumentPart(DBSDocument.java:1069) ~[nuxeo-core-storage-dbs-10.10-HF61.jar:?]
at org.nuxeo.ecm.core.api.DocumentModelFactory.createDataModel(DocumentModelFactory.java:198) ~[nuxeo-core-10.10-HF61.jar:?]
at org.nuxeo.ecm.core.api.AbstractSession.getDataModel(AbstractSession.java:2057) ~[nuxeo-core-10.10-HF61.jar:?]
at org.nuxeo.ecm.core.api.impl.DocumentModelImpl.loadDataModel(DocumentModelImpl.java:439) ~[nuxeo-core-api-10.10-HF61.jar:?]
at org.nuxeo.ecm.core.api.impl.DocumentModelImpl.getDataModel(DocumentModelImpl.java:449) ~[nuxeo-core-api-10.10-HF61.jar:?]
at org.nuxeo.ecm.core.api.impl.DocumentModelImpl.getPart(DocumentModelImpl.java:1215) ~[nuxeo-core-api-10.10-HF61.jar:?]
at org.nuxeo.ecm.core.api.impl.DocumentModelImpl.getPropertyObjects(DocumentModelImpl.java:1241) ~[nuxeo-core-api-10.10-HF61.jar:?]
at org.nuxeo.elasticsearch.io.JsonESDocumentWriter.writeProperties(JsonESDocumentWriter.java:204) ~[nuxeo-elasticsearch-core-10.10-HF62.jar:?]
