-
Type: Bug
-
Status: Resolved
-
Priority: Minor
-
Resolution: Not A Bug
-
Affects Version/s: 9.10, 10.3-SNAPSHOT
-
Fix Version/s: None
-
Component/s: TransientStore
-
Tags:
-
Backlog priority:600
KeyValueBlobTransientStore.putBlobs() does not currently include the providerId in the blobKey.
A conservative, 4-line patch to KeyValueBlobTransientStore that fixes the problem is included below.
Alternate, more invasive patches to BinaryBlobProvider that achieve the same result are also included below.
Thanks,
Ron
Patch
@Override public void putBlobs(String key, List<Blob> blobs) { if (absoluteMaxSize > 0 && getStorageSize() > absoluteMaxSize) { // do the costly computation of the exact storage size if needed doGC(); if (getStorageSize() > absoluteMaxSize) { throw new MaximumTransientSpaceExceeded(); } } // remove previous blobs removeBlobs(key); KeyValueStore kvs = getKeyValueStore(); BlobProvider bp = getBlobProvider(); long totalSize = 0; int i = 0; for (Blob blob : blobs) { long size = blob.getLength(); if (size >= 0) { totalSize += size; } // store blob String blobKey; try { blobKey = bp.writeBlob(blob); } catch (IOException e) { throw new NuxeoException(e); } int colon = blobKey.indexOf(':'); String digest = (colon >= 0) ? blobKey.substring(colon + 1) : blobKey; blobKey = name + ":" + digest; // write blob data Map<String, String> blobMap = new HashMap<>(); blobMap.put(KEY, blobKey); blobMap.put(MIMETYPE, blob.getMimeType()); blobMap.put(ENCODING, blob.getEncoding()); blobMap.put(FILENAME, blob.getFilename()); blobMap.put(LENGTH, String.valueOf(size)); blobMap.put(DIGEST, digest); kvs.put(key + DOT_BLOB_DOT + i, toJson(blobMap), ttl); i++; } Map<String, String> blobInfoMap = new HashMap<>(); blobInfoMap.put(COUNT, String.valueOf(blobs.size())); blobInfoMap.put(SIZE, String.valueOf(totalSize)); kvs.put(key + DOT_BLOBINFO, toJson(blobInfoMap), ttl); addStorageSize(totalSize); markEntryExists(key); }
Alternate Patch #1 - return blobKey not digest from BinaryBlobProvider.writeBlob()
@Override public String writeBlob(Blob blob) throws IOException { // writes the blob and return its blobKey Binary binary = binaryManager.getBinary(blob); String digest = binary.getDigest(); return binary.getProviderId() + ":" + digest; }
Alternate Patch #2 - create BinaryBlob with calculated blobKey within BinaryBlobProvider.readBlob()
@Override public Blob readBlob(BlobInfo blobInfo) throws IOException { String digest = blobInfo.key; // strip prefix int colon = digest.indexOf(':'); if (colon >= 0) { digest = digest.substring(colon + 1); } Binary binary = binaryManager.getBinary(digest); if (binary == null) { throw new IOException("Unknown binary: " + digest); } String blobKey = binary.getProviderId() + ":" + digest; long length; if (blobInfo.length == null) { log.debug("Missing blob length for: " + blobInfo.key); // to avoid crashing, get the length from the binary's file (may be costly) File file = binary.getFile(); length = file == null ? -1 : file.length(); } else { length = blobInfo.length.longValue(); } return new BinaryBlob(binary, blobKey, blobInfo.filename, blobInfo.mimeType, blobInfo.encoding, blobInfo.digest, length); }