Uploaded image for project: 'Nuxeo Platform'
  1. Nuxeo Platform
  2. NXP-26044

Embed providerId into blobKey within KeyValueBlobTransientStore.putBlobs()

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Not A Bug
    • Affects Version/s: 9.10, 10.3-SNAPSHOT
    • Fix Version/s: None
    • Component/s: TransientStore
    • Tags:
    • Backlog priority:
      600

      Description

      KeyValueBlobTransientStore.putBlobs() does not currently include the providerId in the blobKey.

      A conservative, 4-line patch to KeyValueBlobTransientStore that fixes the problem is included below.

      Alternate, more invasive patches to BinaryBlobProvider that achieve the same result are also included below.

      Thanks,

      Ron

      Patch

      KeyValueBlobTransientStore.java
      
          @Override
          public void putBlobs(String key, List<Blob> blobs) {
              if (absoluteMaxSize > 0 && getStorageSize() > absoluteMaxSize) {
                  // do the costly computation of the exact storage size if needed
                  doGC();
                  if (getStorageSize() > absoluteMaxSize) {
                      throw new MaximumTransientSpaceExceeded();
                  }
              }
      
              // remove previous blobs
              removeBlobs(key);
      
              KeyValueStore kvs = getKeyValueStore();
              BlobProvider bp = getBlobProvider();
              long totalSize = 0;
              int i = 0;
              for (Blob blob : blobs) {
                  long size = blob.getLength();
                  if (size >= 0) {
                      totalSize += size;
                  }
                  // store blob
                  String blobKey;
                  try {
                      blobKey = bp.writeBlob(blob);
                  } catch (IOException e) {
                      throw new NuxeoException(e);
                  }
                  int colon = blobKey.indexOf(':');
                  String digest = (colon >= 0) ? blobKey.substring(colon + 1) : blobKey;
                  blobKey = name + ":" + digest;
                  // write blob data
                  Map<String, String> blobMap = new HashMap<>();
                  blobMap.put(KEY, blobKey);
                  blobMap.put(MIMETYPE, blob.getMimeType());
                  blobMap.put(ENCODING, blob.getEncoding());
                  blobMap.put(FILENAME, blob.getFilename());
                  blobMap.put(LENGTH, String.valueOf(size));
                  blobMap.put(DIGEST, digest);
                  kvs.put(key + DOT_BLOB_DOT + i, toJson(blobMap), ttl);
                  i++;
              }
              Map<String, String> blobInfoMap = new HashMap<>();
              blobInfoMap.put(COUNT, String.valueOf(blobs.size()));
              blobInfoMap.put(SIZE, String.valueOf(totalSize));
              kvs.put(key + DOT_BLOBINFO, toJson(blobInfoMap), ttl);
              addStorageSize(totalSize);
              markEntryExists(key);
          }
      
      

      Alternate Patch #1 - return blobKey not digest from BinaryBlobProvider.writeBlob()

      BinaryBlobProvider.java
      
          @Override
          public String writeBlob(Blob blob) throws IOException {
              // writes the blob and return its blobKey
              Binary binary = binaryManager.getBinary(blob);
              String digest = binary.getDigest();
              return binary.getProviderId() + ":" + digest;
          }
      
      

      Alternate Patch #2 - create BinaryBlob with calculated blobKey within BinaryBlobProvider.readBlob()

      BinaryBlobProvider.java
      
          @Override
          public Blob readBlob(BlobInfo blobInfo) throws IOException {
              String digest = blobInfo.key;
              // strip prefix
              int colon = digest.indexOf(':');
              if (colon >= 0) {
                  digest = digest.substring(colon + 1);
              }
              Binary binary = binaryManager.getBinary(digest);
              if (binary == null) {
                  throw new IOException("Unknown binary: " + digest);
              }
              String blobKey = binary.getProviderId() + ":" + digest;
              long length;
              if (blobInfo.length == null) {
                  log.debug("Missing blob length for: " + blobInfo.key);
                  // to avoid crashing, get the length from the binary's file (may be costly)
                  File file = binary.getFile();
                  length = file == null ? -1 : file.length();
              } else {
                  length = blobInfo.length.longValue();
              }
              return new BinaryBlob(binary, blobKey, blobInfo.filename, blobInfo.mimeType, blobInfo.encoding,
                      blobInfo.digest, length);
          }
      

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 10 minutes
                10m