Key-Vary Partitions

Calculating partition measurement and Discovering the center key

Getting the scale of the partition and discovering the center secret is dependent
on what storage engines are getting used. A easy approach of dong this
might be to only scan by your entire partition to calculate its measurement.
TiKV initially used this strategy.
To have the ability to break up the pill, the important thing which is located
on the mid level must be discovered as properly. To keep away from scanning by
the partition twice, a easy implementation can get the center
key if the scale is greater than the configured most.

class Partition…

  public String getMiddleKeyIfSizeCrossed(int partitionMaxSize) {
      int kvSize = 0;
      for (String key : kv.keySet()) {
          kvSize += key.size() + kv.get(key).size();
          if (kvSize >= partitionMaxSize / 2) {
              return key;
          }
      }
      return "";
  }

The coordinator, dealing with the break up set off message replace the
key vary metadata for the unique partition,
and creates a brand new partition metadata for the break up vary.

class ClusterCoordinator…

  non-public void handleSplitTriggerMessage(SplitTriggerMessage message) {
      logger.information("Dealing with SplitTriggerMessage " + message.getPartitionId() + " break up key " + message.getSplitKey());
      splitPartition(message.getPartitionId(), message.getSplitKey());
  }

  public CompletableFuture splitPartition(int partitionId, String splitKey) {
      logger.information("Splitting partition " + partitionId + " at key " + splitKey);
      PartitionInfo parentPartition = partitionTable.getPartition(partitionId);
      Vary originalRange = parentPartition.getRange();
      Record<Vary> splits = originalRange.break up(splitKey);
      Vary shrunkOriginalRange = splits.get(0);
      Vary newRange = splits.get(1);
      return replicatedLog.suggest(new SplitPartitionCommand(partitionId, splitKey, shrunkOriginalRange, newRange));
  }

After the partitions metadata is saved efficiently, it
sends a message to the cluster node that’s internet hosting the mum or dad partition
to separate the mum or dad partition’s knowledge.

class ClusterCoordinator…

  non-public void applySplitPartitionCommand(SplitPartitionCommand command) {
      PartitionInfo originalPartition = partitionTable.getPartition(command.getOriginalPartitionId());
      Vary originalRange = originalPartition.getRange();
      if (!originalRange.coveredBy(command.getUpdatedRange().getStartKey(), command.getNewRange().getEndKey())) {
          logger.error("The unique vary begin and finish keys "+ originalRange + " don't match break up ranges");
          return;
      }

      originalPartition.setRange(command.getUpdatedRange());
      PartitionInfo newPartitionInfo = new PartitionInfo(newPartitionId(), originalPartition.getAddress(), PartitionStatus.ASSIGNED, command.getNewRange());
      partitionTable.addPartition(newPartitionInfo.getPartitionId(), newPartitionInfo);

      //ship requests to cluster nodes if that is the chief node.
      if (isLeader()) {
          var message = new SplitPartitionMessage(command.getOriginalPartitionId(), command.getSplitKey(), newPartitionInfo, requestNumber++, listenAddress);
          scheduler.execute(new RetryableTask(originalPartition.getAddress(), community, this, originalPartition.getPartitionId(), message));
      }
  }

class Vary…

  public boolean coveredBy(String startKey, String endKey) {
      return getStartKey().equals(startKey)
              && getEndKey().equals(endKey);
  }

The cluster node splits the unique partition and creates a brand new partition.
The info from the unique partition is then copied to the brand new partition.
It then responds to the coordinator telling that the break up is full.

class KVStore…

  non-public void handleSplitPartitionMessage(SplitPartitionMessage splitPartitionMessage) {
      splitPartition(splitPartitionMessage.getPartitionId(),
                                  splitPartitionMessage.getSplitKey(),
                                  splitPartitionMessage.getSplitPartitionId());
      community.ship(coordLeader,
              new SplitPartitionResponseMessage(splitPartitionMessage.getPartitionId(),
                      splitPartitionMessage.getPartitionId(),
                      splitPartitionMessage.getSplitPartitionId(),
                      splitPartitionMessage.messageId, listenAddress));
  }

  non-public void splitPartition(int parentPartitionId, String splitKey, int newPartitionId) {
      Partition partition = allPartitions.get(parentPartitionId);
      Partition splitPartition = partition.splitAt(splitKey, newPartitionId);
      logger.information("Including new partition " + splitPartition.getId() + " for vary " + splitPartition.getRange());
      allPartitions.put(splitPartition.getId(), splitPartition);
  }

class Partition…

  public Partition splitAt(String splitKey, int newPartitionId) {
      Record<Vary> splits = this.vary.break up(splitKey);
      Vary shrunkOriginalRange = splits.get(0);
      Vary splitRange = splits.get(1);

      SortedMap<String, String> partition1Kv =
              (vary.getStartKey().equals(Vary.MIN_KEY))
                      ? kv.headMap(splitKey)
                      : kv.subMap(vary.getStartKey(), splitKey);

      SortedMap<String, String> partition2Kv =
              (vary.getEndKey().equals(Vary.MAX_KEY))
                      ? kv.tailMap(splitKey)
                      : kv.subMap(splitKey, vary.getEndKey());

      this.kv = partition1Kv;
      this.vary = shrunkOriginalRange;

      return new Partition(newPartitionId, partition2Kv, splitRange);
  }

class Vary…

  public Record<Vary> break up(String splitKey) {
      return Arrays.asList(new Vary(startKey, splitKey), new Vary(splitKey, endKey));
  }

As soon as the coordinator receives the message, it marks the partitions as on-line

class ClusterCoordinator…

  non-public void handleSplitPartitionResponse(SplitPartitionResponseMessage message) {
      replicatedLog.suggest(new UpdatePartitionStatusCommand(message.getPartitionId(), PartitionStatus.ONLINE));
  }

One of many doable points that may come up when making an attempt to switch
the prevailing partition is that
the shopper can not cache and at all times must get the newest partition
metadata earlier than it will possibly ship any requests to the cluster node.
Knowledge shops use Era Clock for partitions;
that is up to date each single time a partition is break up.
Any shopper requests with an older technology quantity can be rejected.
Shoppers can then reload the
partition desk from the coordinator and retry the request.
This ensures that purchasers that possess older metadata do not get
the improper outcomes.
YugabyteDB
chooses to create two separate new partitions and marks the unique
as defined of their
Automated desk splitting design..