Mask R-CNN Load_weights函数在带有 tensorflow.compat.v1 的 Google Colab 中不起作用

我认为 Mask R-CNN 需要 TensorFlow 1.X，而 Colab 的默认 Python 版本（3.10）不支持

您可以使用此实现，该实现构建在原始 Mask R-CNN 存储库之上，以支持 TF2。此存储库允许使用 TensorFlow 2.14.0 和 Python 3.10.12 训练和测试 Mask R-CNN 模型。

您也可以在 Google Colab 上使用它（当前的 colab 环境也使用 Python 3.10.12 和 TF 2.14.0），它在 GPU 上没有任何问题。请确保您的运行时使用的是 GPU：

然后按照以下确切步骤操作：

# Clone the repo
!git clone https://github.com/z-mahmud22/Mask-RCNN_TF2.14.0.git maskrcnn  

# Change the runtime directory to the cloned repo
import os 
os.chdir('/content/maskrcnn/')

# Download pre-trained weights
!wget https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5

然后使用此代码片段将权重加载到 Mask R-CNN 模型中：

import mrcnn
import mrcnn.config
import mrcnn.model

# create a config file

CLASS_NAMES = ['BG', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush']

class SimpleConfig(mrcnn.config.Config):
    # Give the configuration a recognizable name
    NAME = "coco_inference"
    
    # set the number of GPUs to use along with the number of images per GPU
    GPU_COUNT = 1
    IMAGES_PER_GPU = 1

    # Number of classes = number of classes + 1 (+1 for the background). The background class is named BG
    NUM_CLASSES = len(CLASS_NAMES)

# Initialize the Mask R-CNN model for inference and then load the weights.
# This step builds the Keras model architecture.
model = mrcnn.model.MaskRCNN(mode="inference", 
                             config=SimpleConfig(),
                             model_dir=os.getcwd())

# Load the weights into the model
model.load_weights(filepath="mask_rcnn_coco.h5", 
                   by_name=True)

如果您能正确地遵循所有这些步骤，您应该能够毫无问题地加载预先训练的权重，并通过以下方式验证权重的变化：

from mrcnn import visualize
visualize.display_weight_stats(model)

打印出来：

# Showing the first 10 layers as done in the question
WEIGHT NAME SHAPE   MIN MAX STD
conv1/kernel:0  (7, 7, 3, 64)   -0.8616 +0.8451 +0.1315
conv1/bias:0    (64,)   -0.0002 +0.0004 +0.0001
bn_conv1/gamma:0    (64,)   +0.0835 +2.6411 +0.5091
bn_conv1/beta:0 (64,)   -2.3931 +5.3610 +1.9781
bn_conv1/moving_mean:0  (64,)   -173.0470   +116.3013   +44.5654
bn_conv1/moving_variance:0*** Overflow? (64,)   +0.0000 +146335.3594    +21847.9668
res2a_branch2a/kernel:0 (1, 1, 64, 64)  -0.6574 +0.3179 +0.0764
res2a_branch2a/bias:0   (64,)   -0.0022 +0.0082 +0.0018
bn2a_branch2a/gamma:0   (64,)   +0.2169 +1.8489 +0.4116
bn2a_branch2a/beta:0    (64,)   -2.1180 +3.7332 +1.1786 

下面是一个片段，用于可视化来自预训练的 Mask R-CNN 的预测：

import cv2
import mrcnn.visualize
# load the input image, convert it from BGR to RGB channel
image = cv2.imread("test.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Perform a forward pass of the network to obtain the results
r = model.detect([image], verbose=0)

# Get the results for the first image.
r = r[0]

# Visualize the detected objects.
mrcnn.visualize.display_instances(image=image, 
                                  boxes=r['rois'], 
                                  masks=r['masks'], 
                                  class_ids=r['class_ids'], 
                                  class_names=CLASS_NAMES, 
                                  scores=r['scores'])

其结果为：