POSTED BY
2023-10-08

【Windows】Python3.10+TensorFlow2.6-GPU+CUDA11.4+cuDNN8.2を動かす【１】 【Windows】Python3.10+TensorFlow2.6-GPU+CUDA11.4+cuDNN8.2を動かす【２】 【Windows】Python3.10+TensorFlow2.6-GPU+CUDA11.4+cuDNN8.2を動かす【３】

すべて最新のもので固められてGPUも無事認識できたので、サンプルを動かしてみます。

OREILLYジャパンの書籍

生成 Deep Learning ―絵を描き、物語や音楽を作り、ゲームをプレイする David Foster

の、最初のサンプルーCIFAR10画像の判別コードーを実装してみます。

#ライブラリ

import numpy as np
import tensorflow as tf
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.layers import Input, Flatten, Dense, Activation
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt

#データロード

NUM_CLASSES = 10

(x_train, y_train), (x_test, y_test) = cifar10.load_data()

x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

y_train = to_categorical(y_train, NUM_CLASSES)
y_test = to_categorical(y_test, NUM_CLASSES)

data = np.random.randint(low=0, high=5, size=10)
print(data)
print(to_categorical(data))
print(to_categorical(data, 8))

#モデル定義・コンパイルをMirroredStrategy withブロック内で行い、以降のトレーニングをデュアルGPUで行う

with tf.distribute.MirroredStrategy(cross_device_ops=tf.distribute.HierarchicalCopyAllReduce()).scope():

  input_layer = Input(shape=(32, 32, 3))
  x = Flatten()(input_layer)
  x = Dense(units=200, activation='relu')(x)
  x = Dense(units=150, activation='relu')(x)

  output_layer = Dense(units=10, activation='softmax')(x)

  output_layer2 = Dense(units=200)(x)
  output_layer2 = Activation('relu')(output_layer2)
  model = Model(input_layer, output_layer)
  model.summary()

  opt = Adam(lr=0.0005)
  model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])

#トレーニング実行

model.fit(x_train,
          y_train,
          batch_size = 32,
          epochs = 10,
          shuffle = True)

model.evaluate(x_test, y_test, batch_size=32)

#予測インデックステスト

CLASSES = np.array(['AirPlane', 'Car','Bird','Cat', 'Deer','Dog','Flog','Horse','Ship','Truck'])
preds = model.predict(x_test)
preds_single = CLASSES[np.argmax(preds, axis= -1)]
actual_single = CLASSES[np.argmax(y_test, axis = -1)]

print(preds[0])
print(np.argmax(preds[0]))
print(preds_single[0])
print(actual_single[0])

#予測画像リスト表示テスト

n_to_show = 10
indices = np.random.choice(range(len(x_test)), n_to_show)
fig = plt.figure(figsize=(15, 3))
fig.subplots_adjust(hspace=0.4, wspace=0.4)

for i, idx in enumerate(indices):
  img = x_test[idx]
  ax = fig.add_subplot(1, n_to_show, i+1)
  ax.axis('off')
  ax.text(0.5, -0.35, 'pred = ' + str(preds_single[idx]), fontsize=10, ha='center', transform=ax.transAxes) 
  ax.text(0.5, -0.7, 'act = ' + str(actual_single[idx]), fontsize=10, ha='center', transform=ax.transAxes)
  ax.imshow(img)

複数GPUを使わせるため、モデルの定義～コンパイルまでのブロックをインデントして、

import tensorflow as tf
with tf.distribute.MirroredStrategy(cross_device_ops=tf.distribute.HierarchicalCopyAllReduce()).scope():

の中に入れます。jupyter qtconsole にコメントブロック単位に貼り付け実行した結果は以下です。

Jupyter QtConsole 5.1.1
Python 3.10.0 (tags/v3.10.0:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.28.0 -- An enhanced Interactive Python. Type '?' for help.

[3 1 1 1 3 1 1 1 2 3]
[[0. 0. 0. 1.]
 [0. 1. 0. 0.]
 [0. 1. 0. 0.]
 [0. 1. 0. 0.]
 [0. 0. 0. 1.]
 [0. 1. 0. 0.]
 [0. 1. 0. 0.]
 [0. 1. 0. 0.]
 [0. 0. 1. 0.]
 [0. 0. 0. 1.]]
[[0. 0. 0. 1. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0. 0. 0.]]

INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1')
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 32, 32, 3)]       0         
_________________________________________________________________
flatten (Flatten)            (None, 3072)              0         
_________________________________________________________________
dense (Dense)                (None, 200)               614600    
_________________________________________________________________
dense_1 (Dense)              (None, 150)               30150     
_________________________________________________________________
dense_2 (Dense)              (None, 10)                1510      
=================================================================
Total params: 646,260
Trainable params: 646,260
Non-trainable params: 0
_________________________________________________________________
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).

Epoch 1/10
INFO:tensorflow:batch_all_reduce: 6 all-reduces with algorithm = hierarchical_copy, num_packs = 1
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:batch_all_reduce: 6 all-reduces with algorithm = hierarchical_copy, num_packs = 1
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
1563/1563 [==============================] - 18s 8ms/step - loss: 1.8522 - accuracy: 0.3292
Epoch 2/10
1563/1563 [==============================] - 13s 9ms/step - loss: 1.6648 - accuracy: 0.4022
Epoch 3/10
1563/1563 [==============================] - 12s 8ms/step - loss: 1.5831 - accuracy: 0.4331
Epoch 4/10
1563/1563 [==============================] - 13s 8ms/step - loss: 1.5281 - accuracy: 0.4542
Epoch 5/10
1563/1563 [==============================] - 11s 7ms/step - loss: 1.4924 - accuracy: 0.4691
Epoch 6/10
1563/1563 [==============================] - 12s 8ms/step - loss: 1.4623 - accuracy: 0.4760
Epoch 7/10
1563/1563 [==============================] - 13s 8ms/step - loss: 1.4352 - accuracy: 0.4879
Epoch 8/10
1563/1563 [==============================] - 12s 7ms/step - loss: 1.4109 - accuracy: 0.4974
Epoch 9/10
1563/1563 [==============================] - 12s 8ms/step - loss: 1.3870 - accuracy: 0.5067
Epoch 10/10
1563/1563 [==============================] - 10s 7ms/step - loss: 1.3680 - accuracy: 0.5142
313/313 [==============================] - 3s 5ms/step - loss: 1.4446 - accuracy: 0.4915
Out[4]: [1.4446195363998413, 0.49149999022483826]

[0.10204831 0.21945089 0.03836511 0.33486548 0.09174908 0.12024193
 0.02515405 0.00319037 0.05551995 0.00941479]
3
Cat
Cat

無事トレーニングと予測表示ができています。