更新时间:2023-12-02 20:12:10
您不必加载整个数据.您可以使用 DataSet 类逐段摄取数据. /p>
在您的GPU处理数字时,Tensorflow可以处理加载更多数据的任务.您可以按照以下步骤操作.
您可以在此处中查看示例>.
希望这会有所帮助.
I have a specific case where the networks are relatively tiny and for convergence and generalization matters I should maintain small batch sizes (e.g. 256), which leads to hundreds of batches to process per epoch.
Unfortunately, in this scenario batch, loading, and loss calculation becomes a bottleneck (as timeline
tool tells me).
In TensorFlow, you can write something like this to load the data on the GPU:
with tf.device('/gpu:0'):
train_data = tf.constant(train_data_numpy)
But if I pass train_data
to Keras Model.predict
or Model.fit
functions, I get the following error:
keras/engine/training.pyc in predict(self, x, batch_size, verbose)
1515 f = self.predict_function
1516 return self._predict_loop(f, ins,
-> 1517 batch_size=batch_size, verbose=verbose)
1518
1519 def train_on_batch(self, x, y,
keras/engine/training.pyc in _predict_loop(self, f, ins, batch_size, verbose)
1129 if verbose == 1:
1130 progbar = Progbar(target=samples)
-> 1131 batches = _make_batches(samples, batch_size)
1132 index_array = np.arange(samples)
1133 for batch_index, (batch_start, batch_end) in enumerate(batches):
keras/engine/training.pyc in _make_batches(size, batch_size)
368 A list of tuples of array indices.
369 """
--> 370 num_batches = int(np.ceil(size / float(batch_size)))
371 return [(i * batch_size, min(size, (i + 1) * batch_size))
372 for i in range(0, num_batches)]
AttributeError: 'Dimension' object has no attribute 'ceil'
Which makes sense, since Keras expects only NumPy-like arrays and lists of such.
Having said that, I also tried pyCUDA and cupy arrays, since they say to be NumPy-like... but those produce the following errors:
keras/engine/training.pyc in predict(self, x, batch_size, verbose)
1515 f = self.predict_function
1516 return self._predict_loop(f, ins,
-> 1517 batch_size=batch_size, verbose=verbose)
1518
1519 def train_on_batch(self, x, y,
keras/engine/training.pyc in _predict_loop(self, f, ins, batch_size, verbose)
1139 ins_batch = _slice_arrays(ins, batch_ids)
1140
-> 1141 batch_outs = f(ins_batch)
1142 if not isinstance(batch_outs, list):
1143 batch_outs = [batch_outs]
keras/backend/tensorflow_backend.pyc in __call__(self, inputs)
2266 updated = session.run(self.outputs + [self.updates_op],
2267 feed_dict=feed_dict,
-> 2268 **self.session_kwargs)
2269 return updated[:len(self.outputs)]
2270
tensorflow/python/client/session.pyc in run(self, fetches, feed_dict, options, run_metadata)
893 try:
894 result = self._run(None, fetches, feed_dict, options_ptr,
--> 895 run_metadata_ptr)
896 if run_metadata:
897 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
tensorflow/python/client/session.pyc in _run(self, handle, fetches, feed_dict, options, run_metadata)
1091 feed_handles[subfeed_t] = subfeed_val
1092 else:
-> 1093 np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
1094
1095 if (not is_tensor_handle_feed and
numpy/core/numeric.pyc in asarray(a, dtype, order)
529
530 """
--> 531 return array(a, dtype, copy=False, order=order)
532
533
ValueError: object __array__ method not producing an array
I tried googling this issue, but the only reasonable match is some Chinese blog post, which basically suggests patching Keras, which is impractical obviously.
I wonder what is the correct way to preload the whole dataset on GPU for Keras.
Useful info: I am using Keras 2.0.6 with TF 1.3, upgrading to 2.0.8/1.4 stack is yet unavailable due to crucial API changes, but would definitely be sped up in case it solves this issue.
You don't have to load the whole data. You can ingest the data piece by piece using the DataSet class.
Tensorflow can take care of loading more data while your gpu is crunching your numbers. You can follow the below steps.
You can check the example listed here.
Hope this is helpful.