[D] how to parallelize optimal parameter search for DL NNs on multiple datasets?

Reddit r/MachineLearning / 3/16/2026

💬 OpinionDeveloper Stack & InfrastructureIdeas & Deep AnalysisTools & Practical UsageModels & Research

共有:

Key Points

The post describes having 11 datasets and 5 DL networks, each with its own set of non-DL hyperparameters, and wanting to exhaustively try all combinations across models and datasets.
The author asks how to parallelize this workload on a single GPU, noting that GPUs cannot be split like CPU cores and expressing willingness to run computations overnight.
Two-phase workflow is described (learning and predicting) with a model checkpoint artifact that would need unique suffixes to prevent overwriting when running in parallel.
They consider expanding the search to hyperparameters like epochs and tolerance and seek advice on efficient strategies.

suppose i have 5 and 6 datasets, 11 in total.

then i have a collection of 5 different deep learning networks, each having their own set of free non-DL parameters, ranging from none to 3-4.

imagine i have a list of educated guesses for each parameter (5-6 values) and i wanna try all their combinations for each DL method on each dataset. i’m okay with leaving it computing overnight. how would you approach this problem? is there a way to compute these non-sequentially/in parallel with a single GPU?

* each run has 2 phases: learning and predicting, and there’s the model checkpoint artifact that’s passed between them. i guess these have to now be assigned special suffixes so they don’t get overwritten.

* the main issue is a single GPU. i don’t think there’s a way to “split” the GPU as you can do with CPU that has logical cores. i’ve completed this task for non-DL/NN methods where each of 11 datasets occupied 1 core. seems like the GPU will become a bottleneck.

* should i also try to sweep the DL parameters like epochs, tolerance, etc?

does anyone have any advice on how to do this efficiently?

submitted by /u/Mampacuk
[link] [comments]