Tuning leads to All-NaN axis encountered after long time

[GhostDog98]

Describe the bug
After running a bunch of tunings, I eventually get the following error:

Search: Running Trial #68

Value             |Best Value So Far |Hyperparameter
4                 |1                 |num_layers
32                |96                |lstm_units_0
64                |64                |dense_units
1e-05             |3.9608e-05        |learning_rate
32                |32                |lstm_units_1
32                |128               |lstm_units_2
96                |32                |lstm_units_3

Epoch 1/100
I0000 00:00:1734570907.031189 1848349 service.cc:146] XLA service 0x76d5d4004ae0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
I0000 00:00:1734570907.031210 1848349 service.cc:154]   StreamExecutor device (0): NVIDIA GeForce RTX 3060, Compute Capability 8.6
2024-12-19 12:15:07.104318: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:268] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-19 12:15:07.423862: I external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:531] Loaded cuDNN version 90300
I0000 00:00:1734570908.886075 1848349 device_compiler.h:188] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 46s 33ms/step - loss: 187201.1562 - mae: 192.5935 - val_loss: 64136.8945 - val_mae: 117.8278
Epoch 2/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 40s 31ms/step - loss: 53528.4414 - mae: 98.4317 - val_loss: 30422.4727 - val_mae: 68.2864
Epoch 3/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 21780.5391 - mae: 56.0624 - val_loss: 12904.9023 - val_mae: 43.0659
Epoch 4/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 10389.4004 - mae: 40.3216 - val_loss: 7285.5630 - val_mae: 36.6022
Epoch 5/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 7087.9570 - mae: 35.0648 - val_loss: 6534.5283 - val_mae: 34.3760
Epoch 6/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 5333.1699 - mae: 31.5071 - val_loss: 4140.5713 - val_mae: 28.6819
Epoch 7/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 31ms/step - loss: 4137.9907 - mae: 28.6213 - val_loss: 2916.1416 - val_mae: 23.9534
Epoch 8/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 2356.5020 - mae: 22.2533 - val_loss: 1953.6709 - val_mae: 20.1413
Epoch 9/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 1712.8751 - mae: 19.4396 - val_loss: 1503.3846 - val_mae: 18.8406
Epoch 10/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 1390.0425 - mae: 17.8179 - val_loss: 1104.1555 - val_mae: 16.9162
Epoch 11/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 31ms/step - loss: 1076.9076 - mae: 16.1942 - val_loss: 922.5842 - val_mae: 15.8216
Epoch 12/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 1164.0901 - mae: 16.3073 - val_loss: 1144.8092 - val_mae: 16.7875
Epoch 13/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 1166.4641 - mae: 16.5099 - val_loss: 994.0040 - val_mae: 15.5358
Epoch 14/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 31ms/step - loss: 972.1603 - mae: 15.1948 - val_loss: 1331.9351 - val_mae: 17.0246
Epoch 15/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 30ms/step - loss: 1171.1146 - mae: 15.8957 - val_loss: 1663.4587 - val_mae: 19.4891
Epoch 16/100
1275/1275 ━━━━━━━━━━━━━━━━━━━━ 39s 31ms/step - loss: 1868.6052 - mae: 20.3558 - val_loss: 2075.2771 - val_mae: 23.2808
/home/ghostdog/mambaforge/lib/python3.10/site-packages/keras_tuner/src/engine/metrics_tracking.py:111: RuntimeWarning: All-NaN axis encountered
  np.nanmin(values) if self.direction == "min" else np.nanmax(values)
Traceback (most recent call last):
  File "/home/ghostdog/hypertuner.py", line 103, in <module>
    tuner.search(train_dataset, epochs=100, validation_data=val_dataset, callbacks=[early_stopping])
  File "/home/ghostdog/mambaforge/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 235, in search
    self.on_trial_end(trial)
  File "/home/ghostdog/mambaforge/lib/python3.10/site-packages/keras_tuner/src/engine/base_tuner.py", line 339, in on_trial_end
    self.oracle.end_trial(trial)
  File "/home/ghostdog/mambaforge/lib/python3.10/site-packages/keras_tuner/src/engine/oracle.py", line 108, in wrapped_func
    ret_val = func(*args, **kwargs)
  File "/home/ghostdog/mambaforge/lib/python3.10/site-packages/keras_tuner/src/engine/oracle.py", line 588, in end_trial
    self._check_consecutive_failures()
  File "/home/ghostdog/mambaforge/lib/python3.10/site-packages/keras_tuner/src/engine/oracle.py", line 545, in _check_consecutive_failures
    raise RuntimeError(
RuntimeError: Number of consecutive failures exceeded the limit of 3.

Even if i restart the application to resume from my last checkpoint, it leads to this.

To Reproduce
My relevent code is as such:

# Define the model-building function for KerasTuner
def build_model(hp):
    model = Sequential()

    # Hyperparameter tuning for the number of LSTM layers and units
    for i in range(hp.Int('num_layers', 1, 4)):
        model.add(LSTM(units=hp.Int(f'lstm_units_{i}', min_value=32, max_value=128, step=32),
                       activation='relu', return_sequences=True if i < hp.Int('num_layers', 1, 4) - 1 else False))

    # Dense layer
    model.add(Dense(units=hp.Int('dense_units', min_value=32, max_value=128, step=32), activation='relu'))

    # Hyperparameter tuning for output size (e.g., prediction horizon)
    output_size = 4  # You can set this dynamically if needed
    model.add(Dense(output_size))  # Output size should be set here

    # Compile the model
    model.compile(optimizer=Adam(learning_rate=hp.Float('learning_rate', min_value=1e-5, max_value=1e-2, sampling='LOG')),
                  loss='mean_squared_error', metrics=['mae'])

    return model

# Initialize the tuner
tuner = kt.BayesianOptimization(build_model,
                     objective='val_loss',
                     max_trials=1000,
                     directory='kerastuner_results',
                     project_name='prediction',
                     )

# Define early stopping callback
early_stopping = EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

# Create tf.data.Dataset for training and validation
batch_size = 128  # Adjust batch size as needed, may lead to less stable searches... defaults to 32
train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train)).batch(batch_size).prefetch(tf.data.experimental.AUTOTUNE)
val_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test)).batch(batch_size).prefetch(tf.data.experimental.AUTOTUNE)

# Perform hyperparameter search
tuner.search(train_dataset, epochs=100, validation_data=val_dataset, callbacks=[early_stopping])

Expected behavior
The tuner continues to tune over time without crashing.

Additional context
My Last successful trials trial.json (67):

{"trial_id": "0067", "hyperparameters": {"space": [{"class_name": "Int", "config": {"name": "num_layers", "default": null, "conditions": [], "min_value": 1, "max_value": 4, "step": 1, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "lstm_units_0", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "dense_units", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Float", "config": {"name": "learning_rate", "default": 1e-05, "conditions": [], "min_value": 1e-05, "max_value": 0.01, "step": null, "sampling": "log"}}, {"class_name": "Int", "config": {"name": "lstm_units_1", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "lstm_units_2", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "lstm_units_3", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}], "values": {"num_layers": 4, "lstm_units_0": 32, "dense_units": 64, "learning_rate": 1e-05, "lstm_units_1": 32, "lstm_units_2": 32, "lstm_units_3": 96}}, "metrics": {"metrics": {"loss": {"direction": "min", "observations": [{"value": [NaN], "step": 0}]}, "mae": {"direction": "min", "observations": [{"value": [NaN], "step": 0}]}, "val_loss": {"direction": "min", "observations": [{"value": [NaN], "step": 0}]}, "val_mae": {"direction": "min", "observations": [{"value": [NaN], "step": 0}]}}}, "score": null, "best_step": 0, "status": "RUNNING", "message": null}

And a trial from the middle of the training, just in case:

{"trial_id": "0020", "hyperparameters": {"space": [{"class_name": "Int", "config": {"name": "num_layers", "default": null, "conditions": [], "min_value": 1, "max_value": 4, "step": 1, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "lstm_units_0", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "dense_units", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Float", "config": {"name": "learning_rate", "default": 1e-05, "conditions": [], "min_value": 1e-05, "max_value": 0.01, "step": null, "sampling": "log"}}, {"class_name": "Int", "config": {"name": "lstm_units_1", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "lstm_units_2", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}, {"class_name": "Int", "config": {"name": "lstm_units_3", "default": null, "conditions": [], "min_value": 32, "max_value": 128, "step": 32, "sampling": "linear"}}], "values": {"num_layers": 2, "lstm_units_0": 64, "dense_units": 96, "learning_rate": 0.0012899513615134586, "lstm_units_1": 32, "lstm_units_2": 128, "lstm_units_3": 32}}, "metrics": {"metrics": {"loss": {"direction": "min", "observations": [{"value": [0.04608340561389923], "step": 9}]}, "mae": {"direction": "min", "observations": [{"value": [0.1634569764137268], "step": 9}]}, "val_loss": {"direction": "min", "observations": [{"value": [0.027314456179738045], "step": 9}]}, "val_mae": {"direction": "min", "observations": [{"value": [0.12323067337274551], "step": 9}]}}}, "score": 0.027314456179738045, "best_step": 9, "status": "COMPLETED", "message": null}

All of my installed packages and versions (click to expand)

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
absl-py                   2.1.0              pyhd8ed1ab_0    conda-forge
aiobotocore               1.2.2              pyhd3eb1b0_0    anaconda
aiohappyeyeballs          2.4.4                    pypi_0    pypi
aiohttp                   3.11.9                   pypi_0    pypi
aioitertools              0.11.0             pyhd8ed1ab_0    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
alabaster                 0.7.16             pyhd8ed1ab_0    conda-forge
alembic                   1.14.0                   pypi_0    pypi
alsa-lib                  1.2.12               h4ab18f5_0    conda-forge
anaconda-anon-usage       0.4.4           py310hfc0e8ea_100  
anaconda-client           1.12.3             pyhd8ed1ab_0    conda-forge
anaconda-cloud-auth       0.4.1              pyhd8ed1ab_0    conda-forge
anaconda-project          0.11.1             pyhd8ed1ab_0    conda-forge
androguard                4.1.0                    pypi_0    pypi
annotated-types           0.6.0              pyhd8ed1ab_0    conda-forge
anthropic                 0.40.0                   pypi_0    pypi
antlr4-python3-runtime    4.9.3                    pypi_0    pypi
anyio                     4.3.0              pyhd8ed1ab_0    conda-forge
aom                       3.8.2                h59595ed_0    conda-forge
apkinspector              1.2.1                    pypi_0    pypi
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py310h2372a71_4    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asn1crypto                1.5.1                    pypi_0    pypi
astor                     0.8.1              pyh9f0ad1d_0    conda-forge
astroid                   3.0.3           py310hff52083_0    conda-forge
astropy-iers-data         0.2024.3.18.0.29.47    pyhd8ed1ab_0    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
async-timeout             4.0.3              pyhd8ed1ab_0    conda-forge
atomicwrites              1.4.1              pyhd8ed1ab_0    conda-forge
atpublic                  4.0                      pypi_0    pypi
attr                      2.5.1                h166bdaf_1    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
automat                   22.10.0            pyhd8ed1ab_0    conda-forge
autopep8                  2.0.4              pyhd8ed1ab_0    conda-forge
aws-c-auth                0.8.0                h56a2c13_4    conda-forge
aws-c-cal                 0.8.0                hd3f4568_0    conda-forge
aws-c-common              0.9.31               hb9d3cd8_0    conda-forge
aws-c-compression         0.3.0                hf20e7d7_0    conda-forge
aws-c-event-stream        0.5.0                h68c3b0c_2    conda-forge
aws-c-http                0.9.0                hfad4ed3_3    conda-forge
aws-c-io                  0.15.0               h17eb868_2    conda-forge
aws-c-mqtt                0.11.0               h407ecb8_2    conda-forge
aws-c-s3                  0.7.0                hadeddc1_5    conda-forge
aws-c-sdkutils            0.2.0                hf20e7d7_0    conda-forge
aws-checksums             0.2.0                hf20e7d7_0    conda-forge
aws-crt-cpp               0.29.0               h73f0fd4_6    conda-forge
aws-sdk-cpp               1.11.407             h6a6dca0_6    conda-forge
azure-core-cpp            1.14.0               h5cfcd09_0    conda-forge
azure-identity-cpp        1.10.0               h113e628_0    conda-forge
azure-storage-blobs-cpp   12.13.0              h3cf044e_1    conda-forge
azure-storage-common-cpp  12.8.0               h736e048_1    conda-forge
azure-storage-files-datalake-cpp 12.12.0              ha633028_1    conda-forge
babel                     2.14.0             pyhd8ed1ab_0    conda-forge
banal                     1.0.6                    pypi_0    pypi
base58                    2.1.1              pyhd8ed1ab_0    conda-forge
bcrypt                    4.1.2           py310hcb5633a_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
binaryornot               0.4.4                      py_1    conda-forge
black                     24.3.0          py310hff52083_0    conda-forge
blas                      2.116                       mkl    conda-forge
blas-devel                3.9.0            16_linux64_mkl    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blinker                   1.7.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.6               hef167b5_0    conda-forge
boltons                   23.1.1             pyhd8ed1ab_0    conda-forge
boto3                     1.34.68            pyhd8ed1ab_0    conda-forge
botocore                  1.34.68         pyge310_1234567_0    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
brotli-python             1.1.0           py310hc6cd4ac_1    conda-forge
brotlicffi                1.1.0.0         py310hc6cd4ac_1    conda-forge
brunsli                   0.1                  h9c3ff4c_0    conda-forge
build                     1.2.1                    pypi_0    pypi
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.34.1               heb4867d_0    conda-forge
c-blosc2                  2.13.2               hb4ffafa_0    conda-forge
ca-certificates           2024.12.14           hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.3              pyhd8ed1ab_0    conda-forge
cairo                     1.18.0               hebfffa5_3    conda-forge
certifi                   2024.12.14         pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py310h2fee648_0    conda-forge
chardet                   5.2.0           py310hff52083_1    conda-forge
charls                    2.4.2                h59595ed_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
chess                     1.10.0                   pypi_0    pypi
click                     8.1.7           unix_pyh707e725_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
clyent                    1.2.2              pyhd8ed1ab_2    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
colorcet                  3.1.0              pyhd8ed1ab_0    conda-forge
colorlog                  6.9.0                    pypi_0    pypi
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
conda                     24.11.1         py310hff52083_0    conda-forge
conda-content-trust       0.2.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.1.0             pyhd8ed1ab_0    conda-forge
conda-pack                0.7.1              pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
conda-repo-cli            1.0.88          py310h06a4308_0  
conda-token               0.4.0              pyhd3eb1b0_0    anaconda
configspace               1.2.1                    pypi_0    pypi
constantly                15.1.0                     py_0    conda-forge
contourpy                 1.2.0           py310hd41b1e2_0    conda-forge
cookiecutter              2.6.0              pyhca7485f_0    conda-forge
cryptography              42.0.5          py310h75e40e8_0    conda-forge
cssselect                 1.2.0              pyhd8ed1ab_0    conda-forge
cuda-crt-tools            12.6.77              ha770c72_0    conda-forge
cuda-cudart               12.4.127                      0    nvidia
cuda-cudart_linux-64      12.6.77              h3f2d84a_0    conda-forge
cuda-cupti                12.4.127                      0    nvidia
cuda-libraries            12.4.1                        0    nvidia
cuda-nvcc-tools           12.6.77              he02047a_0    conda-forge
cuda-nvrtc                12.4.127                      0    nvidia
cuda-nvtx                 12.4.127                      0    nvidia
cuda-nvvm-tools           12.6.77              he02047a_0    conda-forge
cuda-opencl               12.6.77                       0    nvidia
cuda-python               12.6.0                   pypi_0    pypi
cuda-runtime              12.4.1                        0    nvidia
cuda-version              12.6                 h7480c83_3    conda-forge
cudf-cu12                 24.12.0                  pypi_0    pypi
cudnn                     9.3.0.75             h93bb076_0    conda-forge
cupy-cuda12x              13.3.0                   pypi_0    pypi
curl                      8.10.1               hbbe4b11_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
cython                    3.0.11          py310h5b1441d_3    conda-forge
cytoolz                   0.12.3          py310h2372a71_0    conda-forge
dask-core                 2024.3.1           pyhd8ed1ab_0    conda-forge
dask-jobqueue             0.9.0                    pypi_0    pypi
dataset                   1.6.2                    pypi_0    pypi
dav1d                     1.2.1                hd590300_0    conda-forge
dbus                      1.13.18              hb2f20db_0    anaconda
debugpy                   1.8.1           py310hc6cd4ac_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
diff-match-patch          20230430           pyhd8ed1ab_0    conda-forge
dill                      0.3.8              pyhd8ed1ab_0    conda-forge
distributed               2024.3.1           pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
dnspython                 2.6.1                    pypi_0    pypi
docstring-to-markdown     0.15               pyhd8ed1ab_0    conda-forge
docutils                  0.20.1          py310hff52083_3    conda-forge
emcee                     3.1.6                    pypi_0    pypi
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
et_xmlfile                1.1.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
expat                     2.6.3                h5888daf_0    conda-forge
fastrlock                 0.8.2                    pypi_0    pypi
ffmpeg                    4.4.0                h6987444_4    conda-forge
ffmpeg-python             0.2.0                    pypi_0    pypi
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
flake8                    7.0.0              pyhd8ed1ab_0    conda-forge
flask                     3.0.2              pyhd8ed1ab_0    conda-forge
flask-cors                5.0.0                    pypi_0    pypi
flatbuffers               24.3.25              h59595ed_0    conda-forge
flufl-lock                8.0.2                    pypi_0    pypi
fmt                       10.2.1               h00ab1b0_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_1    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-anaconda            1                    h8fa9717_0    anaconda
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.49.0                   pypi_0    pypi
fpstimer                  0.0.1                    pypi_0    pypi
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freeglut                  3.2.2                ha6d2627_3    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
frida                     16.2.1                   pypi_0    pypi
frozendict                2.4.6           py310ha75aee5_0    conda-forge
frozenlist                1.4.1           py310h2372a71_0    conda-forge
fsspec                    2024.10.0                pypi_0    pypi
future                    0.18.3                   pypi_0    pypi
gast                      0.5.5              pyhd8ed1ab_0    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
gguf                      0.10.0                   pypi_0    pypi
giflib                    5.2.2                hd590300_0    conda-forge
gitdb                     4.0.11             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.42             pyhd8ed1ab_0    conda-forge
glib                      2.82.1               h2ff4ddf_0    conda-forge
glib-tools                2.82.1               h4833e2c_0    conda-forge
glog                      0.7.1                hbabe93e_0    conda-forge
gmp                       6.3.0                h59595ed_1    conda-forge
gmpy2                     2.1.2           py310h3ec546c_1    conda-forge
gnutls                    3.6.13               h85f3911_1    conda-forge
google-api-core           2.24.0                   pypi_0    pypi
google-api-python-client  2.155.0                  pypi_0    pypi
google-auth               2.37.0                   pypi_0    pypi
google-auth-httplib2      0.2.0                    pypi_0    pypi
google-pasta              0.2.0              pyhd8ed1ab_1    conda-forge
googleapis-common-protos  1.66.0                   pypi_0    pypi
graphite2                 1.3.14               h295c915_1    anaconda
greenlet                  3.0.3           py310hc6cd4ac_0    conda-forge
grpcio                    1.65.5          py310h1a6248f_0    conda-forge
gst-plugins-base          1.24.7               h0a52356_0    conda-forge
gstreamer                 1.24.7               hf3bb09a_0    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
h5py                      3.11.0          nompi_py310hf054cd7_102    conda-forge
harfbuzz                  9.0.0                hda332d3_1    conda-forge
hdf5                      1.14.3          nompi_hdf9ad27_105    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.4              pyhd8ed1ab_0    conda-forge
httplib2                  0.22.0                   pypi_0    pypi
httpx                     0.27.0             pyhd8ed1ab_0    conda-forge
huggingface-hub           0.26.3                   pypi_0    pypi
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
hyperlink                 21.0.0             pyhd3deb0d_0    conda-forge
icu                       75.1                 he02047a_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
imath                     3.1.12               h7955e40_0    conda-forge
importlib-metadata        7.1.0              pyha770c72_0    conda-forge
importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
imutils                   0.5.4                    pypi_0    pypi
incremental               22.10.0            pyhd8ed1ab_0    conda-forge
inflection                0.5.1              pyh9f0ad1d_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
intake                    2.0.4              pyhd8ed1ab_0    conda-forge
intel-openmp              2022.0.1          h06a4308_3633  
intervaltree              3.1.0              pyhd8ed1ab_1    conda-forge
ipykernel                 6.29.3             pyhd33586a_0    conda-forge
ipython                   8.22.2             pyh707e725_0    conda-forge
ipywidgets                8.1.2              pyhd8ed1ab_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
isort                     5.13.2             pyhd8ed1ab_0    conda-forge
itemadapter               0.8.0              pyhd8ed1ab_0    conda-forge
itemloaders               1.1.0              pyhd8ed1ab_0    conda-forge
itsdangerous              2.1.2              pyhd8ed1ab_0    conda-forge
jaraco.classes            3.3.1              pyhd8ed1ab_0    conda-forge
jasper                    4.2.2                he6dfbbe_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jeepney                   0.8.0              pyhd8ed1ab_0    conda-forge
jellyfish                 1.0.3           py310hcb5633a_0    conda-forge
jinja2                    3.1.3              pyhd8ed1ab_0    conda-forge
jiter                     0.8.0                    pypi_0    pypi
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jq                        1.7.1                hd590300_0    conda-forge
json5                     0.9.24             pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py310hff52083_3    conda-forge
jsonschema                4.21.1             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.21.1             pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0             pyhd8ed1ab_10    conda-forge
jupyter-lsp               2.2.4              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.1              pyhd8ed1ab_0    conda-forge
jupyter_console           6.6.3              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py310hff52083_0    conda-forge
jupyter_events            0.10.0             pyhd8ed1ab_0    conda-forge
jupyter_server            2.13.0             pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.5.3              pyhd8ed1ab_0    conda-forge
jupyterlab                4.1.5              pyhd8ed1ab_0    conda-forge
jupyterlab-variableinspector 3.2.1              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.3.0              pyhd8ed1ab_1    conda-forge
jupyterlab_server         2.25.4             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.10             pyhd8ed1ab_0    conda-forge
jxrlib                    1.1                  hd590300_3    conda-forge
keras                     3.7.0                    pypi_0    pypi
keras-tuner               1.4.7                    pypi_0    pypi
keyring                   24.3.1          py310hff52083_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py310hd41b1e2_1    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
kt-legacy                 1.0.5                    pypi_0    pypi
lame                      3.100             h166bdaf_1003    conda-forge
lazy_loader               0.3                pyhd8ed1ab_0    conda-forge
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20240722.0      cxx17_h5888daf_1    conda-forge
libaec                    1.1.3                h59595ed_0    conda-forge
libarchive                3.7.4                hfca40fe_0    conda-forge
libarrow                  18.0.0          h9c5d0aa_0_cuda    conda-forge
libarrow-acero            18.0.0          h530483c_0_cuda    conda-forge
libarrow-dataset          18.0.0          h530483c_0_cuda    conda-forge
libarrow-flight           18.0.0          ha931655_0_cuda    conda-forge
libarrow-flight-sql       18.0.0          heb9aac2_0_cuda    conda-forge
libarrow-gandiva          18.0.0          hf4b9a4e_0_cuda    conda-forge
libarrow-substrait        18.0.0          h8ffff87_0_cuda    conda-forge
libass                    0.17.3               h1dc1e6a_0    conda-forge
libavif16                 1.0.4                h1dcd450_1    conda-forge
libblas                   3.9.0            16_linux64_mkl    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcap                    2.69                 h0f662aa_0    conda-forge
libcblas                  3.9.0            16_linux64_mkl    conda-forge
libclang                  15.0.7          default_h127d8a8_5    conda-forge
libclang-cpp15            15.0.7          default_h127d8a8_5    conda-forge
libclang13                15.0.7          default_h5d6823c_5    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcublas                 12.4.5.8                      0    nvidia
libcudf-cu12              24.12.0                  pypi_0    pypi
libcufft                  11.2.1.3                      0    nvidia
libcufile                 1.11.1.6                      0    nvidia
libcups                   2.3.3                h4637d8d_4    conda-forge
libcurand                 10.3.7.77            hbd13f7d_0    conda-forge
libcurl                   8.10.1               hbbe4b11_0    conda-forge
libcusolver               11.6.1.9                      0    nvidia
libcusparse               12.3.1.170                    0    nvidia
libdeflate                1.19                 hd590300_0    conda-forge
libdrm                    2.4.123              hb9d3cd8_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libegl                    1.7.0                ha4b6fd6_1    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libevent                  2.1.12               hf998b51_1    conda-forge
libexpat                  2.6.3                h5888daf_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libflac                   1.4.3                h59595ed_0    conda-forge
libgcc                    14.1.0               h77fa898_1    conda-forge
libgcc-ng                 14.1.0               h69a702a_1    conda-forge
libgcrypt                 1.10.3               hd590300_0    conda-forge
libgfortran-ng            13.2.0               h69a702a_5    conda-forge
libgfortran5              13.2.0               ha4646dd_5    conda-forge
libgl                     1.7.0                ha4b6fd6_1    conda-forge
libglib                   2.82.1               h2ff4ddf_0    conda-forge
libglu                    9.0.0             ha6d2627_1004    conda-forge
libglvnd                  1.7.0                ha4b6fd6_1    conda-forge
libglx                    1.7.0                ha4b6fd6_1    conda-forge
libgomp                   14.1.0               h77fa898_1    conda-forge
libgoogle-cloud           2.30.0               h438788a_0    conda-forge
libgoogle-cloud-storage   2.30.0               h0121fbd_0    conda-forge
libgpg-error              1.48                 h71f35ed_0    conda-forge
libgrpc                   1.65.5               hf5c653b_0    conda-forge
libhwloc                  2.11.2          default_he43201b_1000    conda-forge
libhwy                    1.0.7                h00ab1b0_0    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libidn2                   2.3.7                hd590300_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
libjxl                    0.10.1               h5b01ea3_0    conda-forge
libkvikio-cu12            24.12.1                  pypi_0    pypi
liblapack                 3.9.0            16_linux64_mkl    conda-forge
liblapacke                3.9.0            16_linux64_mkl    conda-forge
libllvm14                 14.0.6               hcd5def8_4    conda-forge
libllvm15                 15.0.7               hb3ce162_4    conda-forge
libllvm16                 16.0.6               hb3ce162_3    conda-forge
libllvm18                 18.1.8               h8b73ec9_2    conda-forge
libmamba                  1.5.7                had39da4_0    conda-forge
libmambapy                1.5.7           py310h39ff949_0    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnl                     3.9.0                hd590300_0    conda-forge
libnpp                    12.2.5.30                     0    nvidia
libnsl                    2.0.1                hd590300_0    conda-forge
libnvfatbin               12.6.77                       0    nvidia
libnvjitlink              12.4.127                      0    nvidia
libnvjpeg                 12.3.1.117                    0    nvidia
libogg                    1.3.5                h27cfd23_1    anaconda
libopenblas               0.3.26          pthreads_h413a1c8_0    conda-forge
libopenvino               2024.4.0             hac27bb2_1    conda-forge
libopenvino-auto-batch-plugin 2024.4.0             h4d9b6c2_1    conda-forge
libopenvino-auto-plugin   2024.4.0             h4d9b6c2_1    conda-forge
libopenvino-hetero-plugin 2024.4.0             h3f63f65_1    conda-forge
libopenvino-intel-cpu-plugin 2024.4.0             hac27bb2_1    conda-forge
libopenvino-intel-gpu-plugin 2024.4.0             hac27bb2_1    conda-forge
libopenvino-ir-frontend   2024.4.0             h3f63f65_1    conda-forge
libopenvino-onnx-frontend 2024.4.0             he882d9a_1    conda-forge
libopenvino-paddle-frontend 2024.4.0             he882d9a_1    conda-forge
libopenvino-pytorch-frontend 2024.4.0             h5888daf_1    conda-forge
libopenvino-tensorflow-frontend 2024.4.0             h9718a47_1    conda-forge
libopenvino-tensorflow-lite-frontend 2024.4.0             h5888daf_1    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libparquet                18.0.0          hdbc8f64_0_cuda    conda-forge
libpciaccess              0.18                 hd590300_0    conda-forge
libpng                    1.6.44               hadc24fc_0    conda-forge
libpq                     16.4                 h2d7952a_2    conda-forge
libprotobuf               5.27.5               h5b01275_2    conda-forge
libre2-11                 2024.07.02           hbbce691_1    conda-forge
libsndfile                1.2.2                hc60ed4a_1    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsolv                   0.7.30               h3509ff9_0    conda-forge
libspatialindex           1.9.3                h9c3ff4c_4    conda-forge
libsqlite                 3.46.1               hadc24fc_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx                 14.1.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_5    conda-forge
libsystemd0               255                  h3516f8a_1    conda-forge
libta-lib                 0.4.0                hd590300_2    conda-forge
libtasn1                  4.19.0               h166bdaf_0    conda-forge
libthrift                 0.21.0               h0e7cc3e_0    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libutf8proc               2.8.0                h166bdaf_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libva                     2.21.0               hd590300_0    conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    conda-forge
libvpx                    1.11.0               h9c3ff4c_3    conda-forge
libwebp                   1.4.0                h2c329e2_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.17.0               h8a09558_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxkbcommon              1.7.0                h2c5496b_1    conda-forge
libxml2                   2.12.7               he7c6b58_4    conda-forge
libxslt                   1.1.39               h76b75d6_0    conda-forge
libzlib                   1.3.1                hb9d3cd8_2    conda-forge
libzopfli                 1.0.3                h9c3ff4c_0    conda-forge
llvm-openmp               15.0.7               h0cdce71_0    conda-forge
llvmlite                  0.43.0                   pypi_0    pypi
lmdb                      0.9.31               hd590300_1    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
loguru                    0.7.2                    pypi_0    pypi
lxml                      5.3.0           py310h6ee67d5_1    conda-forge
lz4                       4.3.3           py310h350c4a5_0    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mako                      1.3.2                    pypi_0    pypi
mamba                     1.5.7           py310h51d5547_0    conda-forge
markdown                  3.6                pyhd8ed1ab_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5           py310h2372a71_0    conda-forge
matplotlib                3.8.3           py310hff52083_0    conda-forge
matplotlib-base           3.8.3           py310h62c0568_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mccabe                    0.7.0              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
menuinst                  2.0.2           py310hff52083_0    conda-forge
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
mkl                       2022.1.0           hc2b9512_224  
mkl-devel                 2022.1.0           h66538d2_224  
mkl-include               2022.1.0           h06a4308_224  
ml_dtypes                 0.4.0           py310h5eaa309_2    conda-forge
more-itertools            10.2.0             pyhd8ed1ab_0    conda-forge
mpc                       1.3.1                hfe3b2da_0    conda-forge
mpfr                      4.2.1                h9458935_0    conda-forge
mpg123                    1.32.4               h59595ed_0    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.7           py310hd41b1e2_0    conda-forge
multidict                 6.0.5           py310h2372a71_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
mutagen                   1.47.0             pyhd8ed1ab_0    conda-forge
mutf8                     1.0.6                    pypi_0    pypi
mypy                      1.9.0           py310h2372a71_0    conda-forge
mypy_extensions           1.0.0              pyha770c72_0    conda-forge
mysql-common              9.0.1                h266115a_1    conda-forge
mysql-libs                9.0.1                he0572af_1    conda-forge
namex                     0.0.8              pyhd8ed1ab_0    conda-forge
navigator-updater         0.4.0           py310h06a4308_1    anaconda
nbclient                  0.10.0             pyhd8ed1ab_0    conda-forge
nbconvert                 7.16.3               hd8ed1ab_0    conda-forge
nbconvert-core            7.16.3             pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.16.3               hd8ed1ab_0    conda-forge
nbformat                  5.10.3             pyhd8ed1ab_0    conda-forge
nccl                      2.23.4.1             h52f6c39_0    conda-forge
ncurses                   6.4.20240210         h59595ed_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
nettle                    3.6                  he412f7d_0    conda-forge
networkx                  3.2.1              pyhd8ed1ab_0    conda-forge
nh3                       0.2.17                   pypi_0    pypi
nltk                      3.8.1              pyhd8ed1ab_0    conda-forge
nomkl                     2.0                           0    anaconda
notebook                  7.1.2              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.4              pyhd8ed1ab_0    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.105                hd34e28f_0    conda-forge
numba                     0.60.0                   pypi_0    pypi
numba-cuda                0.0.17.1                 pypi_0    pypi
numpy                     1.26.4          py310hb13e2d6_0    conda-forge
numpydoc                  1.6.0              pyhd8ed1ab_0    conda-forge
nvidia-nccl-cu12          2.23.4                   pypi_0    pypi
nvidia-nvcomp-cu12        4.1.0.6                  pypi_0    pypi
nvtx                      0.2.10                   pypi_0    pypi
oauth2client              4.1.3                    pypi_0    pypi
ocl-icd                   2.3.2                hd590300_1    conda-forge
ocl-icd-system            1.0.0                         1    conda-forge
omegaconf                 2.3.0                    pypi_0    pypi
oniguruma                 6.9.9                hd590300_0    conda-forge
openblas                  0.3.26          pthreads_h7a3da1a_0    conda-forge
opencv-python             4.9.0.80                 pypi_0    pypi
openexr                   3.2.2                h04e0de5_2    conda-forge
openh264                  2.1.1                h4ff587b_0  
openjpeg                  2.5.2                h488ebb8_0    conda-forge
openpyxl                  3.1.2           py310h2372a71_1    conda-forge
openssl                   3.4.0                hb9d3cd8_0    conda-forge
opt_einsum                3.4.0              pyhd8ed1ab_0    conda-forge
optree                    0.13.0          py310h3788b33_0    conda-forge
optuna                    4.1.0                    pypi_0    pypi
optuna-integration        4.1.0                    pypi_0    pypi
orc                       2.0.2                h690cf93_1    conda-forge
overrides                 7.7.0              pyhd8ed1ab_0    conda-forge
p11-kit                   0.24.1               hc5aa10d_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.2.2                    pypi_0    pypi
pandoc                    3.1.12.3             ha770c72_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
param                     1.13.0             pyh1a96a4e_0    conda-forge
parsel                    1.9.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.4.1              pyhd8ed1ab_0    conda-forge
pathlib                   1.0.1           py310hff52083_7    conda-forge
pathspec                  0.12.1             pyhd8ed1ab_0    conda-forge
pcre2                     10.44                hba22ea6_2    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pgn2data                  0.0.9                    pypi_0    pypi
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.4.0          py310he228d35_1    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pixman                    0.43.2               h59595ed_0    conda-forge
pkce                      1.0.3              pyhd8ed1ab_0    conda-forge
pkginfo                   1.10.0                   pypi_0    pypi
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
plotly                    5.19.0             pyhd8ed1ab_0    conda-forge
pluggy                    1.4.0              pyhd8ed1ab_0    conda-forge
ply                       3.11                       py_1    conda-forge
prometheus_client         0.20.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
prompt_toolkit            3.0.42               hd8ed1ab_0    conda-forge
propcache                 0.2.1                    pypi_0    pypi
protego                   0.3.0              pyhd8ed1ab_0    conda-forge
proto-plus                1.25.0                   pypi_0    pypi
protobuf                  4.25.5                   pypi_0    pypi
protontricks              1.11.0                   pypi_0    pypi
psutil                    5.9.8           py310h2372a71_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pugixml                   1.14                 h59595ed_0    conda-forge
pulseaudio-client         17.0                 hb77b528_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py-cpuinfo                9.0.0              pyhd8ed1ab_0    conda-forge
pyarrow                   18.1.0                   pypi_0    pypi
pyasn1                    0.5.1              pyhd8ed1ab_0    conda-forge
pyasn1-modules            0.3.0              pyhd8ed1ab_0    conda-forge
pyaudio                   0.2.14                   pypi_0    pypi
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycodestyle               2.11.1             pyhd8ed1ab_0    conda-forge
pycosat                   0.6.6           py310h2372a71_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pycryptodomex             3.19.0          py310h2372a71_1    conda-forge
pyct                      0.5.0              pyhd8ed1ab_0    conda-forge
pycurl                    7.45.3          py310h80730c7_1    conda-forge
pydantic                  2.6.4              pyhd8ed1ab_0    conda-forge
pydantic-core             2.16.3          py310hcb5633a_0    conda-forge
pydispatcher              2.0.5                      py_1    conda-forge
pydocstyle                6.3.0              pyhd8ed1ab_0    conda-forge
pydot                     2.0.0                    pypi_0    pypi
pydrive                   1.3.1                    pypi_0    pypi
pyelftools                0.30                     pypi_0    pypi
pyflakes                  3.2.0              pyhd8ed1ab_0    conda-forge
pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
pyjwt                     2.8.0              pyhd8ed1ab_1    conda-forge
pylibcudf-cu12            24.12.0                  pypi_0    pypi
pylint                    3.0.4              pyhd8ed1ab_0    conda-forge
pylint-venv               3.0.3              pyhd8ed1ab_0    conda-forge
pyls-spyder               0.4.0              pyhd8ed1ab_0    conda-forge
pymongo                   4.6.2                    pypi_0    pypi
pympler                   1.0.1              pyhd8ed1ab_0    conda-forge
pynisher                  1.0.10                   pypi_0    pypi
pynvjitlink-cu12          0.4.0                    pypi_0    pypi
pyodbc                    5.1.0           py310hc6cd4ac_0    conda-forge
pyopenssl                 24.1.0                   pypi_0    pypi
pyparsing                 3.1.2              pyhd8ed1ab_0    conda-forge
pyproject-hooks           1.0.0                    pypi_0    pypi
pyqt                      5.15.10         py310h6a678d5_0    anaconda
pyqt5-qt5                 5.15.2                   pypi_0    pypi
pyqt5-sip                 12.13.0         py310h5eee18b_0    anaconda
pyqtwebengine             5.15.9          py310h704022c_5    conda-forge
pyrfr                     0.9.0                    pypi_0    pypi
pysimplegui               5.0.7                    pypi_0    pypi
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytest                    8.1.1              pyhd8ed1ab_0    conda-forge
python                    3.10.14         hd12c33a_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-dotenv             1.0.1              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
python-flatbuffers        24.3.25            pyh59ac667_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-lsp-black          2.0.0              pyhd8ed1ab_0    conda-forge
python-lsp-jsonrpc        1.1.2              pyhd8ed1ab_0    conda-forge
python-lsp-server         1.10.1             pyhd8ed1ab_0    conda-forge
python-lsp-server-base    1.10.1             pyhd8ed1ab_0    conda-forge
python-magic              0.4.27                   pypi_0    pypi
python-slugify            8.0.4              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python-utils              3.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    4_cp310    conda-forge
pytoolconfig              1.2.5              pyhd8ed1ab_0    conda-forge
pytorch                   2.5.1           py3.10_cuda12.4_cudnn9.1.0_0    pytorch
pytorch-cuda              12.4                 hc786d27_7    pytorch
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyusb                     1.2.1                    pypi_0    pypi
pyviz_comms               3.0.1              pyhd8ed1ab_0    conda-forge
pyxdg                     0.28               pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.2                    pypi_0    pypi
pyzmq                     25.1.2          py310h795f18f_0    conda-forge
qdarkstyle                3.2.3              pyhd8ed1ab_0    conda-forge
qstylizer                 0.2.2              pyhd8ed1ab_0    conda-forge
qt                        5.15.8               hf11cfaa_0    conda-forge
qt-main                   5.15.8              h3155989_26    conda-forge
qt-webengine              5.15.8               h8f589be_8    conda-forge
qtawesome                 1.2.3              pyhd8ed1ab_0    conda-forge
qtconsole                 5.5.1              pyhd8ed1ab_0    conda-forge
qtconsole-base            5.5.1              pyha770c72_0    conda-forge
qtpy                      2.4.1              pyhd8ed1ab_0    conda-forge
queuelib                  1.6.2              pyhd8ed1ab_0    conda-forge
rav1e                     0.6.6                he8a937b_2    conda-forge
rdma-core                 50.0                 hd3aeb46_1    conda-forge
re2                       2024.07.02           h77b4e00_1    conda-forge
readline                  8.2                  h8228510_1    conda-forge
readme-renderer           43.0                     pypi_0    pypi
referencing               0.34.0             pyhd8ed1ab_0    conda-forge
regex                     2023.12.25      py310h2372a71_0    conda-forge
reproc                    14.2.4.post0         hd590300_1    conda-forge
reproc-cpp                14.2.4.post0         h59595ed_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
requests-file             2.0.0              pyhd8ed1ab_0    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_0    conda-forge
retry-decorator           1.1.1                    pypi_0    pypi
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986                   2.0.0                    pypi_0    pypi
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rich                      13.7.1             pyhd8ed1ab_0    conda-forge
rmm-cu12                  24.12.1                  pypi_0    pypi
rope                      1.12.0             pyhd8ed1ab_0    conda-forge
rpds-py                   0.18.0          py310hcb5633a_0    conda-forge
rsa                       4.9                      pypi_0    pypi
rtree                     1.2.0           py310hbdcdc62_0    conda-forge
ruamel.yaml               0.18.6          py310h2372a71_0    conda-forge
ruamel.yaml.clib          0.2.8           py310h2372a71_0    conda-forge
ruamel_yaml               0.15.80         py310h2372a71_1009    conda-forge
s2n                       1.5.6                h0e56266_0    conda-forge
s3fs                      0.6.0              pyhd8ed1ab_0    conda-forge
s3transfer                0.10.1             pyhd8ed1ab_0    conda-forge
safetensors               0.4.5                    pypi_0    pypi
scipy                     1.14.1                   pypi_0    pypi
scrapy                    2.11.1          py310hff52083_2    conda-forge
secretstorage             3.3.3           py310hff52083_2    conda-forge
semver                    2.13.0             pyh9f0ad1d_0    conda-forge
send2trash                1.8.2              pyh41d4057_0    conda-forge
sentencepiece             0.2.0                    pypi_0    pypi
service-identity          21.1.0             pyhd8ed1ab_0    conda-forge
service_identity          21.1.0               hd8ed1ab_0    conda-forge
setuptools                69.2.0             pyhd8ed1ab_0    conda-forge
sip                       6.7.12          py310hc6cd4ac_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smac                      2.2.0                    pypi_0    pypi
smart_open                7.0.3              pyhd8ed1ab_0    conda-forge
smmap                     5.0.0              pyhd8ed1ab_0    conda-forge
snappy                    1.2.1                ha2e4443_0    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
sphinx                    7.2.6              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   1.0.8              pyhd8ed1ab_0    conda-forge
sphinxcontrib-devhelp     1.0.6              pyhd8ed1ab_0    conda-forge
sphinxcontrib-htmlhelp    2.0.5              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1              pyhd8ed1ab_0    conda-forge
sphinxcontrib-qthelp      1.0.7              pyhd8ed1ab_0    conda-forge
sphinxcontrib-serializinghtml 1.1.10             pyhd8ed1ab_0    conda-forge
spyder-kernels            2.5.1           unix_pyh707e725_0    conda-forge
sqlalchemy                1.4.52                   pypi_0    pypi
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
svt-av1                   1.8.0                h59595ed_0    conda-forge
sympy                     1.13.3          pypyh2585a3b_103    conda-forge
ta-lib                    0.4.32          py310h8a78493_0    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tbb                       2022.0.0             hceb3a55_0    conda-forge
tblib                     3.0.0              pyhd8ed1ab_0    conda-forge
tenacity                  8.2.3              pyhd8ed1ab_0    conda-forge
tensorboard               2.17.1             pyhd8ed1ab_0    conda-forge
tensorboard-data-server   0.7.0           py310h75e40e8_1    conda-forge
tensorflow                2.17.0          cuda120py310heb3ae67_203    conda-forge
tensorflow-base           2.17.0          cuda120py310ha9db03a_203    conda-forge
tensorflow-estimator      2.17.0          cuda120py310h1fd330c_203    conda-forge
tensorflow-gpu            2.17.0          cuda120py310hb76ca00_203    conda-forge
termcolor                 2.5.0              pyhd8ed1ab_0    conda-forge
terminado                 0.18.1             pyh0d859eb_0    conda-forge
text-unidecode            1.3                pyhd8ed1ab_1    conda-forge
textdistance              4.5.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.4.0              pyhc1e730c_0    conda-forge
three-merge               0.1.1              pyh9f0ad1d_0    conda-forge
tiktoken                  0.8.0                    pypi_0    pypi
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tldextract                5.1.2              pyhd8ed1ab_0    conda-forge
tokencost                 0.1.17                   pypi_0    pypi
tokenizers                0.20.3                   pypi_0    pypi
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomlkit                   0.12.4             pyha770c72_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
torch                     2.2.2+cpu                pypi_0    pypi
torchtriton               3.1.0                     py310    pytorch
tornado                   6.4             py310h2372a71_0    conda-forge
tqdm                      4.66.2             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.2             pyhd8ed1ab_0    conda-forge
transformers              4.46.3                   pypi_0    pypi
truststore                0.8.0              pyhd8ed1ab_0    conda-forge
twine                     5.0.0                    pypi_0    pypi
twisted                   24.3.0          py310h2372a71_0    conda-forge
types-python-dateutil     2.9.0.20240316     pyhd8ed1ab_0    conda-forge
typing-extensions         4.10.0               hd8ed1ab_0    conda-forge
typing_extensions         4.10.0             pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
tzlocal                   5.2             py310hff52083_0    conda-forge
ucx                       1.15.0               hf604dca_7    conda-forge
ujson                     5.9.0           py310hc6cd4ac_0    conda-forge
unicodedata2              15.1.0          py310h2372a71_0    conda-forge
unidecode                 1.3.8                    pypi_0    pypi
unixodbc                  2.3.12               h661eb56_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
uritemplate               4.1.1                    pypi_0    pypi
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
validators                0.23.2             pyhd8ed1ab_0    conda-forge
vdf                       3.4                      pypi_0    pypi
video-to-ascii            1.2.8                    pypi_0    pypi
w3lib                     2.1.2              pyhd8ed1ab_0    conda-forge
watchdog                  4.0.0           py310hff52083_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
webcolors                 1.13               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.7.0              pyhd8ed1ab_0    conda-forge
websockets                12.0            py310h2372a71_0    conda-forge
werkzeug                  3.0.1              pyhd8ed1ab_0    conda-forge
whatthepatch              1.0.5              pyhd8ed1ab_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.10             pyhd8ed1ab_0    conda-forge
wrapt                     1.16.0          py310h2372a71_0    conda-forge
wurlitzer                 3.0.3              pyhd8ed1ab_0    conda-forge
x264                      1!161.3030           h7f98852_1    conda-forge
x265                      3.5                  h924138e_3    conda-forge
xcb-util                  0.4.1                hb711507_2    conda-forge
xcb-util-image            0.4.0                hb711507_2    conda-forge
xcb-util-keysyms          0.4.1                hb711507_0    conda-forge
xcb-util-renderutil       0.3.10               hb711507_0    conda-forge
xcb-util-wm               0.4.2                hb711507_0    conda-forge
xgboost                   2.1.3                    pypi_0    pypi
xkeyboard-config          2.41                 hd590300_0    conda-forge
xorg-compositeproto       0.4.2             h7f98852_1001    conda-forge
xorg-damageproto          1.2.1             h7f98852_1002    conda-forge
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-inputproto           2.3.2             h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.10               h4f16b4b_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxcomposite        0.4.6                h0b41bf4_1    conda-forge
xorg-libxdamage           1.1.5                h7f98852_1    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxi                1.7.10               h7f98852_0    conda-forge
xorg-libxrandr            1.5.2                h7f98852_1    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-libxtst              1.2.3             h7f98852_1002    conda-forge
xorg-libxxf86vm           1.1.5                hb9d3cd8_2    conda-forge
xorg-randrproto           1.5.0             h7f98852_1001    conda-forge
xorg-recordproto          1.14.2            h7f98852_1002    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-util-macros          1.19.3               h7f98852_0    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xf86vidmodeproto     2.3.1             h7f98852_1002    conda-forge
xorg-xorgproto            2024.1               hb9d3cd8_1    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xtermcolor                1.3                      pypi_0    pypi
xyzservices               2023.10.1          pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yaml-cpp                  0.8.0                h59595ed_0    conda-forge
yapf                      0.40.1             pyhd8ed1ab_0    conda-forge
yarl                      1.18.3                   pypi_0    pypi
yt-dlp                    2024.3.10          pyhd8ed1ab_0    conda-forge
zeromq                    4.3.5                h59595ed_1    conda-forge
zfp                       1.0.1                h59595ed_0    conda-forge
zict                      3.0.0              pyhd8ed1ab_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.3.1                hb9d3cd8_2    conda-forge
zlib-ng                   2.0.7                h0b41bf4_0    conda-forge
zope.interface            6.2             py310h2372a71_0    conda-forge
zstandard                 0.23.0          py310ha39cb0e_1    conda-forge
zstd                      1.5.5.1                  pypi_0    pypi

**Setup details**:

NVCC Version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Sep_12_02:18:05_PDT_2024
Cuda compilation tools, release 12.6, V12.6.77
Build cuda_12.6.r12.6/compiler.34841621_0

Nvidia driver 565.77 on a 3060
Arch linux on 6.6.64-1-lts x86_64 GNU/Linux

Would you like to help us fix it?
Not currently

[GhostDog98]

To preempt this issue being suggested as a fix, none of my npy files contain NaN anywhere...