:
Loading config.json from local directory
Loading weights from local directory
Removing weight norm...
Parsing tokenizer identifier. Schema: None, Identifier: ViT-H-14-378-quickgelu
Attempting to load config from built-in: ViT-H-14-378-quickgelu
Using default SimpleTokenizer.
Loaded MMAudio model weights from D:\StabilityMatrix\Data\Packages\ComfyUI\models\mmaudio\mmaudio_large_44k_v2_fp16.safetensors
clip_frames torch.Size([77, 3, 384, 384]) sync_frames torch.Size([240, 3, 224, 224]) duration 9.633333333333333
!!! Exception during processing !!! Allocation on device
Traceback (most recent call last):
File "D:\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 518, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 329, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 303, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 291, in process_inputs
result = f(**inputs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\nodes.py", line 347, in sample
audios = generate(clip_frames,
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\eval_utils.py", line 53, in generate
sync_features = feature_utils.encode_video_with_sync(sync_video, batch_size=bs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\utils\_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\model\utils\features_utils.py", line 118, in encode_video_with_sync
outputs.append(self.synchformer(x[i:i + batch_size]))
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\ext\synchformer\synchformer.py", line 34, in forward
vis = self.vfeat_extractor(vis)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\ext\synchformer\motionformer.py", line 211, in forward
x = self.forward_segments(x, orig_shape=orig_shape)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\ext\synchformer\motionformer.py", line 220, in forward_segments
x, x_mask = self.forward_features(x)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\ext\synchformer\video_model_builder.py", line 245, in forward_features
x = blk(x,
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\ext\synchformer\vit_helper.py", line 177, in forward
space_output = self.attn(self.norm1(time_residual),
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
File "D:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-mmaudio\mmaudio\ext\synchformer\vit_helper.py", line 89, in forward
k_ = torch.cat((cls_k, k_), dim=1)
torch.OutOfMemoryError: Allocation on device
Memory summary:
|===========================================================================|
| PyTorch CUDA memory summary, device ID 0 |
|---------------------------------------------------------------------------|
| CUDA OOMs: 0 | cudaMalloc retries: 0 |
|===========================================================================|
| Metric | Cur Usage | Peak Usage | Tot Alloc | Tot Freed |
|---------------------------------------------------------------------------|
| Allocated memory | 7148 MiB | 7180 MiB | 0 B | 0 B |
| from large pool | 0 MiB | 0 MiB | 0 B | 0 B |
| from small pool | 0 MiB | 0 MiB | 0 B | 0 B |
|---------------------------------------------------------------------------|
| Active memory | 7148 MiB | 7180 MiB | 0 B | 0 B |
| from large pool | 0 MiB | 0 MiB | 0 B | 0 B |
| from small pool | 0 MiB | 0 MiB | 0 B | 0 B |
|---------------------------------------------------------------------------|
| Requested memory | 0 B | 0 B | 0 B | 0 B |
| from large pool | 0 B | 0 B | 0 B | 0 B |
| from small pool | 0 B | 0 B | 0 B | 0 B |
|---------------------------------------------------------------------------|
| GPU reserved memory | 7488 MiB | 7488 MiB | 0 B | 0 B |
| from large pool | 0 MiB | 0 MiB | 0 B | 0 B |
| from small pool | 0 MiB | 0 MiB | 0 B | 0 B |
|---------------------------------------------------------------------------|
| Non-releasable memory | 0 B | 0 B | 0 B | 0 B |
| from large pool | 0 B | 0 B | 0 B | 0 B |
| from small pool | 0 B | 0 B | 0 B | 0 B |
|---------------------------------------------------------------------------|
| Allocations | 0 | 0 | 0 | 0 |
| from large pool | 0 | 0 | 0 | 0 |
| from small pool | 0 | 0 | 0 | 0 |
|---------------------------------------------------------------------------|
| Active allocs | 0 | 0 | 0 | 0 |
| from large pool | 0 | 0 | 0 | 0 |
| from small pool | 0 | 0 | 0 | 0 |
|---------------------------------------------------------------------------|
| GPU reserved segments | 0 | 0 | 0 | 0 |
| from large pool | 0 | 0 | 0 | 0 |
| from small pool | 0 | 0 | 0 | 0 |
|---------------------------------------------------------------------------|
| Non-releasable allocs | 0 | 0 | 0 | 0 |
| from large pool | 0 | 0 | 0 | 0 |
| from small pool | 0 | 0 | 0 | 0 |
|---------------------------------------------------------------------------|
| Oversize allocations | 0 | 0 | 0 | 0 |
|---------------------------------------------------------------------------|
| Oversize GPU segments | 0 | 0 | 0 | 0 |
|===========================================================================|
Got an OOM, unloading all loaded models.
Prompt executed in 146.94 seconds
got prompt
clip_frames torch.Size([77, 3, 384, 384]) sync_frames torch.Size([240, 3, 224, 224]) duration 9.633333333333333
Flow Matching: 100%|██████████| 25/25 [00:08<00:00, 2.90it/s]
Prompt executed in 29.74 seconds
got prompt
clip_frames torch.Size([38, 3, 384, 384]) sync_frames torch.Size([120, 3, 224, 224]) duration 4.833333333333333
Flow Matching: 100%|██████████| 25/25 [00:08<00:00, 3.02it/s]
Prompt executed in 23.24 seconds
mmaudio_test.json
