Skip to content

[Hardware] Add support for Huawei Ascend NPU#205

Open
shen-shanshan wants to merge 1 commit into
Vchitect:masterfrom
shen-shanshan:npu
Open

[Hardware] Add support for Huawei Ascend NPU#205
shen-shanshan wants to merge 1 commit into
Vchitect:masterfrom
shen-shanshan:npu

Conversation

@shen-shanshan
Copy link
Copy Markdown

@shen-shanshan shen-shanshan commented May 14, 2026

Background

Ascend is a full-stack AI computing infrastructure for industry applications and services based on Huawei Ascend processors and software. For more information about Ascend, see Ascend Community.

CANN (Compute Architecture of Neural Networks), developped by Huawei, is a heterogeneous computing architecture for AI.

Pytorch has officially announced support for Ascend NPU (through key PrivateUse1), please see the PrivateUse1 tutorial here.

Motivation

Currently, the number of developers using Ascend NPU for AI training and inferencing has been significantly increasing, especially in China. Therefore, I would like to add support for Ascend NPU backend for this project.

Tests

Note

To properly install CANN, see here for more details.
In addition, the version of torch-npu should match that of torch, see here for more details.

Example:

from vbench import VBench

my_VBench = VBench("npu", "vbench/VBench_full_info.json", "outputs/evaluation_results")
my_VBench.evaluate(
    videos_path = "sampled_videos/lavie/human_action",
    name = "lavie_human_action",
    dimension_list = ["human_action"],
)

Outputs:

Use device: npu
Evaluation meta data saved to outputs/evaluation_results/lavie_human_action_full_info.json
/root/miniconda3/envs/vbench/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/root/miniconda3/envs/vbench/lib/python3.10/site-packages/timm/models/registry.py:4: FutureWarning: Importing from timm.models.registry is deprecated, please import via timm.models
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.models", FutureWarning)
/vllm-workspace/VBench/vbench/third_party/umt/models/modeling_finetune.py:348: UserWarning: Overwriting vit_large_patch16_224 in registry with vbench.third_party.umt.models.modeling_finetune.vit_large_patch16_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  def vit_large_patch16_224(pretrained=False, **kwargs):
cur_full_info_path: outputs/evaluation_results/lavie_human_action_full_info.json
Use checkpoint: False
Checkpoint number: 16
n_position: 3136
pre_n_position: 1568
Pretraining uses 8 frames, but current frame is 16
Interpolate the position embedding
Use learnable position embedding
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [02:05<00:00,  3.98it/s]
Evaluation results saved to outputs/evaluation_results/lavie_human_action_eval_results.json

Evaluation results:

{
    "human_action": [
        0.964,
        [
            {
                "video_path": "sampled_videos/lavie/human_action/A person is riding a bike-0.mp4",
                "video_results": true,
                "cor_num_per_video": 1
            },
            // ...
        ]
    ]
}

@shen-shanshan
support ascend npu
a480742 
Signed-off-by: Shanshan Shen <[email protected]>
@shen-shanshan
Copy link
Copy Markdown
Author

shen-shanshan commented May 14, 2026

CC @niyaojiayou @zhangfan-p @NattapolChan @JustinYuu @ziqihuangg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@shen-shanshan