Computation Analysis for Ascend NPU

You are analyzing computation patterns for Ascend NPU performance. This skill helps identify:

1. •Computation-intensive operators and their locations
2. •CANN operator library support status
3. •CPU fallback operations (performance impact)
4. •Optimization opportunities with torch_npu
5. •Performance profiling approach

When to Use

Invoke this skill when:

• •User asks about performance or computation
• •Analyzing model operators and operations
• •Looking for performance bottlenecks
• •Planning optimization strategies

Analysis Approach

1. Identify Heavy Operators

Search for compute-intensive patterns:

# Matrix operations
grep -rn "torch\.matmul\|@\|mm" <repo_path>
grep -rn "nn\.Linear" <repo_path>

# Convolutions
grep -rn "nn\.Conv" <repo_path>

# Attention mechanisms
grep -rn "attention\|scaled_dot_product" <repo_path>
grep -rn "F\.scaled_dot_product_attention" <repo_path>

# Normalization
grep -rn "LayerNorm\|BatchNorm" <repo_path>

# Activation functions
grep -rn "relu\|gelu\|silu\|softmax" <repo_path>

2. CANN Operator Support

CANN (Compute Architecture for Neural Networks) provides:

• •Automatic acceleration for standard PyTorch ops
• •TBE (Tensor Boost Engine) operator fusion
• •AI Core acceleration for matrix ops

Natively Supported (High Performance):

• •Matrix multiplication (MatMul, GEMM)
• •Convolutions (Conv1d, Conv2d, Conv3d)
• •Standard activations (ReLU, GELU, SiLU)
• •LayerNorm, BatchNorm
• •Standard attention (scaled_dot_product_attention)

CPU Fallback (Low Performance):

• •Custom CUDA kernels
• •Third-party library operations
• •Unsupported fusion operations

Check CANN documentation for operator support status.

3. Graph Optimization Opportunities

Operator Fusion:

• •Combine multiple operations into single kernel
• •Reduces memory transfers
• •Ascend TBE compiler does automatic fusion

Identify opportunities:

• •Sequential linear + activation
• •Conv + batch norm + activation
• •Multiple element-wise operations

4. Automatic Mixed Precision (AMP)

Performance Benefits:

• •2-4x speedup on supported operations
• •Lower memory bandwidth requirements
• •Better AI Core utilization

Check for AMP usage:

# Existing
torch.cuda.amp.autocast  # → torch.npu.amp.autocast

# Opportunities
# - FP32 models that can use FP16
# - Operations supporting FP16 acceleration

5. Distributed Training

Communication Operations:

grep -rn "all_reduce\|broadcast\|gather" <repo_path>

HCCL (Huawei Collective Communication Library):

• •Replaces NCCL for Ascend
• •Used for multi-NPU training
• •Backend change required

Output Format

Computation-Intensive Operators

List heavy operators with locations:

CANN Support Analysis

Natively Supported (High Performance):

• •List operators with CANN acceleration
• •Expected speedup vs CPU

CPU Fallback (Performance Risk):

• •List operators requiring CPU execution
• •Performance impact assessment
• •Suggested workarounds

Optimization Recommendations

torch_npu AMP:

• •Enable automatic mixed precision
• •Expected speedup: 2-4x
• •Operations supporting FP16

Graph Optimization:

• •Operator fusion opportunities
• •Expected performance gain
• •TBE compiler optimization

Distributed Training:

• •HCCL communication optimization
• •Gradient compression opportunities
• •Overlap compute and communication

Performance Profiling

Recommended Tools:

# NPU monitoring
npu-smi info  # Real-time NPU status
npu-smi info -t usages  # Memory and utilization

# Profiling
torch_npu.npu.profile  # Profile NPU operations
msprof  # CANN profiling tool

# Python profiling
python -m torch_npu.testing  # Benchmark utilities

Profiling Approach:

1. •Run model with small batch
2. •Use npu-smi to monitor utilization
3. •Profile individual operations
4. •Identify bottlenecks
5. •Optimize hotspots

Expected Performance on Ascend

Based on analysis:

• •Bottleneck identification: What limits performance
• •Optimization priority: Rank optimizations by impact
• •Expected speedup: vs GPU baseline
• •Performance model: Operations/sec, memory bandwidth

Tools to Use

Documentation First:

• •Read official Ascend documentation before analysis:
  • •https://www.hiascend.com/doc_center/source/zh/Pytorch/730/ptmoddevg/trainingmigrguide/PT_LMTMOG_0002.html
  • •https://www.hiascend.com/doc_center/source/zh/canncommercial/850/API/aolapi/operatorlist_00001.html

Computation Analysis:

• •Use Grep to search for computation patterns
• •Reference project PDFs:
  • •knowledge/CANN商用版 8.5.0 算子库接口参考 01.pdf

Key Considerations

• •CANN automatically accelerates standard PyTorch ops
• •Custom ops require special handling
• •Profiling essential for optimization
• •AMP provides easy performance wins
• •HCCL required for multi-NPU training
• •Graph fusion provides additional speedup