Skip to content

[ENHANCEMENT] MegaFBD:前向–反向解耦增强 #28

@jennifer88huang

Description

@jennifer88huang
  1. 新增机制支持自由调整$\frac{\#forward\ ranks}{\#backward\ ranks}$ 比例,适配不同负载特征。(预计 1 month)
  2. 恢复支持此前与 MegaFBD 冲突的特性,例如自定义并行组初始化顺序、使用 Transformer Engine 作为 Transformer 实现等。(预计 2 months)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions