PyTorch proposal would cache shared pipeline activations per rank
A rank-local cache would serve reused residuals and return one summed gradient, replacing custom stage subclasses.
By tensorA rank-local cache would serve reused residuals and return one summed gradient, replacing custom stage subclasses.
By tensorA proposed torch.autograd.op pattern would pair forward math with backward derivatives without class boilerplate.
By tensor