Out latest work Lazarus for resilient and elastic training of Mixture-of-Experts models is available. Check it out!