This is a Python library that Madeesh Kannan and I have developed. It started out as a small transformer library to use in spaCy pipelines. The last two months we have extended it to an independent PyTorch library and added support for modern LLMs such as Falcon, LLaMA, and Dolly V2.

Our focus has been on making a library of reusable components, so that new transformer architectures can be added with as little code as possible. We will release a semver-stable 1.0 release over the coming weeks.