Question of today on nature of Magnetism

Does the brittle nature of Permanent Magnet has anything to do with its unlimited supply of Magnetic energy? i.e. Does the mechanism/structure which provide the Magnet which unlimited Magnetic energy is what make the Magnet brittle?