Randomized smoothing has recently emerged as an effective tool that enables certification of deep neural network classifiers at scale. All prior art on randomized smoothing has focused on isotropic `p certification, which has the advantage of yielding certificates that can be easily compared among isotropic methods via `p-norm radius. However, isotropic certification limits the region that can be certified around an input to worst-case adversaries, i.e. it cannot reason about other “close”, potentially large, constant prediction safe regions. To alleviate this issue, (i) we theoretically extend the isotropic randomized smoothing `1 and `2 certificates to their generalized anisotropic counterparts following a simplified analysis. Moreover, (ii) we propose evaluation metrics allowing for the comparison of general certificates – a certificate is superior to another if it certifies a superset region – with the quantification of each certificate through the volume of the certified region. We introduce AnCer, a framework for obtaining anisotropic certificates for a given test set sample via volume maximization. We achieve it by generalizing memory-based certification of data-dependent classifiers. Our empirical results demonstrate that AnCer achieves state-of-the-art `1 and `2 certified accuracy on CIFAR-10 and ImageNet in the data-dependence setting, while certifying larger regions in terms of volume, highlighting the benefits of moving away from isotropic analysis. Our code is available in this repository.