Python ASN.1 library with a focus on performance and a pythonic API

asn1crypto

A fast, pure Python library for parsing and serializing ASN.1 structures.

Features

In addition to an ASN.1 BER/DER decoder and DER serializer, the project includes
a bunch of ASN.1 structures for use with various common cryptography standards:

Why Another Python ASN.1 Library?

Python has long had the pyasn1 and
pyasn1_modules available for
parsing and serializing ASN.1 structures. While the project does include a
comprehensive set of tools for parsing and serializing, the performance of the
library can be very poor, especially when dealing with bit fields and parsing
large structures such as CRLs.

After spending extensive time using pyasn1, the following issues were
identified:

1. Poor performance
2. Verbose, non-pythonic API
3. Out-dated and incomplete definitions in pyasn1-modules
4. No simple way to map data to native Python data structures
5. No mechanism for overridden universal ASN.1 types

The pyasn1 API is largely method driven, and uses extensive configuration
objects and lowerCamelCase names. There were no consistent options for
converting types of native Python data structures. Since the project supports
out-dated versions of Python, many newer language features are unavailable
for use.

Time was spent trying to profile issues with the performance, however the
architecture made it hard to pin down the primary source of the poor
performance. Attempts were made to improve performance by utilizing unreleased
patches and delaying parsing using the Any type. Even with such changes, the
performance was still unacceptably slow.

Finally, a number of structures in the cryptographic space use universal data
types such as BitString and OctetString, but interpret the data as other
types. For instance, signatures are really byte strings, but are encoded as
BitString. Elliptic curve keys use both BitString and OctetString to
represent integers. Parsing these structures as the base universal types and
then re-interpreting them wastes computation.

asn1crypto uses the following techniques to improve performance, especially
when extracting one or two fields from large, complex structures:

• Delayed parsing of byte string values
• Persistence of original ASN.1 encoded data until a value is changed
• Lazy loading of child fields
• Utilization of high-level Python stdlib modules

While there is no extensive performance test suite, the
CRLTests.test_parse_crl test case was used to parse a 21MB CRL file on a
late 2013 rMBP. asn1crypto parsed the certificate serial numbers in just
under 8 seconds. With pyasn1, using definitions from pyasn1-modules, the
same parsing took over 4,100 seconds.

For smaller structures the performance difference can range from a few times
faster to an order of magnitude or more.

Related Crypto Libraries

asn1crypto is part of the modularcrypto family of Python packages:

• asn1crypto
• oscrypto
• csrbuilder
• certbuilder
• crlbuilder
• ocspbuilder
• certvalidator

Dependencies

Python 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8 or pypy. No third-party
packages required.

Installation

pip install asn1crypto

GitHub

https://github.com/wbond/asn1crypto