gh-73613: Support Base32 and Base64 without padding

Add the padded parameter in functions related to Base32 and Base64 codecs
in the binascii and base64 modules.  In the encoding functions it controls
whether the pad character can be added in the output, in the decoding
functions it controls whether padding is required in input.

Padding of input no longer required in base64.urlsafe_b64decode() by default.

Author: serhiy-storchaka

Doc/library/base64.rst

@@ -51,7 +51,7 @@ The :rfc:`4648` encodings are suitable for encoding binary data so that it can b
 safely sent by email, used as parts of URLs, or included as part of an HTTP
 POST request.
 
-.. function:: b64encode(s, altchars=None, *, wrapcol=0)
+.. function:: b64encode(s, altchars=None, *, padded=True, wrapcol=0)
 
    Encode the :term:`bytes-like object` *s* using Base64 and return the encoded
    :class:`bytes`.
@@ -61,6 +61,10 @@ POST request.
    This allows an application to e.g. generate URL or filesystem safe Base64
    strings.  The default is ``None``, for which the standard Base64 alphabet is used.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 4.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not insert any newlines.
@@ -69,11 +73,11 @@ POST request.
    :exc:`TypeError` if *altchars* is not a :term:`bytes-like object`.
 
    .. versionchanged:: 3.15
-      Added the *wrapcol* parameter.
+      Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b64decode(s, altchars=None, validate=False)
-              b64decode(s, altchars=None, validate=True, *, ignorechars)
+.. function:: b64decode(s, altchars=None, validate=False, *, padded=True)
+              b64decode(s, altchars=None, validate=True, *, ignorechars, padded=True)
 
    Decode the Base64 encoded :term:`bytes-like object` or ASCII string
    *s* and return the decoded :class:`bytes`.
@@ -82,6 +86,11 @@ POST request.
    of length 2 which specifies the alternative alphabet used instead of the
    ``+`` and ``/`` characters.
 
+   If *padded* is true, the last group of 4 base 64 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *validate* and *ignorechars*).
+
    A :exc:`binascii.Error` exception is raised
    if *s* is incorrectly padded.
 
@@ -106,7 +115,7 @@ POST request.
    For more information about the strict base64 check, see :func:`binascii.a2b_base64`
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *padded* parameters.
 
    .. deprecated:: 3.15
       Accepting the ``+`` and ``/`` characters with an alternative alphabet
@@ -125,41 +134,52 @@ POST request.
    Base64 alphabet and return the decoded :class:`bytes`.
 
 
-.. function:: urlsafe_b64encode(s)
+.. function:: urlsafe_b64encode(s, *, padded=True)
 
    Encode :term:`bytes-like object` *s* using the
    URL- and filesystem-safe alphabet, which
    substitutes ``-`` instead of ``+`` and ``_`` instead of ``/`` in the
    standard Base64 alphabet, and return the encoded :class:`bytes`.  The result
-   can still contain ``=``.
+   can still contain ``=`` if *padded* is true (default).
+
+   .. versionchanged:: next
+      Added the *padded* parameter.
 
 
-.. function:: urlsafe_b64decode(s)
+.. function:: urlsafe_b64decode(s, *, padded=False)
 
    Decode :term:`bytes-like object` or ASCII string *s*
    using the URL- and filesystem-safe
    alphabet, which substitutes ``-`` instead of ``+`` and ``_`` instead of
    ``/`` in the standard Base64 alphabet, and return the decoded
    :class:`bytes`.
 
+   .. versionchanged:: next
+      Added the *padded* parameter.
+      Padding of input is no longer required by default.
+
    .. deprecated:: 3.15
       Accepting the ``+`` and ``/`` characters is now deprecated.
 
 
-.. function:: b32encode(s, *, wrapcol=0)
+.. function:: b32encode(s, *, padded=True, wrapcol=0)
 
    Encode the :term:`bytes-like object` *s* using Base32 and return the
    encoded :class:`bytes`.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 8.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not add any newlines.
 
    .. versionchanged:: next
-      Added the *wrapcol* parameter.
+      Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32decode(s, casefold=False, map01=None, *, ignorechars=b'')
+.. function:: b32decode(s, casefold=False, map01=None, *, padded=True, ignorechars=b'')
 
    Decode the Base32 encoded :term:`bytes-like object` or ASCII string *s* and
    return the decoded :class:`bytes`.
@@ -175,6 +195,11 @@ POST request.
    digit 0 is always mapped to the letter O).  For security purposes the default is
    ``None``, so that 0 and 1 are not allowed in the input.
 
+   If *padded* is true, the last group of 8 base 32 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *ignorechars*).
+
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
@@ -183,21 +208,21 @@ POST request.
    input.
 
    .. versionchanged:: next
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *padded* parameters.
 
 
-.. function:: b32hexencode(s, *, wrapcol=0)
+.. function:: b32hexencode(s, *, padded=True, wrapcol=0)
 
    Similar to :func:`b32encode` but uses the Extended Hex Alphabet, as defined in
    :rfc:`4648`.
 
    .. versionadded:: 3.10
 
    .. versionchanged:: next
-      Added the *wrapcol* parameter.
+      Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32hexdecode(s, casefold=False, *, ignorechars=b'')
+.. function:: b32hexdecode(s, casefold=False, *, padded=True, ignorechars=b'')
 
    Similar to :func:`b32decode` but uses the Extended Hex Alphabet, as defined in
    :rfc:`4648`.
@@ -210,7 +235,7 @@ POST request.
    .. versionadded:: 3.10
 
    .. versionchanged:: next
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *padded* parameters.
 
 
 .. function:: b16encode(s, *, wrapcol=0)

Doc/library/binascii.rst

@@ -48,15 +48,20 @@ The :mod:`!binascii` module defines the following functions:
       Added the *backtick* parameter.
 
 
-.. function:: a2b_base64(string, /, *, alphabet=BASE64_ALPHABET, strict_mode=False)
-              a2b_base64(string, /, *, ignorechars, alphabet=BASE64_ALPHABET, strict_mode=True)
+.. function:: a2b_base64(string, /, *, padded=True, alphabet=BASE64_ALPHABET, strict_mode=False)
+              a2b_base64(string, /, *, ignorechars, padded=True, alphabet=BASE64_ALPHABET, strict_mode=True)
 
    Convert a block of base64 data back to binary and return the binary data. More
    than one line may be passed at a time.
 
    Optional *alphabet* must be a :class:`bytes` object of length 64 which
    specifies an alternative alphabet.
 
+   If *padded* is true, the last group of 4 base 64 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *strict_mode* and *ignorechars*).
+
    If *ignorechars* is specified, it should be a :term:`bytes-like object`
    containing characters to ignore from the input when *strict_mode* is true.
    If *ignorechars* contains the pad character ``'='``,  the pad characters
@@ -79,14 +84,18 @@ The :mod:`!binascii` module defines the following functions:
       Added the *strict_mode* parameter.
 
    .. versionchanged:: 3.15
-      Added the *alphabet* and *ignorechars* parameters.
+      Added the *alphabet*, *ignorechars* and *padded* parameters.
 
 
-.. function:: b2a_base64(data, *, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)
+.. function:: b2a_base64(data, *, padded=True, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)
 
    Convert binary data to a line(s) of ASCII characters in base64 coding,
    as specified in :rfc:`4648`.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 4.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not insert any newlines.
@@ -98,7 +107,7 @@ The :mod:`!binascii` module defines the following functions:
       Added the *newline* parameter.
 
    .. versionchanged:: 3.15
-      Added the *alphabet* and *wrapcol* parameters.
+      Added the *alphabet*, *padded* and *wrapcol* parameters.
 
 
 .. function:: a2b_ascii85(string, /, *, foldspaces=False, adobe=False, ignorechars=b'')
@@ -190,7 +199,7 @@ The :mod:`!binascii` module defines the following functions:
    .. versionadded:: 3.15
 
 
-.. function:: a2b_base32(string, /, *, alphabet=BASE32_ALPHABET, ignorechars=b'')
+.. function:: a2b_base32(string, /, *, padded=True, alphabet=BASE32_ALPHABET, ignorechars=b'')
 
    Convert base32 data back to binary and return the binary data.
 
@@ -208,6 +217,11 @@ The :mod:`!binascii` module defines the following functions:
    Optional *alphabet* must be a :class:`bytes` object of length 32 which
    specifies an alternative alphabet.
 
+   If *padded* is true, the last group of 8 base 32 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *ignorechars*).
+
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
    If *ignorechars* contains the pad character ``'='``,  the pad characters
@@ -218,14 +232,18 @@ The :mod:`!binascii` module defines the following functions:
 
    .. versionadded:: next
 
-.. function:: b2a_base32(data, /, *, alphabet=BASE32_ALPHABET, wrapcol=0)
+.. function:: b2a_base32(data, /, *, padded=True, alphabet=BASE32_ALPHABET, wrapcol=0)
 
    Convert binary data to a line of ASCII characters in base32 coding,
    as specified in :rfc:`4648`. The return value is the converted line.
 
    Optional *alphabet* must be a :term:`bytes-like object` of length 32 which
    specifies an alternative alphabet.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 8.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not insert any newlines.

Doc/whatsnew/3.15.rst

@@ -657,6 +657,13 @@ base64
 * Added the *pad* parameter in :func:`~base64.z85encode`.
   (Contributed by Hauke Dämpfling in :gh:`143103`.)
 
+* Added the *padded* parameter in
+  :func:`~base64.b32encode`, :func:`~base64.b32decode`,
+  :func:`~base64.b32hexencode`, :func:`~base64.b32hexdecode`,
+  :func:`~base64.b64encode`, :func:`~base64.b64decode`,
+  :func:`~base64.urlsafe_b64encode`, and :func:`~base64.urlsafe_b64decode`.
+  (Contributed by Serhiy Storchaka in :gh:`73613`.)
+
 * Added the *wrapcol* parameter in :func:`~base64.b16encode`,
   :func:`~base64.b32encode`, :func:`~base64.b32hexencode`,
   :func:`~base64.b64encode`, :func:`~base64.b85encode`, and
@@ -686,6 +693,11 @@ binascii
 
   (Contributed by James Seo and Serhiy Storchaka in :gh:`101178`.)
 
+* Added the *padded* parameter in
+  :func:`~binascii.b2a_base32`, :func:`~binascii.a2b_base32`,
+  :func:`~binascii.b2a_base64`, and :func:`~binascii.a2b_base64`.
+  (Contributed by Serhiy Storchaka in :gh:`73613`.)
+
 * Added the *wrapcol* parameter in :func:`~binascii.b2a_base64`.
   (Contributed by Serhiy Storchaka in :gh:`143214`.)
 
@@ -2022,3 +2034,9 @@ that may require changes to your code.
   *dest* is now ``'foo'`` instead of ``'f'``.
   Pass an explicit *dest* argument to preserve the old behavior.
   (Contributed by Serhiy Storchaka in :gh:`138697`.)
+
+* Padding of input no longer required in :func:`base64.urlsafe_b64decode`.
+  Pass a new argument ``padded=True`` or use :func:`base64.b64decode`
+  with argument ``altchars=b'-_'`` (this works with older Python versions)
+  to make padding required.
+  (Contributed by Serhiy Storchaka in :gh:`73613`.)

Include/internal/pycore_global_objects_fini_generated.h

@@ -697,6 +697,7 @@ struct _Py_global_strings {
         STRUCT_FOR_ID(overlapped)
         STRUCT_FOR_ID(owner)
         STRUCT_FOR_ID(pad)
+        STRUCT_FOR_ID(padded)
         STRUCT_FOR_ID(pages)
         STRUCT_FOR_ID(parameter)
         STRUCT_FOR_ID(parent)