gh-137855: Lazy import inspect module in dataclasses

[danielhollas]

inspect module is slow to import (see #117865) and is dragging down dataclasses with it.

There are currently only two uses of inspect in dataclasses, but they are a bit tricky to inline since they are on a direct code path when the @dataclass decorator is executed.

1. inspect.signature is used to autogenerate class docstring (if one is not provided already)
2. inspect.unwrap is used in a rather esoteric code path only for slotted classes, added in gh-90562: Improve zero argument support for super() in dataclasses when slots=True #124692)

For 1. I have used a descriptor protocol to generate the __doc__ attribute on demand (this is my first time messing with descriptors, apologies if I overlooked something).

For 2. can be defferred by calling the unwrap functions only when really necessary (and hopefully this path is not common)

Benchmarks

./python -Ximporttime -c "import dataclasses"

Before

❯ hyperfine -w 10 './python -c "import dataclasses"' 
Benchmark 1: ./python -c "import dataclasses"
  Time (mean ± σ):      19.2 ms ±   2.1 ms    [User: 14.9 ms, System: 4.1 ms]
  Range (min … max):    16.6 ms …  24.1 ms    154 runs

After

❯ hyperfine -w 10 './python -c "import dataclasses"' 
Benchmark 1: ./python -c "import dataclasses"
  Time (mean ± σ):      15.2 ms ±   1.8 ms    [User: 11.2 ms, System: 3.7 ms]
  Range (min … max):    12.9 ms …  19.4 ms    212 runs

Overall seems to be a solid 20-30% improvement.

• Issue: Improve import time of various stdlib modules #137855

[danielhollas]

This check was copied from the while loop in inspect.unwrap

[danielhollas]

Deferring the call to inspect.signature to create __doc__ might also speed up the dataclasses creation a bit. e.g. here's what I see in a cProfile of the _colorize module which uses dataclasses heavily (on main branch)

$ ./python -m cProfile -m _colorize | head 20
         5604 function calls (5484 primitive calls) in 0.007 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      9/1    0.004    0.000    0.007    0.007 {built-in method builtins.exec}
        1    0.000    0.000    0.007    0.007 <string>:1(<module>)
        1    0.000    0.000    0.007    0.007 <frozen runpy>:199(run_module)
        1    0.000    0.000    0.007    0.007 <frozen runpy>:65(_run_code)
        1    0.000    0.000    0.007    0.007 _colorize.py:1(<module>)
        7    0.000    0.000    0.006    0.001 dataclasses.py:1432(wrap)
        7    0.000    0.000    0.006    0.001 dataclasses.py:986(_process_class)
        7    0.000    0.000    0.004    0.001 dataclasses.py:478(add_fns_to_class)
        4    0.000    0.000    0.001    0.000 inspect.py:3343(signature)
        4    0.000    0.000    0.001    0.000 inspect.py:3056(from_callable)
     12/4    0.000    0.000    0.001    0.000 inspect.py:2437(_signature_from_callable)
        4    0.000    0.000    0.001    0.000 inspect.py:2331(_signature_from_function)
    71/11    0.000    0.000    0.000    0.000 annotationlib.py:907(get_annotations)
        8    0.000    0.000    0.000    0.000 dataclasses.py:541(__annotate__)
     17/9    0.000    0.000    0.000    0.000 annotationlib.py:1114(_get_and_call_annotate)

[danielhollas]

This is at this point perhaps the first use of lazy import in stdlib. Not sure where it should go in terms of sorting the imports?

[johnslavik]

Not sure we follow any particular guideline. Personally I'd keep it in the same line as the original import -- the change is then more localized. Not my call though!

[danielhollas]

Agree, I moved it, since another core dev expressed similar sentiment over at #144756

[johnslavik]

cc @JelleZijlstra 😁

[sergey-miryanov]

I believe we should add a NEWS entry, because it is user-facing change (at least in the performance terms).

[DavidCEllis]

As a side effect that may be worth noting, deferring __doc__ generation also 'fixes' the output for self referential dataclasses¹.

@dataclass
class Example:
    examples: list[Example]

print(Example.__doc__)

Before:

"Example(examples: list[ForwardRef('Example', is_class=True, owner=<class '__main__.Example'>)])"

PR:

'Example(examples: list[__main__.Example])'

---

Every dataclass is getting its own _AutoDocString instance, could they share the same instance?

Following on from this, the re and copy imports could also potentially be deferred, copy is only used in some cases for the serialization methods and re is only used for string annotations.

Deferring re only makes sense if inspect is also deferred as inspect currently eagerly imports it (I see you have a PR to change that too though).

Footnotes

1. Arguably dataclasses should probably be doing something with Format.STRING here, other forward references are still ugly but that would be a separate issue. ↩

[johnslavik]

could they share the same instance?

They could; that would be nice.

[danielhollas]

Every dataclass is getting its own _AutoDocString instance, could they share the same instance?
Following on from this, the re and copy imports could also potentially be deferred, copy is only used in some cases for the serialization methods and re is only used for string annotations.

Done, thank you for the suggestions!
The lazy import or re is a bit more annoying since it is used to define a global compiled regex.
I am not sure I can judge whether the extra complexity is worth it --- the module will be imported as soon as you have e.g. from __future__ import annotations anywhere where dataclasses are defined.

Now the import time is dominated only by the "annotationlib" module

EDIT: Ugh, GitHub, you okay? The pushed new commits are still not appearing on the PR.

[DavidCEllis]

I am not sure I can judge whether the extra complexity is worth it --- the module will be imported as soon as you have e.g. from __future__ import annotations anywhere where dataclasses are defined.

Ideally with the new annotations, uses of __future__ annotations will become less common and so they won't need the import. Modules like _colorize should already benefit as they don't use __future__ annotations.

[sergey-miryanov]

I'm not sure we should mix changes for _MODULE_IDENTIFIER_RE and __doc__ in one PR. But I'm not an expert here.