Skip to content

[NOT READY FOR REVIEW] config mk to py, reduce dependency on make#4014

Draft
oharboe wants to merge 9 commits intoThe-OpenROAD-Project:masterfrom
Pinata-Consulting:config-mk-to-py
Draft

[NOT READY FOR REVIEW] config mk to py, reduce dependency on make#4014
oharboe wants to merge 9 commits intoThe-OpenROAD-Project:masterfrom
Pinata-Consulting:config-mk-to-py

Conversation

@oharboe
Copy link
Collaborator

@oharboe oharboe commented Mar 18, 2026

Platform config.py files

Each platform directory now has a config.py alongside config.mk.
The config.mk is a one-liner shim that calls config.py and evals
the output as Make code -- the same pattern as defaults.py in
variables.mk.

Why

Platform config.mk files use Make syntax to define ~30-100
environment variables per platform. These variables flow through to
TCL scripts via $::env(...). The Make syntax is untestable: there
is no way to write unit tests for variable resolution, especially
for asap7's $(foreach)/$(eval) VT variant generation.

Moving the logic to Python enables:

  • Unit testing: 137 pytest tests verify every Make construct
    (conditionals, variable indirection, template expansion, globbing,
    corner selection, VT loops) produces correct output.

  • JSON output for bazel-orfs: config.py --format=json returns a
    flat dict of all variables. bazel-orfs can call this directly
    instead of shelling out to Make, eliminating Make as a runtime
    dependency (Plan E, Phase 2).

  • Piecemeal migration: each platform migrated independently.
    Make users see zero behavior change.

How it works

  Make flow (unchanged):
    Makefile -> variables.mk -> include config.mk (shim)
                                  -> $(shell python config.py)
                                  -> $(foreach)/$(eval) output as Make code

  bazel-orfs (new capability):
    rules.bzl -> python config.py --format=json -> JSON dict

The shim in each config.mk:

$(foreach line,$(shell $(PYTHON_EXE) $(PLATFORM_DIR)/config.py),\
  $(eval export $(subst __SPACE__, ,$(line))))

This is identical to how defaults.py is consumed at variables.mk:48.

Why not port variables.mk too?

variables.mk computes tool paths (OPENROAD_EXE, YOSYS_EXE,
KLAYOUT_CMD), derived output paths (LOG_DIR, RESULTS_DIR), macro
wrappers (WRAPPED_LEFS), stream system selection (GDS vs OAS), and
Make debugging introspection. All of this is Make-flow plumbing that
bazel-orfs replaces with Bazel-native equivalents. The only shared
concern between Make and bazel-orfs consumers is the platform config
variables, which config.py now provides.

Platform migration complexity

Platform Lines Key constructs
nangate45 109 $(sort $(wildcard ...)) for GDS
sky130hs 108 $(PLATFORM) in paths
sky130hd 154 Long DONT_USE_CELLS += list
gf180 149 TRACK/METAL/KVALUE params, corner indirection, $(abspath)
ihp-sg13g2 182 FOOTPRINT_TCL conditional, $(shell sed) SDC parsing, $(origin)
asap7 217 VT variant templates, $(foreach)/$(eval) loop, $(addsuffix), CLUSTER_FLOPS

Reference

Plan E: https://github.com/The-OpenROAD-Project/bazel-orfs/blob/main/docs/plans/drop-make-dependency.md

oharboe and others added 9 commits March 18, 2026 22:35
config.mk files may use $(shell $(PYTHON_EXE) ...) to generate
variables. PYTHON_EXE must be defined before the include directive
at variables.mk:40, not after it at the former line 73.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Migrate all 6 platform configurations from Make syntax to Python:
nangate45, sky130hs, sky130hd, gf180, ihp-sg13g2, asap7.

Each config.py generates identical variable assignments to the
original config.mk. The config.mk becomes a one-liner shim that
calls config.py via $(foreach)/$(eval), the same pattern as
defaults.py in variables.mk.

This enables:
- Unit testing of platform configurations with pytest
- JSON output for bazel-orfs (--format=json) to bypass Make
- Piecemeal migration per Plan E of the bazel-orfs Make dependency
  reduction effort

Make users see zero change in behavior.

variables.mk does not need porting to Python. Its logic (tool
discovery, derived paths, WRAPPED_LEFS, STREAM_SYSTEM, NUM_CORES)
is Make-flow plumbing that bazel-orfs replaces with Bazel-native
equivalents. The only shared concern is platform config, which
config.py now provides.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Tests cover all Make constructs used across the 6 platform config.mk
files: unconditional/conditional assignment, +=, glob, ifeq/ifneq,
variable indirection, PLACEHOLDER template expansion, foreach/eval
VT loop, SDC clock extraction, corner selection, CLUSTER_FLOPS,
design-config overrides, and Make shim round-trip format validation.

Includes __main__ guard so Jenkins CI can run it with plain python.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
CI runs `python "$file"` on every .py in flow/test/.
conftest.py and test_platform_config.py import pytest which
isn't installed in the testUtilScripts CI environment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
…tput

config.py now outputs $(PLATFORM_DIR)/... and $(PLATFORM) in values
instead of baking in absolute paths. Consumers expand on their end:
- Make: $(eval) expands naturally
- bazel-orfs: Python substitution when the rule executes

Glob results are expanded at runtime for filesystem access, then
unexpanded in output. to_dict() still returns expanded paths for
unit test assertions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Verify that config.py output (both Make and JSON) uses symbolic
$(PLATFORM_DIR) and $(PLATFORM) references instead of absolute
paths. Tests cover: JSON output, Make output, glob results, and
$(PLATFORM) in sky130hd/sky130hs filenames.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Documents: symbolic path references, assignment semantics, glob
handling, env var inputs, why variables.mk doesn't need porting,
and the Make functions already resolved by config.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
All other test files in flow/test/ use stdlib unittest. Convert
test_platform_config.py to match and remove the pytest dependency
that broke CI (ModuleNotFoundError: No module named 'pytest').

Delete conftest.py which only existed for pytest marker config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe oharboe marked this pull request as draft March 18, 2026 22:38
@oharboe oharboe changed the title config mk to py, reduce dependency on make [NOT READY FOR REVIEW] config mk to py, reduce dependency on make Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant