Skip to content

Full architecture overhaul: bidirectional HCL2 ↔ JSON pipeline with typed rule classes#203

Merged
kkozik-amplify merged 42 commits intorelease/8.xfrom
transformer-overhaul
Mar 9, 2026
Merged

Full architecture overhaul: bidirectional HCL2 ↔ JSON pipeline with typed rule classes#203
kkozik-amplify merged 42 commits intorelease/8.xfrom
transformer-overhaul

Conversation

@kkozik-amplify
Copy link
Collaborator

@kkozik-amplify kkozik-amplify commented Mar 26, 2025

Summary

Replaces the old DictTransformer (Lark parse tree → Python dict) with a two-stage pipeline that routes through a typed intermediate representation:

Forward:  HCL2 Text → Lark Parse Tree → LarkElement Tree → Python Dict/JSON
Reverse:  Python Dict/JSON → LarkElement Tree → Lark Tree → HCL2 Text

This enables full round-trip fidelity: parse HCL2 to JSON and reconstruct it back to semantically equivalent HCL2.

What changed

  • LarkElement rule classes (hcl2/rules/) — one typed class per grammar rule with property accessors, serialize(), and _children_layout annotations
  • RuleTransformer (hcl2/transformer.py) — maps Lark parse tree nodes to LarkElement tree
  • Deserializer (hcl2/deserializer.py) — Python dict → LarkElement tree (reverse of serialize)
  • HCLReconstructor (hcl2/reconstructor.py) — LarkElement tree → Lark tree → HCL2 text with correct spacing
  • Formatter (hcl2/formatter.py) — vertical alignment and whitespace normalization on LarkElement trees
  • Public API (hcl2/api.py) — load/loads, dump/dumps following the json module convention, plus intermediate-stage functions
  • CLI (cli/) — hcl2tojson and jsontohcl2 entry points with flags mapping to option dataclasses
  • SerializationOptions / DeserializerOptions / FormatterOptions — dataclass-based configuration for all pipeline stages
  • Tests — reorganized into test/unit/ (direct rule instantiation) and test/integration/ (golden-file round-trip suites)

Highlights

  • Every grammar rule has a corresponding typed Python class with child accessors
  • Block vs object semantics preserved via __is_block__ markers
  • Heredoc round-tripping with optional trim support
  • Operator precedence parenthesization (optional force_parens mode)
  • Comment preservation through with_comments serialization option
  • Scientific notation, splat expressions, for-expressions all round-trip correctly
  • 594 tests passing and 99% coverage

Breaking changes

  • hcl2.load() / hcl2.loads() now accept keyword-only option parameters instead of positional booleans
  • Old DictTransformer removed — use hcl2.transform() for the LarkElement tree or hcl2.load() for dicts
  • hcl2tojson CLI flags changed to --flag style (e.g. --with-meta, --wrap-objects)

require heredoc openers to be on their on separate line in lark grammar;
whitespace trimming based on current implementation in dict_transformer.py;
…her fixes and changes:

* preserve order of serialized attributes and blocks
* make RuleTransformer.__default_token__ differentiate between StringToken and StaticStringToken
* add separate ProviderFunctionCallRule class for more accurate reconstruction
* FunctionCallRule.serialize - properly serialize into dollar string
* remove unused import
@kkozik-amplify kkozik-amplify marked this pull request as ready for review March 7, 2026 21:43
@kkozik-amplify kkozik-amplify requested a review from a team as a code owner March 7, 2026 21:43
@kkozik-amplify kkozik-amplify marked this pull request as draft March 7, 2026 21:58
@kkozik-amplify kkozik-amplify removed the request for review from amplify-github March 7, 2026 22:03
@kkozik-amplify kkozik-amplify changed the title transformer overhaul draft Full architecture overhaul: bidirectional HCL2 ↔ JSON pipeline with typed rule classes Mar 7, 2026
@kkozik-amplify kkozik-amplify marked this pull request as ready for review March 7, 2026 22:59
@kkozik-amplify kkozik-amplify changed the base branch from main to release/8.x March 7, 2026 23:41
@kkozik-amplify kkozik-amplify merged commit 285c3a8 into release/8.x Mar 9, 2026
8 checks passed
@kkozik-amplify kkozik-amplify deleted the transformer-overhaul branch March 9, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants