Skip to content

refactor: mv durabletask-python into sdk#963

Open
sicoyle wants to merge 27 commits intodapr:mainfrom
sicoyle:refactor/mv-dt-here
Open

refactor: mv durabletask-python into sdk#963
sicoyle wants to merge 27 commits intodapr:mainfrom
sicoyle:refactor/mv-dt-here

Conversation

@sicoyle
Copy link
Copy Markdown
Contributor

@sicoyle sicoyle commented Mar 24, 2026

Description

Move durabletask python into python sdk
https://github.com/dapr/durabletask-python

  1. Moved durabletask/ into ext/dapr-ext-workflow/dapr/ext/workflow/_durabletask/ — vendored as a private internal package, no more separate PyPI dependency
  2. Rewrote all imports across ext-workflow source, tests, and examples from durabletask.* to dapr.ext.workflow._durabletask.*; removed the try/except import fallbacks
  3. Updated CI — deleted durabletask-validation.yml, removed standalone publish/test steps from build workflows; durabletask now tested and linted alongside everything else
  4. Updated license headers on all vendored files from Microsoft/MIT to Dapr/Apache 2.0 (2026)
  5. Cleaned up build config — removed durabletask-dapr from setup.cfg, dev-requirements.txt, and tox.ini; added pytest for durabletask unit tests; narrowed ruff exclude to only proto files

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation

Signed-off-by: Samantha Coyle <sam@diagrid.io>
@sicoyle sicoyle requested review from a team as code owners March 24, 2026 14:02
Copilot AI review requested due to automatic review settings March 24, 2026 14:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Moves the durabletask-python SDK into this repository as a first-class package, adding its source, tests, examples, and CI/publishing hooks so it can be built and validated alongside the rest of the Python SDK workspace.

Changes:

  • Added the new durabletask/ package (client, task primitives, deterministic helpers, gRPC helpers, async client support, packaging metadata).
  • Added durabletask_tests/ and durabletask_examples/ to validate core behavior and sidecar-based E2E flows.
  • Updated GitHub workflows and publish pipelines to run DurableTask validation and publish artifacts.

Reviewed changes

Copilot reviewed 44 out of 47 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
durabletask_tests/durabletask/test_worker_stop.py Adds unit tests around worker stop/cleanup behavior.
durabletask_tests/durabletask/test_worker_concurrency_loop_async.py Adds async concurrency manager loop tests.
durabletask_tests/durabletask/test_worker_concurrency_loop.py Adds sync + threaded concurrency manager loop tests.
durabletask_tests/durabletask/test_task.py Adds unit tests for when_all / when_any primitives.
durabletask_tests/durabletask/test_serialization.py Adds tests for JSON (de)serialization behavior.
durabletask_tests/durabletask/test_registry.py Adds tests for worker registry validation and version selection.
durabletask_tests/durabletask/test_orchestration_wait.py Adds tests for client wait timeout semantics.
durabletask_tests/durabletask/test_orchestration_e2e_async.py Adds async E2E coverage against a sidecar.
durabletask_tests/durabletask/test_orchestration_e2e.py Adds sync E2E coverage against a sidecar.
durabletask_tests/durabletask/test_deterministic.py Adds determinism helper tests (random/uuid/time sequencing).
durabletask_tests/durabletask/test_concurrency_options.py Adds tests for concurrency options defaults/overrides.
durabletask_tests/durabletask/test_client_async.py Adds tests for async client channel/interceptor behavior.
durabletask_tests/durabletask/test_client.py Adds tests for sync client channel/interceptor behavior.
durabletask_tests/durabletask/test_activity_executor.py Adds tests for activity executor input/error handling.
durabletask_tests/init.py Declares test package marker.
durabletask_examples/human_interaction.py Adds human-interaction example (external event + timeout).
durabletask_examples/fanout_fanin.py Adds fan-out/fan-in example.
durabletask_examples/activity_sequence.py Adds sequence/chaining example.
durabletask_examples/README.md Adds mechanical-markdown driven example validation doc.
durabletask/task.py Introduces task model primitives and retry policy types.
durabletask/pyproject.toml Adds standalone packaging metadata for durabletask distribution.
durabletask/internal/shared.py Adds channel creation helpers, logging helper, JSON codec.
durabletask/internal/orchestrator_service_pb2_grpc.py Adds generated gRPC service bindings.
durabletask/internal/helpers.py Adds protobuf helper constructors and misc helpers.
durabletask/internal/grpc_interceptor.py Adds sync gRPC metadata interceptor.
durabletask/internal/PROTO_SOURCE_COMMIT_HASH Records source commit for generated protos.
durabletask/deterministic.py Adds deterministic helpers/mixin (uuid/random/time sequencing).
durabletask/client.py Adds sync client implementation.
durabletask/aio/internal/shared.py Adds async channel creation helper.
durabletask/aio/internal/grpc_interceptor.py Adds async gRPC metadata interceptor.
durabletask/aio/internal/init.py Declares async internal package.
durabletask/aio/client.py Adds async client implementation.
durabletask/aio/init.py Exposes async client public API.
durabletask/init.py Declares durabletask package root.
durabletask/README.md Adds durabletask package README and usage docs.
durabletask/Makefile Adds durabletask-local dev/test/proto-generation targets.
durabletask/LICENSE Adds durabletask package license file.
durabletask/CHANGELOG.md Adds durabletask package changelog.
.gitignore Updates ignore patterns (tox/nox/IDEs/db artifacts).
.github/workflows/durabletask-validation.yml Adds DurableTask-specific CI workflow (lint/tests/e2e/examples/publish).
.github/workflows/build.yaml Adds DurableTask e2e/tests/examples execution to main build workflow.
.github/workflows/build-tag.yaml Adds DurableTask publish step for tag builds.
.github/workflows/build-push-to-main.yaml Adds DurableTask publish step for push-to-main workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sicoyle added 11 commits March 24, 2026 16:46
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 25, 2026

Codecov Report

❌ Patch coverage is 67.97840% with 1660 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.34%. Comparing base (bffb749) to head (f4096b1).
⚠️ Report is 97 commits behind head on main.

Files with missing lines Patch % Lines
...rkflow/tests/durabletask/test_orchestration_e2e.py 5.93% 444 Missing ⚠️
...-workflow/dapr/ext/workflow/_durabletask/worker.py 61.96% 418 Missing ⚠️
.../tests/durabletask/test_orchestration_e2e_async.py 7.05% 303 Missing ⚠️
...abletask/internal/orchestrator_service_pb2_grpc.py 52.66% 71 Missing ⚠️
.../_durabletask/internal/orchestrator_service_pb2.py 15.38% 66 Missing ⚠️
...kflow/dapr/ext/workflow/_durabletask/aio/client.py 37.62% 63 Missing ⚠️
...-workflow/dapr/ext/workflow/_durabletask/client.py 61.00% 62 Missing ⚠️
...rkflow/_durabletask/internal/history_events_pb2.py 20.68% 46 Missing ⚠️
...xt-workflow/dapr/ext/workflow/_durabletask/task.py 88.29% 33 Missing ⚠️
...w/tests/durabletask/test_orchestration_executor.py 96.45% 33 Missing ⚠️
... and 11 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #963      +/-   ##
==========================================
- Coverage   86.63%   80.34%   -6.29%     
==========================================
  Files          84      136      +52     
  Lines        4473    12568    +8095     
==========================================
+ Hits         3875    10098    +6223     
- Misses        598     2470    +1872     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sicoyle added 10 commits March 25, 2026 15:21
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
@sicoyle
Copy link
Copy Markdown
Contributor Author

sicoyle commented Mar 26, 2026

btw pls ignore the codecov failures as I am copy/pasting durabletask in with changes to make build happy. I do not want to bridge the test caps of durabletask in this PR to limit scope.

Copy link
Copy Markdown
Member

@acroca acroca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to change all license headers to use dapr's license right?

Comment on lines +16 to +17
if not logger.handlers:
logger.addHandler(log_handler)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the check?

Suggested change
if not logger.handlers:
logger.addHandler(log_handler)
logger.addHandler(log_handler)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this was a copilot feedback fix as well as I noticed double logging here and there personally that this addresses

sicoyle added 3 commits March 27, 2026 11:19
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Samantha Coyle <sam@diagrid.io>
@sicoyle
Copy link
Copy Markdown
Contributor Author

sicoyle commented Mar 27, 2026

@acroca I went ahead and better integrated durable task. See the updated PR description :)

sicoyle added 2 commits March 27, 2026 12:56
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants