[API Compatibility] Change compatibility apis to ChangePrefixMatcher, inject paddle.enable_compat(), fix test environment context pollution -part by Manfredss · Pull Request #895 · PaddlePaddle/PaConvert

Manfredss · 2026-06-29T04:02:56Z

PR Docs

For the 24 torch.* APIs that are aligned through paddle.compat.*, switch the converter to the "prefix-only + enable_compat" strategy instead of mapping each to paddle.compat.X explicitly:

api_mapping.json — change those 24 entries from ChangeAPIMatcher → paddle.compat.X to {"Matcher": "ChangePrefixMatcher"} . Converted calls now use the corresponding paddle.* prefix while preserving Torch-style signatures and argument names. For example, torch.sort(x, dim=-1) is converted to: paddle.sort(x, dim=-1). At runtime, enable_compat(level=2) dispatches the external paddle.sort call to its Torch-aligned implementation. Related mappings are adjusted accordingly: torch.nn.Softmax2d uses dim instead of the native Paddle axis; torch.nn.functional.softmin preserves input and dim.
transformer/import_transformer.py — for converted files whose mapped packages include paddle, inject a deduplicated import paddle and a single paddle.enable_compat(level=2). In default mode, the code is inserted after the existing import block. In min mode, it is inserted after the module docstring and any __future__ imports, but before retained Torch imports, ensuring that the compat proxy is enabled before Torch is imported.
tests/apibase.py — paddle.enable_compat(level=2) installs process-wide Torch proxy and Paddle namespace dispatch state. Call the shared disable_paddle_compat() helper immediately before every real PyTorch reference execution, ensuring that the reference always runs against real PyTorch, including when a test invokes run() multiple times. Cross-test process state is restored separately by the autouse fixture in tests/conftest.py.

PR APIs

torch.Tensor.allclose
torch.Tensor.max
torch.Tensor.median
torch.Tensor.min
torch.Tensor.sort

torch.allclose
torch.equal
torch.max
torch.median
torch.min
torch.nanmedian
torch.seed
torch.sort
torch.split
torch.unique

torch.nn.Linear
torch.nn.Softmax
torch.nn.Unfold

torch.nn.functional.linear
torch.nn.functional.log_softmax
torch.nn.functional.pad
torch.nn.functional.scaled_dot_product_attention
torch.nn.functional.softmax
torch.nn.functional.unfold

related Paddle PR:

[API Compatibility] enhance paddle.enable_compat -part Paddle#79391

paddle.Tensor.split is a known failure. It will be fixed in

[API Compatibility] Support ChangePrefixMatcher for aligned nn classes -part Paddle#79539

Also in this PR

The issue of Cross-Test Global State Pollution in conftest.py is fixed. Originally PaConvert tests execute the original torch code and converted Paddle code sequentially in the same pytest worker. Some tests modify process-wide state without restoring it, including:

the compat proxy installed by paddle.enable_compat(level=2)
default device, dtype, and gradient state in PyTorch and Paddle;
environment variables such as CPU_NUM;
runtime monkeypatches applied to paddle.Tensor and paddle.nn.* classes.

Some generated helpers are defined inside temporary exec namespaces. After those namespaces are cleared, the class-level monkeypatches may remain and reference invalid globals, affecting subsequent tests. Therefore, cleaning up only the compat proxy is insufficient.

This change adds an autouse fixture that snapshots the relevant state before each test and restores it afterward. It disables compat, restores PyTorch/Paddle defaults and environment variables, and reverts modified class attributes. This isolates tests without changing PaConvert’s conversion behavior.

Co-authored by Claude and Codex

…to patch_compat

… calls hit the alias The default enable_compat() (level=1) does not alias paddle.*, so prefix-converted calls (torch.X -> paddle.X) would run native and reject torch-style kwargs. level=2 turns on the paddle.* alias; caller-aware dispatch keeps paddle internals native. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…to patch_compat

…eset import_transformer.py: - Inject paddle.enable_compat(level=2) via a transform() post-pass so it lands after the docstring, __future__ imports and the import block (fixes a pre-existing SyntaxError where injected imports preceded `from __future__`). - Gate on real torch imports (torch_packages), not paddle_package_list, so a torch-free file importing only os/einops/setuptools no longer gains a needless paddle import or the process-global compat switch. tests: - Rename confest.py -> conftest.py so the autouse compat-reset fixture is loaded by pytest (was dead due to the typo). - Dedupe the disable logic into conftest.disable_paddle_compat(), imported by apibase and called before each torch reference exec. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…into patch_compat

…evel=2) now.

…ompat Switch Tensor.allclose and Tensor.sort to ChangePrefixMatcher and keep softmin's softmax call using torch arg names (input/dim), since compat paddle APIs now accept torch-style args and reject paddle-native ones. Update tests to inject enable_compat(level=2); fix torchvision model_apibase proxy leak around the torch reference forward.

zhwesky2010 · 2026-07-10T05:07:55Z

+        # enable_compat is injected in transform() (needs all imports in the body).
+        # Gate on real torch imports, not paddle_package_list: the latter also holds
+        # MAY_TORCH packages (os/einops/setuptools) that need no compat switch.
+        if self.imports_map[self.file]["torch_packages"]:


简化下代码，看有无更简单的写法，是否只需要修改visit_Module就可以？

zhwesky2010 · 2026-07-10T05:08:11Z

PR冲突了

zhwesky2010

这些插入操作能不能复用已有的函数record_scope

zhwesky2010 · 2026-07-13T08:12:38Z

注意代码风格简化并贴合已有架构来写，不要让AI随意发挥：

贴合已有架构、功能和风格来开发，不要从0开始（AI不懂代码库架构喜欢重起炉灶）
非代码复用的情况下，不需要封各种类函数（只会增加代码复杂度并无意义）
移除冗余注释

zhwesky2010 · 2026-07-13T08:16:19Z

+                    return True
+        return False
+
+    def _inject_enable_compat(self):


这个使用record_scope不能插入吗？插入这个不需要从0开始重写吧

Bring the latest API compatibility coverage into the branch while preserving the record_scope-based compat injection and test state isolation fixes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

paddle-bot · 2026-07-14T05:58:28Z

Thanks for your contribution!

Align expected diffs with compat injection and intentional ChangePrefixMatcher behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Deduplicate paddle_package_list in ImportTransformer.visit_Module: since imports are now batched into one record_scope call, its cross-call dedup no longer removes duplicates, producing two 'import paddle' lines when a file imports multiple torch-family packages (e.g. torchvision + datasets). Regenerate default-mode goldens for the paddle.enable_compat(level=2) injection and the intentional compat-prefix removals (equal, BatchNorm1d/2d), matching the min-mode baseline update in f7e1c04. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

zhwesky2010

简单问题不用复杂化：这里的node进来时已经是一个处理干净的node，所有torch相关的import都清除干净了，只用在这个基础上插入好paddle相关import即可

zhwesky2010 · 2026-07-14T11:38:53Z

-            line_NO += 1
+        import_nodes = []
+        for paddle_package in dict.fromkeys(paddle_package_list):
+            import_nodes.extend(ast.parse(f"import {paddle_package}").body)


直接在这里判断是否为 paddle_package=='paddle'

如果是，则import_nodes.extend(ast.parse(f"import paddle;paddle.enable_compat(2)").body)

并且该操作仅执行一次（当然执行多次问题也不大，后续会被isort校正）

zhwesky2010 · 2026-07-14T11:39:48Z

+
+        has_torch_package = bool(self.imports_map[self.file]["torch_packages"])
+
+        has_enable_compat = any(


这些判断都需要吗？后面也会经过isort、black的格式化处理，移除多余import，这里不要想的太复杂了

zhwesky2010 · 2026-07-14T11:41:10Z

+        if has_torch_package and not has_enable_compat:
+            import_nodes.extend(ast.parse("paddle.enable_compat(level=2)").body)
+
+        import_end = 1 if node.body and ast.get_docstring(node, clean=False) else 0


for循环，self.record_scope((self.root, "body", line_NO), import_nodes) 就行了吧。

原始的node在visit_Import、visit_ImportFrom时就已经被清除，原始node的import内容应该已经被清理掉了。

另外加上log_info，这个是比较重要的调试信息

…a single paddle.enable_compat(level=2) on top of the already-cleaned node, dropping redundant enable_compat/torch-package scans and adding log_info

…into patch_compat

zhwesky2010 · 2026-07-16T03:43:26Z

+        # Under `paddle.enable_compat(level=2)` the prefix-converted calls
+        # (torch.X -> paddle.X) resolve to the torch-aligned paddle.compat.* impls,
+        # so inject it once when `import paddle` is added. isort/black dedupe repeats.
+        if "paddle" in paddle_package_list:


这个应该得 == "paddle" ，然后做个一次性判断 "enable_compat" not in import_code

我看一下

…to patch_compat

…hangePrefixMathcer.

zhwesky2010 · 2026-07-27T03:22:33Z

+                while (
+                    import_end < len(node.body)
+                    and isinstance(node.body[import_end], ast.ImportFrom)
+                    and node.body[import_end].module == "__future__"


这个写得有点hard code，直接修改diff文件吧

zhwesky2010 · 2026-07-27T03:23:58Z

+
+python -m pytest -v -s -p no:warnings "${PYTEST_IGNORE[@]}" \
+    --ignore=tests/test_cuda_stream.py \
+    --ignore=tests/test_cuda_CUDAGraph.py \


这三个单测不跑了吗，建议在这里加个list维护一个隔离的单测名单，分两批来跑

后面这个list还可以根据实际情况进一步扩容

跑的，分别在 53、59、64 行单独跑了

zhwesky2010 · 2026-07-27T03:25:34Z

+            # internal `import torch` / `torch.SymInt` etc. resolve through the
+            # paddle proxy and blow up. Disable compat for the torch forward, then
+            # restore it so the paddle forward matches how a user runs the output.
+            from paddle.compat.proxy import TORCH_PROXY_FINDER


这个是影响到了paddle内部的paddle.vision.* API吗

不是 paddle.vision.* API 本身受到影响

转换后的 Paddle 代码执行 paddle.enable_compat(level=2) 后会全局启用 Torch proxy；随后执行真实 torchvision 模型 forward 时，其内部的 Torch 引用可能被代理到 Paddle；因此这里临时关闭 compat，完成 Torch 侧计算后再恢复

zhwesky2010 · 2026-07-27T03:26:08Z

                )
                assert paddle_code == expect_paddle_code, error_msg
        elif compared_tensor_names:
+            disable_paddle_compat()


这里为何需要disable

APIBase.run() 会先执行真实 torch 代码，再执行转换后的 paddle 代码。paddle 代码中的 paddle.enable_compat(level=2) 执行后，compat 状态会一直保持到这次 run() 结束之后；如果同一个 pytest case 再次调用 run()，那 autouse fixture 这个时候还没有执行清理，第二次运行的 import torch 就会被代理到 paddle，导致 torch 基准结果实际上由 paddle 计算所以必须在每次执行 torch 代码前关闭 compat。

zhwesky2010 · 2026-07-27T03:26:12Z

                except Exception as e:
                    raise AssertionError(f"Unable to align results: {e}")
        else:
+            disable_paddle_compat()


这里为何需要disable

zhwesky2010 · 2026-07-27T03:27:01Z

+The logic is lazy and best-effort: it only touches a framework that is already in
+``sys.modules``, so it never forces an import of torch/paddle (which would change
+import ordering) for tests that don't use them.
+"""


这个的作用是，看起来是paddle的改动还有些判断不完善吗，是不是应该去改paddle？

尽量是去完善问题的根源，避免在其他地方打补丁，不然就把坑留给用户了

Manfredss added 2 commits June 29, 2026 03:50

change for enable_compat

321f32b

add two left api

9bf7bfb

Manfredss closed this Jun 29, 2026

Manfredss reopened this Jun 29, 2026

Manfredss marked this pull request as draft June 29, 2026 12:59

Manfredss marked this pull request as ready for review June 29, 2026 13:00

paddle-bot Bot added the contributor External developers label Jun 29, 2026

Manfredss and others added 6 commits July 2, 2026 20:37

Merge branch 'master' of https://github.com/paddlepaddle/paconvert in…

204c7a1

…to patch_compat

Merge branch 'master' of https://github.com/paddlepaddle/paconvert in…

1ff4284

…to patch_compat

Merge branch 'master' of https://github.com/paddlepaddle/paconvert in…

020ce44

…to patch_compat

Merge branch 'patch_compat' of https://github.com/manfredss/paconvert …

bfff52a

…into patch_compat

Manfredss force-pushed the patch_compat branch from 74ba7ca to bfff52a Compare July 7, 2026 12:53

Manfredss added 4 commits July 8, 2026 08:22

update ChangePrefixMatcher for those can passed under enable_compat(l…

3dcd2a1

…evel=2) now.

ChangePrefixMatcher for BatchNorm1d/2d/3d

81a3c07

fix code conversion mis-alignment

f04bbf3

zhwesky2010 reviewed Jul 10, 2026

View reviewed changes

zhwesky2010 reviewed Jul 13, 2026

View reviewed changes

Manfredss and others added 2 commits July 14, 2026 02:53

Merge upstream master into patch_compat

9bef7f9

Bring the latest API compatibility coverage into the branch while preserving the record_scope-based compat injection and test state isolation fixes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

restore test script

838a8a5

Manfredss and others added 2 commits July 14, 2026 06:40

update min-mode consistency baselines

f7e1c04

Align expected diffs with compat injection and intentional ChangePrefixMatcher behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

zhwesky2010 reviewed Jul 14, 2026

View reviewed changes

Simplify ImportTransformer.visit_Module to insert paddle imports and …

14d8aeb

…a single paddle.enable_compat(level=2) on top of the already-cleaned node, dropping redundant enable_compat/torch-package scans and adding log_info

Merge branch 'patch_compat' of https://github.com/manfredss/paconvert …

e0808c4

…into patch_compat

zhwesky2010 reviewed Jul 16, 2026

View reviewed changes

paddle_package == paddle and add once

6bd002d

risemeup1111 mentioned this pull request Jul 16, 2026

[API Compatibility] enhance paddle.enable_compat -part PaddlePaddle/Paddle#79391

Merged

Manfredss added 5 commits July 17, 2026 02:02

Merge branch 'master' of https://github.com/paddlepaddle/paconvert in…

f8f344a

…to patch_compat

update after paddle changed

961af0c

fix

d8f6e9d

Merge branch 'master' of https://github.com/paddlepaddle/paconvert in…

62739d7

…to patch_compat

improve

7bb47d1

Manfredss changed the title ~~[API Compatibility] Change compatibility apis to ChangePrefixMatcher, inject paddle.enable_compat() -part~~ [API Compatibility] Change compatibility apis to ChangePrefixMatcher, inject paddle.enable_compat(), fix test environment context pollution -part Jul 24, 2026

Manfredss mentioned this pull request Jul 24, 2026

[TEST] try 24 api align PaddlePaddle/Paddle#79543

Draft

AvgPool1/2/3d, BatchNorm1/2/3d, MultiheadAttention, Tensor.split to C…

67be297

…hangePrefixMathcer.

zhwesky2010 reviewed Jul 27, 2026

View reviewed changes


		has_torch_package = bool(self.imports_map[self.file]["torch_packages"])

		has_enable_compat = any(

Uh oh!

Conversation

Manfredss commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Docs

PR APIs

Also in this PR

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhwesky2010 commented Jul 10, 2026

Uh oh!

zhwesky2010 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhwesky2010 commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paddle-bot Bot commented Jul 14, 2026

Uh oh!

zhwesky2010 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhwesky2010 Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhwesky2010 Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Manfredss commented Jun 29, 2026 •

edited

Loading

zhwesky2010 left a comment •

edited

Loading

zhwesky2010 commented Jul 13, 2026 •

edited

Loading

zhwesky2010 left a comment •

edited

Loading

zhwesky2010 Jul 14, 2026 •

edited

Loading

zhwesky2010 Jul 14, 2026 •

edited

Loading