Skip to content

[CodeStyle][py2] remove six package (part1) - #46965

Merged
luotao1 merged 20 commits into
PaddlePaddle:developfrom
cattidea:py2/fix/remove-six-part-1
Oct 19, 2022
Merged

luotao1 merged 20 commits into
PaddlePaddle:developfrom
cattidea:py2/fix/remove-six-part-1

Conversation

@SigureMo

@SigureMo SigureMo commented Oct 12, 2022 •

Copy link
Copy Markdown
Collaborator

PR types

Others

PR changes

Others

Describe

清理 six 模块的第一步,利用 https://github.com/cattidea/paddle-flake8-project/blob/main/flake8_scripts/six_remover.py 自动清理存量较多的 six 函数,手动清理部分难以自动清理的部分,目前清理的部分有明确的映射关系,与代码上下文无关,可安全清理

转换规则(大多数可参考 six 文档):

  • six.moves 下的函数:明确的映射,文档里有写
  • six.iteritems(d) -> d.items()
  • six.iterkeys(d) -> d.keys()
  • six.b(s)
    • six.b("xxx") -> b"xxx"
    • six.b(var) -> var.encode("latin-1")(与 six.b Python3 分支行为一致,单独为前者加一个 b 前缀是因为可读性更好,而且在源码上就是 bytes 了,不需要运行时转换)
  • six.u(s) -> s
  • six.<attr>
    • six.string_types -> (str, )
    • six.integer_types -> (int, )
    • six.class_types -> (type, ) 由于这三个经常以 isinstance(x, (str, )) 方式使用,因此增加了一个 pass 专门清理这里的括号,也就是清理为 isinstance(x, str)
    • six.text_type -> str
    • six.binary_type -> bytes
  • if six.PY3: <branch_py3>; else: <branch_py2> -> <branch_py3>
  • @six.add_metaclass(Baz); class Foo(Bar) -> class Foo(Bar, metaclass=Baz)

剩余部分(会在后续 PR 中清理):

  • six.ensure_str 需要根据上下文确定修改方式(类似 paddle.compat.to_text)
  • six.reraise 修改方案尚未确定,需要进一步确定下
  • python/paddle/fluid/dygraph/dygraph_to_static/convert_call_func.py 中的 BUILTIN_LIKELY_MODULES
  • 两个有问题的示例代码
  • paddle.compat 模块相关,在 compat 模块清理时一起清理

Note

由于后续要将 six 依赖项彻底清除,因此 six 的清除是针对全部目录的,没有任何目录被 ignore 掉

Related links

@paddle-bot

paddle-bot Bot commented Oct 12, 2022

Copy link
Copy Markdown

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@SigureMo SigureMo changed the title [CodeStyle][py2] remove six package (part 1) [WIP][CodeStyle][py2] remove six package (part 1) Oct 12, 2022
@paddle-bot-old paddle-bot-old Bot added the contributor External developers label Oct 12, 2022
@SigureMo SigureMo changed the title [WIP][CodeStyle][py2] remove six package (part 1) [CodeStyle][py2] remove six package (part 1) Oct 13, 2022
@SigureMo SigureMo changed the title [CodeStyle][py2] remove six package (part 1) [CodeStyle][py2] remove six package (part 1) Oct 13, 2022
@luotao1 luotao1 self-assigned this Oct 13, 2022
@luotao1

luotao1 commented Oct 13, 2022

Copy link
Copy Markdown
Collaborator

@SigureMo Coverage流水线已豁免,static-check流水线有2处API发生变化,可以在对应行标注下,便于 @Ligoml 审核

luotao1
luotao1 previously approved these changes Oct 13, 2022

@SigureMo SigureMo left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ligoml 在这里在这里~两个 docstring 的修改~

max_sizes = []
step = int(math.floor(((max_ratio - min_ratio)) / (num_layer - 2)))
for ratio in six.moves.range(min_ratio, max_ratio + 1, step):
for ratio in range(min_ratio, max_ratio + 1, step):

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里有一处 docstring 修改

print("op inputs are {}".format(op.input_arg_names))
print("op outputs are {}".format(op.output_arg_names))
for key, value in sorted(six.iteritems(op.all_attrs())):
for key, value in sorted(op.all_attrs().items()):

@SigureMo SigureMo Oct 13, 2022 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个 clone 函数示例代码里有几处修改

Ligoml
Ligoml previously approved these changes Oct 17, 2022
@luotao1

luotao1 commented Oct 18, 2022

Copy link
Copy Markdown
Collaborator

需要解决下冲突

XiaoguangHu01
XiaoguangHu01 previously approved these changes Oct 19, 2022

@XiaoguangHu01 XiaoguangHu01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@luotao1

luotao1 commented Oct 19, 2022

Copy link
Copy Markdown
Collaborator

需要解决下冲突

@SigureMo

Copy link
Copy Markdown
Collaborator Author

需要解决下冲突

已解决~

@luotao1
luotao1 merged commit e6fb551 into PaddlePaddle:develop Oct 19, 2022
@SigureMo
SigureMo deleted the py2/fix/remove-six-part-1 branch October 19, 2022 07:57
@SigureMo SigureMo changed the title [CodeStyle][py2] remove six package (part 1) [CodeStyle][py2] remove six package (part1) Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants