Support solidity v0.8.0 grammar by tintinweb · Pull Request #19 · ConsenSysDiligence/python-solidity-parser

tintinweb · 2021-11-26T21:21:00Z

update solidity antlr parser to https://github.com/solidity-parser/antlr/tree/b6b34ca743ba04cc6f976413bc175b2175aa0bc2 (same as js solidity parser) (submodule repo changed as the original is not maintained anymore!)
fix glue code for the statements

note:

obsoletes Specify version in setup.py install_requires parameter #14 - the new grammar works with the latest version without printing a warning
fixes The parser doesn't support receive / fallback functions #16 - failed due to outdated grammar.
fixes Python-solidity-parser failed to handle throw keyword #11 - missing throw visitor.
fixes Read the file using utf8 #17 - hardcoding utf8 encoding.

@phe-sto - appreciate your PRs 🙌 do you mind providing feedback whether this PR solves your problems?

CLAassistant · 2021-11-26T21:21:09Z

All committers have signed the CLA.

phe-sto · 2021-11-26T22:17:29Z

Hi @tintinweb,

Unfortunately installing your branch this way still raise an error message, the same as before.

Installation log:

(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract$ git clone https://github.com/ConsenSys/python-solidity-parser.git
Cloning into 'python-solidity-parser'...
remote: Enumerating objects: 172, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 172 (delta 8), reused 17 (delta 4), pack-reused 144
Receiving objects: 100% (172/172), 265.73 KiB | 509.00 KiB/s, done.
Resolving deltas: 100% (81/81), done.
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract$ cd python-solidity-parser/
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract/python-solidity-parser$ git checkout update/solidity/grammar
Branch 'update/solidity/grammar' set up to track remote branch 'update/solidity/grammar' from 'origin'.
Switched to a new branch 'update/solidity/grammar'
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract/python-solidity-parser$ git pull
Already up to date.
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract/python-solidity-parser$ python3 setup.py install
running install
running bdist_egg
running egg_info
creating solidity_parser.egg-info
writing solidity_parser.egg-info/PKG-INFO
writing dependency_links to solidity_parser.egg-info/dependency_links.txt
writing requirements to solidity_parser.egg-info/requires.txt
writing top-level names to solidity_parser.egg-info/top_level.txt
writing manifest file 'solidity_parser.egg-info/SOURCES.txt'
reading manifest file 'solidity_parser.egg-info/SOURCES.txt'
writing manifest file 'solidity_parser.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/solidity_parser
copying solidity_parser/__main__.py -> build/lib/solidity_parser
copying solidity_parser/__init__.py -> build/lib/solidity_parser
copying solidity_parser/parser.py -> build/lib/solidity_parser
creating build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/__init__.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityListener.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityParser.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityVisitor.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityLexer.py -> build/lib/solidity_parser/solidity_antlr4
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/solidity_parser
copying build/lib/solidity_parser/__main__.py -> build/bdist.linux-x86_64/egg/solidity_parser
copying build/lib/solidity_parser/__init__.py -> build/bdist.linux-x86_64/egg/solidity_parser
copying build/lib/solidity_parser/parser.py -> build/bdist.linux-x86_64/egg/solidity_parser
creating build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/__init__.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityListener.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityParser.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityVisitor.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityLexer.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/__main__.py to __main__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/__init__.py to __init__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/parser.py to parser.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/__init__.py to __init__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityListener.py to SolidityListener.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityParser.py to SolidityParser.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityVisitor.py to SolidityVisitor.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityLexer.py to SolidityLexer.cpython-38.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/solidity_parser-0.0.7-py3.8.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing solidity_parser-0.0.7-py3.8.egg
Copying solidity_parser-0.0.7-py3.8.egg to /home/chrichri/crap-contract/venv/lib/python3.8/site-packages
Adding solidity-parser 0.0.7 to easy-install.pth file

Installed /home/chrichri/crap-contract/venv/lib/python3.8/site-packages/solidity_parser-0.0.7-py3.8.egg
Processing dependencies for solidity-parser==0.0.7
Searching for antlr4-python3-runtime==4.7.2
Best match: antlr4-python3-runtime 4.7.2
Adding antlr4-python3-runtime 4.7.2 to easy-install.pth file

Using /home/chrichri/crap-contract/venv/lib/python3.8/site-packages
Finished processing dependencies for solidity-parser==0.0.7

Execution log:

/home/chrichri/crap-contract/venv/bin/python /home/chrichri/crap-contract/check_safe_math.py
2021-11-26 23:09:02,642 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2021-11-26 23:09:02,642 INFO sqlalchemy.engine.Engine 
DROP TABLE safe_math_absent
2021-11-26 23:09:02,642 INFO sqlalchemy.engine.Engine [no key 0.00005s] ()
2021-11-26 23:09:02,641 INFO Logger initialized
2021-11-26 23:09:02,661 INFO sqlalchemy.engine.Engine COMMIT
2021-11-26 23:09:02,661 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2021-11-26 23:09:02,661 INFO sqlalchemy.engine.Engine 
CREATE TABLE safe_math_absent (
	contract_address VARCHAR, 
	FOREIGN KEY(contract_address) REFERENCES token (contract_address)
)


2021-11-26 23:09:02,661 INFO sqlalchemy.engine.Engine [no key 0.00008s] ()
2021-11-26 23:09:02,671 INFO sqlalchemy.engine.Engine COMMIT
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
line 1:0 mismatched input '{' expecting {<EOF>, 'pragma', 'import', 'from', 'abstract', 'contract', 'interface', 'library', 'error', 'struct', 'function', 'enum', 'address', 'mapping', 'calldata', 'revert', 'var', 'bool', 'string', 'byte', 'callback', Int, Uint, Byte, Fixed, Ufixed, 'leave', 'payable', 'type', 'constructor', 'fallback', 'receive', Identifier}
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3
ANTLR runtime and generated code versions disagree: 4.7.2!=4.9.3

Installing my branch remove this error message:

Installation log:

(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract$ git clone https://github.com/phe-sto/python-solidity-parser.git
Cloning into 'python-solidity-parser'...
remote: Enumerating objects: 155, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 155 (delta 3), reused 1 (delta 0), pack-reused 144
Receiving objects: 100% (155/155), 183.31 KiB | 358.00 KiB/s, done.
Resolving deltas: 100% (76/76), done.
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract$ cd python-solidity-parser/
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract/python-solidity-parser$ git checkout FIX-antlr4-runtime-dep-version
Branch 'FIX-antlr4-runtime-dep-version' set up to track remote branch 'FIX-antlr4-runtime-dep-version' from 'origin'.
Switched to a new branch 'FIX-antlr4-runtime-dep-version'
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract/python-solidity-parser$ git pull
Already up to date.
(venv) chrichri@chrichri-x470aorusultragaming:~/crap-contract/python-solidity-parser$ python3 setup.py install
running install
running bdist_egg
running egg_info
creating solidity_parser.egg-info
writing solidity_parser.egg-info/PKG-INFO
writing dependency_links to solidity_parser.egg-info/dependency_links.txt
writing requirements to solidity_parser.egg-info/requires.txt
writing top-level names to solidity_parser.egg-info/top_level.txt
writing manifest file 'solidity_parser.egg-info/SOURCES.txt'
reading manifest file 'solidity_parser.egg-info/SOURCES.txt'
writing manifest file 'solidity_parser.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/solidity_parser
copying solidity_parser/__main__.py -> build/lib/solidity_parser
copying solidity_parser/__init__.py -> build/lib/solidity_parser
copying solidity_parser/parser.py -> build/lib/solidity_parser
creating build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/__init__.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityListener.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityParser.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityVisitor.py -> build/lib/solidity_parser/solidity_antlr4
copying solidity_parser/solidity_antlr4/SolidityLexer.py -> build/lib/solidity_parser/solidity_antlr4
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/solidity_parser
copying build/lib/solidity_parser/__main__.py -> build/bdist.linux-x86_64/egg/solidity_parser
copying build/lib/solidity_parser/__init__.py -> build/bdist.linux-x86_64/egg/solidity_parser
copying build/lib/solidity_parser/parser.py -> build/bdist.linux-x86_64/egg/solidity_parser
creating build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/__init__.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityListener.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityParser.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityVisitor.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
copying build/lib/solidity_parser/solidity_antlr4/SolidityLexer.py -> build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/__main__.py to __main__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/__init__.py to __init__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/parser.py to parser.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/__init__.py to __init__.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityListener.py to SolidityListener.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityParser.py to SolidityParser.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityVisitor.py to SolidityVisitor.cpython-38.pyc
byte-compiling build/bdist.linux-x86_64/egg/solidity_parser/solidity_antlr4/SolidityLexer.py to SolidityLexer.cpython-38.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying solidity_parser.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/solidity_parser-0.0.7-py3.8.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing solidity_parser-0.0.7-py3.8.egg
Copying solidity_parser-0.0.7-py3.8.egg to /home/chrichri/crap-contract/venv/lib/python3.8/site-packages
Adding solidity-parser 0.0.7 to easy-install.pth file

Installed /home/chrichri/crap-contract/venv/lib/python3.8/site-packages/solidity_parser-0.0.7-py3.8.egg
Processing dependencies for solidity-parser==0.0.7
Searching for antlr4-python3-runtime==4.7.2
Best match: antlr4-python3-runtime 4.7.2
Adding antlr4-python3-runtime 4.7.2 to easy-install.pth file

Using /home/chrichri/crap-contract/venv/lib/python3.8/site-packages
Finished processing dependencies for solidity-parser==0.0.7

Execution log:

/home/chrichri/crap-contract/venv/bin/python /home/chrichri/crap-contract/check_safe_math.py
2021-11-26 23:10:43,893 INFO Logger initialized
2021-11-26 23:10:43,893 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2021-11-26 23:10:43,893 INFO sqlalchemy.engine.Engine 
DROP TABLE safe_math_absent
2021-11-26 23:10:43,893 INFO sqlalchemy.engine.Engine [no key 0.00005s] ()
2021-11-26 23:10:43,906 INFO sqlalchemy.engine.Engine COMMIT
2021-11-26 23:10:43,906 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2021-11-26 23:10:43,907 INFO sqlalchemy.engine.Engine 
CREATE TABLE safe_math_absent (
	contract_address VARCHAR, 
	FOREIGN KEY(contract_address) REFERENCES token (contract_address)
)


2021-11-26 23:10:43,907 INFO sqlalchemy.engine.Engine [no key 0.00011s] ()
2021-11-26 23:10:43,919 INFO sqlalchemy.engine.Engine COMMIT
line 17:0 extraneous input 'abstract' expecting {<EOF>, 'pragma', 'import', 'contract', 'interface', 'library'}
line 322:41 missing ';' at '{'
line 322:48 no viable alternative at input 'value:'
line 388:61 missing ';' at '{'
line 388:68 no viable alternative at input 'value:'
line 1:0 mismatched input '{' expecting {<EOF>, 'pragma', 'import', 'contract', 'interface', 'library'}
line 31:0 extraneous input 'abstract' expecting {<EOF>, 'pragma', 'import', 'contract', 'interface', 'library'}

tintinweb · 2021-11-26T23:43:41Z

Hey @phe-sto,

can you verify that you have antlr4-python3-runtime==4.9.3 installed? In your output it looks like it's 4.7.x while the code was now generated with 4.9.3 (latest), see https://github.com/ConsenSys/python-solidity-parser/blob/update/solidity%2Fgrammar/solidity_parser/solidity_antlr4/SolidityLexer.py#L1

⇒  pip3 freeze | grep antlr
antlr4-python3-runtime==4.9.3

phe-sto · 2021-11-27T00:26:47Z

Ooops the package was already installed in my virtual environment, I removed it. Installing your branch now use antlr4-python3-runtime==4.9.3 and everything is fine 👍

update solidity grammar to support v0.8.0:

f81d384

This was referenced Nov 26, 2021

Specify version in setup.py install_requires parameter #14

Closed

The parser doesn't support receive / fallback functions #16

Closed

tintinweb added 2 commits November 26, 2021 22:28

add throw keyword

76c8abd

hardcode file read utf8 - fixes #17

3ea5c7e

tintinweb added 2 commits November 29, 2021 11:36

pin antlr4-python3-runtime==4.9.3

9abf39c

update readme

925965e

tintinweb merged commit d003c45 into master Nov 29, 2021

tintinweb mentioned this pull request Nov 29, 2021

parser fails on payable(address(variable)) #18

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support solidity v0.8.0 grammar#19

Support solidity v0.8.0 grammar#19
tintinweb merged 5 commits into
masterfrom
update/solidity/grammar

tintinweb commented Nov 26, 2021 •

edited

Loading

Uh oh!

CLAassistant commented Nov 26, 2021 •

edited

Loading

Uh oh!

phe-sto commented Nov 26, 2021 •

edited

Loading

Uh oh!

tintinweb commented Nov 26, 2021

Uh oh!

phe-sto commented Nov 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

tintinweb commented Nov 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Nov 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phe-sto commented Nov 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tintinweb commented Nov 26, 2021

Uh oh!

phe-sto commented Nov 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tintinweb commented Nov 26, 2021 •

edited

Loading

CLAassistant commented Nov 26, 2021 •

edited

Loading

phe-sto commented Nov 26, 2021 •

edited

Loading