Skip to content

DBC VAL_ parser - #280

Closed
dek3rr wants to merge 3 commits into
commaai:develfrom
dek3rr:dbc_vals
Closed

dek3rr wants to merge 3 commits into
commaai:develfrom
dek3rr:dbc_vals

Conversation

@dek3rr

@dek3rr dek3rr commented Jul 1, 2018

Copy link
Copy Markdown

This parser, CANDefine, pulls definition-value pairs from the respective dbc. It shares a similar call structure with CANParser.
For example:
given,
VAL_ 399 STEER_STATUS 5 "fault" 4 "no torque alert 2" 2 " no torque alert 1 " 0 "normal" ;
the value of 2 corresponding to "no torque alert 1" is callable by:
cd.dv["STEER_STATUS"]['NO_TORQUE_ALERT_1']
Repeated lines in the dbc are removed, leading and trailing white space is removed in each def/val pair, and naming is converted to uppercase with underscores for convention.
This is a very forgiving parser, but it requires a unique naming call--this is by design
VAL_ 506 FCW 3 "fcw" 2 "fcw" 1 "fcw" 0 "no_fcw" ;
will only return FCW : 1 and NO_FCW : 0.

This will greatly reduce gear shifter logic in carstate and also removes other hard coding.

@rbiasini

rbiasini commented Jul 9, 2018

Copy link
Copy Markdown
Contributor

Thanks for working on this. This will improve readability of the carstate.

I’m not sure why we need to change the C part of the CANParser though. Why can’t we just use the python regex to build a dict containing all the mappings from int to enum values? When creating the python instance of the CANParser we store this dict as an attribute.

So a lookup in carstate will look like this:
self.stopped = cp.vl["ACC_HUD"]['CRUISE_SPEED'] == cp.dv["ACC_HUD"]["CRUISE_SPEED"]['STOPPED']

@dek3rr

dek3rr commented Jul 9, 2018

Copy link
Copy Markdown
Author

I originally had it as an attribute, but we aren't actually parsing the CAN so I thought it made more sense for it to live separately beside CANParser instead of inside it. The deciding factor for me was that certain definitions may also be useful outside of carstate so it wouldn't always make sense to me to call CANParser in those situations. CANDefine would be more lightweight.

Sorry if I'm not understanding correctly, but I believe you're asking why I didn't just build the dictionary in the regex dbc.py file. I would argue that it makes it clearer to have everything follow the same call path into parser.py and built the same way as CANParser.

Parsing definitions is very closely related to parsing messages and signals but is also distinctly different since they are static. This was just my solution to keep things uniform while maintaining some separability.

@jfrux

jfrux commented Jul 10, 2018

Copy link
Copy Markdown

@dekerr Just gotta step in and say you sound very knowledgable and you sound like you've thought this through. I think we need a bit of separation of concerns for sure in the codebase and a more maintainable and in the end TESTABLE codebase. This codebase is lacking a lot of modern unit testing and could definitely get there if we get methods smaller and make things more approachable.

My random 2 cents... :)

@rbiasini

Copy link
Copy Markdown
Contributor

@dekerr yeah, I actually like CANDefine and in general being able to access dbc file info in a lightweight way, without calling CANParser. Let me think a bit more about it.

@rbiasini

rbiasini commented Aug 4, 2018

Copy link
Copy Markdown
Contributor

@dekerr merged this change internally. The only mod to CANdefine I did is to change the dv dict so that there is an extra layer: you need to specify the message too (either by address or name). Indeed, signal names are not meant to be unique in a dbc.
Basically you look up by cd.dv["GEARBOX"]["GEAR_SHIFTER"] instead of simply cd.dv["GEAR_SHIFTER"]
So, closing this PR as it will be released in 0.5.2. Nice work!

@rbiasini rbiasini closed this Aug 4, 2018
@dek3rr
dek3rr deleted the dbc_vals branch June 11, 2019 01:20
pd0wm pushed a commit that referenced this pull request Oct 9, 2019
30c7ca8 bump version to 1.5.3
9403dbe Need to fix wifi test before re-enabling.
0812362 GPS UART fix until boardd is refactored (#294)
ffbdb87 python2 -> 3 fixes to pedal flasher (#292)
78b75ef Added build type to release version strings
736c2cb Fixed sending of bytes over PandaSerial
0894b28 Fixed USB power mode on black (#291)
4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (#290)
a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (#287)
9486836 UART instability fix with high interrupt load (#283)
9a9e9d4 Fix usb_power_mode missing initialization (#289)
af0960a DFU fix (#288)
70219d7 match safety enum in cereal (#285)
a338d39 Fix build for jenkins test
78ef4a6 Stop charge (#284)
5266a40 Fix typo (#286)
f4787ec Revert "turn on CDP when ignition switches on (#281)"
d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas"
e97b283 NONE and CLIENT should be the same thing in white/grey pandas
8c1df55 turn on CDP when ignition switches on (#281)
847a35d Fix bullet points
fac0277 Misra update (#280)
5a04df6 Added description of regression tests to README
c4aabae Fixed some python3 bugs in the test scripts and PandaSerial
9af0cb3 Bump version
c4ac3d6 Disable GPS load switching on black pandas
078ee58 This is the correct table, actually
578b95e Misra table of coverage added
d383a26 bump panda
b98ca01 fix sdk build in python3 env (#279)
63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails
e951d79 legacy code we don't control can remain python2
11b7151 Merge pull request #276 from commaai/python3
9893a84 Merge pull request #277 from zorrobyte/patch-1
d326869 Revert "revert back esptool to python2 and force to build esptools with python2"
875e760 revert back esptool to python2 and force to build esptools with python2
9c40e62 needed to install python3
ed2ac87 Also moved safety tests to python3
6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat
b5a2cab this hopefully fixes build test
6280509 Fixes safety replay
2c220b6 this fixes language regr test
fdbe789 use python 3 in Docker container
ee1ae4f Better hash print
0de9ef7 Revert "Final 2to3 on the whole repo"
c92fd3b Final 2to3 on the whole repo
5f2bc44 better
b2a30fd make works!
b74005d fix sign.py
fe72770 read file as byte and no tab before sleep
32a344e Update README.md
2dc3409 2to3 applied
ffa68ef undo unnecessary brackets for print
dbc2480 Fix all the prints with 2to3, some need to be undo
5a7aeba xrange is gone
982c4c9 one more python3 env
1e2412a env python -> env python3

git-subtree-dir: panda
git-subtree-split: 30c7ca8
jyoung8607 pushed a commit to jyoung8607/openpilot that referenced this pull request Oct 10, 2019
30c7ca8 bump version to 1.5.3
9403dbe Need to fix wifi test before re-enabling.
0812362 GPS UART fix until boardd is refactored (commaai#294)
ffbdb87 python2 -> 3 fixes to pedal flasher (commaai#292)
78b75ef Added build type to release version strings
736c2cb Fixed sending of bytes over PandaSerial
0894b28 Fixed USB power mode on black (commaai#291)
4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (commaai#290)
a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (commaai#287)
9486836 UART instability fix with high interrupt load (commaai#283)
9a9e9d4 Fix usb_power_mode missing initialization (commaai#289)
af0960a DFU fix (commaai#288)
70219d7 match safety enum in cereal (commaai#285)
a338d39 Fix build for jenkins test
78ef4a6 Stop charge (commaai#284)
5266a40 Fix typo (commaai#286)
f4787ec Revert "turn on CDP when ignition switches on (commaai#281)"
d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas"
e97b283 NONE and CLIENT should be the same thing in white/grey pandas
8c1df55 turn on CDP when ignition switches on (commaai#281)
847a35d Fix bullet points
fac0277 Misra update (commaai#280)
5a04df6 Added description of regression tests to README
c4aabae Fixed some python3 bugs in the test scripts and PandaSerial
9af0cb3 Bump version
c4ac3d6 Disable GPS load switching on black pandas
078ee58 This is the correct table, actually
578b95e Misra table of coverage added
d383a26 bump panda
b98ca01 fix sdk build in python3 env (commaai#279)
63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails
e951d79 legacy code we don't control can remain python2
11b7151 Merge pull request commaai#276 from commaai/python3
9893a84 Merge pull request commaai#277 from zorrobyte/patch-1
d326869 Revert "revert back esptool to python2 and force to build esptools with python2"
875e760 revert back esptool to python2 and force to build esptools with python2
9c40e62 needed to install python3
ed2ac87 Also moved safety tests to python3
6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat
b5a2cab this hopefully fixes build test
6280509 Fixes safety replay
2c220b6 this fixes language regr test
fdbe789 use python 3 in Docker container
ee1ae4f Better hash print
0de9ef7 Revert "Final 2to3 on the whole repo"
c92fd3b Final 2to3 on the whole repo
5f2bc44 better
b2a30fd make works!
b74005d fix sign.py
fe72770 read file as byte and no tab before sleep
32a344e Update README.md
2dc3409 2to3 applied
ffa68ef undo unnecessary brackets for print
dbc2480 Fix all the prints with 2to3, some need to be undo
5a7aeba xrange is gone
982c4c9 one more python3 env
1e2412a env python -> env python3

git-subtree-dir: panda
git-subtree-split: 30c7ca8
jyoung8607 pushed a commit to jyoung8607/openpilot that referenced this pull request Nov 6, 2019
30c7ca8 bump version to 1.5.3
9403dbe Need to fix wifi test before re-enabling.
0812362 GPS UART fix until boardd is refactored (commaai#294)
ffbdb87 python2 -> 3 fixes to pedal flasher (commaai#292)
78b75ef Added build type to release version strings
736c2cb Fixed sending of bytes over PandaSerial
0894b28 Fixed USB power mode on black (commaai#291)
4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (commaai#290)
a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (commaai#287)
9486836 UART instability fix with high interrupt load (commaai#283)
9a9e9d4 Fix usb_power_mode missing initialization (commaai#289)
af0960a DFU fix (commaai#288)
70219d7 match safety enum in cereal (commaai#285)
a338d39 Fix build for jenkins test
78ef4a6 Stop charge (commaai#284)
5266a40 Fix typo (commaai#286)
f4787ec Revert "turn on CDP when ignition switches on (commaai#281)"
d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas"
e97b283 NONE and CLIENT should be the same thing in white/grey pandas
8c1df55 turn on CDP when ignition switches on (commaai#281)
847a35d Fix bullet points
fac0277 Misra update (commaai#280)
5a04df6 Added description of regression tests to README
c4aabae Fixed some python3 bugs in the test scripts and PandaSerial
9af0cb3 Bump version
c4ac3d6 Disable GPS load switching on black pandas
078ee58 This is the correct table, actually
578b95e Misra table of coverage added
d383a26 bump panda
b98ca01 fix sdk build in python3 env (commaai#279)
63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails
e951d79 legacy code we don't control can remain python2
11b7151 Merge pull request commaai#276 from commaai/python3
9893a84 Merge pull request commaai#277 from zorrobyte/patch-1
d326869 Revert "revert back esptool to python2 and force to build esptools with python2"
875e760 revert back esptool to python2 and force to build esptools with python2
9c40e62 needed to install python3
ed2ac87 Also moved safety tests to python3
6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat
b5a2cab this hopefully fixes build test
6280509 Fixes safety replay
2c220b6 this fixes language regr test
fdbe789 use python 3 in Docker container
ee1ae4f Better hash print
0de9ef7 Revert "Final 2to3 on the whole repo"
c92fd3b Final 2to3 on the whole repo
5f2bc44 better
b2a30fd make works!
b74005d fix sign.py
fe72770 read file as byte and no tab before sleep
32a344e Update README.md
2dc3409 2to3 applied
ffa68ef undo unnecessary brackets for print
dbc2480 Fix all the prints with 2to3, some need to be undo
5a7aeba xrange is gone
982c4c9 one more python3 env
1e2412a env python -> env python3

git-subtree-dir: panda
git-subtree-split: 30c7ca8
Vaggysag pushed a commit to Vaggysag/openpilot that referenced this pull request Apr 21, 2020
* Updated cppcheck ref. New rules to be fixed

* changed 2 MACRO names that had more than 31 characters in common

* Fix newly popped 10.4: same type on arithmetic ops

* Fix 18.4: operators should not be applied to an expression

* ensure a NULL pointer isn't dereferenced
nworb-cire pushed a commit to opgm/openpilot that referenced this pull request Aug 2, 2024
66thparallel pushed a commit to 66thparallel/janes_openpilot_fork that referenced this pull request May 1, 2026
30c7ca8 bump version to 1.5.3
9403dbe Need to fix wifi test before re-enabling.
0812362 GPS UART fix until boardd is refactored (commaai#294)
ffbdb87 python2 -> 3 fixes to pedal flasher (commaai#292)
78b75ef Added build type to release version strings
736c2cb Fixed sending of bytes over PandaSerial
0894b28 Fixed USB power mode on black (commaai#291)
4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (commaai#290)
a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (commaai#287)
9486836 UART instability fix with high interrupt load (commaai#283)
9a9e9d4 Fix usb_power_mode missing initialization (commaai#289)
af0960a DFU fix (commaai#288)
70219d7 match safety enum in cereal (commaai#285)
a338d39 Fix build for jenkins test
78ef4a6 Stop charge (commaai#284)
5266a40 Fix typo (commaai#286)
f4787ec Revert "turn on CDP when ignition switches on (commaai#281)"
d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas"
e97b283 NONE and CLIENT should be the same thing in white/grey pandas
8c1df55 turn on CDP when ignition switches on (commaai#281)
847a35d Fix bullet points
fac0277 Misra update (commaai#280)
5a04df6 Added description of regression tests to README
c4aabae Fixed some python3 bugs in the test scripts and PandaSerial
9af0cb3 Bump version
c4ac3d6 Disable GPS load switching on black pandas
078ee58 This is the correct table, actually
578b95e Misra table of coverage added
d383a26 bump panda
b98ca01 fix sdk build in python3 env (commaai#279)
63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails
e951d79 legacy code we don't control can remain python2
11b7151 Merge pull request commaai#276 from commaai/python3
9893a84 Merge pull request commaai#277 from zorrobyte/patch-1
d326869 Revert "revert back esptool to python2 and force to build esptools with python2"
875e760 revert back esptool to python2 and force to build esptools with python2
9c40e62 needed to install python3
ed2ac87 Also moved safety tests to python3
6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat
b5a2cab this hopefully fixes build test
6280509 Fixes safety replay
2c220b6 this fixes language regr test
fdbe789 use python 3 in Docker container
ee1ae4f Better hash print
0de9ef7 Revert "Final 2to3 on the whole repo"
c92fd3b Final 2to3 on the whole repo
5f2bc44 better
b2a30fd make works!
b74005d fix sign.py
fe72770 read file as byte and no tab before sleep
32a344e Update README.md
2dc3409 2to3 applied
ffa68ef undo unnecessary brackets for print
dbc2480 Fix all the prints with 2to3, some need to be undo
5a7aeba xrange is gone
982c4c9 one more python3 env
1e2412a env python -> env python3

git-subtree-dir: panda
git-subtree-split: 30c7ca8

old-commit-hash: 38faf7f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants