DBC VAL_ parser - #280
DBC VAL_ parser#280dek3rr wants to merge 3 commits into
Conversation
|
Thanks for working on this. This will improve readability of the carstate. I’m not sure why we need to change the C part of the CANParser though. Why can’t we just use the python regex to build a dict containing all the mappings from int to enum values? When creating the python instance of the CANParser we store this dict as an attribute. So a lookup in carstate will look like this: |
|
I originally had it as an attribute, but we aren't actually parsing the CAN so I thought it made more sense for it to live separately beside CANParser instead of inside it. The deciding factor for me was that certain definitions may also be useful outside of carstate so it wouldn't always make sense to me to call CANParser in those situations. CANDefine would be more lightweight. Sorry if I'm not understanding correctly, but I believe you're asking why I didn't just build the dictionary in the regex dbc.py file. I would argue that it makes it clearer to have everything follow the same call path into parser.py and built the same way as CANParser. Parsing definitions is very closely related to parsing messages and signals but is also distinctly different since they are static. This was just my solution to keep things uniform while maintaining some separability. |
|
@dekerr Just gotta step in and say you sound very knowledgable and you sound like you've thought this through. I think we need a bit of separation of concerns for sure in the codebase and a more maintainable and in the end TESTABLE codebase. This codebase is lacking a lot of modern unit testing and could definitely get there if we get methods smaller and make things more approachable. My random 2 cents... :) |
|
@dekerr yeah, I actually like CANDefine and in general being able to access dbc file info in a lightweight way, without calling CANParser. Let me think a bit more about it. |
|
@dekerr merged this change internally. The only mod to CANdefine I did is to change the |
30c7ca8 bump version to 1.5.3 9403dbe Need to fix wifi test before re-enabling. 0812362 GPS UART fix until boardd is refactored (#294) ffbdb87 python2 -> 3 fixes to pedal flasher (#292) 78b75ef Added build type to release version strings 736c2cb Fixed sending of bytes over PandaSerial 0894b28 Fixed USB power mode on black (#291) 4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (#290) a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (#287) 9486836 UART instability fix with high interrupt load (#283) 9a9e9d4 Fix usb_power_mode missing initialization (#289) af0960a DFU fix (#288) 70219d7 match safety enum in cereal (#285) a338d39 Fix build for jenkins test 78ef4a6 Stop charge (#284) 5266a40 Fix typo (#286) f4787ec Revert "turn on CDP when ignition switches on (#281)" d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas" e97b283 NONE and CLIENT should be the same thing in white/grey pandas 8c1df55 turn on CDP when ignition switches on (#281) 847a35d Fix bullet points fac0277 Misra update (#280) 5a04df6 Added description of regression tests to README c4aabae Fixed some python3 bugs in the test scripts and PandaSerial 9af0cb3 Bump version c4ac3d6 Disable GPS load switching on black pandas 078ee58 This is the correct table, actually 578b95e Misra table of coverage added d383a26 bump panda b98ca01 fix sdk build in python3 env (#279) 63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails e951d79 legacy code we don't control can remain python2 11b7151 Merge pull request #276 from commaai/python3 9893a84 Merge pull request #277 from zorrobyte/patch-1 d326869 Revert "revert back esptool to python2 and force to build esptools with python2" 875e760 revert back esptool to python2 and force to build esptools with python2 9c40e62 needed to install python3 ed2ac87 Also moved safety tests to python3 6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat b5a2cab this hopefully fixes build test 6280509 Fixes safety replay 2c220b6 this fixes language regr test fdbe789 use python 3 in Docker container ee1ae4f Better hash print 0de9ef7 Revert "Final 2to3 on the whole repo" c92fd3b Final 2to3 on the whole repo 5f2bc44 better b2a30fd make works! b74005d fix sign.py fe72770 read file as byte and no tab before sleep 32a344e Update README.md 2dc3409 2to3 applied ffa68ef undo unnecessary brackets for print dbc2480 Fix all the prints with 2to3, some need to be undo 5a7aeba xrange is gone 982c4c9 one more python3 env 1e2412a env python -> env python3 git-subtree-dir: panda git-subtree-split: 30c7ca8
30c7ca8 bump version to 1.5.3 9403dbe Need to fix wifi test before re-enabling. 0812362 GPS UART fix until boardd is refactored (commaai#294) ffbdb87 python2 -> 3 fixes to pedal flasher (commaai#292) 78b75ef Added build type to release version strings 736c2cb Fixed sending of bytes over PandaSerial 0894b28 Fixed USB power mode on black (commaai#291) 4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (commaai#290) a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (commaai#287) 9486836 UART instability fix with high interrupt load (commaai#283) 9a9e9d4 Fix usb_power_mode missing initialization (commaai#289) af0960a DFU fix (commaai#288) 70219d7 match safety enum in cereal (commaai#285) a338d39 Fix build for jenkins test 78ef4a6 Stop charge (commaai#284) 5266a40 Fix typo (commaai#286) f4787ec Revert "turn on CDP when ignition switches on (commaai#281)" d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas" e97b283 NONE and CLIENT should be the same thing in white/grey pandas 8c1df55 turn on CDP when ignition switches on (commaai#281) 847a35d Fix bullet points fac0277 Misra update (commaai#280) 5a04df6 Added description of regression tests to README c4aabae Fixed some python3 bugs in the test scripts and PandaSerial 9af0cb3 Bump version c4ac3d6 Disable GPS load switching on black pandas 078ee58 This is the correct table, actually 578b95e Misra table of coverage added d383a26 bump panda b98ca01 fix sdk build in python3 env (commaai#279) 63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails e951d79 legacy code we don't control can remain python2 11b7151 Merge pull request commaai#276 from commaai/python3 9893a84 Merge pull request commaai#277 from zorrobyte/patch-1 d326869 Revert "revert back esptool to python2 and force to build esptools with python2" 875e760 revert back esptool to python2 and force to build esptools with python2 9c40e62 needed to install python3 ed2ac87 Also moved safety tests to python3 6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat b5a2cab this hopefully fixes build test 6280509 Fixes safety replay 2c220b6 this fixes language regr test fdbe789 use python 3 in Docker container ee1ae4f Better hash print 0de9ef7 Revert "Final 2to3 on the whole repo" c92fd3b Final 2to3 on the whole repo 5f2bc44 better b2a30fd make works! b74005d fix sign.py fe72770 read file as byte and no tab before sleep 32a344e Update README.md 2dc3409 2to3 applied ffa68ef undo unnecessary brackets for print dbc2480 Fix all the prints with 2to3, some need to be undo 5a7aeba xrange is gone 982c4c9 one more python3 env 1e2412a env python -> env python3 git-subtree-dir: panda git-subtree-split: 30c7ca8
30c7ca8 bump version to 1.5.3 9403dbe Need to fix wifi test before re-enabling. 0812362 GPS UART fix until boardd is refactored (commaai#294) ffbdb87 python2 -> 3 fixes to pedal flasher (commaai#292) 78b75ef Added build type to release version strings 736c2cb Fixed sending of bytes over PandaSerial 0894b28 Fixed USB power mode on black (commaai#291) 4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (commaai#290) a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (commaai#287) 9486836 UART instability fix with high interrupt load (commaai#283) 9a9e9d4 Fix usb_power_mode missing initialization (commaai#289) af0960a DFU fix (commaai#288) 70219d7 match safety enum in cereal (commaai#285) a338d39 Fix build for jenkins test 78ef4a6 Stop charge (commaai#284) 5266a40 Fix typo (commaai#286) f4787ec Revert "turn on CDP when ignition switches on (commaai#281)" d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas" e97b283 NONE and CLIENT should be the same thing in white/grey pandas 8c1df55 turn on CDP when ignition switches on (commaai#281) 847a35d Fix bullet points fac0277 Misra update (commaai#280) 5a04df6 Added description of regression tests to README c4aabae Fixed some python3 bugs in the test scripts and PandaSerial 9af0cb3 Bump version c4ac3d6 Disable GPS load switching on black pandas 078ee58 This is the correct table, actually 578b95e Misra table of coverage added d383a26 bump panda b98ca01 fix sdk build in python3 env (commaai#279) 63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails e951d79 legacy code we don't control can remain python2 11b7151 Merge pull request commaai#276 from commaai/python3 9893a84 Merge pull request commaai#277 from zorrobyte/patch-1 d326869 Revert "revert back esptool to python2 and force to build esptools with python2" 875e760 revert back esptool to python2 and force to build esptools with python2 9c40e62 needed to install python3 ed2ac87 Also moved safety tests to python3 6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat b5a2cab this hopefully fixes build test 6280509 Fixes safety replay 2c220b6 this fixes language regr test fdbe789 use python 3 in Docker container ee1ae4f Better hash print 0de9ef7 Revert "Final 2to3 on the whole repo" c92fd3b Final 2to3 on the whole repo 5f2bc44 better b2a30fd make works! b74005d fix sign.py fe72770 read file as byte and no tab before sleep 32a344e Update README.md 2dc3409 2to3 applied ffa68ef undo unnecessary brackets for print dbc2480 Fix all the prints with 2to3, some need to be undo 5a7aeba xrange is gone 982c4c9 one more python3 env 1e2412a env python -> env python3 git-subtree-dir: panda git-subtree-split: 30c7ca8
* Updated cppcheck ref. New rules to be fixed * changed 2 MACRO names that had more than 31 characters in common * Fix newly popped 10.4: same type on arithmetic ops * Fix 18.4: operators should not be applied to an expression * ensure a NULL pointer isn't dereferenced
30c7ca8 bump version to 1.5.3 9403dbe Need to fix wifi test before re-enabling. 0812362 GPS UART fix until boardd is refactored (commaai#294) ffbdb87 python2 -> 3 fixes to pedal flasher (commaai#292) 78b75ef Added build type to release version strings 736c2cb Fixed sending of bytes over PandaSerial 0894b28 Fixed USB power mode on black (commaai#291) 4b3358c patch to be able to switch from EON to PC with a Panda that has EON b… (commaai#290) a95c44a Made setting of NOOUTPUT on no heartbeat more efficient (commaai#287) 9486836 UART instability fix with high interrupt load (commaai#283) 9a9e9d4 Fix usb_power_mode missing initialization (commaai#289) af0960a DFU fix (commaai#288) 70219d7 match safety enum in cereal (commaai#285) a338d39 Fix build for jenkins test 78ef4a6 Stop charge (commaai#284) 5266a40 Fix typo (commaai#286) f4787ec Revert "turn on CDP when ignition switches on (commaai#281)" d37daee Revert "NONE and CLIENT should be the same thing in white/grey pandas" e97b283 NONE and CLIENT should be the same thing in white/grey pandas 8c1df55 turn on CDP when ignition switches on (commaai#281) 847a35d Fix bullet points fac0277 Misra update (commaai#280) 5a04df6 Added description of regression tests to README c4aabae Fixed some python3 bugs in the test scripts and PandaSerial 9af0cb3 Bump version c4ac3d6 Disable GPS load switching on black pandas 078ee58 This is the correct table, actually 578b95e Misra table of coverage added d383a26 bump panda b98ca01 fix sdk build in python3 env (commaai#279) 63d3dc7 Set python3 env before runnign get_sdk, so we know if it fails e951d79 legacy code we don't control can remain python2 11b7151 Merge pull request commaai#276 from commaai/python3 9893a84 Merge pull request commaai#277 from zorrobyte/patch-1 d326869 Revert "revert back esptool to python2 and force to build esptools with python2" 875e760 revert back esptool to python2 and force to build esptools with python2 9c40e62 needed to install python3 ed2ac87 Also moved safety tests to python3 6842b2d move esptool sdk installation before python3 env is set. Kind of a cheat b5a2cab this hopefully fixes build test 6280509 Fixes safety replay 2c220b6 this fixes language regr test fdbe789 use python 3 in Docker container ee1ae4f Better hash print 0de9ef7 Revert "Final 2to3 on the whole repo" c92fd3b Final 2to3 on the whole repo 5f2bc44 better b2a30fd make works! b74005d fix sign.py fe72770 read file as byte and no tab before sleep 32a344e Update README.md 2dc3409 2to3 applied ffa68ef undo unnecessary brackets for print dbc2480 Fix all the prints with 2to3, some need to be undo 5a7aeba xrange is gone 982c4c9 one more python3 env 1e2412a env python -> env python3 git-subtree-dir: panda git-subtree-split: 30c7ca8 old-commit-hash: 38faf7f
This parser, CANDefine, pulls definition-value pairs from the respective dbc. It shares a similar call structure with CANParser.
For example:
given,
VAL_ 399 STEER_STATUS 5 "fault" 4 "no torque alert 2" 2 " no torque alert 1 " 0 "normal" ;the value of 2 corresponding to "no torque alert 1" is callable by:
cd.dv["STEER_STATUS"]['NO_TORQUE_ALERT_1']Repeated lines in the dbc are removed, leading and trailing white space is removed in each def/val pair, and naming is converted to uppercase with underscores for convention.
This is a very forgiving parser, but it requires a unique naming call--this is by design
VAL_ 506 FCW 3 "fcw" 2 "fcw" 1 "fcw" 0 "no_fcw" ;will only return FCW : 1 and NO_FCW : 0.
This will greatly reduce gear shifter logic in carstate and also removes other hard coding.