Hi @MarkusSchilling
While writing and testing the parser, I found out some stuff that needs clarification. These are very specific, so I will be pointing to the rows in the excel (something like "r4" -> "row 4" in the metadata values). Most of the feedback can be resolved with a "ok, go ahead", but some other require a quick clarification.
You can find the specific properties I am mentioning in this by looking for the codes in the object_types.py and vocabularies.py of the creep_test/ folder.
r4: it seems that CREEP_TEST_LINK_PROJECT is now defined as an OBJECT, but seeing the values in the excel, does it make more sense to simply use a VARCHAR?
r41: right now CREEP_TEST_HEAT_TREATMENT_STATE is defined as a CONTROLLEDVOCABULARY. However, when "Other" is used and specified, this breaks (as the parser cannot read an unexpected vocabulary term). We have two options: 1) map all non fitting cases to "OTHER_SPECIFY" term or 2) change this property to VARCHAR. I am inclined to follow option 2 here, as it seems easier to simply map into a VARCHAR knowing that you have some controlled options anyway. Same happens for (r56, CREEP_TEST_GRAIN_SIZE_DETERMINATION_METHOD), (r126, CREEP_TEST_THERMOCOUPLE_CALIBRATION_STANDARD).
r55: CREEP_TEST_GRAIN_SIZE is defined as a REAL with units of µm. However, I can see in the Excel they type things like "Radial-tangential plane: 46, radial-axial plane: 56". How do you suggest tackling this? I think you have many more examples in your Excels and can make a better decision here.
r90: small clarification, CREEP_TEST_CALIBRATION_CLASS is now a REAL, but in the Excel I have it says "Verification: the machine fulfills the requirements for test forces >2 kN". Should we change it to VARCHAR?
r99: CREEP_TEST_CALIBRATION_VALIDITY_TIME_PERIOD is now a REAL, but the Excel contains "24 Months". Should we change it to VARCHAR? How flexible is this? Or in other words, do people often write "2 days", "20 months", "1 year", and all other variants? The same happens for (r107, CREEP_TEST_CALIBRATION_VALIDITY_TIME_PERIOD), (r129, CREEP_TEST_CALIBRATION_VALIDITY_TIME_PERIOD), (r139, ...).
r102: CREEP_TEST_CALIBRATION_RANGE is a REAL with kN units. However, the Excel shows things like: "2 - 20 kN and 1 - 10 kN (class 1 for both)". I'll suggest to change this to VARCHAR. Same happens for (r111, CREEP_TEST_CALIBRATION_RANGE), (r131, CREEP_TEST_CALIBRATION_RANGE), (r143, ...), (r160, ...).
r112: CREEP_TEST_FORCE_RECORDING is defined as a BOOLEAN, but the field is a Text in the Excel. In the example I have, it is written "Phase-wise (during loading)" making the Yes/No mapping to True/False impossible.
r121: THERMOCOUPLE_TYPE is a CONTROLLEDVOCABULARY which contains terms like "TYPE_S", while the excel is simply "S" and so on. Given that this vocabulary has "OTHER_SPECIFY" and due to our problems with r41 (and others), I'd suggest to simply changing this to a VARCHAR.
r124: CREEP_TEST_CALIBRATION_STATUS is now a CONTROLLEDVOCABULARY with multiple terms (calibrated, not_calibrated, expired, unknown, not_applicable). However, in the Excel I've got it was typed just "Yes". How do we handle this? Same happens with (r135, CREEP_TEST_CALIBRATION_STATUS)
r144: CREEP_TEST_DISPLACEMENT_MEASURING_METHOD is a CONTROLLEDVOCABULARY with two terms, contacting_method or non_contacting_method. But in the excel there are things like "Contacting extensometer". Can you clarify whether this could be a VARCHAR instead?
r154 and r155: CREEP_TEST_EXTENSION_RANGE_UPPER_LIMIT (and the LOWER_LIMIT) are now REAL properties. The units are unclear, in my opinion. In the excel they appear as "... %/... mm". Should we match this to one unit (e.g., mm) only? Or do we better store these as VARCHAR? Something similar happens for r162, but in there the strings are like "... % (... mm)". My suggestion: we store all of these as VARCHAR.
Besides these, some values recorded during the test at the final rows are sometimes "not applicable..." or "not recorded...". In the schema, these are typically REAL, so the parser expects a numerical value. Nevertheless, the parser now handles these by sending a logs message, so it is handled properly and no action is required from our side.
Hi @MarkusSchilling
While writing and testing the parser, I found out some stuff that needs clarification. These are very specific, so I will be pointing to the rows in the excel (something like "r4" -> "row 4" in the metadata values). Most of the feedback can be resolved with a "ok, go ahead", but some other require a quick clarification.
You can find the specific properties I am mentioning in this by looking for the codes in the object_types.py and vocabularies.py of the creep_test/ folder.
r4: it seems that
CREEP_TEST_LINK_PROJECTis now defined as an OBJECT, but seeing the values in the excel, does it make more sense to simply use a VARCHAR?r41: right now
CREEP_TEST_HEAT_TREATMENT_STATEis defined as a CONTROLLEDVOCABULARY. However, when "Other" is used and specified, this breaks (as the parser cannot read an unexpected vocabulary term). We have two options: 1) map all non fitting cases to "OTHER_SPECIFY" term or 2) change this property to VARCHAR. I am inclined to follow option 2 here, as it seems easier to simply map into a VARCHAR knowing that you have some controlled options anyway. Same happens for (r56,CREEP_TEST_GRAIN_SIZE_DETERMINATION_METHOD), (r126,CREEP_TEST_THERMOCOUPLE_CALIBRATION_STANDARD).r55:
CREEP_TEST_GRAIN_SIZEis defined as a REAL with units of µm. However, I can see in the Excel they type things like "Radial-tangential plane: 46, radial-axial plane: 56". How do you suggest tackling this? I think you have many more examples in your Excels and can make a better decision here.r90: small clarification,
CREEP_TEST_CALIBRATION_CLASSis now a REAL, but in the Excel I have it says "Verification: the machine fulfills the requirements for test forces >2 kN". Should we change it to VARCHAR?r99:
CREEP_TEST_CALIBRATION_VALIDITY_TIME_PERIODis now a REAL, but the Excel contains "24 Months". Should we change it to VARCHAR? How flexible is this? Or in other words, do people often write "2 days", "20 months", "1 year", and all other variants? The same happens for (r107,CREEP_TEST_CALIBRATION_VALIDITY_TIME_PERIOD), (r129,CREEP_TEST_CALIBRATION_VALIDITY_TIME_PERIOD), (r139, ...).r102:
CREEP_TEST_CALIBRATION_RANGEis a REAL with kN units. However, the Excel shows things like: "2 - 20 kN and 1 - 10 kN (class 1 for both)". I'll suggest to change this to VARCHAR. Same happens for (r111,CREEP_TEST_CALIBRATION_RANGE), (r131,CREEP_TEST_CALIBRATION_RANGE), (r143, ...), (r160, ...).r112:
CREEP_TEST_FORCE_RECORDINGis defined as a BOOLEAN, but the field is a Text in the Excel. In the example I have, it is written "Phase-wise (during loading)" making the Yes/No mapping to True/False impossible.r121:
THERMOCOUPLE_TYPEis a CONTROLLEDVOCABULARY which contains terms like "TYPE_S", while the excel is simply "S" and so on. Given that this vocabulary has "OTHER_SPECIFY" and due to our problems with r41 (and others), I'd suggest to simply changing this to a VARCHAR.r124:
CREEP_TEST_CALIBRATION_STATUSis now a CONTROLLEDVOCABULARY with multiple terms (calibrated, not_calibrated, expired, unknown, not_applicable). However, in the Excel I've got it was typed just "Yes". How do we handle this? Same happens with (r135,CREEP_TEST_CALIBRATION_STATUS)r144:
CREEP_TEST_DISPLACEMENT_MEASURING_METHODis a CONTROLLEDVOCABULARY with two terms, contacting_method or non_contacting_method. But in the excel there are things like "Contacting extensometer". Can you clarify whether this could be a VARCHAR instead?r154 and r155:
CREEP_TEST_EXTENSION_RANGE_UPPER_LIMIT(and the LOWER_LIMIT) are now REAL properties. The units are unclear, in my opinion. In the excel they appear as "... %/... mm". Should we match this to one unit (e.g., mm) only? Or do we better store these as VARCHAR? Something similar happens for r162, but in there the strings are like "... % (... mm)". My suggestion: we store all of these as VARCHAR.Besides these, some values recorded during the test at the final rows are sometimes "not applicable..." or "not recorded...". In the schema, these are typically REAL, so the parser expects a numerical value. Nevertheless, the parser now handles these by sending a logs message, so it is handled properly and no action is required from our side.